Parsing JSON Strings with Python

What is JSON?

JSON (JavaScript Object Notation) is a lightweight text format for storing structured data in a human-readable format. Compared to XML, JSON data takes up less space, is easier for humans to read, and for computers to process. JSON is often used to transfer information over the network in client/server and server/server communications. JSON itself has no methods or functions; it's just a data storage format. Although JSON came from JavaScript, it is now actively used in almost all programming languages, including Python, Java, PHP, C++, C#, Go, and most of them, like Python, have built-in modules for working with JSON data.

{
    "Id": 78912,
    "Customer": "Jason Sweet",
    "Quantity": 1,
    "Price": 18.00
}

What is JSON deserialization?

Deserialization is the process of converting (decoding) JSON data into appropriate data types in the Python programming language. The JSON decoder converts the JSON data to a Python dictionary using the methods of the corresponding JSON parser module to accomplish this task:

How to work with JSON in Python?

Python has a built-in module called json (JSON Encoder and Decoder) that makes it easy to work with JSON data in Python. Using the JSON Encoder and Decoder module, you can serialize (dump) Python objects to JSON strings or deserialize (parse) JSON strings to Python objects. The JSON encoder can create pretty or compact JSON strings, or convert Python objects directly to JSON files. The JSON decoder can decode (or parse) JSON strings and JSON files into Python objects. You can extend the JSON parser with custom processors by passing a processor to the json.loads() method.

To work with JSON in Python, you need to import the json module first, then call json.loads() method to parse the JSON string to a Python object.

import json

json_str = '{"Id": 12345}'

data = json.loads(json_str)

print(data['Id'])

# output: 12345

How does JSON Decoder convert JSON string to Python object?

The JSON parser can deserialize both JSON strings and JSON files to Python objects using the conversion table below. The json.load() method accepts an optional "object_hook" parameter that can be used to perform custom deserialization of some JSON objects.

How to perform custom deserialization when parsing JSON strings into Python objects?

You can perform custom deserialization by passing the deserialization function to the json.loads() method with the optional "object_hook" parameter. The deserialization function will be called on each JSON object, and the result of the function will be used to provide a custom Python type instead of a dictionary.

import json

class User(object):
    def __init__(self, id, name):
        self.id = id
        self.name = name

def user_decoder(obj):
    return User(obj['id'], obj['name'])

json_str = '{"id": 12345, "name": "John Smith"}'

user = json.loads(json_str, object_hook=user_decoder)

print(f"Id: {user.id}, Name: {user.name}") 

# output: Id: 12345, Name: John Smith

Python JSON parser-related exceptions

The JSONDecodeError class is a subclass of ValueError and represents exceptions when decoding JSON strings into a Python objects. The parser exception indicates a problem with the integrity of your JSON data. For example, the JSON data might have a syntax error (missing closing curly brace) or a key with no value (eg. {"name": }).

import json
from json.decoder import JSONDecodeError

json_str = '{"id": 12345, "name": }'

try:
    user = json.loads(json_str)

except JSONDecodeError as exp:
    print(exp.msg)

# output: Expecting value

The json.loads() syntax

The json.loads() method from the JSON Encoder and Decoder module performs the JSON to Python conversion. The json.loads() method takes a string (and the paired json.load() method takes a file object), parses the JSON data, converts it to the appropriate data types in Python, populates the dictionary with this data, and returns it. The reverse conversion from Python objects to JSON strings is performed by the json.dumps() method from the same library.

json.loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

• s: a string, bytes, or bytearray containing a JSON document.
• object_hook (optional): a function that is called by decoder while processing the JSON. Allows you to get custom Python objects instead of dict.
• cls (optional): a class that contains a custom JSON parsing implementation to be used in place of the standard JSONDecoder.
• parse_float (optional): a custom string to float converting function.
• parse_int (optional): a custom string to int converting function.
• parse_constant (optional): a function that can be used to throw an exception when invalid numbers are encountered in JSON.
• object_pairs_hook (optional): a function will be called to decode an ordered list of pairs and return Python objects instead of a dict. The object_pairs_hook function has a higher priority than object_hook functnion.

import json

json_str = '{"id": 12345, "person":  {"name":  "John Smith"}}'
        
data = json.loads(json_str)

print(f'Name: {data["person"]["name"]}')

# output: Name: John Smith

How to parse a JSON file into a Python object?

To convert a JSON file to a Python object using JSON Decoder, you can use the json.load() method (without the "s"), which has a syntax similar to json.loads() but accepts a file object instead of a string.

import json

with open('data.json') as file_object:
    data = json.load(file_object)

print(data)

How to access JSON array elements in the Python dict?

The JSON Decoder converts a JSON array to a Python list data type. You can access an element from a JSON array by its index in a Python object.

import json

json_str =  """
{
  "Orders": [
    {"Id": 78912},
    {"Id": 88953}
  ]
}
"""''
data = json.loads(json_str)
      
order_1_id = data['Orders'][0]['Id']
order_2_id = data['Orders'][1]['Id']
total = len(data['Orders'])
      
print(f"Order #1: {order_1_id}, Order #2: {order_2_id}, Total Orders: {total}")

# output: Order #1: 78912, Order #2: 88953, Total Orders: 2

How to parse JSON data from an URL to Python object?

First, install the Python Requests Library:

pip install requests

Then use the requests.get() method to download the JSON data from the target URL to a Python string. After this, you can use json.loads() method to convert the resulting JSON string to a Python object.

import json 
import requests 
      
json_str = requests.get("https://reqbin.com/echo/get/json")
      
data = json.loads(json_str.text)
           
print(data)

# output: {'success': 'true'}

How does JSON Parser handle duplicate keys in JSON?

RFC 8259 requires that the key name SHOULD be unique in JSON, but this is not required. The Python JSON Parser library does not throw duplicate key exceptions. It ignores duplicate key-value pairs and takes the last key-value pair.

import json

json_str = '{"key":  1, "key":  2, "key":  3}'
        
data = json.loads(json_str)
        
print(f'Key: {data["key"]}')

# output: Key: 3

What is the difference between json.loads() and json.dumps()?

The json.dumps() method is the opposite of json.loads(). It takes a Python object and returns a JSON string. The json.dumps() method can generate pretty or compact JSON depending on the parameters passed.

import json

obj = {"Id": 12345, "Quantity": 1, "Price": 18.00}

json_str = json.dumps(obj, indent=4)
        
print(json_str)

Python to JSON conversion result:

{
    "Id": 12345,
    "Quantity": 1,
    "Price": 18.0
}

See also

• How do I add comments to JSON?
• How do I split strings in Python?
• How do I reverse strings in Python?
• How do I convert an object to JSON in Python?
• How do I compare strings in Python?
• How can I dump Python object to JSON string using json.dumps()?
• How do I get the length of a list in Python?