Downstream Processing

In addition to the tools that output and format JSON data for the user are several tools that provide downstream processing of JSON as input in other programs (post-processing).

If you receive JSON data via an interface, it is good programming practice to sanity check the received data before further processing. The sanity check includes two stages:

• Syntactic correctness – is the spelling correct? Do all brackets (equal number of opening and closing brackets), commas, and quotation marks fit the bill?
• Correctness of data fields – does the received data structure match the data definition (JSON schema)?

For the first question, it is best to use JSONLint, which is described earlier in this article. For the second stage, you need the JSON schema that describes the data structure. You then compare this description with the received data.

On json-schema.org, you will find an overview of validators [15], sorted by the various programming languages in which they were developed. For example, consider the validate-json tool implemented in PHP [16]. If you are more into Python, jsonschema [17] serves the same purpose. The call to the two tools is identical.

Defining the JSON Schema

Listing 10 shows the JSON schema with which you define the exact format of your data structure. The schema matches the book inventory used earlier in this article. The schema was stored in the book-inventory-schema.json file in the local directory.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "Book",
  "type": "object",
  "required": ["author", "title", "publication"],
  "properties": {
    "author": {
      "type": "string",
      "description": "The author's name"
    },
    "title": {
      "type": "string",
      "description": "The book's title"
    },
    "publication": {
      "type": "number",
      "minimum": 0
    },
    "tags": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  }
}

The schema definition references the JSON standard used (the draft from September 2019, in this case) in the second line. The definition contains a number of keywords. Table 3 explains these keywords in more detail; a complete list of all supported keywords is available at json-schema.org [18].

The next task is to validate the records by checking whether they match the specified schema. Listing 11 shows a single record from the book inventory in readable format. The compact version of the record contains all the parentheses and fields in a single line.

{
  "author": "Stephen Fry",
  "title": "The Hippopotamus",
  "publication": 1994
}

The validate-json tool expects two parameters in the call, the dataset and the schema (Listing 12). If everything goes well, the output does not cause any further feedback (line 2); otherwise, validate-json grumbles (lines 4 and 5). To provoke the error message starting in line 4, we turned the numeric specification for the year of publication (1994) into a string "1994", which means that the data type in the dataset no longer matched the stored data type in the JSON schema. validate-json has every reason to complain.

01 $ validate-json record.json bookinventory-schema.json
02 $
03 $ validate-json record.json bookinventory-schema.json
04 JSON does not validate. Violations:
05 [publication] String value found, but a number is required

Some programming languages also offer suitable helper libraries. In Python, for example, you can use jsonschema, and for NodeJS, you can use the Express framework.

Processing JSON

The list of command-line tools and helpers that read, search on, and modify JSON data is quite extensive. We stopped counting after more than 20 entries (see Table 4 for a sample). Developer Ilya Sher maintains a useful, commented overview of options [19].

Jtbl, for example, takes JSON records and knits a pretty table from them. In Figure 7, you can see how this table looks for the book inventory. Each record is shown in a separate row. Jtbl can only cope with flat JSON structures. It cannot handle nesting so far.

Figure 7: Representing records as a table.