Verifying JSON Data

The formatted output of a tool like jq or aeson-pretty usually helps to detect obvious errors in the JSON structure at a glance, but if you need a closer look, you can check your JSON data with JSONLint. The tool expects the file with the JSON data as a parameter. If everything is correct, it reports back that JSON is valid. If there is an error, it outputs the location that it identified as the error. Figure 2 shows JSONLint output for the book inventory – first with a correct JSON file, and then with a variant that is missing a comma as a separator.

Figure 2: JSONLint helps you identify errors in JSON data.

JSONLint was written in PHP. If you are looking for an alternative, you may also be able to get by with jq. If jq fails to parse the JSON data, it returns a number greater than zero, otherwise zero. Listing 8 shows the output for a deliberate error.

$ cat book-inventory-broken.json | jq .
parse error: Expected separator between values at line 7, column 5
$ echo $?
4

Converting Output to JSON

All Unix/Linux tools have their own specific output format. With a clever combination of grep, sed, and awk, you can break down the output and create the format you need for further processing. This approach sounds simple, but it often feels like walking up the stairs backwards while balancing a crystal vase on your head.

On the other hand, if every Unix/Linux tool had a --json switch and used it to create output in JSON format, the output could be parsed in a standardized way. However, the world still has a long way to go before this happens, so a workaround is needed. Jc and jo can both read the output from a tool, convert the output, and flip it back to the standard output in JSON format.

The list of output formats that jc understands is quite long and includes the output from df, du, lsblk, crontab, netstat, and lsof. Figure 3 shows output from the route command, as processed by the jc web demo page [13]. On the demo page, you can select the desired Unix/Linux command or data format at the top, and then copy the associated output into the input box. Click on Convert to JSON to create the output below – each entry is a JSON element. Use the Pretty Print checkbox to specify whether the output should be a compact one-liner or a prettied-up, longer version.

Figure 3: The demo app shows output converted into JSON format using route as an example.

Building complicated JSON files yourself and counting parentheses – that was yesterday. Today, jo does it for you. Jo expects the key-value pairs as parameters and screws together a corresponding JSON output from them. Figure 4 shows the output for the two parameters magazine and issue.

Figure 4: Jo lets you compile JSON output without any serious overhead.

Because jo receives the key-value pairs as parameters when called, variable content from the shell is no longer a problem. See Listing 9, which shows variables for today's date and home directory.

$ jo timeofday="$(date +%c" home=$HOME
{"time of day":"Mon 12 Oct 2020 17:06:30 CEST","home":"/home/frank"}

Interactive Filters

Tools like jq, jid, and Jshon can filter the output if you only need part of the data. Earlier you learned that passing a . to jq outputs the entire dataset. With the appropriate call, you can filter the data and extract the author, title, and publication data from a JSON file with book publishing data. Using .book[], you first narrow down the search to the book list; then you filter all items with the publication key using a pipe (|).

Figure 5 shows the results from jqplay – a simple playground tool in the web browser whose contents are passed to jq for processing. If you change the filter or the output in the two input fields on the left, the output on the right adjusts.

Figure 5: Jq can filter JSON output by arbitrary criteria.

Jid stands for JSON Interactive Digger. A call to cat book_inventory.json | jid lets you browse a JSON file interactively.

We have not yet discovered any special programs with graphical interfaces for editing JSON data. All text editors offer syntax highlighting and thus simplify editing. We were particularly impressed by the web-based JSON Viewer [14] editor, which offers a graph structure in addition to an object-based display. We fed JSON Viewer our book list for Figure 6.

Figure 6: The web-based JSON Viewer shows data links as a graph.