tidwall/jj

JSON Stream Editor (command line utility)

JJ is a command line utility that provides a fast and simple way to retrieve or update values from JSON documents. It’s powered by GJSON and SJSON under the hood.

It’s fast because it avoids parsing irrelevant sections of json, skipping over values that do not apply, and aborts as soon as the target value has been found or updated.

Getting started

Install

Mac (Homebrew)

brew install tidwall/jj/jj

Build

make

Or download a pre-built binary for Linux, OSX, Windows, or FreeBSD.

Usage

$ jj -h

usage: jj [-v value] [-purOD] [-i infile] [-o outfile] keypath

examples: jj keypath                      read value from stdin
      or: jj -i infile keypath            read value from infile
      or: jj -v value keypath             edit value
      or: jj -v value -o outfile keypath  edit value and write to outfile

options:
      -v value             Edit JSON key path value
      -p                   Make json pretty, keypath is optional
      -u                   Make json ugly, keypath is optional
      -r                   Use raw values, otherwise types are auto-detected
      -n                   Do not output color or extra formatting
      -O                   Performance boost for value updates
      -D                   Delete the value at the specified key path
      -l                   Output array values on multiple lines
      -i infile            Use input file instead of stdin
      -o outfile           Use output file instead of stdout
      keypath              JSON key path (like "name.last")

Examples

Getting a value

JJ uses a path syntax for finding values.

Get a string:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj name.last
Smith

Get a block of JSON:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj name
{"first":"Tom","last":"Smith"}

Try to get a non-existent key:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj name.middle
null

Get the raw string value:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -r name.last
"Smith"

Get an array value by index:

$ echo '{"friends":["Tom","Jane","Carol"]}' | jj friends.1
Jane

JSON Lines

There’s support for JSON Lines using the .. path prefix. Which when specified, treats the multi-lined document as an array.

For example:

{"name": "Gilbert", "age": 61}
{"name": "Alexa", "age": 34}
{"name": "May", "age": 57}

..#                   >> 4
..1                   >> {"name": "Alexa", "age": 34}
..#.name              >> ["Gilbert","Alexa","May"]
..#[name="May"].age   >> 57

Setting a value

The path syntax for setting values has a couple of tiny differences than for getting values.

The -v value option is auto-detected as a Number, Boolean, Null, or String. You can override the auto-detection and input raw JSON by including the -r option. This is useful for raw JSON blocks such as object, arrays, or premarshalled strings.

Update a value:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -v Andy name.first
{"name":{"first":"Andy","last":"Smith"}}

Set a new value:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -v 46 age
{"age":46,"name":{"first":"Tom","last":"Smith"}}

Set a new nested value:

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -v relax task.today
{"task":{"today":"relax"},"name":{"first":"Tom","last":"Smith"}}

Replace an array value by index:

$ echo '{"friends":["Tom","Jane","Carol"]}' | jj -v Andy friends.1
{"friends":["Tom","Andy","Carol"]}

Append an array:

$ echo '{"friends":["Tom","Jane","Carol"]}' | jj -v Andy friends.-1
{"friends":["Tom","Andy","Carol","Andy"]}

Set an array value that’s past the bounds:

$ echo '{"friends":["Tom","Jane","Carol"]}' | jj -v Andy friends.5
{"friends":["Tom","Andy","Carol",null,null,"Andy"]}

Set a raw block of JSON:

$ echo '{"name":"Carol"}' | jj -r -v '["Tom","Andy"]' friends
{"friends":["Tom","Andy"],"name":"Carol"}

Start new JSON document:

$ echo '' | jj -v 'Sam' name.first
{"name":{"first":"Sam"}}

Deleting a value

Delete a value:

$ echo '{"age":46,"name":{"first":"Tom","last":"Smith"}}' | jj -D age
{"name":{"first":"Tom","last":"Smith"}}

Delete an array value by index:

$ echo '{"friends":["Andy","Carol"]}' | jj -D friends.0
{"friends":["Carol"]}

Delete last item in array:

$ echo '{"friends":["Andy","Carol"]}' | jj -D friends.-1
{"friends":["Andy"]}

Optimistically update a value

The -O option can be used when the caller expects that a value at the specified keypath already exists.

Using this option can speed up an operation by as much as 6x, but slow down as much as 20% when the value does not exist.

For example:

echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -v Tim -O name.first

The -O tells jj that the name.first likely exists so try a fasttrack operation first.

Pretty printing

The -p flag will make the output json pretty.

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -p name
{
  "first": "Tom",
  "last": "Smith"
}

Also the keypath is optional when the -p flag is specified, allowing for the entire json document to be made pretty.

$ echo '{"name":{"first":"Tom","last":"Smith"}}' | jj -p
{
  "name": {
    "first": "Tom",
    "last": "Smith"
  }
}

Ugly printing

The -u flag will compress the json into the fewest characters possible by squashing newlines and spaces.

Performance

A quick comparison of jj to jq. The test json file is 180MB file of 206,560 city parcels in San Francisco.

Tested on a 2015 Macbook Pro running jq 1.5 and jj 1.0.0

Get the lot number for the parcel at index 10000

jq:

$ time cat citylots.json | jq -cM .features[10000].properties.LOT_NUM
"091"

real    0m5.486s
user    0m4.870s
sys     0m0.686s

jj:

$ time cat citylots.json | jj -r features.10000.properties.LOT_NUM
"091"

real    0m0.354s
user    0m0.161s
sys     0m0.321s

Update the lot number for the parcel at index 10000

jq:

$ time cat citylots.json | jq -cM '.features[10000].properties.LOT_NUM="12A"' > /dev/null

real    0m13.579s
user    0m16.484s
sys     0m1.310s

jj:

$ time cat citylots.json | jj -O -v 12A features.10000.properties.LOT_NUM > /dev/null

real    0m0.431s
user	0m0.201s
sys     0m0.295s

Contact

Josh Baker @tidwall

License

JJ source code is available under the MIT License.