Sort JSON/JavaScript tuples using command-line tools
If you can guarantee that all fields are same size, you can use sort command. For example, this sorts by column x value numerically.
cat <your file.dat> | sort -n -k 5,7
Data you have here as example is not valid JSON but javascript syntax. One way is to wrap the file so it's valid javascript program and run it in node.js command line,
var l = [
{ x : 12, y : -1.0, as : [ 2, 0, 0 ], str : "xxx", d : 0.041 },
{ x : 27, y : 11.4, as : [ 1, 1, 7 ], str : "yyy", d : 0.235 },
...
]
l.sort(function(o1, o2) { return o1.d < o2.d ? -1 : 1 });
console.log(l);
If there is a one JSON per row in the input data (as shown in the question), then the @Ashley Coolman's solution does not work, as written here:
Sorting JSON by value with jq can easily be done by using the sort_by() function. The main trick with using the sort_by() function is that your JSON input must be in an array. There are different ways to do this in jq (if your data is not already like this), including using the -s or --slurp option. The --slurp option will read all JSON input into a JSON array. From this point the data can be sorted by value. You can use the .[] syntax to return all elements of the array.
It means that right solution for the data from the question is the following:
cat data.json | jq -s -c 'sort_by(.d) | .[]' >> data_sorted.json
It's a hack, but if each JSON record is one line, and you know that the value for d
begins after the same number of whitespace-separated tokens on each line, then you can just use
sort -g -k 20 < in > out
which will compare lines numerically based on the 20th whitespace-separated component. For increased comfort you could specify a different delimiter with -t
(perhaps :
) and adjust the argument to -k
as necessary, but it's still a hack :)
sort
is generally carefully optimised for speed, so you're unlikely to find something faster.
Some time has passed since this question was asked and answered.
These days, a tool-based way would be to use something like jq:
cat data.json | jq 'sort_by(.d)' >> data_sorted.json
For more info check the site:
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
-https://stedolan.github.io/jq/
If for some reason you don't like jq, there are many alternatives
- jsonpipe
- json
- json:select
- json-command
- jsawk
- jshon
- json2