jq count the number of items in json by a specific key
Here's one solution (assuming the input is a stream of valid JSON objects) and that you invoke jq with the -s option:
map({ItemId: .Properties.ItmId}) # extract the ItmID values
| group_by(.ItemId) # group by "ItemId"
| map({ItemId: .[0].ItemId, Count: length}) # store the counts
| .[] # convert to a stream
A slightly more memory-efficient approach would be to use inputs
if your jq has it; but in that case, use -n instead of -s, and replace the first line above by: [inputs | {ItemId: .Properties.ItmId} ]
Efficient solution
The above solutions use the built-in group_by
, which is convenient but leads to easily-avoided inefficiencies. Using the following counter
makes it easy to write a very efficient solution:
def counter(stream):
reduce stream as $s ({}; .[$s|tostring] += 1);
Using the -n command-line option, and applied as follows:
counter(inputs | .Properties.ItmId)
this leads to a dictionary of counts:
{
"1694738780": 1,
"1347809133": 1
}
Such a dictionary is probably more useful than a stream of singleton objects as envisioned by the OP, but if such as stream is needed, one can modify the above as follows:
counter(inputs | .Properties.ItmId)
| to_entries[]
| {ItemId: (.key), Count: .value}
Using jq command
cat json.txt | jq '.Properties .ItmId' | sort | uniq -c | awk -F " " '{print "{\"ItmId\":" $2 ",\"count\":" $1"}"}'| jq .
Here is a variation using reduce, setpath and getpath to do the aggregation and to_entries to do the final formatting which assumes you run jq as
jq --slurp -f query.jq < data.json
where data.json contains your data and query.jq contains
map(.Properties.ItmId)
| reduce .[] as $i (
{}; setpath([$i]; getpath([$i]) + 1)
)
| to_entries | .[] | { "ItemId": .key, "Count": .value }
Here's a super-efficient solution -- in particular, no sorting is required. The following implementation requires a version of jq with inputs
but it is easy to adapt the program to use earlier versions of jq. Please remember to use the -n command-line option if using the following:
# Count the occurrences of distinct values of (stream|tostring).
# To avoid unwanted collisions, or to recover the exact values,
# consider using tojson
def counter(stream):
reduce stream as $s ({}; .[$s|tostring] += 1);
counter(inputs | .Properties.ItmId)
| to_entries[]
| {ItemId: (.key), Count: .value}