How to extract data from a JSON file

You can use jq to process json files in shell.

For example, I saved your sample json file as raul.json and then ran:

$ jq .message.temperature raul.json 
409.5
25.1
409.5
$ jq .message.humidity raul.json 
null
40
null

jq is available pre-packaged for most linux distros.

There's probably a way to do it in jq itself, but the simplest way I found to get both the wanted values on one line is to use xargs. For example:

$ jq 'select(.message.id == 1490) | .message.temperature, .message.humidity' raul.json | xargs
25.1 40

or, if you want to loop through each .message.id instance, we can add .message.id to the output and use xargs -n 3 as we know that there will be three fields (id, temperature, humidity):

jq '.message.id, .message.temperature, .message.humidity' raul.json | xargs -n 3
4095 409.5 null
1490 25.1 40
2039 409.5 null

You could then post-process that output with awk or whatever.


Finally, both python and perl have excellent libraries for parsing and manipulating json data. As do several other languages, including php and java.