Loading a file with more than one line of JSON into Pandas
From version 0.19.0 of Pandas you can use the lines
parameter, like so:
import pandas as pd
data = pd.read_json('/path/to/file.json', lines=True)
You have to read it line by line. For example, you can use the following code provided by ryptophan on reddit:
import pandas as pd
# read the entire file into a python array
with open('your.json', 'rb') as f:
data = f.readlines()
# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)
# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ','.join(data) + "]"
# now, load it into pandas
data_df = pd.read_json(data_json_str)