Write Pandas DataFrame to newline-delimited JSON
If you're trying to write a DF using iterrows
- I suspect you should be looking at:
df.to_json(orient='records') # List of lists of values
# [[1, 2], [3,4]]
Or:
df.to_json(orient='records') # List of dicts with col->val
# [{'A': 1, 'B': 2}, {'A': 3, 'B': 4}]
Or writing a dict of {index:col value}:
df.A.to_json()
# {0: 1, 1: 3}
To create newline-delimited json from a dataframe df
, run the following
df.to_json("path/to/filename.json",
orient="records",
lines=True)
Pay close attention to those optional keyword args! The lines
option was added in pandas 0.19.0
.
You can pass a buffer in to df.to_json()
:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame({"a":[1,3,5], "b":[1.1,1.2,1.2]})
In [3]: df
Out[3]:
a b
0 1 1.1
1 3 1.2
2 5 1.2
In [4]: f = open("temp.txt", "w")
In [5]: for row in df.iterrows():
row[1].to_json(f)
f.write("\n")
...:
In [6]: f.close()
In [7]: open("temp.txt").read()
Out[7]: '{"a":1.0,"b":1.1}\n{"a":3.0,"b":1.2}\n{"a":5.0,"b":1.2}\n'