creating pandas data frame from multiple files
I might try to concatenate the files before feeding them to pandas. If you're in Linux or Mac you could use cat
, otherwise a very simple Python function could do the job for you.
Potentially horribly inefficient but...
Why not use read_csv
, to build two (or more) dataframes, then use join to put them together?
That said, it would be easier to answer your question if you provide some data or some of the code you've used thus far.
The pandas concat
command is your friend here. Lets say you have all you files in a directory, targetdir. You can:
- make a list of the files
- load them as pandas dataframes
- and concatenate them together
`
import os
import pandas as pd
#list the files
filelist = os.listdir(targetdir)
#read them into pandas
df_list = [pd.read_table(file) for file in filelist]
#concatenate them together
big_df = pd.concat(df_list)