How to iterate over pandas multiindex dataframe using index
Late to the party, I found that the following works, too:
for date in df.index.unique("date"):
print(df.loc[date])
It uses the level
optional parameter of the Index.unique
method introduced in version 0.23.0.
You can specify either the level number or label.
One easy way would be to groupby the first level of the index - iterating over the groupby object will return the group keys and a subframe containing each group.
In [136]: for date, new_df in df.groupby(level=0):
...: print(new_df)
...:
observation1 observation2
date Time
2012-11-02 9:15:00 79.373668 224
9:16:00 130.841316 477
observation1 observation2
date Time
2012-11-03 9:15:00 45.312814 835
9:16:00 123.776946 623
9:17:00 153.766460 624
9:18:00 463.276946 626
9:19:00 663.176934 622
9:20:00 763.773330 621
observation1 observation2
date Time
2012-11-04 9:15:00 115.449437 122
9:16:00 123.776946 555
9:17:00 153.766460 344
9:18:00 463.276946 212
You can also use droplevel
to remove the first index (the useless date
index):
In [136]: for date, new_df in df.groupby(level=0):
...: print(new_df.droplevel(0))
...:
observation1 observation2
Time
9:15:00 79.373668 224
9:16:00 130.841316 477
...
Tagging off of @psorenson answer, we can get unique level indices and its related data frame slices without numpy as follows:
for date in df.index.get_level_values('date').unique():
print(df.loc[date])
What about this?
for idate in df.index.get_level_values('date'):
complex_process(df.ix[idate], idate)