Example 1: pandas new df from groupby
df = pd.DataFrame(old_df.groupby(['groupby_attribute'])['mean_attribute'].mean())
df = df.reset_index()
df
Example 2: group by pandas examples
>>> n_by_state = df.groupby("state")["state"].count()
>>> n_by_state.head(10)
state
AK 16
AL 206
AR 117
AS 2
AZ 48
CA 361
CO 90
CT 240
DC 2
DE 97
Name: last_name, dtype: int64
Example 3: groupby in pandas
>>> df = pd.DataFrame({'Animal': ['Falcon', 'Falcon',
... 'Parrot', 'Parrot'],
... 'Max Speed': [380., 370., 24., 26.]})
>>> df
Animal Max Speed
0 Falcon 380.0
1 Falcon 370.0
2 Parrot 24.0
3 Parrot 26.0
>>> df.groupby(['Animal']).mean()
Max Speed
Animal
Falcon 375.0
Parrot 25.0
Example 4: Groups the DataFrame using the specified columns
df.groupBy().avg().collect()
sorted(df.groupBy('name').agg({'age': 'mean'}).collect())
sorted(df.groupBy(df.name).avg().collect())
sorted(df.groupBy(['name', df.age]).count().collect())
Example 5: groupby and list
In [1]: df = pd.DataFrame( {'a':['A','A','B','B','B','C'], 'b':[1,2,5,5,4,6]})
df
Out[1]:
a b
0 A 1
1 A 2
2 B 5
3 B 5
4 B 4
5 C 6
In [2]: df.groupby('a')['b'].apply(list)
Out[2]:
a
A [1, 2]
B [5, 5, 4]
C [6]
Name: b, dtype: object
In [3]: df1 = df.groupby('a')['b'].apply(list).reset_index(name='new')
df1
Out[3]:
a new
0 A [1, 2]
1 B [5, 5, 4]
2 C [6]
Example 6: pandas groupby
data.groupby('month', as_index=False).agg({"duration": "sum"})