Calculate summary statistics of columns in dataframe
describe
may give you everything you want otherwise you can perform aggregations using groupby and pass a list of agg functions: http://pandas.pydata.org/pandas-docs/stable/groupby.html#applying-multiple-functions-at-once
In [43]:
df.describe()
Out[43]:
shopper_num is_martian number_of_items count_pineapples
count 14.0000 14 14.000000 14
mean 7.5000 0 3.357143 0
std 4.1833 0 6.452276 0
min 1.0000 False 0.000000 0
25% 4.2500 0 0.000000 0
50% 7.5000 0 0.000000 0
75% 10.7500 0 3.500000 0
max 14.0000 False 22.000000 0
[8 rows x 4 columns]
Note that some columns cannot be summarised as there is no logical way to summarise them, for instance columns containing string data
As you prefer you can transpose the result if you prefer:
In [47]:
df.describe().transpose()
Out[47]:
count mean std min 25% 50% 75% max
shopper_num 14 7.5 4.1833 1 4.25 7.5 10.75 14
is_martian 14 0 0 False 0 0 0 False
number_of_items 14 3.357143 6.452276 0 0 0 3.5 22
count_pineapples 14 0 0 0 0 0 0 0
[4 rows x 8 columns]
To clarify one point in @EdChum's answer, per the documentation, you can include the object columns by using df.describe(include='all')
. It won't provide many statistics, but will provide a few pieces of info, including count, number of unique values, top value. This may be a new feature, I don't know as I am a relatively new user.
Now there is the pandas_profiling
package, which is a more complete alternative to df.describe()
.
If your pandas dataframe is df
, the below will return a complete analysis including some warnings about missing values, skewness, etc. It presents histograms and correlation plots as well.
import pandas_profiling
pandas_profiling.ProfileReport(df)
See the example notebook detailing the usage.