DataFrame: add column whose values are the quantile number/rank of an existing column?
I discovered it is quite easy:
df['quantile'] = pd.qcut(df['b'], 2, labels=False)
a b quantile
0 1 1 0
1 2 10 0
2 3 100 1
3 4 100 1
Interesting to know "difference between pandas.qcut and pandas.cut"
You can use DataFrame.quantile with q=[0.25, 0.5, 0.75] on the existing column to produce a quartile column.
Then, you can DataFrame.rank on that quartile column.
See below for an example of adding a quartile column:
import pandas as pd
d = {'one' : pd.Series([40., 45., 50., 55, 60, 65], index=['val1', 'val2', 'val3', 'val4', 'val5', 'val6'])}
df = pd.DataFrame(d)
quantile_frame = df.quantile(q=[0.25, 0.5, 0.75])
quantile_ranks = []
for index, row in df.iterrows():
if (row['one'] <= quantile_frame.ix[0.25]['one']):
quantile_ranks.append(1)
elif (row['one'] > quantile_frame.ix[0.25]['one'] and row['one'] <= quantile_frame.ix[0.5]['one']):
quantile_ranks.append(2)
elif (row['one'] > quantile_frame.ix[0.5]['one'] and row['one'] <= quantile_frame.ix[0.75]['one']):
quantile_ranks.append(3)
else:
quantile_ranks.append(4)
df['quartile'] = quantile_ranks
Note: There's probably a more idiomatic way to accomplish this with Pandas... but it's beyond me
df['quantile'] = pd.qcut(df['b'], 2, labels=False)
seems to tend to throw a SettingWithCopyWarning
.
The only general way I have found of doing this without complaints is like:
quantiles = pd.qcut(df['b'], 2, labels=False)
df = df.assign(quantile=quantiles.values)
This will assign the quantile rank values as a new DataFrame
column df['quantile']
.
A solution for a more generalized case, in which one wants to partition the cut by multiple columns, is given here.