Rename the less frequent categories by "OTHER" python
Convert your series to categorical, extract categories whose counts are not in the top 3, add a new category e.g. 'Other'
, then replace the previously calculated categories:
df['Jobrol'] = df['Jobrol'].astype('category')
others = df['Jobrol'].value_counts().index[3:]
label = 'Other'
df['Jobrol'] = df['Jobrol'].cat.add_categories([label])
df['Jobrol'] = df['Jobrol'].replace(others, label)
Note: It's tempting to combine categories by renaming them via df['Jobrol'].cat.rename_categories(dict.fromkeys(others, label))
, but this won't work as this will imply multiple identically labeled categories, which isn't possible.
The above solution can be adapted to filter by count. For example, to include only categories with a count of 1 you can define others
as so:
counts = df['Jobrol'].value_counts()
others = counts[counts == 1].index
Use value_counts
with numpy.where
:
need = df['Jobrol'].value_counts().index[:3]
df['Jobrol'] = np.where(df['Jobrol'].isin(need), df['Jobrol'], 'OTHER')
valCount = df['Jobrol'].value_counts()
print (valCount)
Research Scientist 7
Sales Executive 7
Laboratory Technician 5
OTHER 2
Name: Jobrol, dtype: int64
Another solution:
N = 3
s = df['Jobrol'].value_counts()
valCount = s.iloc[:N].append(pd.Series(s.iloc[N:].sum(), index=['OTHER']))
print (valCount)
Research Scientist 7
Sales Executive 7
Laboratory Technician 5
OTHER 2
dtype: int64