Comparison of a Dataframe column values with a list
You can also do it in one step using .transform('first')
:
In [280]: df['D'] = df.groupby('A')['C'].transform('first').eq(df['C']).astype(np.int8)
In [281]: df
Out[281]:
A B C D
0 1 10 100 1
1 1 15 150 0
2 2 20 200 1
3 2 25 250 0
4 3 30 300 1
5 3 35 350 0
Explanation: GroupBy.transform('func')
returns us a vector with the same length as the original DF with applied func
In [14]: df.groupby('A')['C'].transform('first')
Out[14]:
0 100
1 100
2 200
3 200
4 300
5 300
Name: C, dtype: int64
In [15]: df.groupby('A')['C'].transform('max')
Out[15]:
0 150
1 150
2 250
3 250
4 350
5 350
Name: C, dtype: int64
In [16]: df.groupby('A')['C'].transform('min')
Out[16]:
0 100
1 100
2 200
3 200
4 300
5 300
Name: C, dtype: int64
In [17]: df.groupby('A')['C'].transform('mean')
Out[17]:
0 125
1 125
2 225
3 225
4 325
5 325
Name: C, dtype: int64
In [18]: df.groupby('A')['C'].transform('sum')
Out[18]:
0 250
1 250
2 450
3 450
4 650
5 650
Name: C, dtype: int64
TL;DR:
df['newColumn'] = np.where((df.compareColumn.isin(yourlist)), TrueValue, FalseValue)
Another one-step method would be to use np.where()
and isin
.
import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1, 1, 2, 2, 3, 3],
'B': [10, 15, 20, 25, 30,35],
'C': [100, 150, 200, 250, 300, 350]})
df['D'] = np.where((df.B.isin(firsts)), 1, 0)
We use the return from isin
as the condition in np.where()
to return either
1
whenTrue
0
whenFalse
and assign them to a new column in the same dataframe df['D']
.
Note: np.where
allows more complex conditions with bitwise operators and replacement cases, i.e. 'bypass' on False
df['col1'] = np.where(((df['col1'] == df['col2']) &
(~df['col1'].str.startswith('r'))),
'replace', df['col1'])
You can use isin
method:
df['D'] = df.C.isin(firsts).astype(int)
df
# A B C D
#0 1 10 100 1
#1 1 15 150 0
#2 2 20 200 1
#3 2 25 250 0
#4 3 30 300 1
#5 3 35 350 0
The reason your approach fails is that python in
operator check the index of a Series instead of the values, the same as how a dictionary works:
firsts
#A
#1 100
#2 200
#3 300
#Name: C, dtype: int64
1 in firsts
# True
100 in firsts
# False
2 in firsts
# True
200 in firsts
# False
Modifying your method as follows works:
firstSet = set(firsts)
df['C'].apply(lambda x: 1 if x in firstSet else 0)
#0 1
#1 0
#2 1
#3 0
#4 1
#5 0
#Name: C, dtype: int64