Check for None in pandas dataframe
If you want to get True/False for each line, you can use the following code. Here is an example as a result for the following DataFrame:
df = pd.DataFrame([[None, 3], ["", np.nan]])
df
# 0 1
#0 None 3.0
#1 NaN
How to check None
Available: .isnull()
>>> df[0].isnull()
0 True
1 False
Name: 0, dtype: bool
Available: .apply
==
or is
None
>>> df[0].apply(lambda x: x == None)
0 True
1 False
Name: 0, dtype: bool
>>> df[0].apply(lambda x: x is None)
0 True
1 False
Name: 0, dtype: bool
Available: .values
==
None
>>> df[0].values == None
array([ True, False])
Unavailable: is
or ==
>>> df[0] is None
False
>>> df[0] == None
0 False
1 False
Name: 0, dtype: bool
Unavailable: .values
is
None
>>> df[0].values is None
False
How to check np.nan
Available: .isnull()
>>> df[1].isnull()
0 False
1 True
Name: 1, dtype: bool
Available: np.isnan
>>> np.isnan(df[1])
0 False
1 True
Name: 1, dtype: bool
>>> np.isnan(df[1].values)
array([False, True])
>>> df[1].apply(lambda x: np.isnan(x))
0 False
1 True
Name: 1, dtype: bool
Unavailable: is
or ==
np.nan
>>> df[1] is np.nan
False
>>> df[1] == np.nan
0 False
1 False
Name: 1, dtype: bool
>>> df[1].values is np.nan
False
>>> df[1].values == np.nan
array([False, False])
>>> df[1].apply(lambda x: x is np.nan)
0 False
1 False
Name: 1, dtype: bool
>>> df[1].apply(lambda x: x == np.nan)
0 False
1 False
Name: 1, dtype: bool
You could use applymap
with a lambda
to check if an element is None
as follows, (constructed a different example, as in your original one, None
is coerced to np.nan
because the data type is float
, you will need an object
type column to hold None
as is, or as commented by @Evert, None
and NaN
are indistinguishable in numeric type columns):
df = pd.DataFrame([[None, 3], ["", np.nan]])
df
# 0 1
#0 None 3.0
#1 NaN
df.applymap(lambda x: x is None)
# 0 1
#0 True False
#1 False False