How to identify a pandas column is a list
You can use applymap
, compare and then add all
for check if all values are True
s:
print (df.applymap(type))
X Y Z
0 <class 'int'> <class 'list'> <class 'str'>
1 <class 'int'> <class 'list'> <class 'str'>
2 <class 'int'> <class 'list'> <class 'str'>
a = (df.applymap(type) == list).all()
print (a)
X False
Y True
Z False
dtype: bool
Or:
a = df.applymap(lambda x: isinstance(x, list)).all()
print (a)
X False
Y True
Z False
dtype: bool
And if need list of columns:
L = a.index[a].tolist()
print (L)
['Y']
If want check dtypes
(but strings
, list
, dict
are object
s):
print (df.dtypes)
X int64
Y object
Z object
dtype: object
a = df.dtypes == 'int64'
print (a)
X True
Y False
Z False
dtype: bool
If your dataset is big, you should take a sample before apply the type function, then you can check:
If the the most common type is list:
df\
.sample(100)\
.applymap(type)\
.mode(0)\
.astype(str) == "<class 'list'>"
If all values are list:
(df\
.sample(100)\
.applymap(type)\
.astype(str) == "<class 'list'>")\
.all(0)
If any values are list:
(df\
.sample(100)\
.applymap(type)\
.astype(str) == "<class 'list'>")\
.any(0)