Reshape long to wide using columns names
stack
Stacking drops null values while reshaping the array
df.stack().groupby(level=1).agg(list)
DVD [0.3, 0.15, 0.8, 0.41]
Netflix [0.1, 0.12, 0.4]
TV [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
dtype: object
Remove missing values by Series.dropna
and convert to Series in dictionary comprehension:
s = pd.Series({x: df[x].dropna().tolist() for x in df.columns})
print (s)
Netflix [0.1, 0.12, 0.4]
TV [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
DVD [0.3, 0.15, 0.8, 0.41]
dtype: object
...or in DataFrame.apply
:
s = df.apply(lambda x: x.dropna().tolist())
print (s)
Netflix [0.1, 0.12, 0.4]
TV [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
DVD [0.3, 0.15, 0.8, 0.41]
dtype: object
Last if need 2 columns DataFrame
:
df1 = s.rename_axis('a').reset_index(name='b')
print (df1)
a b
0 Netflix [0.1, 0.12, 0.4]
1 TV [0.2, 0.5, 0.6, 0.5, 0.41, 0.2]
2 DVD [0.3, 0.15, 0.8, 0.41]
I think this is what you are looking for:
> df.T.apply(lambda x: x.dropna().tolist(), axis=1)
Netflix [0.1, 0.12, 0.4, 0.5, 0.41, 0.2]
TV [0.2, 0.5, 0.6, 0.41]
DVD [0.3, 0.15, 0.8]
dtype: object