How to convert a pandas MultiIndex DataFrame into a 3D array
How about using xarray
?
res = df.to_xarray().to_array()
Result is an array of shape (4, 15, 5)
In fact the docs now recommend this as an alternative to pandas Panel
. Note that you must have the xarray
package installed.
Since df.values
is a (15*100, 4)
-shaped array, you can call reshape
to make it a (15, 100, 4)
-shaped array:
arr = df.values.reshape(15, 100, 4)
Then call transpose
to rearrange the order of the axes:
arr = arr.transpose(2, 0, 1)
Now arr
has shape (4, 15, 100)
.
Using reshape/transpose
is ~960x faster than to_xarray().to_array()
:
In [21]: df = pd.DataFrame(np.random.randint(10, size=(15*100, 4)), index=pd.MultiIndex.from_product([range(15), range(100)], names=['A','B']), columns=list('colu'))
In [22]: %timeit arr = df.values.reshape(15, 100, 4).transpose(2, 0, 1)
3.31 µs ± 23.2 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [24]: %timeit df.to_xarray().to_array()
3.18 ms ± 24.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [25]: 3180/3.31
Out[25]: 960.7250755287009