Conceptual reason for why the volume of an ocahedron is four times the volume of a tetrahedron

Consider a cube with a tetrahedron inside it sharing four of its vertices. The cube dissects into this tetrahedron, and four identical triangular pyramids. Look at this picture of the cube standing on one vertex:

A body diagonal of the cube, vertical in this picture, is split into three equal parts by the heights of the vertices. This shows that the inner tetrahedron has twice the height of each small pyramid, and hence twice the volume. Eight of those small pyramids can form an orctahedron, so the tetrahedron is a quarter of the volume of the octahedron.

I don't think this can be generalised to higher dimensions in the direction you are looking for.

If you inscribe two tetrahedra in a cube, their overlap is an octahedron:

Equivalently, the octahedron may be constructed by joining midpoints of a tetrahedron's edges. Notice that within this (say, red) tetrahedron, outside the octahedron $O$ there are four smaller tetrahedra $T$. The side lengths of these smaller $T$s are half the side-length of the original, red tetrahedra, so the red one has eight times the volume, and so $8T=4T+O$.

This is another decomposition proof, but it's more direct. 3D only though.

Join the vertices of the unit-sided octahedron with its centre. That will divide it into eight regular pyramids, having the faces of the octahedron as bases and three lateral edges with length $1/\sqrt2$.

Pythagora's theorem gives then a height of $1/\sqrt6$ for these eight pyramids, whereas the height of a regular unit-sided tetrahedron is $2/\sqrt6$. The volume of the tetrahedron is then double that of each regular pyramid in the octahedron, which explains why the volume of the octahedron is four times the volume of the tetrahedron.