What's that shape? Inferring a 3D shape from random shadows

This is very similar to the cryo-electron microscopy problem: You want to image a certain macromolecule, and the scale of the macromolecule requires the use of an electron microscope. Unfortunately, such an imaging process is harmful to the specimen. In the 80s, they realized you can protect the specimen by freezing it first, but a single exposure will undo the protection. As such, they freeze a bunch of identical macromolecules (which end up at unknown orientations), and then image each one, producing a two-dimensional projection image (similar to your shadow). The cryo-EM problem is to recover the three-dimensional macromolecule from these images (see this paper and references therein)

If you have no information about the vertices and allow nonconvexity there could be problems first you will only get information about the surface of the 3d shape but even there you will have problems. If you remove a tiny cube near the center of one of the faces of the cube the surface will not be the same but I don't think it will make a difference in the projections. So I don't think all nonconvex polyhedra will be inferred from shadows but maybe convex polyhedra will.

Shape-reconstruction from shadows will definitely be impossible for "modified convex" polyhedra, that are topologically equivalent to a sphere and can be generated from a corresponding truly convex polyhedron via excavations, that do not destroy edges or vertices; take for example a model of the surface of a convex polyhedron made of thin plates for the faces and remove one of those faces.
The situation is however different if the surface of the 3D body doesn't contain straight-line segments.

Edit
I dare to conjecture, that a necessary and sufficient criterion for 3D shapes, which can be reconstructed from projections onto 2D planes is, that for every boundary point $p$ there exists a straight line through $p$ that doesn't intersect the interior of the 3D shape.