Using Gauss's law to prove that a field is zero

You are exactly correct. In general $$\iint\mathbf E\cdot\text d\mathbf a=0\nrightarrow\mathbf E=0$$

However, in special instances of symmetry (like your spherical cavity), you can argue that the field must have the same magnitude and point radially outwards at all points on your Gaussian surface, then you do get $$\iint\mathbf E\cdot\text d\mathbf a=\iint E\,\text d a=E\iint\text da=EA=0\to \mathbf E=0$$

This is something to understand about the integral form of Gauss's law: You can only use it to find the field when specific symmetries are present in the system. Without the symmetries, then there is nothing to be done (as the value of an integral does not uniquely determine an integrand in general).

The argument uses a spherical Gaussian surface whose center coincides with that of the sphere. Spherical symmetry then means that any nonzero field would be radial and only a function of the radial coordinate. Thus the flux integral reduces to $4\pi E_r$. The only way it can be zero is for the field to be zero.

In general, if the flux of electric field through a closed surface is zero doesn't always mean that the electric field through that closed surface is zero. For spherically symmetric bodies, it is true that if electric flux through a closed surface is zero then the electric field is also zero. If, $$\iint\mathbf E\cdot\text d\mathbf a=0,$$ then there are two possibilities - $E=0$ (in general for spherical symmetry) or $E \neq 0$ (for any other closed surface).