Analytically compute signed distance of ellipsoid

Thanks to Will Jagy for his clear exposition, and many references. And thanks to pointing to me. But in fact, none of my books address this problem. However, Dave Eberly has addressed it quite extensively in his manuscript "Distance from a Point to an Ellipse, an Ellipsoid, or a Hyperellipsoid," which you may retrieve from this PDF link. Eberly is not only a master of the mathematics, but he cares about actually computing these quantities. He works out various cases, one of which may lead to a critical function having six roots!
     

From what I can see the signed distance you are asking about should not possess a closed form expression. Elliptical coordinates are not a constant distance apart, see http://en.wikipedia.org/wiki/Elliptic_coordinate_system

Evidently this is a difficult computational problem, http://http.developer.nvidia.com/GPUGems3/gpugems3_ch34.html

However, you can get a fair amount of the functionality you need with simple differential geometry, as you do not need to define the ellipsoid as a mesh. You can tell instantly whether a point is outside or inside based on your $f(x,y,z),$ so your comment about $\pm$ sign is entirely correct.

If a point is outside the ellipsoid ($f > 0,$) you can find the closest point on the ellipsoid by fairly rapid numerical methods. The closest point to some $(r,s,t)$ is the point $(x,y,z)$ in the same octant where the normal vector at $(r,s,t)$ is parallel to the vector $(x,y,z) - (r,s,t).$ The normal vector at $(r,s,t)$ is just a positive multiple of $$ \left(\frac{r}{a^2}, \frac{s}{b^2},\frac{t}{c^2}\right).$$ So, take as seed value the multiple $(\lambda x,\lambda y, \lambda z).$ At each stage, update the point on the ellipse by projecting the vector $(x,y,z) - (r,s,t)$ onto the tangent plane at $(r,s,t).$ Then, say, project that radially onto the ellipsoid.

I'll need to think about speed. For computer applications, a fast enough algorithm may be good enough, when a closed-form expression is not available.

Inside the ellipsoid, there is not necessarily a unique closest point on the surface. I might recommend just scaling your $f,$ just $- \sqrt{|f|}.$

Meanwhile, I recommended a certain book to Joseph O'Rourke, who does this sort of thing. John Thorpe, Elementary Topics in Differential Geometry.