Check if a point is inside a rectangle (not knowing the coordinates, but knowing distances to vertices)

It seems it is impossible to know whether X is inside for sure. For example, if we had a rectangle ABCD and a point X inside of it, WLOG so that the vertical distance from X to CD is less than the vertical distance from X to AB, creating a new rectangle ABEF with EF parallel to AB and CD, but EF underneath X with the same vertical distance creates a new rectangle with exactly the same distances to the vertices, but with X outside.

(Low picture quality, which is why I hesitate to put a picture. But I also thought that my above explanation was sort of unclear, so I posted this too.)

It's not possible in general:

A . . . . . D . . A'
.           .     .
.           .     .
B . . . . . C . . B'

Let $X$ be the centre of $ABA'B'$. Then $XA=XA'$ and $XB=XB'$, and of course $XC=XC$ and $XD=XD$. But for suitable choice of $C$ and $D$, $X$ is inside $ABCD$ but not $A'B'CD$.