Question about vectors and geometry in calculus

All of your answers are correct. To see why 3 is impossible, set $\mathbf u =(u_1,u_2,u_3)$. A general property of $\mathbb R^3$ is that the dot product of 2 vectors can be expressed geometrically:

$$\mathbf u \mathbf v = \lvert\mathbf u\rvert\lvert\mathbf v\rvert\cos(\alpha),$$

where $\alpha$ is the angle between $\mathbf u$ and $\mathbf v$.

If we apply this to $\mathbf u$ and $\mathbf i$ (angle $\alpha_1$) we get

$$\mathbf u \mathbf i = u_1=\lvert\mathbf u\rvert\cdot 1\cdot \cos(\alpha_1),$$

and similiarly for $\mathbf u$ and $\mathbf j$ (angle $\alpha_2$):

$$\mathbf u \mathbf j = u_2=\lvert\mathbf u\rvert\cdot 1\cdot \cos(\alpha_2).$$

This means

$$u_1^2+u_2^2=\lvert\mathbf u\rvert^2\cos^2(\alpha_1) + \lvert\mathbf u\rvert^2\cos^2(\alpha_2) = (u_1^2+u_2^2+u_3^2)(\cos^2(\alpha_1) + \cos^2(\alpha_2))$$

and finally

$$\cos^2(\alpha_1) + \cos^2(\alpha_2) = \frac{u_1^2+u_2^2}{u_1^2+u_2^2+u_3^2}\le 1.$$

As you can see that inequality is an equality for $\alpha_1=\alpha_2=45°$, is true for $\alpha_1=\alpha_2=60°$ but not true for $\alpha_1=\alpha_2=30°.$