Why does cross product give a vector which is perpendicular to a plane

The determinant formula isn't so mysterious. Consider the cross product $\mathbf{v} = \langle a,b,c \rangle \times \langle d,e,f \rangle$ as the formal determinant

$$ \det \left(\begin{array}{ccc} \mathbf{i} & \mathbf{j} & \mathbf{k}\\ a & b & c \\ d & e & f \end{array} \right) $$

where $\mathbf{i}, \mathbf{j}, \mathbf{k}$ are the standard basis vectors. If instead one considers $\mathbf{i}, \mathbf{j}, \mathbf{k}$ as indeterminates and substitutes $x, y, z$ for them, this determinant computes the dot product $\mathbf{v} \cdot \langle x, y, z \rangle$. But letting $\langle x, y, z \rangle$ be $\langle a, b, c \rangle$ or $\langle d, e, f \rangle$ gives a zero determinant, so $\mathbf{v}$ is perpendicular to the latter two vectors, hence to the plane they span, as Omnomnomnom says.

According to Prof. Josiah Willard Gibbs, one of the founders of Vector Analysis, the cross (or skew) product is defined as follows in his 1881 pamphlet Vector Analysis - Arranged for the use of Students in Physics

Definition: The skew product of the vector A into the vector B is the vector quantity C whose direction is the normal upon that side of the plane of A and B on which rotation from A to B through an angle of less than one hundred and eighty degrees appears positive or counter- clockwise ; and whose magnitude is obtained by multiplying the product of the magnitudes of A and B by the sine of the angle from A to B.

This definition, coming from the horse's mouth, as it were, is a geometric definition. Why then, do we in the 21st c. insist on defining it in terms of determinants. The determinant "definition" is a direct consequence of the geometric definition and the distributivity of cross products $$\vec{\mathbf{w}}\times(\vec{\mathbf{u}}+ \vec{\mathbf{v}}) = \vec{\mathbf{w}}\times\vec{\mathbf{u}}+\vec{\mathbf{w}}\times\vec{\mathbf{v}}$$ $$(\vec{\mathbf{u}}+ \vec{\mathbf{v}})\times\vec{\mathbf{w}} = \vec{\mathbf{u}}\times\vec{\mathbf{w}}+\vec{\mathbf{v}}\times\vec{\mathbf{w}}$$ now try to expand $$(a_1\vec{\mathbf{i}}+a_2\vec{\mathbf{j}}+a_3\vec{\mathbf{k}})\times(b_1\vec{\mathbf{i}}+b_2\vec{\mathbf{j}}+b_3\vec{\mathbf{k}})$$ to see that cross products, in fact, distribute over addition, see a geometric proof on page 9 of http://www.math.oregonstate.edu/bridge/papers/dot+cross.pdf

Look. I'm not a mathematician, but I have a perspective which can explain why the cross product of two vectors is another vector perpendicular to them. It is not a proof but it will help make that idea fimilar.

One can understand cross product in this way:imagine a line segment that makes colorful marks wherever it moves on a paper. Now make it move in the same direction and for the same distance as another segment, preserving the direction of the first segment. The result would be a parallelogram with area donated by ab sin(θ) where a and b are the lengths of the segments. So we can define cross product of two vectors as "the area of the parallelogram in wich these vectors are adjacent sides".

The first question is: why is it a vector? Imagine a plane containing two vectors a and b and the angle from a to b equals θ, the cross product of a and b equals ||a|| ||b|| sin(θ). That works quite well as a scalar, but the problem comes when you flip the entire plane containing the vectors, which is the same as replacing a for b and vise versa. The angle from a to b becomes -θ, and the cross product becomes ||a|| ||b|| sin(-θ), which equals -||a|| ||b|| sin(θ). Flipping the plane reverses the result. So the cross product can be represented as a vector with its starting point lying on the plane, which points "up or down". If that vector is pointing up, it will point down when you flip the plane and vise versa.

The second question is: why is the cross product perpendicular to the plane? Why doesn't it just point anywhere up or down? That's because when you flip the plane the cross product is completely reversed, which means it's perpendicular to the plane.