Geometric meaning of the quantity $|a|^2 |b|^2 |c|^2 - (a \cdot b)(b \cdot c)(c \cdot a)$ for non-coplanar vectors $a$, $b$, $c$

I don't know if this hint is conclusive, but this is what I got:

\begin{align*} B & = \lVert a\rVert^{2}\lVert b\rVert^{2}\lVert c\rVert^{2} - \langle a,b\rangle\langle b,c\rangle\langle c,a\rangle\\ & = \lVert a\rVert^{2}\lVert b\rVert^{2}\lVert c\rVert^{2} - \lVert a\rVert^{2}\lVert b\rVert^{2}\lVert c\rVert^{2}\cos(\theta_{1})\cos(\theta_{2})\cos(\theta_{3})\\ & = \lVert a\rVert^{2}\lVert b\rVert^{2}\lVert c\rVert^{2}[1 - \cos(\theta_{1})\cos(\theta_{2})\cos(\theta_{3})] \end{align*}

I don't know a geometric interpretation for this quantity, but there's a natural interpretation of this quantity that makes it more transparently analogous to the other candidate you mentioned, namely the square of volume $V$ of the parallelepiped spanned by ${\bf a}, {\bf b}, {\bf c}$, at least for $n = 3$.

Recall that that squared area can be written as $$\begin{align*}V^2 &= \operatorname{det}\pmatrix{{\bf a} \cdot {\bf a} & {\bf a} \cdot {\bf b} & {\bf a} \cdot {\bf c} \\ {\bf b} \cdot {\bf a} & {\bf b} \cdot {\bf b} & {\bf b} \cdot {\bf c} \\ {\bf c} \cdot {\bf a} & {\bf c} \cdot {\bf b} & {\bf c} \cdot {\bf c}} \\ &= ({\bf a} \cdot {\bf a})({\bf b} \cdot {\bf b})({\bf c} \cdot {\bf c}) - \mathfrak{S}[({\bf a} \cdot {\bf a})({\bf b} \cdot {\bf c})^2] + 2 ({\bf b} \cdot {\bf c}) ({\bf c} \cdot {\bf a})({\bf a} \cdot {\bf b}),\end{align*}$$ where $\mathfrak{S}[\cdot]$ denotes the cyclic sum of $\cdot$ in ${\bf a}, {\bf b}, {\bf c}$.

On the other hand, recall that the determinant of an $n \times n$ matrix $M = (m_{ij})$ can be written as $$\det M = \sum_{\sigma} (\operatorname{sign} \sigma) m_{1 \sigma(1)} \cdots m_{n \sigma(n)},$$ where the sum is over all permutations $\sigma$ of $n$. Here, $\operatorname{sign} \sigma$ is $+1$ if the permutation is even and $-1$ if it is odd.

Now, if you know a little representation theory, you know $\operatorname{sign}$ is a representation of the group $S_n$ of permutations of $n$ elements. By replacing $\operatorname{sign}$ in the definition of the determinant with some other representation we get a generalization of the determinant (called an immanant) and---perhaps you see where this is going---can replace $\det$ in the first display equation above to get a new invariant for the (unordered) triple $({\bf a}, {\bf b}, {\bf c})$.

Taking $n = 3$ and the two-dimensional representation $\lambda$ gives the immanant $$\operatorname{imm}_{\lambda}(M) = 2 m_{11} m_{12} m_{13} - m_{23} m_{31} m_{12} - m_{32} m_{13} m_{21} .$$ Replacing $\det$ in the first display equation with $\operatorname{imm}_{\lambda}$ gives exactly twice the quantity in question: $$\boxed{\tfrac{1}{2} \operatorname{imm}_{\lambda} \pmatrix{{\bf a} \cdot {\bf a} & {\bf a} \cdot {\bf b} & {\bf a} \cdot {\bf c} \\ {\bf b} \cdot {\bf a} & {\bf b} \cdot {\bf b} & {\bf b} \cdot {\bf c} \\ {\bf c} \cdot {\bf a} & {\bf c} \cdot {\bf b} & {\bf c} \cdot {\bf c}} = ({\bf a} \cdot {\bf a})({\bf b} \cdot {\bf b})({\bf c} \cdot {\bf c}) - ({\bf b} \cdot {\bf c})({\bf c} \cdot {\bf a})({\bf a} \cdot {\bf b})} .$$

Remark Up to isomorphism there are only three irreducible representations of $S_3$, and the one remaining is the trivial representation. This gives rise to the permanent, $\operatorname{per}$, which is defined just by removing from the definition of $\det$ the sign of the permutation, so $$\begin{multline*}\operatorname{per}\pmatrix{{\bf a} \cdot {\bf a} & {\bf a} \cdot {\bf b} & {\bf a} \cdot {\bf c} \\ {\bf b} \cdot {\bf a} & {\bf b} \cdot {\bf b} & {\bf b} \cdot {\bf c} \\ {\bf c} \cdot {\bf a} & {\bf c} \cdot {\bf b} & {\bf c} \cdot {\bf c}} \\ = ({\bf a} \cdot {\bf a})({\bf b} \cdot {\bf b})({\bf c} \cdot {\bf c}) + \mathfrak{S}[({\bf a} \cdot {\bf a})({\bf b} \cdot {\bf c})^2] + 2 ({\bf b} \cdot {\bf c}) ({\bf c} \cdot {\bf a})({\bf a} \cdot {\bf b}),\end{multline*}$$ where $\mathfrak{S}[\cdot]$ denotes the cyclic sum of $\cdot$ in ${\bf a}, {\bf b}, {\bf c}$. It would be interesting to have a geometric interpretation of this quantity, too.