Can mass be defined in purely mathematical or geometrical terms?

At the moment mass is one of the axiomatically defined quantities in the MKS (meter, kilogram, second) system. As with axioms in mathematics, other units can and have been defined and then the (MKS) units become derivative.

The theoretical models of physics use mathematics with its axioms, and in addition impose additional axioms and axiomatic statements to connect the mathematical solutions to observations and data and to predict new situations. In particle physics it is the standard model.

Since ancient times there are two schools of thought, the Platonic, which can be encapsulated into "mathematics creates reality", and the pragmatic, which can be summed into "mathematics models observations and data".

In my observation, many theorists and theoretically inclined physicists belong to the Platonic school. Usually experimental physicists belong to the pragmatic, because they are aware how one generation's physics theory becomes an emergent one from the current theory in the next generation.

If the platonists are correct and there exists a Theory Of Everything (TOE) and the masses of the elementary particles in the standard model of particle physics come out as predictions of the TOE, the answer to your question will be "yes".

At the moment the pragmatists are on the ascendant so the answer is that no, there isn't any prediction of elementary masses in the current models of physics.

Yes, mass has a geometrical explanation. The mass of a system is the magnitude or “length” of its energy-momentum four-vector $p=(E,p_x,p_y,p_z)$, using the Minkowski metric $\text{diag}(1,-1,-1,-1)$ of four-dimensional spacetime. This is the standard interpretation of

$$m^2=p\cdot p=E^2-p_x^2-p_y^2-p_z^2$$

in units where $c=1$.

Yes; you can derive mass from the representation theory of the Lie group ${\rm Spin}(3,1)$.

It's a long story, but I think I can usefully summarize it. If you need some areas further expanded, please say so in the comments.

First, a definition. The Lie group ${\rm SO}(3,1)$ describes the world that we live in according to special relativity. It is the local gauge group of space-time, so it's the exact meaning of the geometry in your question. The Lie group ${\rm Spin}(3,1)$ is the double cover of ${\rm SO}(3,1)$, and it's simply connected [note 1].

How does representation theory show up? Imagine you do an experiment, with some inputs $x$ and you get some output $y=f(x)$. The results should be invariant under the gauge group $G$, which might be ${\rm SO}(3)$ if we're non-relativistic or ${\rm SO}(3,1)$ if we're using special relativity. It also includes translations by ${\mathbb R}^3$ or ${\mathbb R}^4$ but these are less important to the story of mass. Anyway, invariance under $G$ means that $f(g x)=g f(x)$ for $g\in G$. That means that $f$ is actually a representation of $G$. So a fundamental particle is actually described by a representation of ${\rm SO}(3,1)$.

So why ${\rm Spin}(3,1)$? In physics, we note that we need to include so-called spin representations of ${\rm SO}(3,1)$, by which we mean representations of the double cover ${\rm Spin}(3,1)$. Phenomenologically, this is certainly true, because fermions are described by a spin representation, and they exist!

Okay, so what is the representation theory of ${\rm Spin}(3,1)$? Let's use the sign convention $(+,-,-,-)$. There are 3 types of representation, corresponding to a 4-momentum $p_i=(p_0, p_1, p_2, p_3)$ [note 2] with a squared-length $p^2=p_0^2-p_1^2-p_2^2-p_3^2$ for which $p^2<0$, $p^2=0$ or $p^2>0$. In each case, we look at the subgroup $H_p<G$ which fixes $p_i$, we find the representations of $H_p$, and we induce these representations back up to $G$. This is called Mackey theory. For $p^2<0$, we get tachyons, which may or may not be physical, but in any case we won't discuss this further. For $p^2=0$ (but $p_i$ is not the zero 4-vector), we get massless light-like particles with $H_p\cong{\rm SO}(2)$ [note 3]. The representation theory of ${\rm SO}(2)$ describes polarization. But the important case for our purposes is $p^2>0$. Here, $H_p\cong{\rm Spin}(3)$, which is the double cover of ${\rm SO}(3)$ [note 4]. The Lie group ${\rm Spin}(3)$ has one irreducible representation in each dimension. If a particle corresponds to an irreducible representation of dimension $m$, we say that the particle has spin $(m-1)/2$. If $m$ is odd (so the spin is an integer) we have a boson, otherwise we have a fermion.

We've gone on a bit of a journey into representation theory, but the important take-away for our purposes is that the representation corresponding to a massive particle depends fundamentally on the 4-momentum $p_i$ through its squared-length $p^2>0$. But the length is its mass. The 4-momentum is $p_i=(E, {\bf p})$, where $E$ is the (combined rest-mass and kinetic) energy and ${\bf p}=(p_1, p_2, p_3)$ is the non-relativistic momentum. In the rest-frame of the particle, the 4-momentum is $(E,{\bf 0})$, so $p=E$. Now the possibly most well-known equation in physics, $E=m c^2$, finishes the story and tells you that a fundamental particle must have a well-defined mass.

Notes

• [note 1]: You may see ${\rm Spin}(3,1)$ described as ${\rm SL}(2,{\mathbb C})$. These groups are isomorphic, but I find it more helpful for this story to regard this as a coincidence.
• [note 2]: Physicists actually write $p^i$ rather than $p_i$, but the meaning is essentially the same.
• [note 3]: Actually the double cover ${\rm Spin}(2)$ of ${\rm SO}(2)$, but they are isomorphic, so it's not too important.
• [note 4]: Again, ${\rm Spin}(3)$ is often described as ${\rm SU}(2)$, but for our purposes this is best regarded as a coincidence.

Further reading

• Gerald B. Folland, Quantum Field Theory: A Tourist Guide for Mathematicians