Geodesics on SO(3)

If you are assuming the use of the bi-invariant metric, then the geodesics are right/left translations of the one parameter sub-groups $O(t)=O(0) \exp(tG)$ where $G \in \mathfrak{so}(3)$ (i.e. it is anti-symmetric and traceless) and $O(0)\in SO(3)$.

The curve you want will have $O(0)=M_1$ and thus be of the form $O(t) = M_1 \exp(tG)$ with $G=\log(M_1^{-1}M_2)$ (the matrix log). You need to choose a specific matrix log as generically there will be many, you many wish to read about principal matrix logs if you are going to do any numerical computations. This is what will likely come out of Matlab etc. The curve is parameterised by $t \in [0,1]$, it is smooth, is a geodesic of the aforementioned metric and has the end points you hoped for.


The geodesics (of the unique connection invariant under left and right translations and inversion) are the translates of one-parameter subgroups: see Helgason, exerc. 6(iii), p. 148. So given your $M_1$ and $M_2$, find (by surjectiveness of $\exp$) a $Z\in\mathfrak{so}(3)$ such that $\exp(Z)=M_1^{-1}M_2$, and put $g(t) = M_1\exp(tZ)$.