Dot product of two vectors in spherical polar coordinates, do I have to convert to cartesian coordinates?

The formula $$ \sum_{i=1}^3 p_i q_i $$ for the dot product obviously holds for the Cartesian form of the vectors only. The proposed sum of the three products of components isn't even dimensionally correct – the radial coordinates are dimensionful while the angles are dimensionless, so they just can't be added.

One can't simplify the calculation much. At most, one may realize that the inner product will only depend on $\phi_1,\phi_2$ through their difference $\phi_1-\phi_2$ because one may use the rotational symmetry around the $z$-axis to set e.g. $\phi_2$=0.

While doing so, we may set $\phi_2=0$ i.e. $y_2=0$ and the inner product reduces to $$ x_1 x_2 + z_1 z_2 = r_1r_2 (\sin\theta_1\sin\theta_2 \cos\phi_1 + \cos\theta_1\cos\theta_2) $$ We may restore the form for a general rotation by replacing $\phi_1$ in the formula above by $\phi_1-\phi_2$ to get the inner product $$ r_1r_2 (\sin\theta_1\sin\theta_2 \cos(\phi_1-\phi_2) + \cos\theta_1\cos\theta_2) $$ which is the same as your formula because $\cos(a-b)=\cos a\cos b +\sin a \sin b$.

Notice that the radius vector of a point in space with spherical coordinates $r,\theta,\phi$ can be written as $$\mathbf{r}=r\mathbf{\hat{r}}(\theta,\phi),$$ where $$\mathbf{\hat{r}}(\theta,\phi)=\sin\theta\cos\phi\mathbf{\hat{x}}+\sin\theta\sin\phi\mathbf{\hat{y}}+\cos\theta\mathbf{\hat{z}}.$$ Thus the components of the radius vector with respect to the "spherical basis" $(\mathbf{\hat{r}}(\theta,\phi),\boldsymbol{\hat{\theta}}(\theta,\phi),\boldsymbol{\hat{\phi}}(\theta,\phi))$ AT THE POINT with spherical coordinates $(r,\theta,\phi)$ (it is VERY important to realize that the spherical unit vectors are really vector fields, they vary from point to point!) are NOT $(r,\theta,\phi)$. Instead, they are $(r,0,0)$! Indeed, $\theta$ and $\phi$, having no physical dimension, cannot be the components of a vector.

When $r_1,\theta_1,\phi_1$ and $r_2,\theta_2,\phi_2$ are known for two vectors $\mathbf{p}_1,\mathbf{p}_2$, we have $$\mathbf{p}_1=r_1\mathbf{\hat{r}}(\theta_1,\phi_1)\quad\text{and}\quad\mathbf{p}_2=r_2\mathbf{\hat{r}}(\theta_2,\phi_2).$$ (With respect to the spherical basis, we are forced to use different unit vectors $\mathbf{\hat{r}}(\theta_1,\phi_1)$ and $\mathbf{\hat{r}}(\theta_2,\phi_2)$! This is a striking difference between cartesian coordinates and spherical coordinates.) Hence: \begin{align} \mathbf{p}_1\cdot\mathbf{p}_2&=\left[r_1\mathbf{\hat{r}}(\theta_1,\phi_1)\right]\cdot\left[r_2\mathbf{\hat{r}}(\theta_2,\phi_2)\right]\\ &=r_1r_2\mathbf{\hat{r}}(\theta_1,\phi_1)\cdot\mathbf{\hat{r}}(\theta_2,\phi_2)\\ &=r_1r_2\left(\sin\theta_1\cos\phi_1\mathbf{\hat{x}}+\sin\theta_1\sin\phi_1\mathbf{\hat{y}}+\cos\theta_1\mathbf{\hat{z}}\right)\cdot\left(\sin\theta_2\cos\phi_2\mathbf{\hat{x}}+\sin\theta_2\sin\phi_2\mathbf{\hat{y}}+\cos\theta_2\mathbf{\hat{z}}\right)\\ &=r_1r_2\left(\sin\theta_1\sin\theta_2\cos\phi_1\cos\phi_2+\sin\theta_1\sin\theta_2\sin\phi_1\sin\phi_2+\cos\theta_1\cos\theta_2\right)\\ &=r_1r_2\left[\sin\theta_1\sin\theta_2\cos(\phi_1-\phi_2)+\cos\theta_1\cos\theta_2\right]. \end{align} This is the formula you have given in your post. When we put $r_1=r_2=1$ and call $\omega$ the angle between $\mathbf{p}_1$ and $\mathbf{p}_2$, we get a fairly established formula, namely $$\cos\omega=\sin\theta_1\sin\theta_2\cos(\phi_1-\phi_2)+\cos\theta_1\cos\theta_2$$ since $\mathbf{p}_1\cdot\mathbf{p}_2=r_1r_2\cos\omega=\cos\omega$.

EDIT: never mind, your question ended up at the top of the list because someone recently edited your OP. My bad for not noticing the 2016 date in due time ...