Prove $\sum_\text{cyc}\frac{1}{3-ab}\le\frac{3}{2}$

Your calculation is correct: $m = \frac 65$ is an upper bound. It is not the best upper bound though, because the estimates $\frac{2}{5+c^2} < \frac 25$ are not sharp for all three variables simultaneously.

The best upper bound is $\color{red}{m=\frac 98}$. Proof: The function $f(t) = \frac{1}{3-\sqrt t}$ is increasing and concave on $[0, 1]$: $$ f'(t) = \frac{1}{2 \sqrt t (3-\sqrt t)^2} \ge 0 \, ,\\ f''(t) = -\frac{3(1-\sqrt t)}{4 t^{3/2}(3-\sqrt t)^3} \le 0 \, . $$ Therefore Jensen's inequality gives $$ \frac{1}{3-ab}+\frac{1}{3-bc}+\frac{1}{3-ca} = f(a^2b^2)+f(b^2c^2)+ f(c^2a^2) \\ \le 3 f\left( \frac{a^2b^2+b^2c^2+c^2a^2}{3}\right) \underset{(*)}{\le} 3 f\left( \bigl( \frac{a^2+b^2+c^2}{3} \bigr)^2\right) = 3 f(\frac 19) = \frac 98 \, . $$ At $(*)$ we have used that $$ \frac{a^2b^2+b^2c^2+c^2a^2}{3} \le \left( \frac{a^2+b^2+c^2}{3} \right)^2 \, , $$ which is MacLaurin's inequality applied to $$ (x_1, x_2,x_3) = (a^2, b^2, c^2) \, . $$

So $m= \frac98$ is an upper bound, and it is best possible since equality holds for $a=b=c=1/\sqrt 3$.

Hint.-Because of $a^2+b^2+c^2=1$, each $a,b,c$ must be in the interval $[-1,1]$ then one has for positive values $$\frac13\le\frac{1}{3-ab}\le\frac12\\\frac13\le\frac{1}{3-ac}\le\frac12\\\frac13\le\frac{1}{3-bc}\le\frac12$$ Thus the sum is less or equal than $\dfrac32$.

If some of $a,b,c$ is negative, the fraction is less than $\dfrac13$ then one has the same majorant $\dfrac32$ for the sum.