Limit exists v. Differentiable

You're right. $f(x)$ as written has a discontinuity at $x=1$, so it is not considered to be differentiable. However, if you modify $f(x)$ by inserting the removable discontinuity, $f(x)$ becomes a differentiable function. That is $$ g(x)=x-1 = \begin{cases} \frac{(x-1)^2}{x-1} & x\neq 1\\ 0 & x = 1 \end{cases} $$ is a differentiable function

As you said, the existence of a limit at a point (even along with differentiability in the neighborhood of that point) does not guarantee that the function is differentiable at that point. In order for a function to be differentiable at a point, it must first be continuous at that point.

Related: differentiability implies continuity.

Continuity is necessary for the existence of a $f'$, that means if a function $f$ has discontinuity at $x=a$ then $f$ is not differentiable at $x=a$. Your function is not continuous at $x=1$, therefore it's not differentiable at $x=1$. Moreover, continuity is not a sufficient condition.For example $|x|$ is continuous over $\mathbb{R}$ but it's not differentiable at $x=0$.

For the example that you have given, the function is not differentiable at that point because of the discontinuity that exists at that point.

Remember that the definition of a derivative at point uses a limit e.g.

$$f' (a) = \lim_{h\rightarrow 0}\frac{f(a+h) - f(a)}{h}$$

Thus, it is only differentiable when a limit exists so yes, you are right when you say that if a limit exists, the function is differentiable and when it doesn't exist, the function is not differentiable.