Difference between "undefined" and "does not exist"

In the particular example you gave: The derivative is defined as $\lim_{h\to 0} \frac{f(x+h) - f(x)}{h}$ and, as it is with limits, this limit may or may not exist. In the case $f(x) = |x|$ and $x=0$ the limit just does not exist and hence, this is the right wording. On the other hand there are possible definitions of a derivative of $f(x) = |x|$ at zero (e.g. using convex analysis one may define it to be the whole interval $[-1,1]$) and hence, it seems appropriate to say that the derivative is undefined.

In general "does not exists" and "is undefined" are very different things at a practical level. The former says that there is a definition for something which does not lead to a mathematical object in a specific case. The latter says that there is just no definition for a specific case. Of course, one can interchange both formulation some times (as in you example, at least in my opinion).

There is a little subtlety not addressed so far:

Given a function $f:\ A\to\Bbb R$ on an open set $A\subset\Bbb R$ and a point $x\in A$, you can ask whether $f$ is differentiable at $x$. The function $f$ is differentiable at $x$ (or: has a derivative at $x$) if the limit $$\lim_{h\to0}{f(x+h)-f(x)\over h}\tag{1}$$ exists and is finite. This limit is called the derivative of $f$ at $x$ and is denoted by $f'(x)$.

The set $A'$ of all $x\in A$ where the limit $(1)$ exists, is the domain of a new function $f'$ associated to $f$. This new function is called the derivative of $f$.

When $x\in A'$ then we say that $f'$ is defined at $x$.