Why doesn't derivative difference quotient violate the epsilon-delta definition of a limit?
The difference quotient $\dfrac{f(x+h) - f(x)} h$ is of course undefined when $h=0$. You have \begin{align} \frac{f(x+h)-f(x)} h & = 2x+h & & \text{when } h\ne 0 \\[10pt] \text{and } 2x+h & = 2x & & \text{when }h=0. \end{align} LATER NOTE: In NO edition of this answer has there every been a piecewise definition of any function. That is NOT what appears above.
If the value of $f'(x)$ is $2x$, then you want to prove that $$ \left| \frac{f(x+h) - f(x)} h - 2x \right| $$ can be made less than $\varepsilon$ by making $|h|$ less than $\delta$, but not equal to $0$.
That's the same as saying $$ |(2x+h) - 2x| $$ can be made less than $\varepsilon$ by making $h$ small enough, but not equal to $0$.
And that's the same as saying $|h|$ can be made less than $\varepsilon$ by making $|h|$ less than $\delta$ but not $0$.
Why, then, is it permissible to evaluate the limit by making $h=0$?
The answer is that it's easy to see that $|h|$ can be made small by making $|h|$ small. In other words, the limit as $h$ approaches $0$ of $h$ is simply the value that $h$ has when $h=0$.
In somewhat conventional language, $2x+h$ is a continuous function of $h$, so its limit as $h\to0$ is the same as its value when $h=0$.
It appears that you are still thinking of limits as almost equivalent to plugging. Perhaps you should have a look at these answers here and here to understand that limits are something very different from plugging.
Your question is ultimately: how $\lim_{h \to 0}2x + h = 2x$? Your argument is that the $\epsilon$-$\delta$ definition of limit prevents $h$ to become $0$ then how $2x + h$ becomes $2x$ if $h$ is not $0$. Well, $2x + h$ is not equal to $2x$ unless $h = 0$ but limit of $2x + h$ is $2x$ as $h \to 0$. Trying to think of limits as values of functions always creates this problem. Apply the definition of limit here and you can see that for any given $\epsilon > 0$ it is possible to find $\delta > 0$ such that $$|2x + h - 2x| < \epsilon$$ whenever $0 < |h| < \delta$. Here you can see that $\delta = \epsilon$ suffices. Hence $\lim_{h \to 0}2x + h = 2x$.
A limit does not mean evaluating the values of a function, rather it means verifying whether a set of inequalities regarding the values of a function hold under a certain set of circumstances or not. Unless you adopt this view you are going to get confused and ask similar questions.
Update: User Dave L. Renfro has mentioned a very nice example of a limit evaluation where it is possible to plug (plugging does not result in undefined expression), yet plugging gives the wrong answer. The example was originally given by David Sanders in A cautionary counter-example, The Mathematical Gazette 59 #407 (March 1975) 44-45.
Let $\lfloor x\rfloor$ denote the greatest integer not exceeding $x$ and then we define $$f(x) = x\lfloor -x^{2}\rfloor$$ Clearly $f(0) = 0$ and hence $$f'(0) = \lim_{x \to 0}\frac{f(x) - f(0)}{x} = \lim_{x \to 0}\frac{x\lfloor -x^{2}\rfloor}{x} = \lim_{x \to 0}\lfloor -x^{2}\rfloor$$ In the last step after cancelling $x$ from numerator and denominator we get an expression which is well defined at $x = 0$ and we can plug the value $x = 0$ to get the limit as $0$. But this gives a wrong answer! The right answer is $-1$ because we can make $-x^{2}$ to lie between $-1$ and $0$ by choosing $x$ near $0$ and therefore $\lfloor -x^{2}\rfloor$ stays constant with value $-1$.
If the reader has visited my linked answers (given in beginning of this answer) then he may well notice that the plugging does not work in above example because the greatest integer function $\lfloor x \rfloor$ is not an elementary function.
We can do this because the function $g(h)=2x+h$ (for some fixed value of $x$) is continuous at $h=0$. This means that $\lim_{h\to 0}g(h)=g(0)$, so we can evaluate the limit by just plugging in $h=0$.
Of course, this is ultimately circular: we only know that $g(h)$ is continuous at $h=0$ because we can evaluate the limit by using the $\epsilon$-$\delta$ definition and find that it coincides with the value of $g(0)$. So you should think of the "plug in $h=0$" rule as only a heuristic; the true rigorous way to compute limits is by using the $\epsilon$-$\delta$ definition. However, if by previous work you have done you know that the expression you get is continuous in $h$ (e.g., in this case, if you have proven that all polynomial functions are continuous), then you can compute the limit by plugging in $h=0$.