Question about the derivative definition

Instead of $h$ here I will use $\Delta x$ as I think it makes more sense to consider a change in $x$.

For $$\lim\limits_{\Delta x\to 0} \frac{f(x+\Delta x) - f(x)}{\Delta x}\tag{1}$$ at no point do we ever substitute $\Delta x=0$. We can only say that in the limit $\Delta x$ tends to zero; this is why there is the limit notation directly in-front of the fraction.

Consider the graph below which is an identical graphical representation of Equation $(1)$:

You can see from the graph that the Secant line, which is the line that intersects the purple curve at $(x,f(x))$ and at $(x+\Delta x,f(x+\Delta x))$ is an approximation to the derivative (tangent) of the purple curve at $(x,f(x))$.

Now imagine $\Delta x$ getting smaller and smaller until eventually it becomes vanishingly small; which we call 'infinitesimal' (loosely speaking this is as small as you can possibly get but still greater than zero).

At this point you can see that the Secant line approaches the Tangent line or derivative at $(x,f(x))$ as $\Delta x$ tends to zero. Thus, the approximation of the Secant line to the Tangent line is optimal at $(x,f(x)+\Delta x)$ and this is the exact meaning of Equation $(1)$.

You seem to conflate the two notions of "take the limit as $h$ approaches $0$" and "plug in $0$ for $h$". In some situations, the latter is an easy way of doing the former. But in other situations, including the definition of derivative, the latter is (as you noticed) nonsense, and so you have to actually do the former. (The first step toward doing the former is, of course, to make sure you thoroughly understand the concept of limit.)

When I taught Calculus, I got this question a lot. One simple exercise that helped many students understand the concept of a limit is the following: Pick your favorite function, e.g. $\sin(x)$, then evaluate the fractional part of the limit with a calculator choosing some small $h$ and some value for $x$. For example with $x=1$ and $h=0.1$, we get: $${\sin(1.1) - \sin(1) \over{0.1} }= 0.497364$$ Then choose a smaller $h$, but keep $x$ the same. For example, for $h=0.01$: $${\sin(1.01) - \sin(1) \over{0.01} }= 0.536086$$ And continue with smaller and smaller values for $h$. \begin{array}{|c|c|} \hline h & (\sin(x+h) - \sin(x))/h \\\hline 0.1 & 0.497364 \\\hline 0.01 & 0.536086 \\\hline 0.001 & 0.539881 \\\hline 0.0001 & 0.540260 \\\hline 0.00001 & 0.540298 \\\hline 0.000001 & 0.540302 \\\hline \end{array} You can clearly see that the value is converging to the derivative which is: $$\cos(1) = 0.5403023059$$