Proof of the chainrule: is this proof correct and did I use the right notation?

Your case 1 is irreparably flawed.$ \def\lfrac#1#2{{\large\frac{#1}{#2}}} $ In that case you claimed that if $g(x+h)=g(x)$ for some $h$ then $\lim_{h\to0} \lfrac{f(g(x+h))−f(g(x))}{h} = 0$. That is false. For example let $f$ be the identity function, and $g = \sin$ and $x = 0$ and $h = π$. Then $g(x+h) = g(x)$ but $\lim_{h\to0} \lfrac{f(g(x+h))−f(g(x))}{h}$ $= \lim_{h\to0} \lfrac{\sin(h)-\sin(0)}{h} = 1$, contradicting your claim.

Your case 2 is also completely broken for the same logical reason, because even if $g(x+h) \ne g(x)$ for some $h$ it does not mean that $\lim_{h\to0} \lfrac{f(g(x+h))−f(g(x))}{g(x+h)-g(x)}$ exists, so your first line in that case is already wrong. For example let $f$ be the identity function again, and $g(t) = |t-1|+|t+1|$ for every real $t$, and $x = 0$. Then $g(2) \ne g(0)$ but $\lim_{h\to0} \lfrac{f(g(x+h))−f(g(x))}{g(x+h)-g(x)}$ does not exist because $g(x+h)-g(x) = 0$ for every $h \in [-1,1]$. Note that $g'(0) = 0$ and $f'(2) = 1$, and the chain-rule still holds, but the limit you wrote down does not exist.

This answer will attempt to elaborate further on the problem that Medo pointed out in the comments:

Fix some value of $x$ and consider the sets $$ H=\{h\mid\ g(x+h)= g(x)\} \\ H^c=\{h\mid\ g(x+h)\neq g(x)\} $$ Here $H$ corresponds to your case 1, and $H^c$ corresponds to your case 2.

• Either set may be infinite (countable or uncountable) or empty and $H$ may also be finite.
• So first of all, there is no guarantee that $h=0$ is a limit point of both sets meaning $h\to 0$ may not even make sense within both sets. In that case one can simply argue using the set in which $h=0$ is in fact a limit point, and everything should be fine.
• Second of all, if $h=0$ is a limit point of both $H$ and $H^c$ so that $h\to 0$ makes senses for both, we need an extra argument to make sure that the two limits are equal. This is where some work remains to be done.

---

The way I have developed for dealing with the problem uses a more Leibnizian approach, namely define: $$ \frac{\Delta f(g)}{\Delta x} = \begin{cases} 0 & \text{for }\Delta g=0 \\ \quad\\ \frac{\Delta f}{\Delta g}\cdot\frac{\Delta g}{\Delta x} & \text{for }\Delta g\neq 0 \end{cases} $$ where $\Delta g$ and $\Delta f$ denote the corrsponding changes of $g(x)$ and $f(g(x))$ when $x$ is changed by $\Delta x$. This can be shown to be continuous and always equal to $$ \frac{f(g+\Delta g)-f(g)}{\Delta x} $$ and so it provides a continuous alternative to the problematic fatorization by "filling in" the missing values when $\Delta g=0$.