Can we apply L'Hospital's rule where the derivative is not continuous?

L'Hospital's rule says under certain conditions: IF $\lim_{h\to 0} \frac{f'(h)}{g'(h)}=c$ exists, then also $\lim_{h\to 0} \frac{f(h)}{g(h)}=c$. It does not say anything about the existence of the former limit.

In this case - yes, you need derivative to be continuous. In general, you need $\lim \frac{f'(x)}{g'(x)}$ to exist to apply L'Hospital's rule. As in your case $g'(x) = 1$, you proved that if there is a limit of $f'(a + h)$, then the limit is equal to $f'(a)$.