If $f,g: [a,b] \to \mathbb{R}$ are bounded with $g$ continuous and $f>g$, is there a continuous $h:[a,b] \to \mathbb{R}$ with $f>h>g$?

The claim is false. Consider any interval $[a,b]$ of non-zero length and define on it

$$g(x)=0$$

and

$$ f(x)= \begin{cases} 1, & \text{ if } x \notin \mathbb Q \\ 1, & \text{ if } x =0 \\ \frac1q, & \text{ if } x =\frac{p}q;p,q\in \mathbb Z, q>0, p\neq 0, \gcd(p,q)=1. \\ \end{cases} $$

You can take a similar example as in your other question: