Upper semicontinuity on a normal space
Thinking out loud: consider the set $U(f):=\{g \in C(X): f\le g\}$. This has a lower bound $f$, trivially. The $\inf$ of $U(f)$ is the largest such upper bound. So consider $h(x) = \inf U(f)$, where the $\inf$ is taken pointwise, so is well-defined by order-completeness of $\mathbb{R}$. Then we know that $h$ is upper semi-continuous (for a proof see here, e.g.)
Suppose we have $p \in X$ where $f(p) < h(p)$. Then $O = \{x: f(x) < h(p) \}$ is open by usc and non-empty, as $p$ is in it. Maybe this will allow you to construct new functions in between but larger than $f$.
If $f$ were lsc we'd have $f \le h$ $f$ lsc and $h$ usc and then normality gives us a continuous $g$ with $ f \le g \le h$ (this is classical for normal spaces, an interpolation theorem). But we only know $f$ is usc not lsc.
It is possible that $f$ is upper semicontinuous and there is no continuous $g$ above $f$, e.g. $f(x)=1/x$ for $x>0$ and $f(x)=0$ for $x\le 0$. (Edit: This example is wrong. A corrected example is appended at the bottom.) So the statement you want to prove is not valid.
If you add the assumption that there exists a continuous $g\ge f$ then the complete regularity of $X$ will suffice: we can assume that $f(x)<g(x)$ for every $x$. Then for every $u\in X$ and every $r$ such that $f(u)<r<g(u)$ there exists an open neighborhood $U$ of $u$ such that $f(x)<r<g(x)$ for every $x\in U$. Since $X$ is completely regular, there exists a continuous function $t\colon X\to [0,1]$ such that $t(u)=1$ and $t(x)=0$ for $x\in X\setminus U$. Define $h(x)=g(x)-t(x)\big(g(u)-r\big)$. Then $h$ is continuous, $f(x)<h(x)\le g(x)$ for every $x\in X$, and $h(u)=r$. It then follows that $f=\inf\{g\in C(X)\!:g\ge f\}$.
In fact, the following theorem holds.
Let $X$ be a $T_1$-space. The following conditions are equivalent.
- For every upper semicontinuous $f\colon X\to\mathbb{R}$, if there is $g\in C(X)$ such that $g\ge f$ then $f=\inf\{g\in C(X)\!:g\ge f\}$.
- $X$ is completely regular.
Edit: Let $X$ be a Dowker space. Then there exists a decreasing sequence $\langle D_n\!:n\in\omega\rangle$ of closed sets with $\bigcap_{n} D_n=\emptyset$ such that $\bigcap_n U_n\neq\emptyset$ whenever $\langle U_n\!:n\in\omega\rangle$ is a sequence of open sets such that $D_n\subseteq U_n$ for every $n$. Let us define $f\colon X\to\mathbb{R}$ by $f(x)=-1$ iff $x\in X\setminus D_0$ and $f(x)=n$ iff $x\in D_n\setminus D_{n+1}$. Then $f$ is an upper semicontinuous function such that there is no continuous $g$ above $f$. Otherwise, $U_n=\{x\!:g(x)>n\}$ would be open sets such that $D_n\subseteq U_n$ for every $n$, hence $\bigcap_n U_n\neq\emptyset$, which is impossible. Conversely, it can be proved that if $X$ is a normal space such that for some upper semicontinuous $f\colon X\to\mathbb{R}$ there exists no continuous $g$ above $f$, then $X$ must be a Dowker space.