Notation of the second derivative - Where does the d go?

Gottfried Wilhelm Leibniz, who introduced this notation in the 17th century, intended $dx$ to be an infinitely small change in $x$ and $du$ to be the corresponding infinitely small change in $u$, so that if, for example, $du/dx=3$ at a particular point that means $u$ is changing $3$ times as fast as $x$ is changing at that point.

The notation $\dfrac{d^2u}{dx^2}$ actually means $\dfrac{d\left(\dfrac{du}{dx}\right)}{dx}$, the infinitely small change in $du/dx$ divided by the corresponding infinitely small change in $x$. Thus the second derivative is the rate of change of the rate of change.

Notice that if $u$ is in meters and $x$ in seconds, then $du/dx$ is in $\dfrac{\text{m}}{\text{sec}}$, i.e. meters per second, and $d^2 u/dx^2$ is in $\dfrac{\text{m}}{\text{sec}^2}$, i.e. meters per second per second. Thus $dx^2$ means $(dx)^2$, so the units of measurement of $x$ get squared, and $d^2y$ is in the same units of measurement that $y$ is in, consistently with the fact that $y$ is not a part of what gets squared in the numerator.

where does the $d$ go?

Physicist checking in. All the other answers seem to focus on whether $d$ is a variable and are neglecting the heart of your question.

Simply put, $dx$ is the name of one thing, so in your example

$$\frac{d^2u}{dx^2}=\frac{d^2u}{\left(dx\right)^2}$$

In your words, the "second $d$" is inside the implied parentheses.

$d$ is not a variable, and neither is $dx$ for that matter.

It is confusing because in some case, like the chain rule, differentials act like variables which can cancel:

$$\frac{dy}{dx}\frac{dx}{dt}=\frac{dy}{dt}$$

However, it is most appropriate to think of $\frac{d}{dx}$ as an operator that does something.

Thus, $\frac{d}{dx}(\frac{d}{dx} y)=\frac{d^2}{dx^2}y$.

Somewhat similarly, you wouldn't say that $\sin^2 x=s^2i^2n^2x$

Edit: In case it isn't from the example, you cannot separate $dx$. That is, $dx$ is not $d$ times $x$. This is very much analogous to chemistry when we say things like $\Delta H$. This isn't $\Delta$ times $H$. It is $\Delta$ (change) of $H$.