What does the denominator in the second derivative mean?

You are completely correct. When we write $\frac{d^2y}{dx^2}$, we really mean to write $\frac{d^2y}{(dx)^2}$, but those parentheses make the denominator difficult to read and write. Mathematicians accept the form $\frac{d^2y}{dx^2}$ without question, because it is not exactly an algebraic expression (although sometimes it can be treated as one), rather it is a notation that represents the concept of finding the rate of change of the rate of change.

You're right that it should really be $(dx)^2$ (if you were asking "why should that be?", see my answer here).

I suspect the brackets/parentheses aren't written because Leibniz himself didn't write them when he first presented the notation and everyone just followed his lead.