Is it $\mu(dx)$ or $d\mu(x)$? or they are equal?

Let me start by saying that unfortunately I have no references at hand.

---

Both are notations for the same thing: the integral of function $f$ with respect to measure $\mu$.

Which to use is a matter of taste.

Personally I prefer: $$\cdots\mu(dx)$$ because somehow a measurement of the infinitesimal small $dx$ takes place.

In the special case where $\lambda$ denotes the Lebesgue measure on $\mathbb R$ you could say that we have the equality: $$\lambda(dx)=dx$$

i.e. the measure of $dx$ equals $dx$ itself.

If $\mu$ is also a measure on $\mathbb R$ and this with a density $f$ wrt the Lebesgue measure then we can state:$$\mu(dx)=f(x)\lambda(dx)=f(x)dx$$