Find the closed path of given system of ordinary differential equation

In this problem, it is not necessary to solve a system of differential equations. Use the Bendixson–Dulac theorem instead: $$ \frac{\partial f}{\partial x}=12x^2y^2-5x^4y^4 $$ $$ \frac{\partial g}{\partial y}=5x^4y^4+6x^2y^2 $$ The function $$ \frac{\partial f}{\partial x}+\frac{\partial g}{\partial y}=12x^2y^2+6x^2y^2=18x^2y^2, $$ has the same sign almost everywhere, therefore, the conditions of the theorem are satisfied. Hence, there are no closed orbits.

Starting from this line in your answer: $$-(x^2y^2+2)ydx+(4-x^2y^2)xdy=0$$ $$-x^2y^2(ydx+xdy)-2ydx+4xdy=0$$ $$-x^2y^2d(xy)-2ydx+4xdy=0$$ Divide by $xy$: $$-xyd(xy)-2\dfrac {dx}x+4\dfrac {dy}y=0$$ $$-(xy)^2-4\ln x+8\ln y=0$$ $$-(xy)^2+4\ln \dfrac {y^2}{x}=C$$ Or if you prefer: $$(xy)^2+4\ln \dfrac x{y^2}=C_1$$ Our answers are different. There is a sign difference. This may explain AVK's comment:

There is a mistake somewhere in your solution. The picture of the phase plane of the original system does not coincide with the curves that you obtained.

---

Your mistake is here: $$\frac{1}{6} \left ( xy^2+\frac{2}{x} \right ) dx-\frac{1}{6} \left ( \frac{4}{y}\color {red}{+x^2y} \right )dy=0$$

The term in red should be negative from the precedent line: $$-(x^2y^2+2)ydx+(4-x^2y^2)xdy=0$$ You multiply by $-\dfrac 1 {6xy}$: $$\dfrac 1 {6xy}(x^2y^2+2)ydx-\dfrac 1 {6xy}(4 \color{blue}{-x^2y^2})xdy=0$$

HINT:

It's worth getting an idea about the field, using a phase plotter, say here. Once you notice the rough shape of the movements along the paths, you can check for instance that along any path starting in the upper half plane we have $\frac{d y}{d t} > 0$. What can you conclude? What about a path starting in the lower half plane? On the $x$ axis?