Determine all functions satisfying $f\left ( f(x)^{2}y \right )=x^{3}f(xy)$

Ok, I'll complete with your solution...

$$f\left ( f(x)^{2}y \right )=x^{3}f(xy) \tag{1}.$$

$$f\left ( f(x)^{2} \right )=x^{3}f(x) \tag{2}.$$

Now replace $x$ by $xy$ in $(2)$ , now apply $(2)$ twice, second time to $\left (y, f(x)^{2} \right )$ instead of $(x,y)$ $$f\left ( f(xy)^{2} \right )=(xy)^{3}f(xy)=y^{3}f\left ( f(x)^{2}y \right )=f\left ( f(x)^{2} f(y)^{2} \right )$$ Since $f$ is injective, we get $$f(xy)^{2}=f(x)^{2}f(y)^{2}$$ $$f(xy)=f(x)f(y)$$

Therefore, $f$ is multiplicative. This also implies $f(1)=1$ and $f(x^n)=f(x)^n$ for all integers $n.$

Then the function equation $(1)$ can be re-written as $$f\left ( f(x) \right )^{2} f(y)=x^{3}f(x)f(y)$$ $$f\left ( f(x) \right )=\sqrt{x^{3}f(x)} \tag{3}$$

Let $g(x)=xf(x)$. Then, by $(3),$ we have $$g\left ( g(x) \right )=g\left (xf(x) \right )=xf(x)\cdot f\left (xf(x) \right )=xf(x)^{2} f\left ( f(x) \right )=xf(x)^{2}\sqrt{x^{3}f(x)}=\left ( xf(x) \right )^{5/2}=\left ( g(x) \right )^{5/2}$$

and, by induction, $$\underset{n+1}{\underbrace{g ( g(....g}}(x)....) )=\left ( g(x) \right )^{(5/2)^{n}} \tag{4}$$

for every positive integer $n.$

Consider $(4)$ for a fixed $x.$ The left-hand side is always rational, so $\left ( g(x) \right )^{(5/2)^{n}}$ must be rational for every $n$. We show that this is possible only if $g(x)=1$. Suppose that $g(x)\neq 1$ and let the prime factorization of $g(x)$ be $g(x)=p_{1}^{\alpha _{1}}... p_{k}^{\alpha _{k}}$ where $p_{1}...p_{k}$ are distinct primes and $\alpha _{1}...\alpha _{k}$ are nonzero integers. Then the unique prime factorization of $(4)$ is

$$\underset{n+1}{\underbrace{g ( g(....g}}(x)....) )=\left ( g(x) \right )^{(5/2)^{n}} =p_{1}^{(5/2)^{n}{\alpha _{1}}}...p_{k}^{(5/2)^{n}{\alpha _{k}}}$$

where the exponents should be integers. But this is not true for large values of $n$, for example $(\frac{5}{2})^{n} \alpha _{1}$ cannot be a integer number when $2^{n}\not{\mid } \alpha $. Therefore $g(x) \neq 1$ is impossible.

Hence, $g(x)=1$ and thus $f(x)=\frac{1}{x}$ for all $x$.

The function $f(x)=\frac{1}{x}$ satisfies the equation $(1)$:

$$f\left ( f(x)^{2}y \right )=\frac{1}{f(x)^{2}y}=\frac{1}{(\frac{1}{x}^{2})y}=\frac{x^{3}}{xy}=x^{3}f(x).$$

AND WE'RE DONE. "SP3ED"

Let $P(x,y)$ be the assertion: $$ f(f(x)^2y)=x^3f(xy)\space\forall x,y\in\mathbb{Q^+} $$ We get (due to the injectivity): $$ P(1,y): f(f(1)^2y)=f(y)\iff f(1)^2y=y\iff f(1)=1 $$ $$ P(x,f(y)^2): f(f(x)^2f(y)^2)=x^3f(xf(y)^2)=x^3y^3f(xy) $$ $$ P(xy,1): f(f(xy)^2)=x^3y^3f(xy) $$ If we combine the last to results, we get: $$ f(f(xy)^2)=f(f(x)^2f(y)^2)\iff f(xy)=f(x)f(y) $$ Now define $g(x):=xf(x)$: $$ P(x,x^2): f(x^2f(x)^2)=x^3f(x^3)\iff f(x)^2f(f(x))^2=x^3f(x^3)\iff g(f(x))=g(x)^{\frac{3}{2}} $$ If we iterate this relation, we obtain: $$ g(x)^{\frac{3^n}{2^n}}=g(f^n(x)) $$ Where $f^n$ denotes the $n$-th iteration of $f$. Therefore, since $g(x)^{\frac{3^n}{2^n}}$ is rational, $g(x)^{\frac{1}{2^n}}$ needs to be rational to for all $n\in\mathbb{N_0}$ and thus $g(x)=1$. Therfore, we obtain the only solution: $$ f(x)=\frac{1}{x}\space\forall x\in\mathbb{Q^+} $$