Limit $\frac{x^2y}{x^4+y^2}$ is found using polar coordinates but it is not supposed to exist.

The limit is not defined because in order for the limit to exist, the value of the function for every possible path to $(0,0)$ must tend to the same finite value. When $y = x^2$, you have not necessarily shown that the limit is in fact $0$. When you transformed to polar coordinates and then took the limit as $r \to 0$, you are assuming that $\theta$ is a fixed constant. Therefore, you are looking only at paths that follow a straight line to the origin.

Mathematica code:

F[x_, y_] := x^2 y/(x^4 + y^2)
op = ParametricPlot3D[{r Cos[t], r Sin[t], F[r Cos[t], r Sin[t]]},
     {r, 0, Sqrt[2.1]}, {t, -Pi, Pi}, PlotPoints -> 40, MaxRecursion -> 8,
     Mesh -> {10, 48}, PlotRange -> {{-1, 1}, {-1, 1}, {-1/2, 1/2}}, 
     SphericalRegion -> True, Axes -> False, Boxed -> False];
an = Show[op, ViewPoint -> {{Cos[2 Pi #], Sin[2 Pi #], 0}, {-Sin[2 Pi #], 
     Cos[2 Pi #], 0}, {0, 0, 1}}.{1.3, -2.4, 2}] & /@ (Range[40]/40);

You haven't taken into account what happens if $\theta$ is variable as a function of $r$ when $r$ goes to $0$. Choose $\theta$ so that $\sin \theta = r$, i.e. $\theta$ is approximately $r$ and you will get $\cos \theta$ is about 1 for small $r$, and then the limit will not be zero, so the limit doesn't exist.

If you want to use polar coordinates to show that a limit exists, particularly in the case where you want to show the limit is $0$ as $r \to 0$, then if you factor out a positive power of $r$ then you need to bound the remaining factor by either a constant or a multiple of a negative power of $r$ that is lower than the positive power you factored out. In your case you can't do this because when $\sin \theta = r$ you can't produce such a bound for the expression after you factor out $r$. If you had something like $r/(\cos^4 \theta + \sin^4 \theta)$ then you could bound $1/(\cos^4 \theta + \sin^4 \theta)$ by a constant for all $\theta$ and so you would then get that the limit is indeed $0$ as $r \to 0$.

Let $\alpha >0$, and consider the path $\gamma_\alpha(t) = (t,\alpha t^2)$. Then we have $f \circ \gamma_\alpha (t) = {\alpha t^4 \over t^4+ \alpha^2 t^4 }$, and the limit as $t \to 0$ is ${\alpha \over 1+\alpha^2}$ (in fact, it is constant along this path).

The limit exists along all of these paths, but is different. If the limit exists, its value must be independent of how $(x,y) \to 0$.