Question about equilibrium points of non-linear system of ODEs.

More likely than not, this is just a system that is defined on $\mathbb{R}^2/(0,0)$; that is, the origin is not part of the domain of the dynamical system.

However, that doesn't say anything about whether or not the neighborhood of the origin behaves as if the origin was a fixed point, and it is very likely that the singularity at the origin is removable.

To prove it, we can show using L'Hopital's rule that:

$$\lim_{y\rightarrow0} \frac{\ln(4y^2)}{1/y} = \lim_{y\rightarrow0} \frac{2/y}{-1/y^2} = \lim_{y\rightarrow0} -2y = 0$$

Therefore the singularity at the origin is removable, and you can analyze the system just as you would in the "normal" case as long as you take care to indicate that you are actually "tacking on" the $0$ value at the origin to define it over all of $\mathbb{R}^2$. In that situation, the origin is indeed a fixed point.

It is worthwhile to do this analysis, because as you can see below, the system possesses a stable limit cycle! You can formally prove it by constructing a trapping region and applying the Poincare-Bendixson theorem.

Let $V((x,y)) = {1 \over 2} (x^2+y^2)$, and $\phi(t) = V((x(t),y(t)))$. Note that $\phi'(t) = -y^2 \ln(x^2+4y^2)$.

Let $A= \{ (x,y) | {1 \over 4} \le x^2+y^2 \le 4 \}$.

Note that $\phi'(t) \ge 0$ if $x(t)^2+y(t)^2= {1 \over 4}$.

Note that $\phi'(t) \le 0$ if $x(t)^2+y(t)^2= 4$.

$A$ contains no equilibrium points.