What breaks the symmetry in a Keplerian orbit?

1. In general if EOMs possess a symmetry, it is not necessarily true that a solution to the EOMs possesses this symmetry. See also this related Phys.SE post.

2. It might make the reader more happy to know that the 3D Kepler problem is maximally superintegrable (i.e. has 5 independent integrals of motion), and via a coordinate transformation it is equivalent to a free particle on a 3-sphere $\mathbb{S}^3$, which has $SO(4)$ symmetry.

The eccentricity, $e$, of the orbit of the reduced mass, $\mu = m_1m_2/(m_1 + m_2)\,,$ is $$ e = \sqrt{1 - \frac{L^2}{GMa\mu^2}}\,, $$ where $a$ is the length of the semi-major axis of the ellipse, $M$ is the total mass of the system and $L$ is the magnitude of the total orbital angular momentum of the system.

The total angular momentum of the system is $$ {\bf L} = m_1{\bf r}_1\times{\bf v}_1 + m_2{\bf r}_2\times{\bf v}_2\,. $$ So it is the angular momentum that causes an uncircular orbit, i.e. an elliptical orbit with $e\ne 0$. So if the positions and velocities of the masses are such that $$ L < \sqrt{GMa\mu^2} $$ the orbit of the reduced mass will not be circular, but elliptical. This is what ``breaks the symmetry." Orbits with $$ 0 < L < \sqrt{GMa\mu^2} $$ will be elliptical.

${\bf Edit:}$ So yes, if you take given positions, the initial velocities will determine the eccentricity of the orbit. If these are such that $$ L < \sqrt{GMa\mu^2} $$ then the circular symmetry will be broken.