Trajectory of a satellite whose velocity is less than circular orbital velocity?

The orbit of the satellite would be elliptical, as it must honour 2 elementary conservation laws:

• Law of conservation of total energy

• Law of conservation of angular momentum ( measure of rotational motion )

But depending on particulars, the elliptic orbit may formally cross the physical boundary of the central object or its atmosohere, i.e. a crash would happen.

I.e the GPS satellites have altitude of circular orbits about 20000 km.

If their speed suddenly drops, their orbit would transform to elliptical ones, with apogee (farthest point) at those 20000 km, and perigee (nearest point) e.g just 10000 km.

The bigger the speed drop would be, the lover the perigee would be.

With some speed drop threshold, the satellite at perigee would collide with dragging atmosphere, lowering the ( higher than orbital ) speed at perigee, what would be lowering the subsequent apogees.

Progressively, the elliptical orbit transforms to less and less prolonged orbit, finally gradually transforming to the spiral of death.

With some even bigger initial speed drop, the satellite orbit would cross the Earth surface and what would not be burnt as a fireball, would crash to Earth.

Let suppose the initial satellite speed $v$ is tangential and lower then the speed needed for the circular orbit.

Then it's energy

$$E=\frac 12 mv_\mathrm{ap}^2 - \frac{ GmM}{r_\mathrm{ap}}\lt - \frac{ GmM}{2r_\mathrm{ap}}$$

$$\frac 12 mv_\mathrm{ap}^2 \lt \frac{ GmM}{2r_\mathrm{ap}}$$

$$v_\mathrm{ap} \lt \sqrt {\frac{ GM}{r_\mathrm{ap}}}$$

$$v_\mathrm{ap} = k \cdot \sqrt {\frac{ GM}{r_\mathrm{ap}}}$$

As it's velocity at both perigee and apogee is perpendicular to the position vector, its angular momentum

$$L=m \cdot (\vec r \times \vec v)=m\cdot r_\mathrm{ap}\cdot v_\mathrm{ap}\\=m\cdot r_\mathrm{per}\cdot v_\mathrm{per}$$

Therefore

$$v_\mathrm{per}=v_\mathrm{ap}\cdot \frac{r_\mathrm{ap}}{r_\mathrm{per}}$$

$$E=\frac 12 m\left({v_\mathrm{ap}\cdot \frac{r_\mathrm{ap}}{r_\mathrm{per}}}\right)^2 - \frac{ GmM}{r_\mathrm{per}}=\frac 12 mv_\mathrm{ap}^2 - \frac{ GmM}{r_\mathrm{ap}}$$

$$\frac 12\left({v_\mathrm{ap}\cdot \frac{r_\mathrm{ap}}{r_\mathrm{per}}}\right)^2 - \frac{ GM}{r_\mathrm{per}}=\frac 12 v_\mathrm{ap}^2 - \frac{ GM}{r_\mathrm{ap}} $$

$$ \left(\frac 12 v_\mathrm{ap}^2 - \frac{ GM}{r_\mathrm{ap}}\right)\cdot {r_\mathrm{per}}^2 + GMr_\mathrm{per} - \frac 12 {v_\mathrm{ap}}^2\cdot {r_\mathrm{ap}}^2=0$$

$$ \frac{ GM}{r_\mathrm{ap}} \left(\frac 12 k^2 - 1\right)\cdot {r_\mathrm{per}}^2 + GMr_\mathrm{per} - \frac 12 k^2\cdot \frac{ GM}{r_\mathrm{ap}} \cdot {r_\mathrm{ap}}^2=0$$

$$ \left(1-\frac 12 k^2 \right)\cdot {r_\mathrm{per}}^2 - r_\mathrm{per}\cdot r_\mathrm{ap} + \frac {k^2\cdot {r_\mathrm{ap}}^2}{2}=0$$

This leads to the solving of quadratic equation for $r_\mathrm{per}$

$$ r_\mathrm{per}= \frac{r_\mathrm{ap} \pm \sqrt {{r_\mathrm{ap}}^2- \left(2-k^2 \right)\cdot { k^2\cdot {r_\mathrm{ap}}^2}}}{2-k^2}$$

$$ r_\mathrm{per}=r_\mathrm{ap} \frac{1 \pm \sqrt {1- \left(2-k^2 \right)\cdot k^2}}{2-k^2}$$

$$ r_\mathrm{per}=r_\mathrm{ap} \frac{1 \pm \left(1-k^2\right)}{2-k^2}$$

The solution is $$ r_\mathrm{per}=r_\mathrm{ap} \frac{k^2}{2-k^2}$$

as the other one is the trivial

$$ r_\mathrm{per}=r_\mathrm{ap}$$

If we want to get k for given apogee and perigee:

$$k=\sqrt{\frac{2r_\mathrm{per}}{r_\mathrm{per} + r_\mathrm{ap}}}$$

Note that for $k\gt 1$ and $k\lt \sqrt{2}$, it is the reverse calculation of apogee from perigee.

$$ r_\mathrm{ap}=r_\mathrm{per} \frac{k^2}{2-k^2}$$

For $k=\sqrt{2}$, respectively $k\gt \sqrt{2}$ the trajectory would not be an ellipse any more, but satellite would get itself a parabolic, respectively hyperbolic trajectory.

Depending on apogee and perigee radius, compared to Earth and Earth atmosphere (negligible drag) radius, these cases happen:

For both radii above drag region - ellipse

For perigee within drag region+ apogee out if drag region - ellipse shortening then spiralling.

For perigee within Earth radius - direct crash.

For both radii within drag region - spiralling.

To illustrate what was answered by @Poutnik, consider a satellite whose speed changes in $k$ times as $\vec {v_1}=k\vec {v}$ at some point of the trajectory. Figure 1 shows the trajectory before the speed reduction (blue) and after (orange). We see an elliptical trajectory that is getting closer to the central body with decreasing $k$.

Fig. 2 shows how a collision occurs with a central body (green disk).

If the orbit is an ellipse, then the collision looks like in Figure 3

It is clear that there can be no helix for orbits of type MEO,GEO and HEO. But a fall in a spiral orbit is possible for type LEO. Figure 4 shows a fall along a spiral trajectory with many turns (left) and with one quarter turn (right) from a type LEO. Light blue color highlighted the atmosphere.