Why don't we feel the subtle speed change of Earth's elliptical orbit?

We don't feel any acceleration because the Earth and all of us humans on it is in free fall around the Sun. We don't feel the centripetal acceleration any more than the astronauts on the ISS feel the acceleration of the ISS towards the Earth.

This happens because of the way general relativity describes motion in gravitational field. The motion of a freely falling object is along a line called a geodesic, which is basically the equivalent of a straight line in curved spacetime. And because the freely falling object is moving in a straight line it experiences no force.

To be a bit more precise about this, the trajectory followed by a freely falling object is given by the geodesic equation:

$$ \frac{\mathrm d^2x^\alpha}{\mathrm d\tau^2} = -\Gamma^\alpha_{\,\,\mu\nu}U^\mu U^\nu \tag{1} $$

Explaining what this means is a bit involved, but actually we don't need the details. All we need to know is that the four-acceleration of a body $\mathbf A$ is given by another equation:

$$ A^\alpha = \frac{\mathrm d^2x^\alpha}{\mathrm d\tau^2} + \Gamma^\alpha_{\,\,\mu\nu}U^\mu U^\nu \tag{2} $$

But if use equation (1) to substitute for $d^2x^\alpha/d\tau^2$ in equation (2) we get:

$$ A^\alpha = -\Gamma^\alpha_{\,\,\mu\nu}U^\mu U^\nu + \Gamma^\alpha_{\,\,\mu\nu}U^\mu U^\nu = 0 $$

So for any freely falling body the four acceleration is automatically zero. The acceleration you feel, the "g force", is the size of the four-acceleration - technically the norm of the four-acceleration or the proper acceleration.

Nothing in this argument has referred to the shape of the orbit. Whether the orbit is hyperbolic, parabolic, elliptical or circular the same conclusion applies. The orbitting observer experiences no acceleration.

You might be interested to read my answer to How can you accelerate without moving?, where I discuss this in a bit more detail. For an even more technical approach see How does "curved space" explain gravitational attraction?.

John Rennie's answer is right from the viewpoint of General Relativity -- but since the question is tagged with Newtonian mechanics, it deserves a Newtonian answer too.

In the Newtonian framework, I think the best answer to "why don't we experience this force" is that we can't feel forces that apply to our body at all. What we actually experience with our senses are only forces between different parts of our body.

When you stand on the surface of Earth, you feel neither the gravitational pull of the Sun nor the gravitational pull of the Earth. Strictly speaking you don't even feel the contact force between your footsoles and the ground -- but you do feel the compressive force between the skin of your feet and the bones inside the foot. And to a lesser extent you feel your bones being compressed, and your flesh being stretched by hanging from your skeleton. All these internal forces balance out the gravitational pull on your body, such that there's zero net force applying to each part of it (ignoring the pull from the sun and moon), and you stay in place compared to the earth.

This is what produces the sensation of being pulled towards the earth: the internal forces in your body that resist that pull.

However, for the pull from the sun, there's nothing that balances it out. Every particle in your body simply falls towards the sun, with the acceleration given by the strength of the sun's gravitational field -- and every particle in the earth and in the air around you is doing the same, so no internal forces are needed anywhere to keep the various parts of your body in the same relative position. Therefore there is nothing to feel.

According to the Equivalence Principle a free falling system cannot locally detect a gravitational field. However Earth is a large enough system such that non-local effects turn out to be appreciable. Solar tides are - although small - detectable. So in principle one can experience the Sun's gravitational field even though we are in free fall. What I claim is that the acceleration change through the elliptical orbit is too small.

The Earth's angular momentum, with respect to the focus of the ellipse, is $L=mr^2\dot\phi$, where $m$, $r$ and $\dot\phi$ are the mass, the distance to the center of the Sun and the angular velocity, respectively. Therefore $$mr_p^2\dot\phi_p=mr_a^2\dot\phi_a,$$ where the indices $p$ and $a$ denotes "perihelion" and "aphelion". For an ellipse, $$r_p=\frac{r_0}{1+\epsilon},\quad r_a=\frac{r_0}{1-\epsilon}.$$ Hence $$\frac{\dot\phi_p}{\dot\phi_a}=\left(\frac{1+\epsilon}{1-\epsilon}\right)\approx 1,0340,$$ since the Earth's orbit eccentricity is, $\epsilon\approx 0,0167$. We have a change of about three per cent in six months. The mean angular acceleration is $$\bar\alpha=\frac{0.0340\cdot\phi_a}{180\cdot 24\cdot 60\cdot 60}\sim 10^{-9}\phi_a\, \mathrm{rad/s^2}.$$ Notice that $\phi_a$ is of order $$\frac{2\pi}{365\cdot 24\cdot 60\cdot 60}\sim 10^{-7}\, \mathrm{rad/s}.$$ So the angular acceleration is of order $10^{-16}\mathrm{rad/s^2}$. If you multiply this value by the mean distance to the Sun, $r\sim 10^{11}\, \mathrm{m}$, we get an acceleration of order $10^{-6}\, \mathrm{m/s^2}$. That is negligible compared to the acceleration due to the Earth's gravity, $9.8\, \mathrm{m/s^2}$.