Why doesn't orbital body keep going faster and faster?
You have to consider the limit of infinitesimally short time, in which the (vertical on the paper) component of velocity is infinitely short, and thus also the angle changes for an infinitesimal amount. In this limit, the correction to the length is quadratic in the time step and vanishes exactly in the physical limit of continuous time. Pythagoras:
$$v_2=\sqrt{v^2+(adt)^2}\approx v + \frac{a^2}{2v}dt^2 +\cdots$$
Because the direction of the velocity changes. The velocity will start to point less and less 'towards' point A and when the distance between A and B is the smallest, the velocity will make a right angle with the radius, which means acceleration vector also makes a right angle with the velocity. At this point the radial component of the speed is zero and the total speed is the highest. After this point the acceleration vector will point slightly away from the speed vector and its length will only decrease until it reaches the highest point again.
I have made this image to help you understand with the velocity (red) and radial velocity vector (blue) drawn. Keep in mind that when the radial velocity is decreasing but still pointing towards A, the total velocity is still increasing.
This question points out the importance of symplecticity in physics.
In an orbital simulation, suppose one simply advances state via $$\begin{align} \boldsymbol x(t+\!\Delta t) &= \boldsymbol x(t) + \boldsymbol v(t)\, \Delta t \tag 1 \\ \boldsymbol v(t+\Delta t) &= \boldsymbol v(t) + \boldsymbol a(t)\, \Delta t \end{align}$$ where $\Delta t$ is a finite (non-infinitesimal) quantity and $\boldsymbol a(t)$ is calculated via Newton's law of gravitation. This makes the orbiting body spiral outward and gain speed. This is what is vexing @TylerDurden.
This is outward spiraling is clearest when one starts with an object in a circular orbit. The initial step is along the tangent, so away from the circular orbit. The speed increases as well; the change in velocity is orthogonal to the initial velocity. Something is clearly amiss.
What's amiss is the discretization performed above, as suggested by simple numerical integration theory. Any second order degree differential equation can be converted to a first order differential equation by making the first derivative (velocity in this case) a part of the state and then applying numerical integration techniques for solving first order ODEs to the resultant differential equation. The simplest numerical solution to solving a first order initial value problem is to advance state via $\boldsymbol s(t+\Delta t) = \boldsymbol s(t) + \Delta t \, d \boldsymbol s(t)/dt$. This is the Euler method, and it results in equations (1) above when applied to an orbiting body.
The problem is that this discretization isn't symplectic (i.e., it violates the conservation laws). Another way to look at is that this approach ignores geometry. (The conservation laws are "geometry.") There are other non-symplectic techniques such as canonical Runge Kutta integration that make an orbiting body spiral inward.
The issue at hand is that converting a second order differential equation to a first order differential equation and then using first order initial value techniques to numerically solve the ODE comes at a cost, and that cost is tossing geometry out the window. What's needed are techniques that do not toss geometry out the window. A very simple approach is to apply equations (1) in a slightly different order: $$\begin{align} \boldsymbol v(t+\!\Delta t) &= \boldsymbol v(t) + \boldsymbol a(t)\, \Delta t \tag 2 \\ \boldsymbol x(t+\!\Delta t) &= \boldsymbol x(t) + \boldsymbol v(t+\!\Delta t)\, \Delta t \end{align}$$ This is the symplectic Euler method. Notice how the velocity and position calculations are now braided. This is one of the meanings of "symplectic."
If you work out the math with regard to applying equations (2) to gravitation, you'll find that this alternate formulation of the Euler method explicitly obey's Kepler's second law, that a line drawn from the Sun to a planet sweeps out equal areas in equal times. This is geometry! Kepler's second law is of course a special instance of the conservation of angular momentum. The conservation laws and geometry are closely coupled.