Covariant Derivative: What does changing direction mean in curved space?

I am going to use the technical term “geodesic” to refer to a “straight line” in a curved manifold. There are two ways to understand this. One is a global way and one is a local way.

Global

The global way may be the easiest (at least to me). Globally a geodesic is the shortest distance* between two points. Once you have a geodesic any slight deviation from that path in any direction will increase your distance. When you have a flat manifold then a geodesic is a straight line, i.e. the shortest distance is a straight line. So the global notion of a geodesic in a curved manifold shares the same minimum-distance property as a straight line in a flat manifold.

For example, on a sphere the geodesics are great circles. If you pick two points on the sphere and attach a rubber band between them then that rubber band will try to minimize the distance and will naturally assume a great circle path. Similarly a rubber band stretched between two points on a flat plane will form a straight line.

*technically it extremizes the distance, so it can be a minimum or maximum

Local

The local concept is a bit more difficult, in my opinion, because it requires two new concepts. One is called parallel transport, and the other is the tangent vector.

Parallel transport is used to map vectors at one point in the manifold to vectors at another nearby point. The idea is to move the vector from one point to the next without turning it. Think about laying a piece of tape smoothly along the path (no wrinkles) and then flattening the tape and making the vector at one point on the path parallel to the vector at any other point on the path. That is the parallel in parallel transport. The mathematical function that maps vectors at one point to the parallel vector at a nearby point is called a connection.

The other concept is the tangent vector. At each point on a path you can form a vector that points along the path. It shows which direction you need to step if you want to stay on the path. Combining the ideas of parallel transport and tangent vectors a geodesic is a curve that parallel transports its tangent vector. Intuitively, this is the concept of never turning either left or right but always stepping straight ahead.

Returning to the example of the sphere. If you walk along a great circle then you never turn to the right or left at any point but you always step straight ahead.

So those are the two concepts of geodesics: geodesics minimize the path length between two points and they parallel transport their tangent vector. Those are the concepts of “the shortest distance between two points is a straight line” and “straight lines don’t turn anywhere” both applied to a curved manifold.