Why is the second derivative of an inflection point zero?

Short answer: The second derivative at an inflection point may be zero but it also may be undefined.

Longer answer:

One definition of an inflection point is where the second derivative changes sign (from positive to negative or the reverse).

The Intermediate Value Theorem for derivatives says that if a derivative is defined on a closed interval then it assumes any intermediate value between the values at the endpoints of that interval. The second derivative is obviously a derivative, so if it is defined on any interval including the inflection point, if we look at a value on one side of the inflection point, which must be positive, and another value on the other side, which must be negative, there is some place where the second derivative is zero. The only place it can be zero is at the inflection point. Therefore, it is commonly said that the second derivative at the inflection point must be zero.

However, there is one more possibility. The second derivative may not be defined at the inflection point. This does not satisfy the assumptions of the Intermediate Value Theorem so there is no problem.

One example where the second derivative is undefined at an inflection point is $y=\sqrt[3] x$, where there is an inflection point at $x=0$ but the first and second derivatives are not defined there. You can see the curve is concave up on the left and concave down on the right.

By definition, inflection points are where a function changes concavity, or in other words where a function is neither concave up nor down but is (often) moving from one to the other. A positive second derivative corresponds to a function being concave up, and a negative corresponds to concave down, so it makes sense that it is when the second derivative is 0 that our function is changing concavity, and hence corresponds to an inflection point.

Perhaps this image can give you a visual idea of when you can see concavity changing, and a corresponding inflection point. It is also important to note (as several answers have done already) that an inflection point can also have an undefined second derivative.

Think about what the second derivative means. A positive second derivative means concave up, negative means concave down. Well, an inflection point is when the concavity switches. So naturally the second derivative has to equal zero at some point if our second derivative is going to switch signs.

It's very analogous to a critical value.