Why is heat transfer reversible when temperature difference is infinitesimal?

To do it reversibly, you can heat the body from $T_1$ to $T_2$ (i.e., over a finite temperature change) using an infinite sequence of constant temperature reservoirs, in which each reservoir in turn is only dT higher in temperature than the body at any time (and also only dT higher in temperature than the reservoir before it in the sequence). Each increment in heat transfer would take place with only a differential temperature driving force between the body and the current reservoir. To reverse the process, and bring both the body and the reservoirs back to their original states, you would simply contact the body with the reservoirs in the reverse sequence, in which case the reservoirs would be dT lower in temperature than the body in each step of the process). The only difference would be with regard to the very first and very last reservoirs (which could not be returned to their original states). But this would be insignificant.

In the case where the body is heated from $T_1$ to $T_2$ by bringing it into contact with a constant temperature reservoir at $T_2$ for the entire time until the body equilibrated at $T_2$, all the heat transfer would take place with a finite temperature driving force, and there would be no way to return both the body and the reservoir to their original states without causing a significant change in something else (like using other reservoirs).

A reversible process is one in which the system is only slightly removed from being at thermodynamic equilibrium throughout the change. Thus, a reversible process can be viewed as imposing a continuous sequence of thermodynamic equilibrium states.

It is reversible in the first case because it satisfies the reversibility definition. A thermodynamic process is called reversible if an infinitesimal change of the external condition reverses the process. Consider a system at temperature $T$ in thermal contact with a thermal reservoir at same temperature. By an infinitesimal increase $dT$ of the reservoir's temperature you get heat flowing to the body. With a further infinitesimal decrease, let us say $2dT$ you reverse the flow. The same will not happen if there was a finite difference of temperature.