Two guys and a horse

Let $t$ be the minimum time for the trip in hours. The optimal solution is for the horse to take one person $x$ miles, drop him off, go back to pick up the other person, and ride $x$ miles with the second person in such a way that they all reach the destination at the same time. In this way, each person rode for $x$ miles and walked for $20-x$ miles. Since the points where the people switch from walking to riding are each $20-x$ miles from an endpoint, the total number of miles the horse traveled is $$2x+(20-2(20-x))=2x+(2x-20)=4x-20$$

Since time is distance divided by speed, we have the following equations (the first since the horse travelled for a total of $t$ hours and the second since the people rode and walked respectively for a total of $t$ hours): $$\frac{4x-20}{10}=t\\\frac x{10}+\frac{20-x}{4}=t$$

The solution to this system of equations is $x=\frac{140}{11}$ miles and $t=\frac{34}{11}$ hours, as the book suggested.

To get an answer of $3\frac1{11}$ hours, we have to assume that (1) the horse is sufficiently trained to run back down the road on its own to pick up the walking person, (2) the initial rider knows the exact moment to dismount to achieve the optimal arrival time, and (3) the time needed to mount or dismount the horse is negligible. It's probably easier to see what's going on if we look at the general case rather than jumping straight into the arithmetic. Let the distance to travel be $d$, the walking speed be $u$, and the trotting speed $v$ (respectively 20 miles, 4 mph, and 10 mph in the question). Person A sets out on foot, with person B on the horse. At an intermediate time $t$, B dismounts, sends the horse back to pick up A, and continues the journey on foot. Person A meets the horse later at time $t'$, mounts it, and arrives at the destination at the same time $T$ as B does.

Since the total distance travelled by A is $d$, we have $ut'+v(T-t')=d,$ or$$(v-u)t'=vT-d\qquad(1)$$and similarly for B, we get $vt+u(T-t)=d$, or:$$(v-u)t=d-uT.\qquad(2)$$The net distance of the horse from the start, after its backward leg, equals the distance covered by A when they meet at time $t'$: namely $vt-v(t'-t)=ut'$, or$$(u+v)t'=2vt.\qquad\qquad(3)$$Considering the ratio of $t'$ to $t$ from eqns 1 and 2, and then from eqn 3, we have$$\frac{t'}t=\frac{vT-d}{d-uT}=\frac{2v}{u+v},$$from which we obtain$$T=\frac{u+3v}{3u+v}\cdot\frac dv.$$Substituting the given values for the distance and speeds then gives the required result.