Why do ships lean to the outside, but boats lean to the inside of a turn?

Your question became clear after you posted the images.

This corresponds to the difference in a turn as accomplished by a bike/motorcycle and a car/bus/truck. So let's study this case first.

During the turn within the accelerated reference frame there's an imaginary "centrifugal" force, which is directed outward the turn of course. OTOH the force applied by the ground has a component toward the turn (due to friction to prevent/reduce sliding). This creates a momentum that tends to lean the object outward the turn.

This is indeed what happens with most 4-wheel vehicles. They lean outward the turn, this transfers the pressure from the inner wheels to the outers, which causes appropriate change in the normal force applied to the wheels. This in turn creates a momentum which tends to lean the object toward the turn. By such the vehicle is leaned to some angle, after which equilibrium is achieved.

Now let's see what happens with 2-wheel vehicles (like bike). Since the normal force is applied in just 2 points, leaning outward the turn does not transfer the pressure, there's no change in the normal force, hence no momentum toward the turn is created. Moreover, leaning outward the turn displaces the mass from the 2-wheel axis, hence the gravitational force creates even more momentum outward the turn. The bike would just fall.

To accomplish the turn however, the biker leans the bike toward the turn deliberately. They do this by counter-steering, which involves microscopic inputs on the steering pushing the handlebars in the opposite direction to which they want to turn, thus leaning the bike into the turn. Displacing the mass causes gravitational force to create a momentum toward the turn. Which is in equilibrium with the momentum of the "centrifugal" force.

Now let's see what happens with vessels.

As with vehicles, the "centrifugal" force is applied outward the turn, the force applied by the water has a component toward the turn (due to the viscosity). Hence the "centrifugal" force's momentum is outward the turn. The difference is that there's a considerable part of the vessel under the water. Moreover, the center of mass is not required to sit above the water level. Another difference is that there are no discrete contact points with the water, instead water pressure is applied on all the underwater part of the vessel.

When the vessel leans (to either side) its configuration changes: its center of mass is displaced, its underwater part is changed, the volume and shape of the water "pushed out" changes as well. If the center of mass of the vessel + "pushed out" water raises - there's a momentum that tends to return the vessel back to its original state, hence it's stable.

Theoretically all the vessels are stable when at rest (otherwise they'd turn around). However during the motion some vessels are raised (like the small boat in your question) and become unstable. Such vessels definitely may not perform the turn unless deliberately leaned toward the turn. Simply because there's nothing to compensate for the "centrifugal" momentum. OTOH big vessels may remain stable even during the motion, with enough reserve to perform the turn as-is.

So, the factors to consider are:

• Vehicle configuration (sunk level, mass distribution, shape of the underwater part) during the motion.
• Required centripetal acceleration to perform the turn (velocity and radius).
• Exact forces imposed by the water (hydrodynamics).

Based on those one may see to which side the vessel leans during the turn.

There's however another interesting moment. If the vessel is unstable it should lean toward the turn. But how does this actually happen? Bicycle rider leans intentionally, otherwise he'd fall. He does it by counter-steering or displacing his own mass toward the turn, which is considerable WRT the mass of the bicycle.

But is this the case with the motor boats? I doubt if the mass of the rider is considerable WRT the mass of the boat. Plus, if this was the case, unskilled riders would turn around frequently, and I personally never saw this. There may be two explanations of this:

• Perhaps such boats are designed such that steering alone makes them lean toward the turn (due to a specific shape of the underwater tail, some hydrodynamical trick).
• During improper turn the vessel leans outward the turn, than it sinks a little, and in this new configuration there's an adequate momentum. So that the vessel doesn't turn around, it just passes the turn with a lower speed.

The above two answers both have flaws. valdo doesn't answer the key part of the original question, but asks why does the small boat lean in? The second response says that small boats turn in because the drive force is being applied at the rear of the vessel, which isn't quite relevant.

Turning is accomplished by applying a sideways force at the stern of the boat. Seen looking down from above, this is a torque, and attempts to rotate the boat about its COM. As noted in the other responses, the water must push against the side of the hull towards the inside of the turn to turn a moving boat in a circle. (If the boat was not moving, and it had, say, trolling motors at the bow and stern pointing in opposite directions, it could rotate in place, and the water wouldn't have to provide any force.)

We can see the rolling effect come into play if we look horizontally at the boat from behind. In a large ship, where the sideways force is provided by a rudder that extends pretty much from the water line to the keel, the center of the force is operating at a point below the water, about half way between the water line and the keel. The direction of this force is towards the outside of the turn (we are pushing the stern to the outside to turn the bow to the inside). If the center of mass is above this force, a roll-in torque is generated. However, the force provided by the water on the hull to turn the moving boat acts to the inside. This creates a roll-out torque (again, if below the COM). At speed, this is greater than the rudder torque and so the ship rolls to the outside.

In a small boat, the turn-rotation force is provided by changing the thrust direction, in the case of an outboard engine or water jet, or deflecting a low rudder in the prop stream below the keel in the case of an inboard engine. This turning force is very low, not centered between the keel and the waterline, but below the keel. This longer distance below the boat center of mass creates a high roll-in torque that is greater than the roll-out torque created by the water on the hull. This is even more true as the boat rolls, when the thrust vector of the engine actually has an upward component.

By the way, re stability, the center of buoyancy does not generally stay in a fixed location as a boat rolls. If the center of buoyancy is directly below the center of mass, then you would expect that when a boat rolls about its COM, this would position the COB toward high side of the boat. However, when a boat rolls, the shape of the hull that is under water changes, and the center of buoyancy often moves toward the lowered side. This provides a righting torque.

Nice question.

In both boats and ships, the center of mass of the vessel tends to be above the water line. This is a result of design; one wishes to have as little of the vessel as possible below the water line as it is the part of the vessel that is below the water line that causes the most friction in movement. At the same time, the humanly usable portions of a boat tend to have a density less than that of water. (An exception is a submarine; note that submarines tend to lean towards the inside and are subject to a problem called "snap roll" where they roll too far. For more, read this MIT thesis.)

To turn a vessel, the water must apply a force to the vessel. Since that force is applied by the water, it is typically applied below the center of mass. Thus one expects that a typical vessel will lean out during a turn. And in fact, this is what a large ship does.

A small boat, when turned without power, will do the same thing as a large ship. To counter the effect, one applies power and this makes a small boat lean in. This is due to the fact that the drive force is being applied at the rear of the vessel. This pushes the rear out on a wider turn and the small boat leans in. Try turning a small boat with no throttle.

Sail boats can either lean out or in depending on which way the wind is blowing. I.e. they lean out on the first part of a tack and lean in on the first part of a jibe, and then reverse. But the natural tendency is to lean out and so the force on a sail boat will be making it lean out as it goes through the wind during a tack. (I.e. at the point in the turn where it would be in irons if it stopped.) Since sail boats do not have propulsion at the rear of the boat (as in a small powered boat), these tendencies apply to both sail boats and sailing ships.