Why do non-rigid bodies try to increase their moment of inertia?

This happens to isolated rotating system that is not a rigid body.

Inside such a body (for example, steel chain in free fall) the parts move relatively to each other and there is internal friction that dissipates kinetic energy of the system, while angular momentum is conserved. The dissipation goes on until the parts stop moving with respect to each other, so body rotates as a rigid body, even if it is not rigid by constitution.

The rotating state of the body that has the lowest kinetic energy for given angular momentum is that in which the body has the greatest moment of inertia (with respect to center of mass). For example, a long chain thrown into free fall will twist and turn until it is all straight and rotating as rigid body.

This can be seen as follows. Rotational energy of a system in state of rigid rotation around fixed axis $a$ is given, in general, by the formula

$$ E = \frac{1}{2}I_a \Omega^2 $$ where $I_a$ is moment of inertia of the system with respect to $a$ and $\Omega$ is angular velocity of rotation.

Since angular momentum is given by

$$ L = I_a \Omega $$

we can express energy as

$$ E = \frac{L^2}{2I_a}. $$

If $L$ is constant (net torque of external forces acting on the system is zero) and the constitution and initial conditions allow it, the system's dissipation will work to diminish energy until it has the minimum value, which happens for maximum $I_a$ possible.

The moment-of-inertia is found by:

$$I=\sum m r^2$$

The fact that each particle tries to increase its distance $r$ to the rotation centre during rotation is a different topic — that is the centrifugal effect:

A particle in motion will always try to keep up its velocity (speed and direction), because it takes force to change that. So the natural response of a particle is to continue in a straight path.

When rotating, the straight path happens to be away from the orbit. It just happens to be in the direction which increases the distance.

So, it just happens to be the case that rotating things tend to increase their moment of inertia because of the kinematics and dynamics of nature.

Moment of inertia is known as the angular mass or rotational inertia,of a rigid body is a tensor that determines the torque needed for a desired angular acceleration about a rotational axis.

If a mechanical system is constrained to move parallel to a fixed plane, then the rotation of a body in the system occurs around an axis k perpendicular to this plane. In this case, the moment of inertia of the mass in this system is a scalar known as the polar moment of inertia. The definition of the polar moment of inertia can be obtained by considering momentum, kinetic energy and Newton's laws for the planar movement of a rigid system of particles.

If a system of n particles, Pi, i = 1,...,n, are assembled into a rigid body, then the momentum of the system can be written in terms of positions relative to a reference point R, and absolute velocities vi.

https://wikimedia.org/api/rest_v1/media/math/render/svg/9a82ae33e8e6d485a106fca031040e1839c1de03 where ω is the angular velocity of the system and V is the velocity of R.

For planar movement the angular velocity vector is directed along the unit vector k which is perpendicular to the plane of movement. Introduce the unit vectors ei from the reference point R to a point ri , and the unit vector.

ti = k × ei so https://wikimedia.org/api/rest_v1/media/math/render/svg/ff4a46e0a6d0c4eb43981fc94554b8a7426d8522

This defines the relative position vector and the velocity vector for the rigid system of the particles moving in a plane.