Why is Lebesgue measure theory asymmetric?

I have a (possibly idiosyncratic) view that the natural form of measure theory is for finite measure spaces and bounded functions. Other cases are obviously very important, but we have to work harder to get them. You can see this is many of the proofs, where the finite case is easier, and we have to work a bit more to generalize it. For example, the usual proof of the Radon-Nikodym theorem works that way.

In the finite measure space case, with bounded functions, everything can be made symmetric. The symmetry is broken in the general case, because allowing infinity breaks it. In integration, this asymmetry shows up in the way we have to have separate theorems for the non-negative measurable functions and the integrable functions. For bounded functions on finite measure spaces you don't need to impose any extra conditions.

I think your statement about Jordan is actually wrong. If $m_*(E) = \infty$ and $m^*(E) = \infty$, then $E$ need not be Jordan measurable. If you talk only about bounded sets $E$, then your characterization is correct. But it is also correct for Lebesgue measure (using Lebesgue inner and outer measure).

The reason for Caratheodory's criterion is to define measurability when even bounded sets could have infinite measure, so that restricting to bounded sets no longer helps. One of Caratheorory's examples was an "arc length" measure for sets in $\mathbb R^n$. In that case, there is no obvious way to define inner measure. But we still can define outer measure. And then we need a criterion for measurability that uses only outer measure.

More recent mathematicians have developed a way to start only with an "inner measure" and go from there.

I think, the reason is that if the ground space has infinite measure, you can not define the measurable sets as those for which inner measure equals the outer measure: it may happen that both are infinite, while the set is still not measurable.

Note also (this may be related) that outer and inner regarity behave differently in general. For example, the sigma-finite Borel measure on the Polish space is inner regular, but not always outer regular (example: counting measure of rational numbers as a measure on $\mathbb{R}$.)