Splitting of Automorphism Group
About 20% of the finite groups of order at most 60 have a non-split automorphism group.
The smallest example is the dihedral group of order 10. Its automorphism group is the Frobenius group of order 20, the normalizer of a Sylow 5-subgroup in $S_5$. This automorphism group has an element of order 4, but the outer automorphism group is order 2. If the extension was split, then it would be split when restricted to the Sylow 2-subgroups, but a cyclic group of order 4 is not a split extension of a group of order 2 by a group of order 2.
The smallest nilpotent examples have order 16, and include the dihedral and quaternion groups of order 16, as well as the direct product of the cyclic group of order 2 with the quaternion group of order 8.
The smallest non-supersolvable example has order 36, and is an index 2 subgroup of AGL(1,9), so is basically identical to the order 10 example, an index 2 subgroup of AGL(1,5).
The smallest non-solvable example is the simple group of order 360, $A_6 \cong \operatorname{PSL}(2,9)$ with automorphism group $\operatorname{P\Gamma L}(2,9)$. The outer automorphism group is elementary abelian of order 4, and the preimages of two of the three maximal subgroups are split, but the third, giving rise to the Mathieu group of degree 10. There are similar examples for every odd prime power squared $q^2$. They are called $M(q^2)$ and are Zassenhaus groups of degree $q^2+1$.