Examples of loops which have two-sided inverses.

I think most of your questions are answered by looking at Moufang loops.

A loop in which the left and right inverse agree (a loop with two-sided inverses) is called an IP-loop. Sometimes people replace a loop by an isotope, which basically scrambles and relabels the multiplication table (apply a row and column permutation, and a permutation of the underlying set). For groups, that would basically be crazy, but loops are not terribly messed up by such an operation.

A loop is a Moufang loop iff every isotope has two-sided inverses.

Non-associative, commutative, Moufang loops have order a multiple of 81, and there are two non-isomorphic such loops. They were constructed by M. Hall Jr.

See also the Parker Loop which is a finite loop of order $2^{13}$ related to the binary Golay code, $M_{24}$ (largest sporadic Mathieu group), Conway's construction of the Monster group, etc.