Non-abelian Ext functor and non-abelian $H^2$

EDITED, taking into account the comments of Donu Arapura.

As JLA wrote, a homomorphism $f\colon G\to N$ gives an extension \begin{equation}\label{e:E} 1\to K\to E\to G\to 1.\tag{E} \end{equation} This extension defines a homomorphism $$b\colon G\to \operatorname{Out} K$$ called the band (lien, kernel) of \eqref{e:E}. By definition, $H^2(G,K,b)$ is the set of isomorphism classes of extensions \eqref{e:E} bound by $b$.

A cohomology class $\eta(E)\in H^2(G,K,b)$ is called neutral if the extension \eqref{e:E} splits, that is, there exists a homomorphism $G\to E$ such that the composite homomorphism $G\to E\to G$ is the identity automorphism of $G$. In this case we obtain an action $\varphi$ of $G$ on the normal subgroup $K$ of $E$, and we obtain an isomorphism $E\overset{\sim}{\to}K\rtimes_\varphi G$ with the semidirect product.

There may be more that one neutral class in $H^2(G,K,b)$: they correspond to semidirect products with different actions $\varphi$ of $G$ on $K$. I have read that there may be no neutral elements, but I don't know examples. (In the Galois cohomology setting, for a connected reductive group, by Douai's theorem there always exists a neutral element in nonabelian $H^2$; see [2], Proposition 3.1).

If $K$ is abelian, then $\operatorname{Out} K = \operatorname{Aut} K$, so $b$ is just an action of $G$ on $K$, and $H^2(G,K,b)$ is the usual abelian group cohomology $H^2(G,K)$, where $G$ acts on $K$ via $b$.

The set $H^2(G,K,b)$ can be described in terms of cocycles. See Section 1.14 in Springer [1].

The band $b$ defines an action of $G$ on the center $Z=Z(K)$, and we may consider the usual (abelian) group cohomology $H^2(G,Z)$. From the cocyclic description of $H^2(G,K,b)$ it is clear that $H^2(G,Z)$ naturally acts on $H^2(G,K,b)$.

Moreover, if the set $H^2(G,K,b)$ is nonempty, then $H^2(G,Z)$ acts on it simply transitively; see Mac Lane, Homology, Theorem IV.8.8. The set $H^2(G,K,b)$ is nonempty if and only if a certain obstruction $\operatorname{Obs}(G,K,b)\in H^3(G,Z)$ vanishes; see Mac Lane, Theorem IV.8.7.

Note that we should not think that $H^2(G,K,b)$ "equals" $H^2(G,Z)$. First, $H^2(G,K,b)$ does not have a distinguished unit element. Secondly, $H^2(G,K,b)$ has a distinguished subset $N^2(G,K,b)$ of neutral elements. This is important because in many applications one uses nonabelian $H^2$ in order to determine whether a given extension \eqref{e:E} is split or not.

As far as I know, nonabelian $H^2$ is mostly used in the Galois cohomology setting. Namely, if $k$ is an algebraic closure of a field $k_0$ of characteristic 0, $G=\operatorname{Gal}(k/k_0)$, and $Y$ is a quasi-projective $k$-variety with additional structure (say, an algebraic group or a homogeneous space) such that for any $\sigma\in G=\operatorname{Gal}(k/k_0)$ there exists an isomorphism $\alpha\colon\sigma Y\overset{\sim}{\to}Y$, then it defines an extension $$1\to \operatorname{Aut} Y\to E\to G\to 1,$$ where $E$ is the set of such pairs $(\alpha,\sigma)$ with a suitably defined composition law. We obtain the cohomology class $\eta(Y)\in H^2(k_0,\operatorname{Aut} Y,b)$ of this extension for a suitable band $b$. The variety $Y$ (with additional structure) admits a $k_0$-model if and only if $\eta(Y)$ is neutral, that is, the extension splits; see this question.

For nonabelian $H^2$ in Galois cohomology see:

[1] T. A. Springer, Non-abelian $H^2$ in Galois cohomology, in: Algebraic Groups and Discontinuous Subgroups, Proc. Sympos. Pure Math. 9, Amer. Math. Soc., Providence, 1966, 164-182.

[2] M. Borovoi, Abelianization of the second nonabelian Galois cohomology. Duke Math. J. 72 (1993), 217-239.

[3] Flicker, Scheiderer, Sujatha, Grothendieck's theorem on non-abelian $H^2$ and local–global principles. J. Amer. Math. Soc. 11 (1998), no. 3, 731–750.

See also newer papers (they refer to these three), and this preprint.