What's the origin of the naming convention for the standard basis of $\mathfrak{sl}_2(\mathbb{C}) $?

The letters $\mathrm X$ and $\mathrm Y$ are already used by Cayley in what Dieudonné (in MR) calls the first description of all finite-dimensional irreducible $\mathfrak{sl}_2$-modules: A Second Memoir upon Quantics (1856, §§29–31). He apparently has no name for $\mathrm{XY-YX}$. Same in e.g. Faà di Bruno (1876, §§113–114).

$\mathrm H$ seems to stand for Hauptmatrix, as introduced by Weyl (1922, p. 125; see also 1931, pp. 114, 122) to describe Cartan subalgebras, roots (Multiplikatoren) and root spaces (Länder):

Kommt in $\mathfrak g$ eine „Hauptmatrix“ $H$ vor, in der alle Elemente außerhalb der Hauptdiagonale verschwinden, während in der Hauptdiagonale die Zahlen $\alpha_1,\alpha_2,\dots,\alpha_n$ stehen, so bilde man die Differenzen $\alpha_i - \alpha_k$ und teile mit Bezug auf $H$ das Schema einer beliebigen Matrix in „Länder“ ein, indem man jedem Feld $(ik)$ des Schemas ($i$ der Zeilen-, $k$ der Kolonnenindex) die Zahl $\alpha_i - \alpha_k$ als „Multiplikator“ zuordnet (...)

$\mathfrak{sl}_2$-triples with your bracket relations appear in Killing (1888, p. 281), denoted $(X_{r-1}, X_r, X_{r-2})$; Cartan (1894, p. 116), denoted $\mathrm{(Y, X,X')}$; Weyl (1925, p. 276), denoted $(h_\alpha,e_\alpha,e_{-\alpha})$, with the $h_\alpha$ again called “Diagonal- oder Hauptmatrizen”; Dynkin (1952, §8.1), denoted $(f,e_+,e_-)$; Chevalley (1955, p. 28; 1955, p. 96), denoted $(D,N,N')$, $(H_r,X_r,X_{-r})$, and finally the desired $\mathrm{(H,X,Y)}$.

(Standardization was slow: Lie (1876, p. 53; 1890, p. 353) used $(X_1,X_2,X_3)=(\mathrm{-Y,\smash{\frac12}H,X})$, and similarly rescaled bases and bracket relations still appear in Pauli (1927, p. 614), Born-Jordan (1930, p. 135), Casimir-van der Waerden (1931, p. 46; 1935, p. 4), Bauer (1933, p. 126), Harish-Chandra (1950, p. 301; 1952, p. 337), Jacobson (1951, p. 107; 1958, p. 825), Séminaire “Sophus Lie” (1955, p. 10-01), Kostant (1959, p. 977), etc. Settling on the “Chevalley” basis $\mathrm{(H,X,Y)}$ over Lie’s seems ultimately motivated by the smaller $\mathbf Z$-form $\mathfrak g_\mathrm{sc}\subset\mathfrak g_\mathrm{ad}$ it spans, cf. Borel (1970, §2.7).)


Both terminology and notation in Lie theory have varied over time, but as far as I know the letter H comes up naturally (in various fonts) as the next letter after G in the early development of Lie groups. Lie algebras came later, being viewed initially as "infinitesimal groups" and having labels like those of the groups but in lower case Fraktur. Many traditional names are not quite right: "Cartan subalgebras" and such arose in work of Killing, while the "Killing form" seems due to Cartan (as explained by Armand Borel, who accidentally introduced the terminology). It would take a lot of work to track the history of all these things. In his book, Thomas Hawkins is more concerned about the history of ideas. Anyway, both (h,e,f) and (h,x,y) are widely used for the standard basis of a 3-dimensional simple Lie algebra, but I don't know where either of these first occurred. Certainly h belongs to a Cartan subalgebra.

My own unanswered question along these lines is the source of the now standard lower case Greek letter rho for the half-sum of positive roots (or sum of fundamental weights). There was some competition from delta, but other kinds of symbols were also used by Weyl, Harish-Chandra, ....

ADDED: Looking back as far as Weyl's four-part paper in Mathematische Zeitschrift (1925-1926), but not as far back as E. Cartan's Paris thesis, I can see clearly in part III the prominence of the infinitesimal "subgroup" $\mathfrak{h}$ in the structure theory of infinitesimal groups which he laid out there following Lie, Engel, Cartan. (Part IV treats his character formula using integration over what we would call the compact real form of the semisimple Lie group in the background. But part III covers essentially the Lie algebra structure.) The development starts with a solvable subgroup $\mathfrak{h}$ and its "roots" in a Fitting decomposition of a general Lie algebra, followed by Cartan's criterion for semisimplicity and then the more familiar root space decomposition. Roots start out more generally as "roots" of the characteristic polynomial of a "regular" element. Jacobson follows a similar line in his 1962 book, reflecting his early visit at IAS and the lecture notes he wrote there inspired by Weyl.

In Weyl you also see an equation of the type $[h e_\alpha] = \alpha \cdot e_\alpha$, though his notation is quite variable in different places and sometimes favors lower case, sometimes upper case for similar objects. Early in the papers you see an infinitesimal group $\mathfrak{a}$ with subgroup $\mathfrak{b}$. All of which confirms my original view that H is just the letter of the alphabet following G, as often encountered in modern group theory. (No mystery.)