Is there a natural group of subgroups of a group?

I think for such a construction to be called "natural", it would need to have at least the following the property:

The induced action of $\mathrm{Aut}(G)$ on the set of subgroups of $G$ induces a subgroup of the automorphism group of the group of subgroups.

(For example, subgroups of $G$ that are "the same" with respect to the structure of $G$ should become elements of the group of subgroups that are also "the same" with respect to its structure, i.e. subgroups of $G$ that are conjugate under $\mathrm{Aut}(G)$ should also be conjugate under the automorphism group of the group of subgroups.)

But this is not always possible. For example, let $G$ be the Klein group. It has $5$ subgroups, so its group of subgroups should be cyclic of order $5$, but the automorphism group of the Klein group is $S_3$ which does not embed into the automorphism group of $C_5$.


We can open up our conditions a little bit to get something "natural." Namely, we can take the vector space $\mathcal{S}(G)$ spanned by the subgroups of $G$, over some field $F$, with the product being the linear extension of $$A\cdot B = AB$$ Then if we have a homomorphism $f:G\to H$, we can extend this to a homomorphism of $F$-algebras $f_*:\mathcal{S}(G)\to\mathcal{S}(H)$ given by $$f_*(A) = f(A)$$ In particular, automorphisms $a:G\to G$ induce automorphisms $a_*:\mathcal{S}(G)\to \mathcal{S}(G)$.

Note this only works directly for abelian groups, or the rare nonabelian groups like the quaternion group where every subgroup is normal. If you want to do this for general groups, you have to extend the vector space to consist of all nonempty subsets of the group. Either that, or you can declare the subgroups to just be generators. Then the vector space is actually larger, with basis elements that are not subgroups but rather are products of subgroups. Then you'll end up with something noncommutative, with $HK\neq KH$ if $H$ and $K$ are not normal. Then normal subgroups will be in the center of the algebra.

Note that a subgroup in this case is never invertible. In fact, subgroups are idempotent elements, with $A^2=A$ for all subgroups $A$. Then $$A(A-\{e\}) = 0$$ for all $A$, so every subgroup is a zero divisor. We thus lose invertibility, but we do have an algebraic construction that preserves homomorphisms (in the category theory sense, it is functorial). You also maintain your equation given by the isomorphism theorem. Namely, if $\langle H-\{e\}\rangle$ is the ideal generated by a normal subgroup, then $\mathcal{S}(G)/\langle H-\{e\}\rangle\cong \mathcal{S}(G/H)$, and $HH'\mapsto (HH')/H$, which is the same as the image of $H'$. This is really isomorphic to $H'/(H\cap H')$ however, and this is the subgroup it maps to in the quotient. If $H$ and $H'$ intersect trivially, this is just $H'$.

In general, if we want to do constructions involving groups we may get to a point where we can't do what we want because groups are extremely rigid. However, we have a lot of freedom with algebras over a field, and this can is an example where it allows us to basically do what we want. A similar situation is that of quantum groups, where we deform a group to obtain a $q$-analog.