Integral cohomology of $G/N(T)$

There are people on MO more knowledgeable about this topic than I, but since it has been a few days, I will say a few words.

In this case there is a "third isomorphism" of spaces:

Proposition $G/N(T)\cong \mathcal{F}/W$

Proof The flag variety $\mathcal{F}\cong G/T\cong G_{\mathbb{C}}/B$ and $W=N(T)/T$ is the Weyl group. Define $\varphi:G/T\to G/N(T)$ by $gT\mapsto gN(T)$. This is clearly a well-defined continuous surjection. The group $W$ acts on the right of $G/T$ (and $G/N(T)$) by $gT\cdot nT=gTnT=gnT$ and $\varphi$ is $W$-equivariant. So $\varphi$ descends to a map $\varphi:\mathcal{F}/W\to G/N(T)$ which is easily seen to be injective, and hence a homeomorphism (since $\mathcal{F}/W$ is compact and $G/N(T)$ is Hausdorff). $\Box$

As shown here there is an inclusion $H^k(\mathcal{F}/W;\mathbb{Z})\hookrightarrow H^k(\mathcal{F};\mathbb{Z})^W$ modulo torsion. Since you know $H^*(\mathcal{F}/W;\mathbb{Q})\cong H^*(\mathcal{F};\mathbb{Q})^W$ (by a result in Bredon's book on Transformation Groups; page 120) and the latter is $\mathbb{Q}$ (by Borel's 1953 Annals paper, page 193) you know the free parts since $\mathrm{rk} \ H^k(\mathcal{F}/W;\mathbb{Z})=\mathrm{rk}\ H^k(\mathcal{F};\mathbb{Z})^W.$

That leaves torsion. I spoke with an expert in Shubert varieties who told me that (a) they did not know the answer or a reference to this, and (b) they expected complicated torsion.

So I decided to do an example. First note that $W$ acts freely on $\mathcal{F}$ since if $gwT=gT$, then $w\in T$. Thus, $\mathcal{F}/W$ is a smooth manifold.

Example: $\mathrm{SU}(2)/T\cong S^2$ with $W=\mathbb{Z}/2\mathbb{Z}$ acting freely, so the quotient must be $\mathbb{RP}^2$. That gives torsion in $H^2(\mathcal{F}/W;\mathbb{Z})$ at the first opportunity.

More generally, when $G$ is simply connected $G/T$ is simply connected (by path lifting), and so $H_1(\mathcal{F}/W;\mathbb{Z})\cong W/[W,W]$. This can be used with Poincaré duality (or a non-orientable version) to garner some further information (in particular, lots of torsion as $W$ gets larger).

Here are some comments about how this calculation can be organised.

First, let $T(n)$ be the standard maximal torus in $U(n)$, and let $N(n)$ be its normaliser, so $W(n)=N(n)/T(n)$ is the symmetric group. Put $F(n)=U(n)/T(n)$ and $E(n)=F(n)/W(n)=U(n)/N(n)$, so we are interested in the (co)homology of $E(n)$. We know that $H^*(E(n);\mathbb{Q})=\mathbb{Q}$. I think it will be best to study $H_*(E(n);\mathbb{Z}/p)$ (where $p$ is prime) and recover information about $H^*(E(n);\mathbb{Z})$ at the end, using the universal coefficient theorems. From now on I'll write $H_*(X)$ for $H_*(X;\mathbb{Z}/p)$. It will also be best to study all the groups $H_*(E(n))$ together as a bigraded object $H_*(E(*))$.

Note that there are sum and tensor product operations $\oplus\colon U(n)\times U(m)\to U(n+m)$ and $\otimes\colon U(n)\times U(m)\to U(nm)$. These are group homomorphisms, so they induce maps $BU(n)\times BU(m)\to BU(n+m)$ and $BU(n)\times BU(m)\to BU(nm)$. They preserve monomial matrices, so they give maps $BN(n)\times BN(m)\to BN(n+m)$ and $BN(n)\times BN(m)\to BN(nm)$. We can regard $E(n)$ as the fibre of the map $BN(n)\to BU(n)$, so we also get maps $E(n)\times E(m)\to E(n+m)$ and $E(n)\times E(m)\to E(nm)$. This will give two different products on the object $H_*(E(*))$, which will make it into a Hopf ring. Also, using block matrices we get a standard inclusion $\Sigma_n\wr U(m)^n\to U(nm)$. This gives an action of an $E_\infty$ operad on $BU(*)$ and $BN(*)$ and $E(*)$, and so gives Dyer-Lashof operations on the corresponding homology groups.

Now $BN(*)$ is essentially the free $E_\infty$ space generated by $BN(1)=\mathbb{C}P^\infty$, which implies that $H_*(BN(*))$ is the free commutative algebra with Dyer-Lashof operations generated by $H_*(\mathbb{C}P^\infty)$. On the other hand, $H_*(BU(*))$ is just the free commutative algebra generated by $H_*(\mathbb{C}P^\infty)$. This implies that the map $H_*(BN(*))\to H_*(BU(*))$ is surjective (which can also be seen using the Becker-Gottlieb transfer map). Note also that the Dyer-Lashof operations in $H_*(BU(*))$ were worked out by Priddy. It seems likely that $H_*(E(*))$ is just the kernel (in the Hopf algebra sense) of the map $H_*(BN(*))\to H_*(BU(*))$. Some care is needed here because the relevant Hopf algebras do not have antipodes; they correspond to commutative monoids (rather than groups) in the category of coalgebras. Anyway, the message is that we expect that $H_*(E(*))$ will have structure closely related to that of the Dyer-Lashof algebra, and of similar size and complexity.

So far we have not used the tensor product maps $U(n)\times U(m)\to U(nm)$, or equivalently, the second product in the Hopf ring structure. It is a theorem of Paul Turner that when $p=2$, the object $H_*(BW(*))$ has a simple presentation as a Hopf ring, which is in some ways more tractable than the presentation using Dyer-Lashof operations. I expect that there is a similar story for $H_*(BN(*))$, although I do not know whether that has ever been written down. It is also straightforward to write down the Hopf ring structure on $H_*(BU(*))$, and this may well give a nice description of $H_*(E(*))$. I am not sure whether a similar approach would work for $p>2$.