Integral cohomology (stable) operations
$HZ^nHZ$ is trivial for $n<0$. $HZ^0HZ$ is infinite cyclic generated by the identity operation. For $n>0$ the group is finite. So you know everything if you know what's going on locally at each prime. For $n>0$ the $p$-primary part is not just finite but killed by $p$, which means that you can extract it from the Steenrod algebra $H(Z/p)^{*}H(Z/p)$ and Bocksteins.
EDIT Here's the easier part: The integral homology groups of the space $K(Z,n)$ vanish below dimension $n$, and by induction on $n$ they are all finitely generated. Also $H_{n+k}K(Z,n)$ is independent of $n$ for roughly $n>k$, so that in this stable range $H_{n+k}K(Z,n)$ is $HZ_kHZ$, which is therefore finitely generated. This plus the computation of rational (co)homology gives that $HZ_kHZ$ is finite for $k>0$. Here's the funny part: Of course one expects there to be some elements of order $p^m$ for $m>1$ in the (co)homology of $K(Z,n)$, and in fact there are; the surprise is that stably this is not the case.
A possibly interesting analogue of the formula $H\mathbb{F}_{2*} H\mathbb{F}_2 = \otimes_{i\ge1} \mathbb{F}_2[\xi_i]$ is $H\mathbb{Z}_{(2)*} H\mathbb{Z}_{(2)} = \bigotimes^\mathbb{L}_{i\ge1} \mathbb{Z}_{(2)*}[\xi_i^2]/(2\xi_i^2)$, where $\otimes^{\mathbb{L}}$ means the derived tensor product. In other words, resolve $\mathbb{Z}_{(2)*}[\xi_i^2]/(2\xi_i^2)$ by (flat or) free $\mathbb{Z}_{(2)}$-modules, tensor the resolutions together, and pass to homology. If I recall correctly, the "first" interesting class $\xi_2^3 + \xi_1^2 \xi_3$ (in degree 9) arises as a torsion product of $\xi_1^2$ and $\xi_2^2$. I needed this for a Shukla homology calculation once. Presumably there is also an odd story.
That is right.
The slightly easier calculation, I think, is $H\mathbb{Z}_\ast H\mathbb{Z}$, and this is easier to approach one prime at a time, i.e., by calculating $H\mathbb{Z}_\ast H\mathbb{Z}_{(p)}$, which is something you can do using the classical Adams spectral sequence. I don't have it handy to check, but I suspect that this calculation is carried out in Part III of Adams's "blue book" (Stable homotopy and generalised homology). The main thing to take away is that $H\mathbb{Z}_nH\mathbb{Z}_{(p)}$ is $p$-torsion (i.e., in the kernel of multiplication by $p$) for all $n>0$.
Steenrod's original definition was by a chain level construction, called the cup-i product. This is discussed in some other questions, such as here.
Note that I'm discussing the (stable) homology of the Eilenberg MacLane spectrum. The homology of the integral Eilenberg MacLane spaces $H_\ast K(\mathbb{Z},n)$ are quite a bit more complicated.