What is the difference between $\ell$-adic cohomology and cohomology with coefficient in $Z_\ell$?

$\def\ZZ{\mathbb{Z}}$I now understand the issue that user10676 is concerned about: For $X$ a reasonable topological space, why is $H^k(X, \ZZ_{\ell})$ isomorphic to $\lim_{\infty \leftarrow n} H^k(X, \ZZ/\ell^n)$? I thought that this would be most easily done from the universal coefficient theorem but, now that I try to write out a proof, I find it easiest to do directly.

Every complex algebraic variety is homeomorphic to a finite simplicial complex. (If I recall correctly, you can find a proof in "Algorithms in Real Algebraic Geometry".) So I'll use simplicial rather that singular cohomology. We work with a fixed triangulation for the rest of this proof.

Let $C^k$ be the group of $k$-co-chains with $\ZZ$ coefficients. Let $Z^k$ be the co-cycles and $B^k$ the co-boundaries. So $H^k(X, \ZZ) = Z^k/B^k$. Let the finitely generated abelian group $H^k(X, \ZZ)$ be $\bigoplus_{1 \leq i \leq r} \ZZ/d_i \oplus \ZZ^{\oplus s}$ where $d_1$, $d_2$, ..., $d_r$ is a sequence of positive integers where $d_1$ divides $d_2$ divides ... divides $d_r$. The following lemma is useful in many computations about cohomology:

Lemma Let $C^{k-1} \to C^{k} \to C^{k+1}$ be a complex of free finitely generated abelian groups. Then we can write this complex as a direct sum of complexes of the following forms: $$\ZZ \longrightarrow 0 \longrightarrow 0 \quad (1)$$ $$0 \longrightarrow \ZZ \longrightarrow 0 \quad (2)$$ $$0 \longrightarrow 0 \longrightarrow \ZZ \quad (3)$$ $$\ZZ \stackrel{d}{\longrightarrow} \ZZ \longrightarrow 0 \quad (4)$$ $$0 \longrightarrow \ZZ \stackrel{d}{\longrightarrow} \ZZ \quad (5)$$ where $d$ is a positive integer. (The case $d=1$ is permitted.)

Proof First, choose arbitrary bases for $C^{k-1}$ and $Z^k$. The map $C^{k-1} \to Z^k$ is given by an integer matrix. Now change bases so that this matrix is in Smith normal form. In these bases, $C^{k-1} \to Z^k \to 0$ is a sum of complexes of types (1), (2) and (4).

Now, $B^{k+1}$ is a subgroup of the free $\ZZ$-module $C^{k+1}$, so it is free. Thus, we can choose a splitting of $C^k \to C^k/Z^k \cong B^{k+1}$. Let $A$ be the image of this splitting, so $C^k \cong Z^k \oplus A$. Put the map $A \to C^{k+1}$ into Smith normal form as before. Then $0 \to A \to C^{k+1}$ is a direct sum of complexes of the form (3) and (5). (Since $A \to C^{k+1}$ is injective, there are no zeroes on the diagonal of the Smith normal form, and (2) does not occur.) Our original complex is the direct sum of $C^{k-1} \to Z^k \to 0$ and $0 \to A \to C^{k+1}$, so it breaks up as a direct sum of complexes of the five types listed above. $\square$

If we now change coefficients to a new abelian group $G$, we get the same complexes with $\ZZ$ replaced by $G$. All our computations distribute over direct sum, so we just need to work out what each of these five complexes do for the groups $G = \ZZ_{\ell}$ and $G = \ZZ/\ell^n$.

For complexes (1) and (3), we get $0$ in both cases.

For complex (2), we get $\lim_{\infty \leftarrow n} \ZZ/\ell^n$ on one side and $\ZZ_{\ell}$ on the other (with the obvious surjections in the inverse limit).

For complex (4), let $d = \ell^i m$ with $GCD(\ell, m) =1$. For $n \geq i$, we get $\lim_{\infty \leftarrow n} \ZZ/\ell^i$ on one side and $\ZZ/\ell^i$ on the other, where the maps in the inverse limit are the identity for $n > i$.

Complex (5) is the hard one. On the $\ZZ_{\ell}$ side, we get $0$. Write $d = \ell^i m$ as above. For $n \geq i$, we again get $\lim_{\infty \leftarrow n} \ZZ/\ell^i$. However, this time the map in the inverse limit is multiplication by $\ell$ (for $n>i$). This inverse limit is $0$, so we win.

Remark It is probably worth stating the general version of this result: If $R$ is a PID and $C^{\bullet}$ is a complex of free $R$-modules, then $C^{\bullet}$ can be written as a direct sum of complexes of the forms $\cdots \to 0 \to 0 \to R \to 0 \to 0 \to \cdots$ and $\cdots \to 0 \to 0 \to R \stackrel{d}{\longrightarrow} R \to 0 \to \cdots$ for various $d \in r$, and where the nonzero terms can occur in any position. The proof is basically the same: submodules of a free module over a PID are free; surjections to free modules split; Smith normal form is valid over a PID. A lot of modern ring theory can be thought of as classifying the types of complexes which exist over different rings.

I think the answer to your question is "the universal coefficient theorem", but I might not understand your question correctly.

To remove ambiguity, every cohomology group below will have a subscript, either $et$ to mean etale cohomology, $sing$ for singular or $DR$ for deRham.

The universal coefficient theorem says that, for a topological space $Y$ and an abelian group $A$, there is a non-canonically split short exact sequence: $$0 \to Ext(H^{sing}_{k-1}(Y, \mathbb{Z}), A) \to H_{sing}^k(Y,A) \to Hom(H^{sing}_k(Y,\mathbb{Z}), A) \to 0.$$

From this sequence, one can compute $$\lim_{\infty \leftarrow n} H_{sing}^k(X(\mathbb{C}),\mathbb{Z}/(\ell^n)) \cong H_{sing}^k(X(\mathbb{C}),\mathbb{Z}_{\ell}) \cong Hom(H^{sing}_k(X(\mathbb{C}),\mathbb{Z}_{\ell})). \quad (\ast)$$ Even if $H_{k-1}^{sing}(X(\mathbb{C}), \mathbb{Z})$ contains some $\ell$-torsion, so that $Ext(H^{sing}_{k-1}(X(\mathbb{C}), \mathbb{Z}), \mathbb{Z}/\ell^n)$ is nonzero for all $n$, the inverse limit of the $Ext$ groups is still zero.

Now, you say you already understand that $H^k_{et}(X_{\overline{\mathbb{Q}}}, \mathbb{Z}/\ell^n) \cong H^k_{sing}(X(\mathbb{C}), \mathbb{Z}/\ell^n)$, so all the terms in $(\ast)$ are also isomorphic to $\lim_{\infty \leftarrow n} H^k_{et}(X_{\overline{\mathbb{Q}}}, \mathbb{Z}/\ell^n)$.

Also from universal coefficients, $$H^k_{sing}(X(\mathbb{C}), \mathbb{C}) \cong Hom(H^{sing}_k(X(\mathbb{C}),\mathbb{C})) $$

So, chaining together the isomorphisms we know, you want to show that $$Hom(H^{sing}_k(X(\mathbb{C}),\mathbb{Z}_{\ell})) \otimes_{\mathbb{Z}_{\ell}} \mathbb{C} \cong Hom(H^{sing}_k(X(\mathbb{C}),\mathbb{C}))$$

This has nothing to do with topology; it's just that $$Hom(G, R) \otimes_R \mathbb{C} \cong Hom(G, \mathbb{C})$$ for any subring $R$ of $\mathbb{C}$ and any finitely generated abelian group $G$.

Finally, you ask about bringing in de Rham cohomology. If you just mean de Rham cohomology of smooth differential forms, this is a question of differential geometry; you can read that $H^k_{sing}(Y, \mathbb{C}) \cong H^k_{DR}(Y, \mathbb{C})$ in Bott and Tu's book, for example. If you meant algebraic de Rham cohomology, you want Grothendieck, On the de Rham cohomology of algebraic varieties.