Making $\mathbb{Q}$-cohomology integral

This is not an answer but some comments on the topological aspects of the problem. Probably I'm just telling you stuff you already know.

Since $H_i(X, \mathbb{Z})$ is finitely generated, any element of $H^i(X, \mathbb{Q}) \cong \text{Hom}(H_i(X, \mathbb{Z}), \mathbb{Q})$ can be lifted to an element of $\text{Hom}(H_i(X, \mathbb{Z}), \frac{1}{m} \mathbb{Z})$ for some $m$, and hence to an element of $H^i(X, \frac{1}{m} \mathbb{Z})$. So in the question we can replace $\mathbb{Q}$ by $\frac{1}{m} \mathbb{Z}$. Equivalently we can say the following. The short exact sequence

$$0 \to \mathbb{Z} \xrightarrow{m} \mathbb{Z} \to \mathbb{Z}_m \to 0$$

(permit me the sin of using $\mathbb{Z}_m$ for $\mathbb{Z}/m\mathbb{Z}$) induces a long exact sequence

$$\cdots \to H^i(X, \mathbb{Z}) \xrightarrow{m} H^i(X, \mathbb{Z}) \to H^i(X, \mathbb{Z}_m) \to \cdots$$

showing that a given $\alpha \in H^i(X, \mathbb{Z})$ is an $m^{th}$ root, or equivalently lies in the image of the multiplication-by-$m$ map above, iff it lies in the kernel of the reduction-mod-$m$ map to $H^i(X, \mathbb{Z}_m)$. So our problem reduces, more or less to the following question:

Given nice $X$ and $\alpha \in H^i(X, \mathbb{Z}_m)$, when can we find a nice $f : Y \to X$ such that $f^{\ast}(\alpha) = 0$?

(I'm being vague here so that the statement can apply both to algebraic and topological versions of the problem; I have in mind $f$ a fiber bundle and $X, Y$ compact manifolds. Strictly speaking, for this question to be equivalent to the original question $\alpha$ needs to lie in the kernel of the Bockstein but I can't imagine that condition is particularly important.)

For any $i$ there is a universal topological answer to this question: namely, thinking of $\alpha$ as a map to an Eilenberg-MacLane space $X \to B^i \mathbb{Z}_m$, we can take its homotopy fiber, which lies in a fiber sequence

$$B^{i-1} \mathbb{Z}_m \to \widetilde{X} \to X \xrightarrow{\alpha} B^i \mathbb{Z}_m.$$

$\widetilde{X}$ is, by definition, the universal space equipped with a map to $X$ such that the pullback of $\alpha$ is trivial, and can be thought of as a fiber bundle over $X$ with fiber $B^{i-1} \mathbb{Z}_m$. Unfortunately $B^{i-1} \mathbb{Z}_m$ is not really a manifold in general. Presumably a higher stacky version of this construction exists and is what you were alluding to in the comments.

When $i = 1$ the homotopy fiber $\widetilde{X}$ can be taken to be the finite cover you've mentioned already; here $B^0 \mathbb{Z}_m = Z_m$ is discrete. When $i = 2$ we can think of $B \mathbb{Z}_m$ as the infinite lens space $S^{\infty} / \mathbb{Z}_m$, and we might hope that we can replace the fiber sequence above with a fibration

$$S^n / \mathbb{Z}_m \to \widetilde{X} \to X$$

with fiber a finite lens space. That might solve the problem in the topological setting but I guess not in the algebraic setting.

When $i = 2$ and $m = 2$ this is the question of whether every class in $H^2(X, \mathbb{Z}_2)$ can be represented by a bundle of real projective spaces / real matrix algebras. This is true and is the real analogue of Serre's theorem that every torsion class in $H^3(X, \mathbb{Z})$ is representable by a bundle of complex projective spaces / complex matrix algebras; it is apparently due to E. Strickland.

For higher $i$ it's not clear to me that even the topological version of the problem is solvable; I guess a suitable negative answer to that problem would imply a negative answer to the original problem?

Edit: When $i = 2$ we can do a bit more than the question asks for, at least in the topological setting: instead of causing any element of $H^2(X, \mathbb{Z})$ to have an $m^{th}$ root we can in fact cause any of $H^2(X, \mathbb{Z})$ to vanish. If $\alpha$ is such an element, thought of as a map $X \to B^2 \mathbb{Z}$, its homotopy fiber fits into a fiber sequence

$$B \mathbb{Z} \to \widetilde{X} \to X \to B^2 \mathbb{Z}$$

where here our saving grace is that $B \mathbb{Z} \cong S^1$ can be modeled by a compact manifold. Said another way, $\alpha$ is the first Chern class of a unique principal $\text{U}(1)$-bundle, and it vanishes when pulled back to the total space of this bundle.

For example, when we do this to a generator of $H^2(S^2, \mathbb{Z})$ we get the Hopf fibration $S^1 \to S^3 \to S^2$. More generally, when we do this to a generator of $H^2(\mathbb{CP}^n, \mathbb{Z})$ we get the familiar fibration $S^1 \to S^{2n+1} \to \mathbb{CP}^n$. The construction in this case can be interpreted as computing the $2$-connected cover, and if we could take $i$-connected covers of compact manifolds that were still compact manifolds for all $i$ then we would be done, but I don't think that's possible; e.g. the $3$-connected cover of $\text{Spin}(n), n \ge 3$ is $\text{String}(n)$, which can't be modeled by a finite-dimensional Lie group.

As Jason Starr remarks in the comments, this answer to a question of his implies the answer to both of my questions is "no." For the latter, one may take:

$X=\mathbb{P}^1, \mathcal{L}=\mathcal{O}(1)$

Then there is no smooth proper morphism $f: Y\to X$ such that $f^*\mathcal{L}$ has a square root, for example. (Jason's question is about projective morphisms, but I think it's fairly easy to deduce the proper case.)

For the first question, $X=\mathbb{P}^1, i=2$ is a counterexample.

I would be curious to see a simpler proof of this result. I would also be curious to know the answer with the smoothness condition relaxed to flatness.