Quasi-isometric groups without common virtual geometric model
The question was already answered by ThiKu and YCor. I am adding this just to make a conceptual remark.
The OP asks about groups which are QI, but not in an "elementary fashion". As YCor pointed out in his answer, if $G$ acts properly and cocompactly on a geodesic metric space $X$ then it could be seen as a cocompact lattice in the (locally compact group) of isometries of $X$. It thus makes sense to extend our notion of "elementary-ness" as follows:
Definition: Two groups are elementary equivalent if they are in the relation (E) which is the transitive closure of the relation "being cocompact lattices in the same locally compact group".
Finding groups which are QI but not E will answer the OP question. Still, the relation E is defined in a non explicit way. It makes sense to extend it. That is, to find a courser equivalence relation that contains it, and is easier to handle. A good example is the following:
Definition: Two groups are Measure Equivalent (ME) if they act on an infinite measure space, preserving the measure, the actions commute and each has a finite fundamental domain.
The exact relationship between QI and ME is tricky. Both contain the relation E defined above, but none include the other. However, it is not hard to see that property (T) is an ME invariant (this is Corollary 1.4 in Furman's paper), while it is possible for a (T) group to be QI to a non-(T) group. For an example, see (1) in YCor's answer. This example thus addresses all OP's challenges.
The relation ME admits various refinements called ME$^p$, where $1\leq p\leq \infty$ refers to some inegrability property of an associated function (the norm of the rearrangement cocycle). Again, these are all natural equivalence relation containing E. In Theorem 1.1 of Das-Tessera's paper it is shown that the groups given by ThiKu in his answer are not ME$^p$. Again, it follows that they are not E.
A potential pair of candidates might be $$\pi_1\Sigma_g\times{\mathbf Z}\mbox{ and }\pi_1(T_1\Sigma_g)$$ for a closed surface $\Sigma_g$ and its unit tangent bundle $T_1\Sigma_g$.
According to https://www.intlpress.com/site/pub/files/_fulltext/journals/cag/2001/0009/0002/CAG-2001-0009-0002-a001.pdf
it was observed by Epstein, Gersten, and Mess that any extension of a Fuchsian group by ${\mathbf Z}$ is quasi-isometric to ${\mathbf H}^2 \times {\mathbf R}$, and such extensions are typically not finite extensions of lattices in $Isom({\mathbf H}^2 \times {\mathbf R})$.
Of course that does not yet prove that these groups do not act simultaneously on any other space...
Here are two examples of different nature:
(1) let $G$ be a simple Lie group with infinite fundamental group and Property T, e.g., $\mathrm{Sp}_{2n}(\mathbf{R})$ for $n\ge 2$. Let $\Gamma$ be a cocompact lattice in $G$. Let $\tilde{\Gamma}$ its inverse image in the universal covering $\tilde{G}$ of $G$. Then $\tilde{\Gamma}$ and $\Gamma\times\mathbf{Z}$ are quasi-isometric but don't act geometrically (=properly cocompactly) on the same proper metric space.
(1') The conclusion of (1) holds without assuming that $G$ has Property T, covering the case $G=\mathrm{SL}_2(\mathbf{R})$, but the proof is a little longer.
(2) Let $H$ be a simple Lie group, not compact and not locally isomorphic to $\mathrm{SL}_2(\mathbf{R})$. Let $\Gamma_i$, $i=1,2$ be cocompact lattices in $G$ that are not abstractly commensurable (these exist precisely because we have excluded $\mathrm{SL}_2(\mathbf{R})$. Then $\Gamma_1\ast\mathbf{Z}$ and $\Gamma_2\ast\mathbf{Z}$ are quasi-isometric but don't act geometrically (=properly cocompactly) on the same proper metric space.
The examples in (1) and (2) are quite different because in (1) they are commable but not in (2). Morally "commable" is the transitive closure of "acting geometrically on the same space"; see precise definitions below.
Proofs:
(1) If discrete groups $\Gamma$, $\Lambda$ act geometrically on the same proper metric space $X$, say with finite kernels $K,L$, then $\Gamma/K$ and $\Lambda/L$ are cocompact lattices in $\mathrm{Isom}(X)$. Since (Kazhdan's) Property T is invariant (= stable in both directions) under taking quotients by finite normal subgroups and passing from a lattice to a locally compact group, we deduce that $\Gamma$ has Property T iff $\Lambda$ has Property T.
Now in this case, $\tilde{\Gamma}$ is known to have Property T (see Bekka-Harpe-Valette's book), while $\Gamma\times\mathbf{Z}$ doesn't.
On the other hand (in the context of (1) or (1')), they are quasi-isometric because they are commable. Recall a copci homomorphism $G\to H$ between compactly generated locally compact (CGLC) groups means a continuous proper homomorphism with cocompact image. Then every copci homomorphism is a quasi-isometry.
Two CGLC groups $G,H$ are commable if there exists a finite sequence (called commability) of copci homomorphisms (in both directions): $$G\to G_1\leftarrow G_2\to G_3\leftarrow\dots\to H$$ This implies that $G$ and $H$ are quasi-isometric.
In the setting of (1), let $T$ be a simply connected solvable cocompact subgroup in $G$; the its inverse image in $\tilde{G}$ is a direct product $T\times\mathbf{Z}$. Then we get the commability $$\tilde{\Gamma}\to\tilde{G}\leftarrow T\times\mathbf{Z}\to G\times\mathbf{Z}\leftarrow\Gamma\times\mathbf{Z}.$$ Hence $\Gamma\times\mathbf{Z}$ and $\tilde{\Gamma}$ are commable, hence quasi-isometric (this is classical: this shows failure of QI-invariance of Property T).
(2) Since $\Gamma_1,\Gamma_2$ are quasi-isometric and non-amenable, they are bilipschitz by a result of Whyte. Hence $\Gamma_1$ and $\Gamma_2$ are bilipschitz, hence quasi-isometric.
That $\Gamma_1$ and $\Gamma_2$ are not commable is an observation of Carette and Tessera, see Section 5B here (arXiv link).
(1') For these additional examples (which include those suggested by ThiKu) it remains to prove that $\tilde{\Gamma}$ and $\Gamma\times\mathbf{Z}$ have no geometric action on a common proper metric space, without assuming that $G$ has Property T. It's not immediate; I'll write the proof later.