Quasi-isometries vs Cayley Graphs

There was a conjecture by Woess that every infinite vertex-transitive graph is quasi-isometric to a Cayley graph. A slightly more sophisticated counter-example for your first question is the counter-example to this conjecture that was proposed by R.Diestel and I. Leader in "A conjecture concerning a limit of non-Cayley graphs". It was later proved by A. Eskin, D. Fisher, and K. Whyte in "Quasi-isometries and rigidity of solvable groups".

I guess that a star (a tree with $n$ infinite branches issued from a single vertex) should answer at least your first question. It should have $n$ ends, whatever meaningful definition you use, an we know that a group has $1$, $2$ or an infinity of ends.

Since quasi-isometry is an equivalence relation, you do not need to invoke a space $Z$ in your first question and the answer of your second question is obviously positive.