A generously vertex transitive graph which is not Cayley?

Take the graph product $G = P \times \mathbb{Z}$ of the Petersen graph with the infinite path graph. This is clearly infinite, finite-degree, and generously vertex-transitive.

Then we have two distinguishable types of edge, namely $P$-edges (ones which belong to $5$-cycles) and $\mathbb{Z}$-edges (ones which do not). This means that the automorphisms of $G$ must preserve the obvious product structure, and can uniquely be written as compositions of automorphisms of $P$ with automorphisms of $\mathbb{Z}$.

If $G$ were a Cayley graph of some group $H$, then there are three generators corresponding to $P$-edges and two corresponding to $\mathbb{Z}$-edges. The actions of the $P$-generators and $\mathbb{Z}$-generators must, respectively, preserve the $\mathbb{Z}$-coordinate and $P$-coordinate of each vertex.

Hence the $P$-generators and $\mathbb{Z}$-generators each generate disjoint normal subgroups of $H$ (and $H$ is their internal direct product). The normal subgroup generated by the $P$-generators is sharply vertex-transitive on $P$, implying that the Petersen graph is a Cayley graph.

Contradiction.

So $G$ has all the properties you requested.

It seems that any distance-transitive graph satisfies your condition (of course, distance-transitivity is much stronger), and you can find many examples (both finite and infinite) in Peter Cameron's nice paper. (A census of infinite distance-transitive graphs, Discrete Math, 1998) Many of these are not Cayley graphs, it would appear.