Affine Transformations isomorphic to Heisenberg group

One method, that eliminates the appearance of the sticky quadratic term, is to show instead that the Lie algebra $\mathfrak g$ of $G$ and the Lie algebra $\mathfrak h$ of the Heisenberg group $H$ are isomorphic. This fact, together with the fact that both $G$ and $H$ are connected and simply connected (the manifolds underlying both are diffeomorphic to $\Bbb R^3$), implies that $G \cong H$ as Lie groups.

Now, $\mathfrak g$ consists exactly of the infinitesimal affine transformations of the form $${\bf u} \mapsto \pmatrix{\cdot&C&\cdot\\\cdot&\cdot&-C\\ \cdot&\cdot&\cdot} {\bf u} + \pmatrix{A\\B\\C}, \quad \textrm{which we respectively denote $(A, B, C)$,}$$ and $\mathfrak h$ consists of the matrices $$\pmatrix{\cdot&X&Z\\\cdot&\cdot&Y\\\cdot&\cdot&\cdot}, \quad \textrm{which we respectively denote $(X, Y, Z)$.}$$

Computing Lie brackets gives $$\begin{align}[(A, B, C), (A', B', C')] &= (C B' - B C', 0, 0),\\ \quad [(X, Y, Z), (X', Y', Z')] &= (0, 0, X Y' - Y X'). \end{align}$$

Comparing components immediately shows that $$\phi((A, B, C)) := (C, B, A)$$ is a Lie algebra isomorphism. Alternatively it's a straightforward to check that both Lie algebras admit bases $(E_1, E_2, E_3)$ such that $[E_1, E_2] = E_3$, $[E_2, E_3] = [E_3, E_1] = 0$. Either way we're done, and we've established the existence of an isomorphism without messy computation.

Remark If you prefer to produce an explicit group map, one can exponentiate $\phi$ to yield a Lie group isomorphism $\tilde{\phi} : G \to H$ that maps $\exp W$ to $\exp \phi(W)$ for all $W \in \mathfrak g$. Generically computing matrix exponentials is complicated, but because $\mathfrak g$ (and therefore $\mathfrak h$) is nilpotent, one only needs finitely many terms in the usual expansion $\exp W \sim I + W + \tfrac{1}{2} W^2 + \cdots$---the rest are zero. Carrying this out gives that $$\tilde\phi(A + \tfrac{1}{2} B C - \tfrac{1}{6} C^3, B - \tfrac{1}{2} C^2, C) = (C, B, A + \tfrac{1}{2} BC).$$ (Notice that triples here represent elements of the groups, as in the statement of the original problem, not of their Lie algebras.) Solving to write $\tilde\phi(a, b, c)$ in terms of a generic element $(a, b, c) \in G$ recovers precisely the isomorphism in Steve D's answer: $$\color{#bf0000}{\boxed{\tilde\phi(a, b, c) = (c, b + \tfrac{1}{2} c^2, a + \tfrac{1}{6} c^3)}} .$$

Note that we computed the exponential $\exp : \mathfrak g \to G$, but I only mentioned the formula for the exponential of a matrix (Lie algebra), and $\mathfrak g$ is not given as a set of matrices. This is no problem, as we can apply a standard trick: It's straightforward to check that we can identify $\mathfrak g$ with a matrix algebra via the map $$(A, B, C) \leftrightarrow \left({\bf u} \mapsto \pmatrix{\cdot&C&\cdot\\\cdot&\cdot&-C\\ \cdot&\cdot&\cdot} {\bf u} + \pmatrix{A\\B\\C}\right) \mapsto \pmatrix{\cdot&\cdot&\cdot&\cdot\\A&\cdot&C&\cdot\\B&\cdot&\cdot&-C\\C&\cdot&\cdot&\cdot} .$$

Here is an isomorphism: $$ \phi(x,y,z) = \left(z-\frac{x^3}{6}, y-\frac{x^2}{2}, x\right) $$

Note this has an inverse, namely $$ \phi(a,b,c) = \left(c, b+\frac{c^2}{2}, a+\frac{c^3}{6}\right)$$

So to show it is an isomorphism, we must show it preserves products. First we have $$ (x,y,z)\cdot(\hat{x},\hat{y},\hat{z}) = (x+\hat{x}, y+\hat{y}, z+\hat{z}+x\hat{y}) $$ and so we have $$ \phi((x,y,z)\cdot(\hat{x},\hat{y},\hat{z})) = (z+\hat{z}+x\hat{y}-\frac{(x+\hat{x})^3}{6}, y+\hat{y}-\frac{(x+\hat{x})^2}{2}, x+\hat{x})$$

Meanwhile $$ (a,b,c)\ast (a',b',c')=(a+a'+b'c-c'\frac{c^2}{2},b+b'-cc',c+c')$$ and thus \begin{align*} \phi(x,y,z)\ast\phi(\hat{x},\hat{y},\hat{z}) &=(z+\hat{z}-\frac{x^3}{6}-\frac{\hat{x}^3}{6}+x(\hat{y}-\frac{\hat{x}^2}{2})-\frac{\hat{x}x^2}{2},y+\hat{y}-\frac{x^2}{2}-\frac{\hat{x}^2}{2}-x\hat{x}, x+\hat{x})\\ &= (z+\hat{z}+x\hat{y}-\frac{(x+\hat{x})^3}{6}, y+\hat{y}-\frac{(x+\hat{x})^2}{2}, x+\hat{x})\\ &= \phi((x,y,z)\cdot(\hat{x},\hat{y},\hat{z})) \end{align*}

Where did this come from?

Well, if these groups were indeed isomorphic, they should have a 1-dimensional commutator subgroup that was central. Computing the commutators then showed the subgroup generated by $(0,0,1)$ should go to the subgroup generated by $(1,0,0)$. From the comments above, it was clear what should happen to the $x$ and $y$ components; where to send $z$ was just a matter of looking at the difference between the $a$-component of $\phi((x,y,z)\cdot(\hat{x},\hat{y},\hat{z}))$ and of $\phi(x,y,z)\ast\phi(\hat{x},\hat{y},\hat{z})$.

Note that once you know both are 3-dimensional simply-connected Lie groups with a central commutator, general Lie theory shows they must be isomorphic. I don't, however, know of a high-level argument that also shows this explicit isomorphism works. Namely, I don't know how to show $\phi$ works without doing the (messy) computations.

This is not a solution... just some initial thoughts

The commutator of two elements of $G$ is $$[(a,b,c),(a^\prime,b^\prime, c^\prime)] = \left(b^\prime c-bc^\prime-c^\prime \dfrac{c^2}{2} + c\dfrac{(c^\prime)^2}{2},0,0\right)=\left(\left(b^\prime + \dfrac{(c^\prime)^2}{2}\right)c -\left(b + \dfrac{c^2}{2}\right)c^\prime,0,0 \right)$$

While the commutator of two elements of $H$ (the Heisenberg group) is $$[(x,y,z),(x^\prime,y^\prime, z^\prime)] = (0,0,x y^\prime - y x^\prime)$$

This encourages us to map an element $(\cdot,b,c) \in G$ to $(c,b+\dfrac{c^2}{2}, \overline{\cdot}) \in H$.

But I don't know how to move forward...