Can a group be a universal Turing machine?
Update. Here is a more direct construction. (See edit history for previous version.)
There is such a universal computable group as you request. Let $F$ be the free group on infinitely many generators $\langle a_p\rangle_p$, indexed by the Turing machine programs $p$. Let $G$ be the quotient of this group by all the $k^{th}$ powers $a_p^k$, whenever the program $p$ halts (on trivial input) in exactly $k$ steps.
Let us represent the group $G$ by reduced words in the generators $a_p$ and their inverses, but in the case that we took the quotient by $a_p^k$, then in these words we use exponents on $a_p$ in the interval $(-k/2,k/2]$. (The reason for using this exponent format is that if we were to use only the positive powers of the finite-order generators, then we wouldn't be able to compute inverses in $G$, since we cannot compute whether $a_p$ has finite order or not.) First of all, we can computably recognize whether a word in the generators fits this description, simply by checking whether it is reduced and whether any of the exponents is too large. The point of this last issue is that we can tell if the exponent $a_p^r$ is too large by checking if program $p$ halts in $2r$ steps or not. Similarly, we can easily compute the inverse of a word from the word, and we can computably multiply words. Again, whenever we have a word with some new exponents on the generators, we need to check whether they reduce because of our quotient, and this is possible by running the relevant computation for sufficient number to steps to determine it.
Thus, we have a computable representation of the group $G$.
Finally, I claim that it is universal in the sense you requested. Given any Turing machine program $p$, let $x_p=a_p$ and let $y_p=a_q$ for some other program $q$ known not to halt. Thus, by design, the group generated by $x_p,y_p$ will be the free group on these generators if and only if $p$ does not halt.
An essentially equivalent presentation of the group can be made without reference to Turing machines or computations, but only to Diophantine equations, simply by using the Diophantine representation of the universal Turing machine. That is, since every c.e. set is the solution set of a Diophantine equation, there is a fixed Diophantine equation $d(y,\vec x)=0$, such that Turing machine program $p$ halts on trivial input if and only if $d(p,\vec x)=0$ has a solution in the integers, viewing the program as its Gödel code. So we may define the group $G$ as above, with infinitely many generators $a_n$, but taking the quotient by $a_n^k$, if $k$ is the size of the smallest integer solution of $d(n,\vec x)=0$. I'm not sure this makes the group "natural," (and my opinion is that this word has no robust, coherent mathematical meaning), but it does omit any mention of Turing machines, using instead a fixed Diophantine equation.
Lastly, let me observe that my group is not finitely generated, and it may be interesting to have a finitely generated example, or even a finitely presented example. I suspect that one can apply one of the embedding theorems to place this example into a finitely generated or even finitely presented group.
If I've followed this thread correctly, then there is still a question of whether a finitely presented example can be constructed. This follows from standard facts.
Specifically, let $G=G_0$ be the group constructed by Joel or Jason. By the standard Higman--Neumann--Neumann argument, $G_0$ embeds into a finitely generated group $G_1$ because $G_0$ is countable. It's a fact that $G_1$ is recursively presentable if $G_0$ was. By Higman's Embedding Theorem, $G_1$ can be embedded into a finitely presentable group $G_2$ because $G_1$ is recursively presentable.
The embedding of $G_0$ into $G_1$ is computable, and the embedding of $G_1$ into $G_2$ is obviously computable (since both groups are finitely generated).
Finally, one would like all this to be done so that $G_2$ has solvable word problem. That the Higman Embedding can be made to preserve solvability of the word problem follows from a result of Clapham.
(Strictly speaking, one also needs to check that $G_1$ can be taken to have solvable word problem, as I think Clapham starts with a finitely generated group. This must be well known, but Bridson and I wrote out the details in Section 7 of this paper.)
FURTHER
To explain why $G_0\to G_1$ is computable, I need to describe the construction. I'll give one version here, which I think makes the point fairly cleanly. If I recall correctly, it's very similar to the original argument given by Higman, Neuman and Neumann, though they do a little better and get an embedding into a two-generator group. I'll stick with three generators for simplicity.
I'll take $G_0$ to be given by a recursive presentation $\langle (a_m\mid m\in\mathbb{N})\mid (r_n\mid n\in\mathbb{N})\rangle$.
First, note that by replacing $G_0$ with $G_0*\langle x\rangle$ and by replacing each $a_m$ by $a_mx^{m+1}$, I may assume that the $a_m$ are all of infinite order and distinct.
Let
$G_{\frac{1}{2}}=G_0*\langle s\rangle$
and consider the subgroups
$H_M=\langle b_m=s^ma_{m-1}s^{-m}\mid m\geq M\rangle$
for $M\geq 1$. An easy argument with normal forms in free products proves:
Lemma: $H_M$ is free on the given generating set.
Therefore, the assignment $b_m\mapsto b_{m+1}$ defines an isomorphism $\phi:H_1\to H_2$. We can therefore define $G_1$ to be the HNN extension
$G_1=G_{\frac{1}{2}}*_\phi$
which has presentation $\langle G_{\frac{1}{2}},t\mid tht^{-1}=\phi(h)~\forall h\in H_1 \rangle$ (relative to $G_{\frac{1}{2}}$). We now appeal to Britton's Lemma to prove that $G_1$ contains a copy of $G_0$.
Britton's Lemma: The natural map $G_{\frac{1}{2}}\to G_{\frac{1}{2}}*_\phi$ is injective.
But $G_1$ is finitely generated; indeed it's generated by $a_0,s,t$.
The map $G_0\to G_1$ is given recursively by the equations
$a_m=s^{-m}b_ms^m$ and $b_m=tb_{m-1}t^{-1}$.
(Of course, $a_0\mapsto a_0$.) In particular, it's certainly computable.
To prove that the word problem is solvable in $G_1$, you need to argue that you can solve the membership problems for $H_0$ and $H_1$ in $G_\frac{1}{2}$.
I think the answer is yes, there is such a universal group. Let $G$ be the direct sum group $\bigoplus_{n \in \mathbb{N}} G_n$, where $G_n$ is $\mathbb{Z}$ if the $n$th Turing machine does not halt, and $G_n$ is cyclic otherwise. Just construct each $G_n$ by adding $0$, $1$, $-1$, ... until the $n$th Turing machine halts. Then make $G_n$ the smallest cyclic group it could possibly be. The key is that you can run everything in parallel and don't have to define addition (since I am thinking of an Abelian group) when you define the element (for example, $5$ may not be created yet when $2$ and $3$ are). You just have to eventually get around to it.
It is easy to see that you can enumerate the elements of this group, and enumerate the multiplication table. This enumeration can then be turned into the code for each element of the group.
Update: In my construction above, there is only one generator associated with each Turning machine, namely the group element with 1 in the $n$th coordinate, where $n$ is the index of the Turing machine. To have two generators (as was requested), the easiest modification would be to have $G_n$ look like the free group on two elements $a$ and $b$. If the associated Turing machine halts, then make $a$ and $b$ have finite order, leaving the rest free. Again, I think this is similar to Joel's new construction.
(I specialize more in computable analysis than algebra, but I imagine there are a number of standard groups representing the halting problem.)
I am not sure if there is a "natural" such group.
Update: I should also add that usually, when you have a property like that---i.e. you know when it holds, but don't always know when it doesn't hold---then you can make it code a universal Turing machine. Such sentences are called $\Sigma^0_1$ sentences.*