About the commutativity of the $1^{st}$ homotopy group of the space of knots

There is a fairly clean statement about which components of the "long knot space" have abelian fundamental group.

$$K_{3,1} = \{ f : \mathbb R \to \mathbb R^3 : f(t)=(t,0,0) \text{ for } |t|>1 \} \subset Emb(\mathbb R, \mathbb R^3)$$

This is what I call the "long knot space".

As Mike mentioned $Emb(S^1, S^3) = SO_4 \times_{SO_2} K_{3,1}$.

So you can work out the answer in this case, as well, although it will require a bit more leg-work.

Let me state the answer for $K_{3,1}$.

As you've noticed, for the unknot, torus knots, and cables of torus knots, the fundamental groups are abelian. In general, the fundamental group of any component of $K_{3,1}$ has a finite-index subgroup that's a product of pure braid groups. So step one: how do you avoid non-abelian braid groups? Answer: in the satellite decomposition, at no stage can you have a connect sum of three or more knots.

Okay, but if you limit yourself to such "restricted satellites", how do you stay with abelian groups?

As has been noted, the fundamental group of hyperbolic knot components is abelian (free rank 2) in $K_{3,1}$.

But you get some interesting things if you take, for example, the Whitehead double of an invertible knot, like a torus knot. That component has the fundamental group of the Klein bottle cross a circle. A very nice non-abelian group.

If you take a hyperbolic satellite operation with $n$ input knots, the fundamental group of that component has the form

$$\mathbb Z \times \left( \mathbb Z \ltimes_A \left( A_1 \times \cdots \times A_n \right) \right)$$

Where $A$ is a homomorphism from $\mathbb Z$ to a fairly simple automorphism group of the product $A_1 \times \cdots A_n$. The groups $A_1, \cdots , A_n$ are the fundamental groups of the components of the satellite (input) knots in the satellite operation.

So if the input groups $A_1, \cdots, A_n$ were all abelian to begin with, you would get a non-abelian semi-direct product if and only if the homomorphism $A$ acts non-trivially on the product.

This happens if and only if the "pattern link" has a symmetry group that is incompatible with the symmetries of the satellite knots. Rather than write it out as a formula, let me give some examples.

Example 1: Whitehead double of an invertible knot, the homomorphism A : Z --> Aut(A_1) factors through a faithful representation Z --> Z_2 --> Aut(A_1).

Example 2: Whitehead double of a non-invertible knot, the homomorphism A is trivial.

Example 3: The thing I call the "Borromean satellite operation"

In this picture the borromean rings are the "pattern". The two trefoils are the satellite knots.

In the pictured case, the homomorphism factors through a faithful rep $\mathbb Z_4 \to Aut(A_1 \times A_2)$. Both $A_1$ and $A_2$ are $\mathbb Z$. The automorphism in this case would rotate the rank-2 lattice $\mathbb Z^2$ by $\pi/2$. If the satellites were distinct invertible knots (say a trefoil and a figure-8) then the homomorphism would factor $\mathbb Z_2 \to Aut(A_1 \times A_2)$ and it would act by inversion on the individual factors, i.e. it would not interchange them. [Technically I think I need to modify that image slightly, this is what might be called a "twisted" Borromean satellite operation, and I should change that to the un-twisted version]

My paper called "An operad for splicing" covers these kinds of issues in the most detail.

So a recursive "closed form" answer is that you get an abelian group if and only if: (1) when you do a hyperbolic satellite operation it must have "restricted symmetries" where the satellite knots themselves have abelian groups (in the sense of spaces of knots -- perhaps I should say "motion group"). And (2) you are never allowed to take a connect-sum of three knots. And (3) if you take a connect-sum of two prime knots, they must be distinct and those prime summands must have abelian motion groups.

This is the quick answer. If you're comfortable with the language in the splicing operad paper I can be more specific. And of course, the start of the induction operation for the above "closed form" is that torus knots and hyperbolic knots have abelian motion groups. I failed to mention cabling operations -- on the level of the fundamental group that is cartesian product with $\mathbb Z$, so it preserves abelian-ness.

Yes, the fundamental group may be nonabelian.

For the purpose of this question, we may largely move freely between the spaces of long knots and knots in $S^3$:

There is a fibration $$\text{Emb}_v(S^1, S^3) \to \text{Emb}(S^1, S^3) \to S^3 \times S^2,$$ the last term being the unit tangent bundle of $S^3$ and the first term being the space of embeddings with both $f(1)$ and the direction of $f'(1)$ fixed. The fiber is homotopy equivalent to the space of long knots in $\Bbb R^3$; therefore, if the fundamental group of the space of long knots is abelian, so is the fundamental group of the space of all knots. Conversely, the fundamental group of the space of knots can only be abelian if the fundamental group of the space of long knots is abelian or is an extension of an abelian group by $\Bbb Z$.

It is false for the space of long knots, by Theorem 11 of Ryan Budney's "Little cubes and long knots", from which it follows that if $K$ is prime long knot and $K_n$ denotes the $n$-fold connected summand of $K$ with itself, then the space of long knots $\mathcal K_{K_n}$ with fundamental group $\pi$ admits a surjection $\pi \to S_n$, coming from the covering space $$S_n \to \mathcal C_2(n) \times \prod_{i=1}^n \mathcal K_K \to \mathcal C_2(n) \times_{S_n} \prod_{i=1}^n \mathcal K_K \simeq \mathcal K_{K_n},$$ where $\mathcal C_2(n)$ is the configuration space of $n$ distinct ordered points in $\Bbb R^2$.

It follows that the space of knots in $S^3$ may also have non-abelian fundamental group. If we had an extension $0 \to \Bbb Z \to \pi \to A \to 0$ for $A$ abelian, then the surjection $f: \pi \to S_n$ would send the cyclic subgroup $\Bbb Z$ to a normal subgroup of $S_n$. This is impossible for $n \geq 5$, as the only non-trivial normal subgroup of $S_n$ is $A_n$ whenever $n \geq 5$, and the alternating group is hardly cyclic! (There is also no cyclic normal subgroup of $S_4$, but it has one more normal subgroup.) Of course, it is also impossible for $f(\Bbb Z)$ to be trivial, as that would imply that $f$ factors as $f: \pi \to A \to S_n$, and the symmetric group is non-abelian.

So the connected sum of a prime long knot with itself $n \geq 3$ times gives an example in the case of long knots, and the connected sum of a prime long knot with itself $n \geq 4$ times gives an example for knots in $S^3$.