Product of permutation cycles, transpositions. Are there different conventions in the order?

Yes, there are competing conventions for multiplication in the symmetric group.

• Some people read left-to-right, which is consistent with the right-action and English reading order. EDIT: And, importantly, it is the convention used by GAP, Magma, and Sage.
• Some people read right-to-left, which is consistent with the left-action and the usual function notation.

But some people reverse the function notation, as well. Reader beware…

I prefer the second one, for the same reasons that Olivia outlines. But my understanding is that the first one is particularly common in combinatorial or geometric group theory.

It's no error, it stems from the fact that there are two different ways to compose permutations (and functions, more generally).

Each person/book/article has their own convention about whether functions act on the left or on the right. That is, if you want to apply a function $f$ to a point $x$, do you choose the more universal $f(x)$, or the "backwards" $(x)f$ way that is essentially unique to (some) algebraists?

One benefit of the latter is that $f \circ g$ really means "do $f$ first, then $g$": we would compute $(x)(f \circ g) = ((x)f)g$. It's also a bit nicer for permutations, in some sense. We always work left to right. Anyone choosing this convention is going against the grain, and should be specifying this somewhere in the book.

As a personal anecdote, a few of my algebra professors were students of the group-theorist Martin Isaacs, Isaacs being the biggest proponent of the "functions act on the right" convention (that I'm personally aware of). They preferred his notation, accordingly. In the basic abstract algebra courses, they used the usual "on the left" convention (as do most textbooks, including the one we were using). But then when the courses reached the graduate level, they switched to their more familiar "on the right" style. Because of this slightly-jarring transition, I've been rather aware of the issue. I learned somewhat recently that one of the classic algebra texts also use this "algebraist's" convention; I think it's Hungerford, but I'm not sure (hopefully somebody can remind me).

Now that I think about it, this explains why the following is the second exercise in Isaacs' Algebra:

Let $G$ be any group. For $x \in G$, let $r_x$ and $l_x$ be the mappings $G \to G$ defined by $$(g)r_x = gx \quad \text{and} \quad (g)l_x = xg,$$ or in other words, by right and left multiplication by $x$ on $G$. Let $R = \{r_x \mid x \in G\}$ and $L = \{l_x \mid x \in G\}.$ Show that $R$ and $L$ are permutation groups on $G$ and that $R \cong G \cong L$.

to show that the two different conventions are equivalent in the sense that they yield isomorphic groups!

See here for the discussion on MathOverflow.

The first cycle representation $(x_1x_2x_3)=(x_1x_3)(x_1x_2)$ is correct.

Essentially, $(x_1x_2x_3)$ is the permutation that takes $x_1$ to $x_2$, $x_2$ to $x_3$ and $x_3$ back to $x_1$.

So when we have the two transpositions, we read them right to left, meaning we start at the rightmost cycle and move our way in.

$x_1 \rightarrow x_2$ but $x_2$ is not present in the left cycle so we stop here. $x_2 \rightarrow x_1$ in the right cycle and $x_1 \rightarrow x_3$ in the left cycle, so $x_2\rightarrow x_3$.

Thus, this product of cycles is correct.

However, a key fact is products of cycles do not necessarily commute although disjoint cycles commute. So, it should be evident that $(x_1x_3)(x_1x_2)\neq (x_1x_2)(x_1x_3)$

In fact, if we have $(x_1x_2)(x_1x_3)$ then the product is $(x_1x_3x_2)\neq(x_1x_2x_3)=(x_1x_3)(x_1x_2)$