What is the meaning of "formal" in math-speak?

"Formal" means, roughly, "without semantic content". For example, in category theory an arrow is usually a function; if we just say "reverse the arrows", there arises a natural question of "wait, what's the reversal of a function?" Saying "formally reverse the arrows" means that we don't need to answer that question - a formally reversed arrow is just an arrow going backwards, nothing else. Likewise, a "formal sum" of two objects is just the two of them written with a $+$ between them - the formal sum of $a$ and $b$ is "$a+b$", the formal sum of "apple" and "orange" is "apple $+$ orange", and the formal sum of $1$ and $1$ is "$1+1$" - not $2$, just the string "$1 + 1$".

Basically, we use "formal" when we don't want to do anything other than just let an operation make sense - when we want to be able to add elements of a set, for example, without wanting to introduce any new relationships between them. We don't impose any semantics, any "meaning" to "sums" or "reversals" or whatever we're talking about; we just do the operation we want to do, and leave it there.

One characteristic of formal operations is that there's no "other way" to get the same result. For example, the only way to use a formal sum to get "$1 + 1$" is to take the formal sum of $1$ and $1$; you can't sum $2$ and $0$, or $-1$ and $3$, to get the same answer. A formal sum carries absolutely no information beyond what was necessary to build it.

"Formal" here is meant in the sense of "relating to form".

In regards to the first example, given a set $X$, a "Formal sum of things in $X$" means an element of the free abelian group generated by $X$.

There's a general construction in universal algebra of free algebraic structures where the element of a free algebra on $n$ elements are precisely the algebraic expressions one can write down in $n$ variables. (modulo the relations such structures satisfy)

For example, elements of the free commutative ring generated by one element are polynomials — the kinds of expressions you can construct from a single variable using just addition and multiplication (and $0$, $1$, and negation), simplified by imposing the algebraic identities defining rings.