How come rearrangement of a convergent series may not converge to the same value / or does not converge at all
Consider the following
$$\sum_{n=1}^\infty \frac 1n =\infty,\qquad \sum_{n=1}^\infty \frac 1{2n}=\infty,\qquad \sum_{n=1}^\infty \frac 1{2n-1}=\infty,\qquad \sum_{n=1}^\infty \frac{(-1)^{n-1}}n=\ln(2).$$
In short: the sum of the reciprocals of all numbers diverges (harmonic series). The sum of the reciprocals of even/odd numbers diverges. But if you sum up even and odd reciprocals with alternating signs, it converges.
So here you have a specific infinite sum that seems to converge: $$1-\frac12+\frac13-\frac14+\cdots =\ln(2).$$
Of course, if you just rearrange finitely many of the summands, you will end up with the same sum. But if you shuffle up all the numbers, you can get everywhere with your limit. I think this is done in any proof of Riemann's rearrangement theorem, but let me line out the proof on this example. Lets say you want the rearranged sum to converge to $\pi$ (for fun). Then take some of the positive terms (the odd reciprocals) and add enough of them up until you are just greater than $\pi$: $$1+\frac13+\frac15+\cdots>\pi.$$ You can do this, as we know that the sum of the odd reciprocals diverges. In the next step, only take the negative (even) reciprocals and subtract them from your sum until you are just below $\pi$. Again you can do this, because the sum of the even reciprocals diverges. Now again take positiv terms, then negatives, then positives, and so on. In "the end" you will have used all the terms of the original sum, but you rearranged them in a way, so that they converge to $\pi$. And there is nothing special about $\pi$, so you can use this method to converge to anything, including $\pm\infty$.
Here a description of how to rearrange the sum to make it divergent, e.g. divergent to $\infty$. Sum up enough positive (odd) terms to make the sum greater than $1$. Add only a single negative term. Add positive terms until the sum exceed $2$. Add a single negative term. Add positive terms to exceed $3$, ... and so on. You will exceed any natural number, hence diverge to $\infty$.
Here's a concrete example where rearrangement of a convergent series results in a series that converges to a different value.
Consider $$ 1 - 1 + \frac{1}{2} - \frac{1}{2} + \frac{1}{3} - \frac{1}{3} + \frac{1}{4} - \frac{1}{4} + \frac{1}{5} - \frac{1}{5} + \ldots $$ The partial sums of this series alternate 0 between values of the form $1/n$, so it converges to zero.
Now consider $$ 1 + \frac{1}{2} - 1 + \frac{1}{3} + \frac{1}{4} - \frac{1}{2} + \frac{1}{5} + \frac{1}{6} - \frac{1}{3} + \frac{1}{7} + \frac{1}{8} - \frac{1}{4} + \ldots $$ This is clearly a rearrangement of the first, and it's not too hard to see that you don't change the convergence behavior if you sum the terms in groups of 3:
$$ \left(1 + \frac{1}{2} - 1\right) + \left(\frac{1}{3} + \frac{1}{4} - \frac{1}{2}\right) + \left(\frac{1}{5} + \frac{1}{6} - \frac{1}{3}\right) + \left(\frac{1}{7} + \frac{1}{8} - \frac{1}{4}\right) + \ldots \\ = \frac{1}{2} + \frac{1}{12} +\frac{1}{30} + \frac{1}{56} + \ldots \\ $$ Now by standard methods you can check that this converges to some value greater than zero. However, expanding each term as a difference shows that it actually converges to a known value. $$ \frac{1}{2} + \frac{1}{12} +\frac{1}{30} + \frac{1}{56} + \ldots \\ =\left(1 - \frac{1}{2} \right) + \left(\frac{1}{3} - \frac{1}{4} \right) + \left(\frac{1}{5} - \frac{1}{6} \right) + \left(\frac{1}{7} - \frac{1}{8} \right) + \ldots \\ = \ln(2) $$
I think you need to understand the nature of a conditionally convergent series. I restrict the discussion to series whose terms are real numbers.
If a series $\sum a_{n}$ is conditionally convergent and we separate the positive terms of this series into $\sum b_{n}$ and negative terms into another series $\sum c_{n}$ then we have to understand following two things:
- $\lim_{n \to \infty} a_{n} = \lim_{n \to\infty}b_{n} = \lim_{n \to \infty}c_{n} = 0$
- $\sum b_{n}$ diverges to $\infty$ and $\sum c_{n}$ diverges to $-\infty$.
Consider the expression $$S(M,N) =\sum_{n=1}^{M} b_{n} + \sum_{n=1}^{N} c_{n}$$ and we can see that it is of the indeterminate form $\infty - \infty$ as $M, N$ tend to $\infty $ independently. Since the series on right diverge it is possible to take desired number of terms from $\sum b_{n}$ and $\sum c_{n}$ to add up to any particular sum (including $\pm\infty$). To add some details, suppose I want to achieve the sum $2$. Then I choose terms from $\sum b_{n}$ so that the sum of these chosen terms is greater than $2$ (this is possible because $\sum b_{n}$ diverges and we can exceed any number by choosing sufficient number of terms). Next I add terms from the series $\sum c_{n}$ (note the terms here are negative so that adding them effectively reduces the sum) so that the sum is less than $2$. Repeating this procedure ad infinitum I get a series whose sum is $2$.
Formally given any extended real number $S$, we can prove that there exist sequences $m_{k}, n_{k} $ taking positive integer values and tending to $\infty$ as $k\to\infty$ such that $S(m_{k}, n_{k}) \to S$ as $k\to\infty$. The above argument when formalized with all the details constitutes a proof of Riemann's rearrangement theorem.