Why can you chose how to align infinitely long equations when adding them?

We can only shift and rearrange infinite sums if both of them converge absolutely. Otherwise strange things happen, just like in your example.

I feel like the answers to this question are not appropriate because they address arbitrary rearrangements of series instead of shifting. Here is an answer that is just concerned with shifting. The key lies in two facts:

Fact 1: Let $S_1 = a_1 + a_2 + \cdots$ and $S_2 = b_1 + b_2 + \cdots$ be two convergent series. Then the sum $(a_1 + b_1) + (a_2 + b_2) + \cdots$ converges and is equal to $S_1 + S_2$.

Fact 2: The series $a_1 + a_2 + \cdots$ converges if and only if the series $0 + a_1 + a_2 + \cdots$ converges.

The second fact is really just justifying the "shift" operation for a series. This proves that if you take two convergent series, you can do all of the shifting, adding (or subtracting) that you want and you will still have a valid answer. Nothing about absolute convergence is necessary for this problem.

This question about the nature of $\infty - \infty$ might be somewhat relevant.