The sum of powers of two and two's complement – is there a deeper meaning behind this?

Yes. What you are doing is known as working in the $2$-adic numbers.

The $2$-adic numbers are equipped with a curious notion of distance given by the $2$-adic metric. In this metric, two numbers are close together if their difference is divisible by a large power of $2$. In particular, large powers of $2$ are very small. So relative to the $2$-adic metric the geometric series you wrote down really does converge, and the value it converges to really is $-1$.