St. Petersburg Paradox

Suppose you pay a trillion dollars to enter the game. The following table contains some of the values of the net payoff you can possibly end up with and the corresponding probabilities:

\begin{align*} \begin{array}{rr} 1/2&-\,1\mathord{,}000\mathord{,}000\mathord{,}000\mathord{,}000\\ 1/4&-\,999\mathord{,}999\mathord{,}999\mathord{,}998\\ 1/8&-\,999\mathord{,}999\mathord{,}999\mathord{,}996\\ 1/16&-\,999\mathord{,}999\mathord{,}999\mathord{,}992\\ \vdots\\ 1/2^{10}&-\,999\mathord{,}999\mathord{,}999\mathord{,}488\\ \vdots\\ 1/2^{20}&-\,999\mathord{,}999\mathord{,}475\mathord{,}712\\ \vdots\\ 1/2^{30}&-\,999\mathord{,}463\mathord{,}129\mathord{,}088\\ \vdots\\ 1/2^{40}&-\,450\mathord{,}244\mathord{,}186\mathord{,}112\\ 1/2^{41}&99\mathord{,}511\mathord{,}627\mathord{,}776\\ 1/2^{42}&1\mathord{,}199\mathord{,}023\mathord{,}255\mathord{,}552\\ \vdots\\ 1/2^{50}&561\mathord{,}949\mathord{,}953\mathord{,}421\mathord{,}312\\ \vdots\\ 1/2^{100}&633\mathord{,}825\mathord{,}300\mathord{,}114\mathord{,}114\mathord{,}699\mathord{,}748\mathord{,}351\mathord{,}602\mathord{,}688\\ \vdots\\ 1/2^{200}&803\mathord{,}469\mathord{,}022\mathord{,}129\mathord{,}495\mathord{,}137\mathord{,}770\mathord{,}981\mathord{,}046\mathord{,}170\mathord{,}581\mathord{,}301\mathord{,}261\mathord{,}101\mathord{,}496\mathord{,}890\mathord{,}396\mathord{,}417\mathord{,}650\mathord{,}688\\ \vdots \end{array} \end{align*}

The paradox lies in the following observation. If you take out a loan of one trillion dollars to play this game, you will go bankrupt with a very large probability. However, once in a lifetime (not even of a human but of the universe) you win an unspeakably large amount of money, so large you can't even imagine.

In the light of this observation, a reasonable person would never play this game only once. It is worth playing only if you can play it indefinitely, while you have access to unlimited borrowing. What will happen is that you will keep playing for billions of years, accumulating an enormous debt using your infinite line of credit. But after a very long time, you will win so much money that is sufficient for you to pay off this large debt and still purchase the whole world. As @IanColey put it, this is because the chances of winning so much money are very, very tiny, but the payoffs associated with these very, very tiny probabilities are much, much, much more enormous than the probabilities are tiny.

Expectations are statements about behavior of a random variable as you draw from it infinitely many times. If you had an infinite bankroll, you could play the game infinitely many times at $1,000,000,000,000 a pop, winning money in the long run (even if you lost 10 times in a row). At the same time, we are also assuming (rather incorrectly) that the bankroll of the person handing out the reward is infinite. If either of these assumptions is invalidated, new dynamics enter the equation. If you don't have infinite money, you must consider risk (e.g. if the bet is as big as your entire net worth, you have a 50/50 of losing it all in one toss), and if the person offering the game has finite money, the expectation of playing the game to infinity cannot be infinity since the game ends (as well as your opportunity to accrue new winnings) when he runs out of money.

Wikipedia has a small overview of the mathematical reason it is infinity as well as the problems I just brought up - as well as alternative explanations.

http://en.wikipedia.org/wiki/St._Petersburg_paradox