Linearity in Expected Value

Long story short: The random variables have identical expectations when they follow identical distributions.


I have a deck of these four cards, with values 1,2,3,4.   I shuffle the deck and place two cards face down, one on your left and one on your right.

What is your expectation for the value of the card on your left?

What is your expectation for the value of the card on your right?


Hint: I have not told you which I placed down first.   Does it matter?

Indeed, let us go ahead an place the other two cards face down.   What is your expectation for the values of each of these as well?


You see, the dependency of the cards does means that the conditional expectation, $\mathsf E(X_2\mid X_1)$ will indeed not be constant but rather a function of $X_1$; however we are not discussing the conditional expectations, just the expectation.

Still, let us examine the conditional expectation and use the law of iterated expectation:

$$\begin{align}\mathsf E(X_2\mid X_1=k) ~&=~\begin{cases} \tfrac 13(~~~~~~~2+3+4)&:&k=1\\\tfrac 13(1+~~~~~~~3+4)&:&k=2\\\tfrac 13(1+2+~~~~~~~4)&:&k=2\\\tfrac 13(1+2+3~~~~~~~)&:&k=4 \end{cases}\\[2ex] \mathsf E(X_2) ~&=~ \mathsf E(\mathsf E(X_2\mid X_1)) \\[1ex] &=~ \tfrac 14\big(\mathsf E(X_2\mid X_1=1)+\mathsf E(X_2\mid X_1=2)+\mathsf E(X_2\mid X_1=3)+\mathsf E(X_2\mid X_1=4)\big)\\[1ex] &=~ \tfrac 14(1+2+3+4) \\[1ex] &=~ \mathsf E(X_1)\end{align}$$


Contrast with Coupon Collection. Why is it not the same?

Where as the random variables for the cards are the values of the cards drawn, the random variables used in the coupon collection are: the count of draws (with replacement) after the $k-1$-th new value until the $k$ new value is draw ("new value" meaning a value not previously drawn).

That is, with the same deck of cards, let $Y_1$ be the count of draws until the first new value is drawn, $Y_2$ be the count of draws after that until the second new value is drawn, $Y_3$ be the count of draws after that until the third new value is drawn, and $Y_4$ be the count of draws after that until the last new value is drawn.

$$\mathsf E(Y_1)=1, \mathsf E(Y_2)=4/3, \mathsf E(Y_3)=4/2, \mathsf E(Y_4)=4$$

These random variables have different expectations not because they are dependent or independent,† but rather it is because the random variables have different distributions.‡

(† They are in fact independent.)

(‡ They follow Geometric Distributions with different rates.)


The answer is that $X_i$ are copies of one and the same random variable, namely, the random variable that returns the number on a card randomly drawn from your card collection.


$X_1$ and $X_2$ follow the same discrete uniform distribution (since $X_1$ and $X_2|X_1$ are uniform, so is $X_2$). Hence they have identical expectations.