Bayes' Rule broken?!?!

The problem is that seeing heads changes your estimate of the probability that you have seen the $HH$ or $HT$ coins.

To see this intuitively, suppose that, instead of coins, you had a pair of trillion sided dice. Say one die has all $H's$ and the other has one $H$ and all the rest $T$. You choose a die uniformly at random and toss $H$. Which die do you have? While it is possible that you have the one with a single $H$, that is astonishingly unlikely.

To the given problem, I've always thought that the easiest way to see the answer was to note that, when you toss a randomly selected coin, each side (of all the coins) is equally likely to come up. Given that you see a $H$, it's equally likely that it's any given $H$ side. There are $4$ $H$ sides that come from $HH$ coins, and $2$ that come from $HT$ coins, hence the answer is $$\frac 4{4+2}=\frac 23$$

Applying this logic to the trillion sided dice we see that the probability that you have the all $H$ die is $$\frac {10^{12}}{10^{12}+1}\approx 1-10^{-12}$$


The probability space is composed of heads that might be witnessed after the selection and toss.   Every head is equally likely to be witnessed.   If you like, think of each head as being labelled with a unique mark in invisible ink.

There are six heads that the witness could be viewing after the toss, and four of them have another head on the other side of their coin, while two do not.   So the probability that a double sided coin had been selected when given that a head was witnessed is $2/3$.


The probability that there is a head on the upper side of the first coin and a tail on the lower side is not the probability that the first coin is normal, but half as much. Indeed if the first coin is normal, then there is an equal chance that the upper side of the first coin be a head or a rail. That corrects your closing computation.

As was said in the comments, while it’s true that there are four outcomes in the probability space of coins, given that the first flip was a head, the outcomes are not all equally likely!

Tags:

Bayesian

Bayes Theorem

Related

Is a characteristic polynomial we consider in Linear Algebra a polynomial or a polynomial function? Minimize $|a-1|^3+|b-1|^3$ with constant product $ab=s$ Is $\int_{\mathbb{R}} |F(x)-G(x)|^2(f(x)-g(x))dx =0$ true, for $f,g$ probability densities and $F,G$ the corresponding distribution functions? Fourier transform of measure on $\mathbb{T}$ Constructing a simplicial map from a diagram How to check these terms in $\int{x^x dx}$ power series are correct? Improving set intersection lexicographically Show that $\sum_{k=1}^n \frac{X_k}{k^2}$ converges a.s Computing the Galois group of the extension $\mathbb{C}(x,y)/\mathbb{C}(x^a y^b, x^c y^d)$ Find general solution of $xx''=(x')^3$ Let $T,U:V\to W$ be linear transformations. Prove that if $W$ is finite-dimensional, then $\text{rank}(T+U)\leq\text{rank}(T) + \text{rank}(U)$. Find the value of $a$ where $a^x = \frac{\log(x)}{\log(a)}$ has only one solution

Recent Posts