Calculating the probability of reaching each absorbing state in Markov Chain
The state transition diagram for the given problem is given by:
Consider transitions from A to D:
It could happen in several ways, as listed in the following. All these paths are mutually exclusive. Invoking the definition of of first passage time, we can obtain the required probability.
\begin{eqnarray*} A\rightarrow B \rightarrow D &=& p_{AB}\cdot p_{BD}\\ &=&\left(\frac{1}{2}\right)\left(\frac{3}{9}\right)=\frac{3}{18}\\ A\rightarrow B\rightarrow A\rightarrow B \rightarrow D &=& p_{AB}\cdot p_{BA}\cdot p_{AB}\cdot p_{BD}\\ &=&\left(\frac{1}{2}\right)\left(\frac{4}{9}\right)\left(\frac{1}{2}\right)\left(\frac{3}{9}\right)=\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)\left(\frac{3}{9}\right)\\ A\rightarrow B\rightarrow A\rightarrow B\rightarrow A\rightarrow B \rightarrow D &=& p_{AB}\cdot p_{BA}\cdot p_{AB}\cdot p_{BA}\cdot p_{AB}\cdot p_{BD}=p_{AB}\cdot \left(p_{BA}p_{AB}\right)^{2}\cdot p_{BD}\\ &=&\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)^{2}\left(\frac{3}{9}\right)\\ \end{eqnarray*} and several more such transitions. The probability of ever reaching from state A to D is obtained from the following probability:
\begin{eqnarray*} &&\frac{3}{18}+\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)\left(\frac{3}{9}\right)+\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)^{2}\left(\frac{3}{9}\right)+\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)^{3}\left(\frac{3}{9}\right)+\cdots\\ &=&\frac{3}{18}+\left(\frac{1}{2}\right)\left(\frac{2}{9}\right)\left(\frac{3}{9}\right)\left\{1+\frac{2}{9}+\left(\frac{2}{9}\right)^{2}+\cdots\right\}\\ &=&\frac{3}{18}+\frac{1}{3\times 9}\left\{\dfrac{1}{1-2/9}\right\}=\frac{27}{126}=\frac{3}{14} \end{eqnarray*} In a similar manner, we can find other probabilities.