Is there a quick way to justify that this elementary probability is equal to $\frac23$?
The following technique from Is there a clever solution to this elementary probability/combinatorics problem? works here too:
Instead of $n$ urns, have one urn with $n$ numbered balls. You pick one ball and keep it, and the urn is then taken away while an assistant paints all the balls with a number below yours red and the rest magenta. Then you pick two more balls, and we then ask what is the probability that the last ball was magenta, given that the middle ball was.
A bit of thought should convince you that all the balls you never pick are completely irrelevant for the outcome -- the only thing that matters is the numeric ordering between the three balls you do pick.
So instead of picking balls one by one, we can start by selecting a set of three balls to be picked; then among those select a sequence to pick them in. The probabilities of the entire experiment will be the the same as when the picked balls are $\{1,2,3\}$, and we can analyze that simply by listing case:
1, 2, 3 -- 2 is magenta, 3 is magenta
1, 3, 2 -- 3 is magenta, 2 is magenta
2, 1, 3 -- 1 is red, case is excluded
2. 3. 1 -- 3 is magenta, 2 is red
3. 1. 2 -- 1 is red, case is excluded
3. 2. 1 -- 2 is red, case is excluded
Of the three cases where the first of the colored balls is magenta, two of them have the other colored ball magenta too.
We could get through the case analysis a bit faster if we had started by observing that $$P(\text{second ball magenta}\mid\text{first ball magenta})= P(\text{second ball red}\mid\text{first ball red})$$ by symmetry, and then both of these are the same as the (unconditional) probability that the two colored balls have the same color.
The two colored balls have the same color exactly when the ball you pick to determine colors is either the highest numbered between the three balls-to-be-picked or the lowest numbered between the three balls-to-be-picked. And this happens, of course, in $2/3$ of all cases.
Imagine a row of $n$ urns with a single uncolored ball in each of them. One of the urns is selected at random, and its ball is colored white. The balls to the left of the white ball are colored red, and the balls to the right of the white ball are colored magenta. Now two more urns are selected at random, and their contents inspected. With probability ${1\over3}$ the white ball is the middle of the three, hence with probability ${2\over3}$ the contents of the other two urns are of the same color.
The probability that the first ball is magenta is $\frac12$ as by symmetry half the balls are magenta and they are each equally likely to be chosen. It could be expressed as $\displaystyle \sum_{r=1}^n \frac{r-1}{n-1}\times \frac1n = \frac{r(r-1)/2}{n(n-1)}=\frac12$
Similarly the probability that the first and second balls are magenta is $\displaystyle \sum_{r=1}^n \frac{r-1}{n-1}\times\frac{r-2}{n-2}\times \frac1n = \frac{r(r-1)(r-2)/3}{n(n-1)(n-2)} = \frac13$
So the conditional probability that the second ball is magenta given that the first is magenta is $\dfrac{1/3}{1/2} = \dfrac23$