Probability question (Birthday problem)

The basic idea was right, and a small modification is enough.

Line up the people in some arbitrary order. There are, under the usual simplifying assumption that the year has $365$ days, $365^{23}$ possible birthday sequences. Under the usual assumptions of independence, and that all birthdays are equally likely, all these sequences are equally likely. The assumption "equally likely" is not correct, though it is more correct for people than for eagles.

Now we count how many ways we can have precisely $2$ people have the same birthday, with everybody else having a different birthday, meaning different from each other and also different from the birthday of our birthday couple.

The couple can be chosen in $\binom{23}{2}$ ways. For each of these ways, the couple's birthday can be chosen in $365$ ways. And the birthdays of the others can be chosen in what is sometimes called $P(364,21)$ ways. (I have always avoided giving it a name.) So the number of birthday assignments that satisfy our condition is $$\binom{23}{2}(365)P(364,21), \quad\text{that is,}\quad \binom{23}{2}(365)(364)(363)\cdots (344).$$ For the probability, divide by $(365)^{23}$.

Remark: Or else argue this way, which may be closer in spirit to your thinking. Pick two people, $i$ and $j$. What is the probability these two have the same birthday, and all other birthdays are different from this one, and different from each other? Whatever $i$'s birthday is, the probability that $j$'s matches it is $\frac{1}{365}$. By the usual birthday argument, the probability that the birthdays of the other people are different, and different from the birthdays couple's, is $$\frac{364}{365}\frac{363}{365}\frac{363}{365}\cdots\frac{344}{365}.$$ Multiply the above expression by $\frac{1}{365}$. Finally, sum the result over the $\binom{23}{2}$ ways to choose $i$ and $j$, that is, multiply by $\binom{23}{2}$.