$12$ points are arranged around a circle. How many ways can you create two pairs of distinct triangles that overlap?

Pick any of the $12$ points as a base point and number the points $1$ through $12$ clockwise starting at the base point. There are $\binom{12}6=924$ ways to choose $6$ of them to be the vertices of the two triangles.

Say that we choose points $p_1,\ldots,p_6$, where $p_1<\ldots<p_6$. There are $7$ ways to assign them to two triangles so that the triangles overlap. If we label a point $0$ for one triangle, $1$ for the other, and then list the labels of $p_1$ through $p_6$ in that order, the acceptable possibilities are $001011$, $001101$, $010011$, $010101$, $010110$, $011001$, and $011010$: these are the labelings that are not cyclic permutations of $000111$, since it is precisely the cyclic permutations of $000111$ that correspond to non-overlapping triangles. And

$$7\binom{12}6=7\cdot 924=6468\,$$

as it should.

This process is much easier than what you listed, but still essentially counts the "non-overlapping triangles first".

• We want to count the number of pairs of triangles that overlap.
• Such pairs are defined (not necessarily uniquely) by their vertices.
• Given any 6 points (out of 12), how many ways are there to get overlapping triangles?
• Given any 6 points (out of 12), how many ways are there to get pairs of triangles? $\rightarrow { 6 \choose 3 } / 2 $.
• Given any 6 points (out of 12), how many ways are there to get non-overlapping triangles? There are only 3 such pairs, formed by consecutive vertices.
• So, there are $10 - 3 = 7 $ overlapping pairs.
• How many ways are there to choose 6 points out of 12?
• Hence, there are ${ 12 \choose 6 } \times 7 = 6468$ ways.