Chemistry - Method to know the number of isomers of metal complexes
Solution 1:
Do it many times over, so as not to get confused.
Alternatively, there is a thing called Polya's formula, but you won't be able to use it anyway. In trivial cases like this, the said formula is about 100 times more complicated than counting isomers by hand. It is not before polysubstituted fullerenes that its use in chemistry starts making any sense.
Solution 2:
For an easier method without going much into mathematics you can use simple logic to do find the number of geometrical isomers of [Ma2b2c2]. Though such logics are very much situation based, and I believe there's no such generic one.
Here you can start by choosing no. of pairs of same ligands to be in trans (i.e. 180 degrees to each other). After some thought, its clear that one can choose 0,1 or 3 such pairs, because choosing 2 will automatically force the 3rd pair to be in trans. For choosing 1 such pair there are 3 ways, namely (a,a),(b,b),(c,c); and only 1 way each for choosing none (0) or all (3) such pairs. Thus accounting for all 5 isomers. Take the following example of $\ce{[Co(NH3)2Cl2(NO2)2]-}$.
For further references you can visit this site.