Counting the ways to form 4 different teams

You're on the right track.

The number of ways to pick Team 1 is equal to $\binom{40}6\cdot\binom{34}4$. After that, the number of ways to pick out Team 2 is $\binom{30}6\cdot \binom{24}4$. And so on.

Ultimately, the number of ways to pick out the full $4$ teams is $$ \binom{40}6\cdot\binom{34}4\cdot\binom{30}6\cdot\binom{24}4\cdot\binom{20}6\cdot\binom{14}4\cdot\binom{10}6\cdot\binom{4}4\\ = \frac{40!}{(6!)^4\cdot (4!)^4} $$ (Also known as the multinomial coefficient $\binom{40}{6, 4, 6, 4, 6, 4, 6, 4}$.)

However, we don't care which team is Team 1 and which team is Team 4. We just care which four teams are picked. Since the same four teams can be picked out in $4!$ ways, the total number of different team compositions is $$ \frac{40!}{(6!)^4\cdot (4!)^5} $$

I would do it a little different. First split 40 players in to groups of 10 players, that we can do on $$a= {1\over 4!} \cdot {40\choose 10}\cdot {30\choose 10}\cdot {20\choose 10}\cdot {10\choose 10}$$ ways.

Now in each group choose 4 reserve players, so we can do that on $b={10\choose 4}^4$ ways.

Thus the result is $$ a\cdot b = {40!\over 4!^5\cdot 6!^4}$$