Why do tea bags sometimes float and sometimes sink?

The material from which the teabag is made is porous and allows air to pass through it. So if you gently lower a teabag into liquid the rising liquid expels the air through the (dry) upper part of the teabag and the teabag sinks.

However, if you wet the material of the teabag then the water spreads out over the material to form a liquid film that seals all the pores in the material, so the wet material will not allow air to pass through it. If you managed to uniformly wet the material of teabag before dropping it into the tea then you have in effect created a bubble of air and the teabag will float.

These two scenarios are extremes, and in practice the wetting is rather more chaotic. Whether any air is trapped in the teabag will depend on how exactly the water pours over it, and I'm not sure I care to make any predictions without trying the experiment (and since I'm a coffee drinker not a tea drinker I have no tea bags to hand).