Do bubbles get created when you move something in water?

Certainly, the mechanism is called cavitation and it works like this:

As an object moves through water, a pressure distribution gets built up around the walls of the object and depending on its shape, it is possible for pressures at some points to be lower than ambient pressure- the best example of this is on the backwards face of a rotating propeller blade.

If that propeller blade moves fast enough through the water, that "negative pressure" grows until it equals the vapor pressure of the water at that ambient condition and the water there explodes into vapor- in essence, it boils. As soon as those cavitation bubbles full of vapor form, any dissolved air in the water nearby will diffuse into the vapor bubbles and collect there. Because the pressure in the cavitation bubble is below ambient, as soon as the moving object sheds a bubble it tends to quickly collapse as the vapor inside the bubble condenses back into liquid water- but the re-dissolution process for the air in the bubble is much slower, and so after the vapor bubble is gone a tiny air-filled bubble remains.

Note that the lifetime of a cavitation bubble is measured in milliseconds, and those air remnants are far tinier than anything portrayed in a cartoon.

Yes this is theoretically possible, and it is also observed in practice. Gases, including oxygen and carbon dioxide certainly, and probably also nitrogen, are absorbed in solution by water from the air (if oxygen were not absorbed fish would not be able to breathe underwater). Once in solution, the distribution of dissolved particles diffuses naturally through the whole body of the water, to any depth. If water is agitated into turbulent motion, then gases are ejected from solution because turbulence generates rapidly varying pressure. Gas is less soluble in liquid at lower pressure. You can also observe the effect simply by shaking a fizzy drink.

Cavitation is a similar effect, but in this case the bubble contains water vapour, and is produced because the pressure dropped to the point where water boils. Cavitation bubbles generally collapse extremely rapidly.

Yes, cavitation is real, but my understanding is that fish are not cavitating very much (dolphins are known to avoid swimming fast enough to cause cavitation, because the collapsing cavitation bubbles are painful to them).

Plants, bacteria, and other forms of marine life slowly generate gases as part of their metabolic processes. Some of these gases dissolve directly, but some form small bubbles that collect on various surfaces (due to surface tension, being trapped in silt, trapped under an overhang, etc.). When a fish comes by, the sudden motion of the water knocks the bubbles loose to float upwards. In this case the motion isn't creating bubbles, it's releasing bubbles that are continuously forming in the environment.