Why is the fish in the water swifter than the bird in the air?
The speed of any object is a balance between the drag force on the object and the thrust the object can create. Attaining high speeds, or possibly more relevant in this case, high acceleration requires making the thrust as high as possible while keeping the drag as low as possible.
Water is a lot denser than air, but this affects the drag mostly when the flow is turbulent and the drag is dominated by inertial forces. In this regime the drag is effectively due to having to push the medium out of the way, and it's easier to push aside low density air than high density water. However if you can keep the flow laminar the density isn't as big a factor and the drag is dominated by the viscosity of the medium. Water is a lot more viscous than air (as well as denser) but for streamlined objects the drag due to viscosity can be kept remarkably low.
Where water wins is that it's much easier to develop a high thrust in water than in air. In a fluid medium, where there is nothing solid to push against, you produce thrust in basically the same way that a rocket does. If you push away some mass of water $m$ with a velocity $v$ then the momentum of the water changes by $mv$, which means that your momentum changes by $-mv$. So you push the water in one direction and you accelerate in the other direction. The thrust you generate is simply the rate of change of momentum of the water.
And it should now be clear why it's easier to generate a high thrust in water than in air. Because air is low density you can't push a high mass of it (unless you're very large) so it's hard to change its momentum by very much.
So to summarise:
in water the drag is high but it's easy to generate a high thrust
in air the drag is low but it's hard to generate a high thrust.
How the speeds in water and air compare depend on the exact tradeoff between drag and thrust. Sailfish can reach speeds of 68 mph, but they do this mainly by being very streamlined so they can keep the drag as low as possible while exploiting the high thrust they can get from water. Birds generally don't reach speeds this high because although the drag in air is small they simply can't generate the thrust required for high speeds. Peregrine falcons can reach speeds of 200 mph, far faster than a sailfish, but they do it only in dives where gravity provides the thrust.
In addition to @JohnRennie and @WetSavannaAnimalakaRodVance that try to answer the question based on the same logic as my first point below I have some other aspects. In fact I realized that my two other points must be crucial, otherwise there would be no reason fish and birds currently function very differently.
Since water is more viscous media to travel in a slight change of shape (of the traveler) results in larger angle of curvature; turning is quicker.
Fish move more like snakes turning their bodies from side to side so their muscles are then equally as good for turning as the required gesture is similar. Birds move forward by flapping their wings and turning requires tilting of their wings so they would need secondary muscles to do that. And so fish must be superior to birds anatomically to change lateral direction.
Birds have to use energy to constantly stay on the same altitude by movement (flapping their wings). Whereas fish have an organ called swim bladder to regulate their depth. If birds would be using the same kind of organ for altitude (and be effectively as dense as the air that surrounds them) they would've most likely evolved to be able to move more agilely laterally - just like fish. Edit: some birds can hold altitude with very minimal energy consumption by extending their wings while soaring. However, this does not dispute my point; while soaring birds cannot make agile moves without stopping their soaring and losing altitude. Then again fish can utilize their swim bladder and simultaneously perform swift moves.
From your comment:
It is agility here, not speed
it seems you are talking about a fish's higher nimbleness, i.e. its ability to change its direction, reorient itself and accelerate swiftly. You can answer this question by a thought experiment: what happens if we drain the air from around the bird (assuming he/she can still breath)? We'd approach the situation where the bird were in outer space; there is then nothing for it to thrust against and therefore it could not change the motion state of its center of mass at all.
Birds and fish change their motion state by "throwing" the fluid they are steeped in. They thrust on the fluid, and the fluid thrusts back on them, by Newton's third law (see also the question What Really Allows Airplanes to Fly). In the case of the fish, the fluid has a much greater mass per unit volume, so it has to thrust far less of that fluid to get the same impulse. It's a bit like the thought experiment: imagine you are in space and have to get yourself back to your spaceship by throwing a bucket of balls you have with you. Which scenario would let you achieve your goal faster: a given mass of steel balls or the same mass made out of the same size styrofoam balls (which would have a much bigger total volume)? I think you can understand that the steel balls would make this task much quicker and easier.