Why is air invisible?

I think the pithy answer is that our eyes adapted to see the subset of the electromagnetic spectrum where air has no absorption peaks. If we saw in different frequency ranges, then air would scatter the light we saw, and our eyes would be less useful.

I don't think one can just state that particle size smaller than the wavelength of light implies no interaction with light. You have to look at the quantum mechanical modes of the atoms/molecules. If they have modes with frequencies in the wvaelength range you are interested in, then you will get interaction/absorption. I think that even clean air does have nonzero absorption in the visible. Have a large enough column density along a beam of light, and there will be some absorption.

Also do note, that the speed of light is slower in air than in a vacuum, so the air does have some effect. You do get effects of refraction, such as mirages, and heat waves seen looking across convection over a hor surface. Also if you point a telescope at a star, you see the mess the atmosphere makes of the image.

Air is mostly composed of first two row small molecules like O2 and N2, and atoms like Argon. For all these, the absorption is in the deep ultraviolet. Molecules which absorb in the visible have smaller energy differences in their first absorption bands, although chlorine(2) and Iodine(2) molecules are visible. This answer is meant to be complementary to and supportive of the evolutionary answer given above