How to efficiently mask an image?

You'll want to use the image processing functions that handle the image in its entirety, not work with individual pixels. The most straightforward way is to use a black and white "mask image" and then use ImageMultiply. This is not too hard, although you have to be careful to get the width and height of the mask right. For example:

img = Import["http://www.anderson-tours.co.uk/wp-content/uploads/2013/02/Eiffel-Tower-Paris-France.jpg"];
{w, h} = ImageDimensions[img];
Show[ImageMultiply[img, ColorNegate@Graphics[Disk[{0, 0}, {w, h}], ImageSize -> {w, h}]], ImageSize -> 200]

The mask in this example is ColorNegate@Graphics[Disk[{0, 0}, {w, h}].

To generate arbitrary mathematically defined masks RegionPlot might be a good option:

Binarize@RegionPlot[x^2 + y^3 < 2, {x, -2, 2}, {y, -2, 2}, Frame -> False]