Is it possible to create a circular (or any other non-rectangular) image?

You're right that any non-rectangular graphic really does live inside a bounding rectangle that is aligned with the axes. It's done that way because rectangles are so simple to deal with. The whole display itself is just a rectangular arrangement of pixels.

Determining whether a point is inside a rectangle is quite easy: if the X coordinate lies between a given Xmin and Xmax point, and the Y coordinate lies between a Ymin and Ymax, that point is in the rectangle. And those two tests are independent -- the Xmin and Xmax values don't depend on the Y value, and vice-versa. That's easier than determining whether a point lies within a circle, triangle, or any other shape, where you would need operations like multiplication or a large lookup table.

And think about the basic operations that happen in a windowing system. First it has to render the complete picture on the screen. The system internally has a bunch of overlapping windows to represent, and in order to form the picture, it has to decide what color each individual pixel on the screen needs to be. That's easiest with rectangles. The system scans over each row and column, and determines the uppermost window that contains a given X,Y coordinate, using the simple bounds test. Then it's up to that window to choose the color for the pixel.

Conversely, when the mouse is clicked somewhere on the screen, the system has to determine which window or object was clicked on, and then send it a click message. It's really the same problem, easily handled by walking down the list of overlapping objects, and testing the mouse pointer coordinates against the rectangular limits of each one.

Those two basic operations can be done easily in software, or even in dedicated hardware. Some other method not based on rectangles would be much more work.

I have never come across a raster graphics file format that stored anything other than a rectangular array of pixels -- or a compressed version thereof. To store some arbitrary shape the file would have to contain a specification, in some form, of the shape into which the pixels in the file would be filled. I can see how it could be done, but I've never seen it done.

One way would be very simple:

• store a rectangular array of 0s and 1s, where the 1s represent the pixels for which the file contains a specification;
• store the pixels themselves, one for each 1 in the aforementioned array.

Another way would be to store:

• the dimensions (in pixels) of the rectangular bounding box of the image;
• the position, in the data section of the file, of the first pixel in each line wrt the rectangular bounding box;
• the pixels themselves.

It's difficult to see a compelling reason for dealing with the complications that this sort of approach throws up.

Vector graphics, of course, are a different kettle of fish.