Why are circuits considered loops?

Most circuits are considered loops because charge in conductive materials tends to equalize electrostatic potential differences relatively quickly. Take a long wire/rod for instance. Let's say you can add electrons to one side of it. At first you start with 0 electrons. When you add the first electron, there's nothing else around so it can go basically wherever. When you add the second electron, it will push the first electron as far away as possible to try and create charge balance in the rod. This first electron moving is actually a tiny current and its movement could be used to extract work from it (because it took work to add the second electron to the system). Adding a third electron will push the second electron to the middle. The second electron's movement is half of the first so you could only extract half the amount of work out of it. The first electron is at the other end and hasn't moved at this point. If you keep adding electrons to the rod at one end, the movement of the other electrons will be less and less. Soon, you'll be at thousands of volts and not capable of extracting any work out of it because there's simply nowhere for the electrons to go.

Instead, what if we took electrons off of one side and added it to the other side? Now every electron you do this to will cause all the other electrons to move in response in one direction the same amount. Now you can extract a uniform amount of work out of the system for each electron you move. But what have you done? You've created a loop with your hand moving single electrons at a time. This is why most circuit utilize a loop. There's something that pushes electrons in one (or maybe both) direction. In your case, it's a battery, but generators and various other methods can be used to "pump" electrons to extract work from them at a different location.

You can think of a battery as an electron pump, it chemically moves electrons from the positive terminal to the negative so as to maintain a certain 'electrical pressure' (That is what the early guys called potential difference in some old books, and it is not a bad model).

To actually make this thing do anything useful you need to provide a path for the electrons to flow that happens to use the moving electrons to do some kind of interesting task[1]. This might be heating a thin wire to make light, or powering some other electrochemical reaction to recharge another battery, or making a magnetic field in a motor or whatever. This path must clearly be a loop if you want the system to run more then very briefly (think nano seconds).

Note that at no point is there any mention of ground or such, all voltages are measured relative to some arbitrary point in your doings, and for that voltage to do anything useful there must be a loop for current to flow [2].

Ground is one of those really crap words that means at least 3 different things in a highly context dependent way, ignore for now.

[1] Electrons in a copper conductor at any sort of current you want to play with move on average really slowly, think less then a mm per second, but a wire is like a tube full of ball bearings, you push one in at one end, one pops out the other far faster then any ball actually moves down the tube.

[2] Yea, I know, flash memory gates, electrostatic lenses, laser printers, all sorts of slight exceptions, but roll with it for now.

Sorry, no. A battery or any power supply must, in the long run, remain electrically neutral. The restoring forces on separated charge are very large, and permanent separation of charge at a circuit scale simply isn't going to happen. This means that if current flows out of one terminal it must flow in through another. About the closest you'll get to an exception is an electret microphone, which contains permanently separated charges - just not a lot.