What are the Windows A: and B: drives used for?
Short Version: A: & B: are reserved by floppy disk drives, so C: is used by hard drives for backwards compatibility reasons.
Once upon a time, the early CP/M and IBM PC style computers had no hard drive. You had one floppy drive, and that was it. Unless you spent another $1k or so on a second floppy drive, then your system was smokin'! If you only had one drive it was common to boot from one disk, put in the other disk with your programs and data, then run the program. Once the program finished, the computer would request that you reinsert the boot disk so you could use the command line again. Copying data from one disk to the other was a series of
Please insert source disk into drive A:...
Please insert destination disk into drive A:...
Please insert source disk into drive A:...
By the time hard drives became cheap, the "expensive" computers typically had two floppy drives (one to boot and run common programs, one to save data and run specific programs). And so it was common for the motherboard hardware to support two floppy drives at fixed system addresses. Since it was built into the hardware, it was thought that building the same requirement into the OS was acceptable, and any hard drives added to the machine would start with disk C: and so forth.
During the transition from 5.25" disks (which were actually, physically floppy) to 3.5" disks (which were encased in a harder plastic shell) it was common to have both drives in one system, and again it was supported on the motherboard with hardware, and in the OS at fixed addresses. As very few systems ran out of drive letters, it was not thought to be important to consider making those drives re-assignable in the OS until much later when drives were abstracted along with addresses due to the plug'n'play standard.
A lot of software was developed since that time, and unfortunately much of it expected to see long-term storage on the C: drive. This includes the BIOS software that boots the computer. You can still attach two floppy drives, boot into DOS 6.1, and use it as you would have in the early 90's, with floppy drives A: and B:.
So largely the reason for starting the hard drive at C is for backwards compatibility. While the OS has abstracted data storage to some degree, it still treats A: and B: differently, in such a way that allows them to be removed from the system without altering the OS, caching them differently, and due to early viruses treating their boot sector with more caution than the hard drive's boot sector.
For Windows specifically, it's worth mentioning that you can use A: and B: as the names for volumes, be it a flash drive or an internal hard drive.
A: and B: were used for floppy disks.
Edit: Someone asked for pics so here's an 8", 5.25" and 3.5" floppy disk. (8" disks were not used as standard on IBM compatible personal computers.)
Less an answer, more of an anecdote. In this Microsoft article, it says:
"You can assign the letters C through Z to each drive on your computer. A and B are usually reserved for floppy disk drives, but if your computer does not have floppy disk drives, you can assign A and B to volumes."
So when I built a new computer recently with two internal drives, one for the OS and one for data, I thought, hey!, I'll make my data drive "A". I felt all rebellious until I discovered that Windows will not index drives lettered A or B. :( Took me quite a while to figure out what the problem was, but I found some other people who suffered the same issue when they used A or B for a drive. As soon as I assigned that drive a different letter, windows indexed the drive. So much for being rebellious. lol.