How much space does my little file take up, exactly?

As Christopher points out this question is very similar to Why is a text file taking up at least 4kB even when there's just one byte of text in it?

I'm not sure if I personally class it as a duplicate or not.

But where does this 1KB come from

This is more commonly 4KB

File systems are allocated in blocks of bytes (AKA allocation units) not individual bytes. So to store a single byte in a file, that file will need an entire bock. This means the rest of the block is left blank, but no other file can use it.

The origin of this number is unclear, but there's a number of things it fits with. For example at a low level, it's not possible to write single bytes to disk, you can only write a block of them. Modern HDs and even SSDs often have a 4KB limit. Meaning that if you want to write one byte, you must first load 4KB, change that 1 byte and write the entire block back. If you attempt to write a whole block, there is no need to read it's original contents. So file systems aligned to hardware limits are much more efficient.

As Stephen Kitt points out, 4KB is the maximum block size supported for ext3 by many kernels. (Also discussed here). In general larger block sizes have more efficient access times meaning "larger blocks are better".

does it differ across filesystems (this is ext4)

Once apon a time 512 was a common block size, and this figure still comes up occasionally as a default value. Tar files are very old and have this same 512 byte block size (presumably in an attempt to align with the file systems and hardware making disk writes very fast). As such tar files are still very wasteful with very small files (<512 bytes)

It's much more common now to have file systems that are 4KB aligned (not 1KB).

And yes, file systems can be configured when you format them to use a different block size. Different file systems have different limits but most can be configured.

and does a 1.01 KB file take up 2KB?

Assuming a 1KB block size, yes that correct.