Why does computer name change require a reboot in Windows?

You don't have to restart immediately after renaming the machine... just don't expect every service and function to pick up on the new name.

Things such as system environment variables are read once, at system startup. If you change those variables, the various components of Windows won't pick up on them until they're restarted. Some of those components are so deeply integrated with the OS, you might as well restart the entire OS.

Many Windows services and applications, both those written by Microsoft as well as 3rd party, that use the system's hostname, typically only read it once at startup and never read it again.

I know if I was writing an application that read the machine's hostname, it would be pretty silly of me to periodically poll for it just in case it changed.

In an operating system such as Linux, you see the same thing. You can change the hostname without rebooting, but you do have to restart some very basic components of the system in order to get them to pick up on the new name. Linux is more modular than Windows, although Windows has come a long way in terms of modularity.

One way to detect whether a Windows system is pending a computer rename operation is to check the registry. If the contents of

HKLM\SYSTEM\CurrentControlSet\Control\ComputerName\ComputerName

and

HKLM\SYSTEM\CurrentControlSet\Control\ComputerName\ActiveComputerName

are not the same, that means the system has a pending rename operation that will complete the next time the system reboots.