IP ADDRESS request vs. DOMAIN NAME request?
I think I found my answer.
So, the thing is, because I bought a shared hosting package, they also gave me a shared IP. My guess is, this IP has many websites next to mine and they're all accessible with this same shared IP address.
The trick is the HTTP GET request's Host header!
When I entered the shared IP directly in the address bar, the Host header is the same as the shared IP address. So the host doesn't know which shared website to display as a response.
And when I edited Windows's Hosts file and then examined the HTTP GET request, I saw that the Host header is set to my domain name, even though I requested the same shared IP! So this time, the host knows which website that this request belongs to!
Yes, it is very unusual to get a dedicated website IP address these days unless you specifically ask for one, and usually at a cost.
Typically the hosts node apache/nginx server will run a default config page on the public IP address of the node and create "vhosts" for each website they need to run (generally managed by something like cpanel).
The webserver responds appropriately to the request it receives, so if the request is jazzhands.com it will check if it hosts that site and then serve it up.
You'll need to create an A record in your DNS zone and point it to the public IP address of the node to make this work for other users.
Sometimes hosts will try and convince you to just set your domains Nameservers to their infrastructure "to make it easier" but I generally advise against this as the name servers provided by reputable domain registrars tend to be more reliable and remain much more under your control.