What enables Cloudflare to disable direct IP address access?
There's nothing special in the cloudflare setup. This is just a property of HTTP.
When a client opens a URL, there are three important steps:
- If required, it makes a DNS (or other resolution method) to turn a hostname into an IP address. If the URL specifies an IP address for the host, use that.
- It makes a connection to that IP address on a well-known port number, normally 80 (unless it's overridden in the URL)
- It asks the server for the page, including the desired hostname.
A classical example looks like this:
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.w3.org
Consider a large host with many web sites on it. For simplicity let's say it has a single IP address. Hundreds of domain names resolve to this address. How does the server decide which pages to deliver? It uses the host detail given by the client in the HTTP request. If you ask for something it doesn't have or want to give you, it will give you an error response.
In your case, the request contains an IP address for the host specifier.
GET /whatever HTTP/1.1
Host: a.b.c.d
Very many hosts decide not to give out pages when the host is specified by IP address. There's nothing special about Cloudflare here, nor is it to do with DNS. It's about how the server responds to requests for the host specified by IP address, and you can see that this error message specifies that A valid Host header must be supplied.
Here's an answer which describes how to configure a server in this way: https://serverfault.com/a/607222
You can easily verify this kind of behaviour by using telnet to connect to a server and issue the HTTP request manually.
PS. The same general answer applies to an HTTPS request, but using Server Name Indication in the setup. It's worth noting that Host came in with HTTP 1.1 (1997). Prior to that, the mechanism described here didn't exist, and a server had no way to reliably tell if the client had asked for a name which legitimately resolved to its IP address, or had asked for the host by IP address directly. As this was an important development for the explosive growth in web sites, many older clients were updated to send Host. [Thanks commenters for picking up on details.]