Why would an IIS hosted site prompt for AD account credential if accessed through a hostname or IP, but not through servername?

This looks like a Kerberos issue; if I'm correct, then the server's AD computer account(*) has a registered SPN for HTTP/SERVERNAME (this can be verified with the SETSPN command-line tool), thus automatic Kerberos authentication can happen when the web service is called using the computer's name; however, when calling the web service with any other name, this won't work.

If you want to be able to call the web service using a different name, then you need to add another SPN to the same AD computer account(*), with a command such as SETSPN -S HTTP/service.company.com SERVERNAME.

More info here.

(*) Or the user account which runs IIS's application pool, if you manually configured one.

Service.company.com would need to be a registered Service Principal Name on the IIS server.

Also, Kerberos integrated authentication by default will not work with an IP address unless the system is configured to do so, and the IP address is a registered Service Principal Name.

https://docs.microsoft.com/en-us/windows-server/security/kerberos/configuring-kerberos-over-ip

One other check - it is possible your browser isn't passing credentials, forcing the login prompt.

In Internet Options, there is a security zone for Local Intranet; this zone will automatically pass your Windows credentials without the need for the AD dialog. The default setting will detect the server name as part of your network, but the full domain name is probably considered Internet zone and doesn't send the credentials. In the configuration page for Local Intranet, you can add the FQDN to no longer be prompted (or set it via Group Policy).

This all assumes you intend the site to run with Windows Credentials, and you are running internally; I'm not a security expert but personally, I wouldn't want to send my credentials automatically to a site across the internet.