Why should I use an FQDN instead of the server's IP address?
Solution 1:
Using an IP address ensures that you are not relying on a DNS server. It also has the benefit of preventing attacks through DNS spoofing.
Using a FQDN instead of an IP address means that, if you were to migrate your service to a server with a different IP address, you would be able to simply change the record in DNS rather than try and find everywhere that the IP address is used.
This is especially useful when you have many servers and services configured by multiple individuals.
Solution 2:
DNS is not just FQDN = IP
The important thing about DNS is that it provides more than just A records (hostname = IP). DNS provides different types of records such as MX, CNAME, TXT, etc... that may be required by some software, sometimes. It allows multiple address records, IPv4 + IPv6 records, dynamic addresses, load balancing, geo location based resolution, fail-over/redundancy, etc... DNS tells you what things are (www.google.com is google's web service, 172.217.4.110? What's that?) It allows you to change these settings/records and have them picked up by clients without making changes on all the clients. DNS can do complex things.
There's often a clear advantage to using DNS over a direct IP address.
FQDNs can be a requirement
Some things like web servers that use name based virtual hosting or load balancers, etc... absolutely require that you address them via an FQDN or hostname. They determine how to respond to your request based on the FQDN that you are connecting to. Connecting via an IP may not work at all.
SSL certificates are issued based on domain names, so you may not be able to use some SSL enabled services (properly) without DNS.
This is a dig query for the google.com domain to give you a glimpse into the complexity of DNS
google.com. 299 IN A 172.217.0.174 google.com. 299 IN AAAA 2607:f8b0:400b:807::200e google.com. 599 IN MX 10 aspmx.l.google.com. google.com. 599 IN MX 40 alt3.aspmx.l.google.com. google.com. 59 IN SOA ns2.google.com. dns-admin.google.com. 126990955 900 900 1800 60 google.com. 599 IN MX 30 alt2.aspmx.l.google.com. google.com. 21599 IN NS ns2.google.com. google.com. 599 IN MX 20 alt1.aspmx.l.google.com. google.com. 599 IN MX 50 alt4.aspmx.l.google.com. google.com. 21599 IN NS ns1.google.com. google.com. 3599 IN TXT "v=spf1 include:_spf.google.com ~all" google.com. 21599 IN CAA 0 issue "symantec.com" google.com. 21599 IN NS ns3.google.com. google.com. 21599 IN NS ns4.google.com.
Yahoo responds with 3 IP addresses
$ host -ta yahoo.ca yahoo.ca has address 77.238.184.24 yahoo.ca has address 74.6.50.24 yahoo.ca has address 98.137.236.24
Advantage of using an IP address
For me it's usually when DNS could get in the way somehow or is not available. Generally, I would use DNS for most things.
One example of where an IP address might be better would be when you have two machines with a direct link between them (no switch) with private network addresses (say 192.168.1.1 and 192.168.1.2) and they are using it for high availability communications or DRBD or another very specific service. In this case, setting up things in DNS probably doesn't make any sense. It's not necessary, would add complexity, performance issues and could introduce a point of failure.
Another example is routing. Routing tables record IP addresses for various reasons.
Another is referencing name servers (like in /etc/resolv.conf). Since without a name server, you cannot resolve anything.