Why does telnet on a non existent port not directly reject, but time out?

It depends on what the remote end sends back.

For a port where no process is listening, the remote sends a packet with the reset (RST) bit set, which results in the "Connection refused" error on the client. Another possibility is an ICMP "Port unreachable" message, which e.g. Linux's iptables -j REJECT sends by default. That would also result in "Connection refused".

On the other hand, if the remote sends back nothing, then the client can't know what the issue is, and will probably retry and/or wait perhaps a significant amount of time.

An example with iptables on Linux:

# iptables -I INPUT -p tcp --dport 3001 -j REJECT
# iptables -I INPUT -p tcp --dport 3002 -j DROP

$ nc -v 127.0.0.1 3000
nc: connect to 127.0.0.1 port 3000 (tcp) failed: Connection refused

$ nc -v 127.0.0.1 3001
nc: connect to 127.0.0.1 port 3001 (tcp) failed: Connection refused

$ nc -v 127.0.0.1 3002
(waits...)

So, to find out that the backend is not alive, you need to make sure there's someone in there to send an error back. That, of course may be hard to do if the whole host goes down, so you may just have to arrange for a shorter timeout.

The TCP stack decides how to respond to a connection, based on a set of rules (it could be at the firewall level). You can REJECT the connection package (SYN), but you can also DROP it. Dropping it makes sense because of port scanning, for example.