The first email bounces; the second and subsequent emails go through

Your email timeout could well be due to your daemon trying IPv6 first.

The IPv6 stack implementation by default has priority over the IPv4 stack so when programs/daemons try to communicate they will try first to use the IPv6 address, when the destination has both public IPv4 and IPv6 address.

Even if you do not have public IPv6, you have IPv6 localhost and link local addresses.

It would be not the first time, and probably not the last time, that I do catch Internet daemons trying first the link local address as source IP address to communicate to another address, and only after timing out, if they still have time/tries allotted, might fall back to sending the data to the IPv4 destination. (In the past, I already had DNS and email problems due to this in an ISP I used to run.)

So for exim, you can disable IPv6 at application/daemon level, using the directive disable_ipv6=true in /etc/exim4/exim4.conf.template or /etc/exim4/update-exim4.conf.conf depending on whether you are using the non-split or the split configuration scheme.

From Exim Internet Mailer-Chapter 14 - Main configuration

disable_ipv6 Use: main Type: boolean Default: false
If this option is set true, even if the Exim binary has IPv6 support, no IPv6 activities take place. AAAA records are never looked up, and any IPv6 addresses that are listed in local_interfaces, data for the manualroute router, etc. are ignored. If IP literals are enabled, the ipliteral router declines to handle IPv6 literal addresses.

An alternative approach might be also binding it only to IPv4 addresses, however the disadvantage is having to hardcode the IPv4 address(es) in the configuration:

local_interfaces = <; 127.0.0.1 ; \
                  192.168.23.65

As for the system itself, as you are not actively using IPv6:

add as the last line to give priority by default to IPv4, to the file /etc/gai.conf

precedence ::ffff:0:0/96  100 

Add to /etc/sysctl.conf to disable by default the IPv6 stack (setting supported from kernel 3 onwards):

net.ipv6.conf.all.disable_ipv6=1

The sysctl will be applied at boot time. To activate it before booting, do:

sudo sysctl -p

Whilst they call it an IPv6 deactivation, the module is still loaded, and while the interfaces do not have IPv6 addresses anymore, you can still see applications connected to their IPv6 sockets. You can also pass to the kernel an option to disable IPv6, and the IPv6 kernel module won't be loaded. Editing /etc/default/grub:

GRUB_CMDLINE_LINUX="ipv6.disable=1"

And then to apply it, if you have grub (your grub partition may vary or you might not have it; I do not have it in my ARM servers, and have to edit other file for the kernel options):

sudo update-grub
sudo grub-install /dev/sda

You may have to configure one or other daemon to disable IPv6 at the application level (from the top of my head, xinetd, if you have it installed).

The answer of @RuiFRibeiro is the important one. Read that one first. However, once you have understood that answer, here is some additional information that may help.

If your relay host requires that your laptop authenticate itself before relaying mail (which your relay host probably should do), then you should configure Exim4 on your laptop always to authenticate itself to the relay server.

The relay server is also known as a "smarthost." Whichever name you call it by, here is the problem. The default behavior of your laptop's Exim4 is [i] to try to authenticate itself before transmitting mail, but, if that fails, [ii] to send the mail unauthenticated.

If it falls through to [ii] on your setup, then your mail will bounce, permanently failing. Of course, you would think that it would never fall through to [ii], but apparently in your case, sometimes it does.

The fix is in the transport/30_exim4-config_remote_smtp_smarthost section of your Exim4 configuration (which on my system is in the file /etc/exim4/exim4.conf.template). In this section, look for a stanza that begins:

remote_smtp_smarthost:

In this stanza, you may discover a line that begins:

  hosts_try_auth =

Some things will follow the = sign, but it is the symbol hosts_try_auth that interests you. Change this to read:

  hosts_require_auth =

Change nothing after the = sign.

The effect is to cause outbound mail transfer to fail whenever [i] fails, ensuring that [ii] is never reached. How does this help? It helps because, unlike a failure of [ii], a failure of [i] is temporary, not permanent. If you have made the indicated change, then your system will respond to failure by retrying with authentication, never abandoning without authentication. This is the behavior you want.

HOW TO COMMIT THE CONFIGURATION CHANGE

Just making the change in the configuration file won't fix the problem. You have to commit the change using exim4-config, for example by issuing the obscure-seeming but actually important Debian command dpkg-reconfigure -phigh exim4-config. I do not know whether it is actually necessary to restart the server after committing the configuration change, but I always restart it. To restart, use the command invoke-rc.d exim4 restart. [Incidentally, if you are a beginner, then you might feel more confident rebooting the system to just restart everything. I did that too when I was a beginner, so I don't blame you, but rebooting to restart a single service really isn't the Debian way. You're a bit like a rock climber with good ropes and hardware: eventually, you learn when it's appropriate to just trust your equipment. Someday, rather than always rebooting, you should learn to become comfortable with the use of the significant command invoke-rc.d(8).]

If you wish to learn more about the symbol hosts_require_auth, consult the detailed specification manual that comes with your Exim4 software.

(For information, I am the OP. When I posted the question, I lacked the answer, but after @RuiFRibeiro oriented me I discovered the information this answer gives.)