Servers fail to bind to addresses at boot

It may be better to not configure system services to listen on specific IP addresses, and to control access to them via the host firewall if necessary.

If you really need to be able to bind to specific IP addresses before they are configured on a network interface, you can work around the timing issue by setting the sysctl net.ipv4.ip_nonlocal_bind for IPv4 and the sysctl net.ipv6.ip_nonlocal_bind for IPv6. Services can then bind to IP addresses not configured on any network interface, but they will not be accessible until those IP addresses are configured on an interface.


If you're using NetworkManager, then in order for network-online.target to work as expected, you need to enable service NetworkManager-wait-online.service, which is the one that actually waits for the network to be online to satisy that target.

The network-online.target needs to be "hooked" into your network manager (since NetworkManager is not the only alternative, there is also systemd-networkd which can be used to manage the network.)

For network-online.target to work with NetworkManager, you need to have a symlink under /etc/systemd/system/network-online.target.wants/ pointing to /usr/lib/systemd/system/NetworkManager-wait-online.service.

Which you can actually create by enabling that service:

$ sudo systemctl enable NetworkManager-wait-online.service
Created symlink from /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service to /usr/lib/systemd/system/NetworkManager-wait-online.service.

Once that's in place, dependencies on network-online.target should start working, waiting until NetworkManager is done bringing up all interfaces it's supposed to bring up at boot.

To help diagnose any issues with that setup, you might want to look at output of systemctl status network-online.target and systemctl status NetworkManager-wait-online.service as well, as they might have more clues about what is going on. (In particular, the timestamps might be helpful, if the daemons that depend on network-online.target are starting before NetworkManager-wait-online.service is finished, then you might have an issue with your configuration.)


Of the solutions you listed, I'd recommend this one:

# mkdir /etc/systemd/system/sshd.service.d
# tee /etc/systemd/system/sshd.service.d/wait.conf << 'EOF'
[Unit]
Wants=network-online.target
After=network-online.target
EOF

Since network-online.target is the one you actually want (to ensure all IPs are up, etc.) and including Wants= makes sure its startup will be requested.

From the other methods, this one won't work: systemctl add-wants multi-user.target network.target, since it's not creating any dependencies between the services themselves (SSH daemon, etc.) and the network being fully up. It's just saying you want the network to be up...

And the one involving the /etc/systemd/system/sshd.service.requires/ directory is missing the After= dependency (which I believe is essential, and not implied by just having a .requires/ on it.) If you think Requires= is better than Wants= (it's stronger, causes the unit to fail if the dependency fails), then I'd recommend just using that in /etc/systemd/system/sshd.service.d/wait.conf instead, the override file is definitely a more flexible way to manage this configuration.

Adding a dependency on sys-subsystem-net-devices-eth1.device doesn't help either, since that only indicates that the device exists (from the point of view of udev), which says nothing about it being up and configured yet. So that's not an option either.

Tags:

Systemd

Rhel7