"No buffer space available" on connect

Well, I don't know what exactly the problem is, but I'll try to figure out the right direction for solving it.

The ENOBUFS code is returned when either sk_alloc() or dst_alloc() is failed. I can't find any other occurrences of ENOBUFS in the source code related to the sockets.

Also I can't find any paths from the SYSCALL_DEFINE3(connect) to sk_alloc(), and I think the socket should not be allocated during connect() call where you get the error, so I think it is not likely that the sk_alloc() caused the problem.

The dst_alloc() is likely used for checking routes during the connect(), I can't find the exact path to it, it must be somewhere inside: SYSCALL_DEFINE3(connect) -> .connect() -> ip4_datagram_connect() -> ip_route_connect()

The dst_alloc() allocates an entry in a corresponding SLAB cache, and it may actually fail if the cache is full. Actually old entries should be purged if that happens, but perhaps there are cases when it still returns an error.

So I think you can move to this direction. The dst cache size may changed through /proc/sys/net/ipv4/route/max_size. First, check if the setting (or any other settings in sys.net.ipv4.route) is changed by "random TCP tweaks shown on Google".

In the kernels prior to 3.6, you could've been hit by ENOBUFS for the regular IPv4/v6 traffic, when the net.ipv4.route.max_size or net.ipv6.route.max_size limit was depleated, accordingly.

Starting with the kernel 3.6, routing cache was removed, and net.ipv4.route.max_size lost it's influence on amount of dst entries. So, generally, that would no longer be possible.

However, you can still run into this error like myself, when using IPSec. After certain amount of created IPSec tunnels, I was unable to ping the remote host:

# ping 10.100.0.1
connect: No buffer space available

ping from iputils creates test file descriptor, and uses connect() on it, to bind the dst ip. When it happens, dst cache entry for the given AF is created by the kernel, and xfrm4 dst cache entries limit was already depleated in my case. This limit is controlled by sysctl setting:

xfrm4_gc_thresh - INTEGER
    The threshold at which we will start garbage collecting for IPv4
    destination cache entries.  At twice this value the system will
    refuse new allocations.

I had run into this using kernel 3.10.59, where default limit is very low - 1024. Starting from kernel 3.10.83, this limit was increased to 32768, and would be much harder to hit.

So, I issued:

# sysctl net.ipv4.xfrm4_gc_thresh=32768

and it did the thing for me.

Approximate path in kernel for my case with IPSec:

ip4_datagram_connect() -> ip_route_connect() -> ip_route_output_flow() ->
xfrm_lookup() -> xfrm_resolve_and_create_bundle() ->
... -> xfrm_alloc_dst() -> dst_alloc() with xfrm4_dst_ops, where gc is set.