Effect of SO_SNDBUF

The effect of setting SO_SNDBUF option is different for TCP and UDP.

• For UDP this sets the limit on the size of the datagram, i.e. anything larger will be discarded.
• For TCP this just sets the size of in-kernel buffer for given socket (with some rounding to page boundary and with an upper limit).

Since it looks like you are talking about TCP, the effect you are observing is explained by the socket being in blocking mode, so send(2) blocks until kernel can accept all of your data, and/or the network stack asynchronously de-queueing data and pushing it to the network card, thus freeing space in the buffer.

Also, TCP is a stream protocol, it does not preserve any "message" structure. One send(2) can correspond to multiple recv(2)s on the other side, and the other way around. Treat it as byte-stream.

SO_SNDBUF configures the buffer that the socket implementation uses internally. If your socket is non-blocking you can only send up to the configured size, if your socket is blocking there is no limitation for your call.