Set TCP_QUICKACK and TCP_NODELAY

Use TCP_QUICKACK, not TCP_NODELAY

Turning on TCP_NODELAY has similar effects, but can make throughput worse for small writes. If you write a loop which sends just a few bytes (worst case, one byte) to a socket with "write()", and the Nagle algorithm is disabled with TCP_NODELAY, each write becomes one IP packet. This increases traffic by a factor of 40, with IP and TCP headers for each payload. Tinygram prevention won't let you send a second packet if you have one in flight, unless you have enough data to fill the maximum sized packet. It accumulates bytes for one round trip time, then sends everything in the queue. That's almost always what you want. If you have TCP_NODELAY set, you need to be much more aware of buffering and flushing issues. None of this matters for bulk one-way transfers, which is most HTTP today. (I've never looked at the impact of this on the SSL handshake, where it might matter.) Short version: set TCP_QUICKACK. If you find a case where that makes things worse, let me know. John Nagle

https://news.ycombinator.com/item?id=10608356

There's no direct relationship between those two options, they are just for different purposes.

TCP_NODELAY is intended to disable/enable segment buffering so data can be sent out to peer as quickly as possible, so this is typically used to improve network utilisation. TCP_QUICKACK is used to send out acknowledgements as early as possible than delayed under some protocol level exchanging, and it's not stable/permanent, subsequent TCP transactions (which may happen under the hood) can disregard this option depending on actual protocol level processing or any actual disagreements between user setting and stack behaviour.

NOTE TCP_NODELAY is portable while TCP_QUICKACK is not (only works under Linux 2.4.4+).

TCP_QUICKACK and TCP_NODELAY affect different operations in TCP. The tcp(7) man page describes which socket options for TCP interfere with each other, e.g. TCP_CORK and TCP_NODELAY.

Short answer

• To disable Nagle's buffering algorithm, use the TCP_NODELAY socket option.
• To disable Delayed ACKs, use the TCP_QUICKACK socket option.

Details

• Nagle's algorithm

  • Nagle's algorithm, named after its creator John Nagle, is one mechanism for improving TCP efficiency by reducing the number of small packets sent over the network.
  • The goal was to prevent a node from transmitting many small packets if the application delivers data to the socket rather slowly.
  • If a process is causing many small packets to be transmitted, it may be creating undue network congestion. This is especially true if the payload of a packet is smaller than the TCP header data.

• Delayed ACK

  • TCP delayed acknowledgment or Delayed ACK is another technique used by some implementations of the TCP in an effort to improve network performance and reduce congestion.
  • Delayed ACK was invented to reduce the number of ACKs required to acknowledge the segments and reduce the protocol overhead.
  • Delayed ACK means TCP doesn't immediately acknowledge every single received TCP segment. Several ACK responses may be combined together into a single response, reducing protocol overhead.

• Nagle's Algorithm and Delayed ACK Do Not Play Well Together in a TCP/IP Network

  • Delayed ACK tries to send more data per segment if it can. But part of Nagle's algorithm depends on an ACK to send data.
  • Nagle's algorithm and Delayed ACKs together create a problem because Delayed ACKs are waiting around to send the ACK while Nagle's is waiting around to receive the ACK

• How can I resolve the issues caused by Nagle's algorithm and Delayed ACKs

  • Enable TCP_NODELAY to disable Nagle's algorithm via global socket options on the servers
  • Make profile tweaks on proxy servers and Load Balancers: This is especially relevant if you're running applications or environments that only sometimes have highly interactive traffic and chatty protocols. By dynamically switching Nagle's Algorithm and TCP_NODELAY on and off at the load balancer level, you can keep even highly heterogeneous traffic mixes running optimally.
  • Reduce the Delayed ACK timer on your servers and load balancers. Sometimes, this kind of optimization is handled in software, at the application level, but when that's not the case, you may still be able to dynamically manage the ACK timer at the server or load balancer level.
  • As you're making these changes, keep careful watch on your network traffic and see how each tweak impacts congestion.

For more details please refer to this