TCP - What if client call close() before server accept()

Generally speaking, if a client calls close(), the clients protocol stack will send a FIN to indicate that the client is done sending, and will wait for the server to send a FIN,ACK back to the client (which won't happen before the server accepts the connection, as we shall see), and then the client will ACK that. This would be a normal termination of a TCP connection.

However, since a TCP connection consists of two more or less independent streams, sending a FIN from the client really is only a statement that the client is done sending data (this is often referred to as "half closed"), and is not actually a request at the TCP protocol level to close the connection (although higher level protocols often will interpret it that way, but they can only do so after the connection has been accepted and they have had a read return 0 bytes in order to learn that the client is done writing). The server can still continue to send data, but since the client has called close(), it is no longer possible for this data to be delivered to the client application. If the server sends further data, the protocol stack on the client will respond with a reset, causing an abnormal termination of the TCP connection. If the client actually wished to continue receiving data from the server after declaring that it was done sending data, it should do so by calling shutdown(sock,SHUT_WR) rather than calling close().

So what this means is that the connections that time out and that are normally closed by clients will generally remain active at the server, and the server will be able to accept them, read the request, process the request, and send the reply and only then discover that the application can no longer read the reply when the reset is returned from the client. The reason I say "generally" is that firewalls, proxies, and OS protocol stacks all place limits on how long a TCP connection can remain in a half closed state, generally in violation of the relevant TCP RFCs but for "valid" reasons such as dealing with DDOS.

I think your concern is that a server that is overloaded will be further overloaded by clients timing out and retrying, which in my view is correct based on my preceding explanation. In order to avoid this, a client timing out could set SO_LINGER to 0 prior to calling close() which would cause a reset to be sent to cause an immediate abnormal termination. I would also suggest using an exponential back-off on timeout to further mitigate the impact on an overloaded server.

Once the 3way handshake is complete, the connection is in an ESTABLISHED state. On the client side, it can start sending data immediately. On the server side, the connection is placed in a state/queue that accept() can then pull from so the application can use the connection (see How TCP backlog works in Linux).

If the server doesn't accept() the connection, the connection is still ESTABLISHED, it's inbound buffer will simply fill up with whatever data the client sends, if any.

If the client disconnects before accept() is called, then the connection still enters the CLOSED state, and will be removed from the queue that accept() pulls from. The application will never see the connection.