Why is setting Nginx as a reverse proxy a good idea?

Solution 1:

I'm going to focus on slow client behavior, and how your configuration handles it, but don't be tempted to believe that is the only benefit. The same method that benefits slow clients also has benefits for fast clients, SSL handling, dealing with traffic surges, and other aspects of serving HTTP on the Internet.

Gunicorn is pre-forking software. For low latency communications, such as load balancer to app server or communications between services, pre-fork systems can be very successful. There is no cost in spinning up a process to handle the request, and a single process can be dedicated to handling a single request; the elimination of these things can lead to an overall faster, more efficient system until the number of simultaneous connections exceeds the number of available processes to handle them.

In your situation, you are dealing with high latency clients over the internet. These slow clients can tie up those same processes. When QPS matters, the application code needs to receive, handle, and resolve the request as quickly as possible so it can move on to another request. When slow clients communicate directly with your system, they tie up that process and slow it down. Instead of handling and disposing of the request as quickly as possible, that process now also has to wait around for the slow client. Effective QPS goes down.

Handling large numbers of connections with very little cpu and memory cost is what asynchronous servers like Nginx are good at. They aren't affected in the same negative manner by slow clients because they are adept at handling large numbers of clients simultaneously. In Nginx's case, running on modern hardware it can handle tens of thousands of connections at once.

Nginx in front of a pre-forking server is a great combination. Nginx handles communications with clients, and doesn't suffer a penalty for handling slow clients. It sends requests to the backend as fast as the backend can handle those requests, enabling the backend to be as efficient with server resources as possible. The backend returns the result as soon as it calculates it, and Nginx buffers that response to feed it to slow clients at their own pace. Meanwhile, the backend can move on to handling another request even as the slow client is still receiving the result.

Solution 2:

@blueben is right. A specific and common example of what can happen when a reverse proxy is not used is that a backend database can run out database connection handles where there's no proxy and there's a traffic spike. This is due to the connections being slow to release as @blueben described.

A first instinct to seeing database handles running out might be to support more database connections. But by adding a reverse proxy in front of the app you'll see the number of required database connections for high load both drop significantly and stabilize-- the database connection level won't spike nearly as much when there's a traffic spike.

Nginx is also great at serving static content, caching and variety of other HTTP tasks, letting your app server focus on being an app server.