nginx automatic failover load balancing

Solution 1:

I think that it's because nginx is not detecting that the upstream is down because it's on the same machine.

The options that you're looking for are: proxy_next_upstream and proxy_connect_timeout.

Try this:

location / {
        proxy_pass              http://lb;
        proxy_redirect          off;
        proxy_next_upstream     error timeout invalid_header http_500;
        proxy_connect_timeout   2;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}

Solution 2:

Hey, please see the wiki: http://wiki.nginx.org/NginxHttpUpstreamModule#server

Basically if a failure is detected the backend will be marked as down for x seconds and it will try again. So if you keep seeing connections it's probably nginx that keeps checking if the backend has become available.

It should, however, try the next entry in the upstream block, so you shouldn't actually see that no backends are available if only one is down.