Understanding “408 Request Timeout” on Apache with PHP
After some studies on the subject I came to the following answer. It is provided by our lead developer and I think it gives good explanation of the subject.
These errors are perfectly normal. They aren't a sign of a larger issue, but normal connections that are holding Apache open for longer than allowed.
For example, client's queries running them over and over kept Apache open. Apache responded by shutting him down appropriately.
If it hadn't, than a handful of people could take over our server and not allow anyone else to connect.
Most often these errors are coming from systems looking for exploits, and you can recreate it by opening a telnet session and leaving it open.
At the same time, tail -f the access log, and within X time (KeepAliveTimeout) you'll see your IP popup with the same error codes.
Back in the days of Apache 1.3, this error was common, but then 2.2 came out and they had it removed until enough of us asked for it to be returned since it give us ideas on how many people are holding open just the port, and not requesting an actual resource, etc.
I think nothing else should be done here except to make sure to set Timeout to some reasonable value as I have described in original question.
Actually a lot of 408 messages in apache logs are a result pre-fetching mechanism in modern browsers. From looking in apache logs in the last 3 years the amount of 408 errors had more than doubled for the same traffic.