Strange performance problem with SQL Server 2016
If your primary wait is ASYNC_NETWORK_IO
, then the issue is not with SQL Server. It is almost always due to an application bottleneck. I don't mean a bottleneck on the application server, but rather a bottleneck in the application.
The application bottleneck is usually because of row-by-row processing while SQL Server is sending the data:
- The application is requesting data from SQL Server
- SQL Server is sending the data fast
- The application is telling SQL Server to wait while it processes each row
- SQL Server records waiting time on
ASYNC_NETWORK_IO
while the application is telling it to wait
Instead of that, the application needs to consume all of the data from SQL Server and THEN do its row-by-row processing. SQL Server is out of the picture at that point.
sp_BlitzFirst
output
The LCK_M_S
wait is not high. Only 2 seconds of the 30-second sample are on it, and its average is only 400ms. That is very, very unlikely to be the problem. ASYNC_NETWORK_IO
is your top wait in that sample. Still an application issue. If you want help with the LCK
stuff, we'd need to see the queries involved.
Even ASYNC_NETWORK_IO
isn't that bad in that sample. My eyes get big when the waiting time is equal to or greater than the sample size. That's when I dig in.
Your entire issue is ASYNC_NETWORK_IO
. This is not a SQL Server problem. It's a problem with either the application (doing row-by-row processing while SQL Server is sending the data), the application server (you already said it's fine) or the network (you've said that the network is fine). So the issue is with the application. The C++ app needs to be fixed.
To answer my own question:
The main reason for ASYNC_NETWORK_IO appearing on our SQL Server as the top wait type, was that the energy saving
setting of the windows server was set to 'balanced'
instead of 'high performance'
. We talked to some vm ware admins afterwards, and they all said, that this setting kills performance.
Solutions for this are either:
- Don't install energy control when installing windows server
- Set energy saving mode to high performance for all server via group policy
All other issues/stats regarding ASYNC_NETWORK_IO are related to our ERP app being badly written. Thanks to all who helped me with solving this problem, your comments, suggestions and advice were very welcome and helpful!