ServicePointManager.DefaultConnectionLimit in Worker Role

We hit a problem similar to this on our team. By default, the default number of outgoing connections per domain is set to 2 by default. This limits the number of concurrent connections that you can have and can cause perform issues. I'd take a look at the blog below, which goes into some detail on the problem.

https://docs.microsoft.com/en-us/archive/blogs/jpsanders/understanding-maxservicepointidletime-and-defaultconnectionlimit

In terms of adjusting the number of instances, while can scale out the number of instances that are running inside of Azure, simply scaling out will not address bottlenecks on a single instance.

This setting most directly applies to using Windows Azure storage. You have a single host endpoint (e.g. .table.windows.core.net) that will be limited to only 2 connections if you do not set this policy. However, if you know about Windows Azure storage then you know you achieve scale by parallelizing requests. You want many simultaneous connections in this case. In our environment, where we have extreme partitioning and many IO bound operations, we have this limit around 100, IIRC.