HttpWebRequest is extremely slow!

I was having a similar issue with a VB.Net MVC project.
Locally on my pc (Windows 7) it was taking under 1 second to hit the page requests, but on the server (Windows Server 2008 R2) it was taking 20+ seconds for each page request.

I tried a combination of setting the proxy to null

  System.Net.WebRequest.DefaultWebProxy = Nothing
  request.Proxy = System.Net.WebRequest.DefaultWebProxy

And changing the config file by adding

 <system.net>
   .......
   <connectionManagement>
     <add address="*" maxconnection="20"/>
   </connectionManagement>
 </system.net>

This still did not reduce the slow page request times on the server. In the end the solution was to uncheck the “Automatically detect settings” option in the IE options on the server itself. (Under Tools -> Internet Options select the Connections tab. Press the LAN Settings button)

Immediately after I unchecked this browser option on the server all the page request times dropped from 20+ seconds to under 1 second.

It may have to do with the fact that you are opening several connections at once. By default the Maximum amount of open HTTP connections is set to two. Try adding this to your .config file and see if it helps:

<system.net>
  .......
  <connectionManagement>
    <add address="*" maxconnection="20"/>
  </connectionManagement>
</system.net>

I started observing a slow down similar to the OP in this area which got a little better when increasing the MaxConnections.

ServicePointManager.DefaultConnectionLimit = 4;

But after building this number of WebRequests the delays came back.

The problem, in my case, was that I was calling a POST and not bothered about the response so wasn't picking up or doing anything with it. Unfortunately this left the WebRequest floating around until they timed out.

The fix was to pick up the Response and just close it.

WebRequest webRequest = WebRequest.Create(sURL);
webRequest.Method = "POST";
webRequest.ContentLength = byteDataGZ.Length;
webRequest.Proxy = null;
using (var requestStream = webRequest.GetRequestStream())
{
    requestStream.WriteTimeout = 500;
    requestStream.Write(byteDataGZ, 0, byteDataGZ.Length);
    requestStream.Close();
}
// Get the response so that we don't leave this request hanging around
WebResponse response = webRequest.GetResponse();
response.Close();

What I have found to be the main culprit with slow web requests is the proxy property. If you set this property to null before you call the GetResponse method the query will skip the proxy autodetect step:

request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}

The proxy autodetect was taking up to 7 seconds to query before returning the response. It is a little annoying that this property is set on by default for the HttpWebRequest object.