HttpResponseMessage Content won't display PDF

After hours of Google searching as well as trial and error, I have resolved the issue here.

instead of setting the content of the response to a StreamContent I have changed it to ByteArrayContent on the Web Api side.

byte[] fileBytes = System.IO.File.ReadAllBytes(pdfLocation);
response.Content = new ByteArrayContent(fileBytes);
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = fileName;
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");

By doing it this way, my MVC 4 application is able to download the PDF using a WebClient and the DownloadData method.

internal byte[] DownloadFile(string requestUrl)
{
    string serverUrl = _baseAddress + requestUrl;
    var client = new System.Net.WebClient();
    client.Headers.Add("Content-Type", "application/pdf");
    return client.DownloadData(serverUrl);
}

The Byte[] array that is returned can easily be converted to a MemoryStream before returning the File for output...

Response.AddHeader("Content-Disposition", "inline; filename="+fileName);
MemoryStream outputStream = new MemoryStream();
outputStream.Write(file, 0, file.Length);
outputStream.Position = 0;
return File(outputStream, "application/pdf");

I hope this can be useful to others as I have wasted a lot of time to get it working.