Can I directly stream from HttpResponseMessage to file without going through memory?
It looks like this is by-design - if you check the documentation for HttpClient.GetAsync()
you'll see it says:
The returned task object will complete after the whole response (including content) is read
You can instead use HttpClient.GetStreamAsync()
which specifically states:
This method does not buffer the stream.
However you don't then get access to the headers in the response as far as I can see. Since that's presumably a requirement (as you're getting the file name from the headers), then you may want to use HttpWebRequest
instead which allows you you to get the response details (headers etc.) without reading the whole response into memory. Something like:
public async Task<bool> DownloadFile()
{
var uri = new Uri("http://somedomain.com/path");
var request = WebRequest.CreateHttp(uri);
var response = await request.GetResponseAsync();
ContentDispositionHeaderValue contentDisposition;
var fileName = ContentDispositionHeaderValue.TryParse(response.Headers["Content-Disposition"], out contentDisposition)
? contentDisposition.FileName
: "noname.dat";
using (var fs = new FileStream(@"C:\test\" + fileName, FileMode.Create, FileAccess.Write, FileShare.None))
{
await response.GetResponseStream().CopyToAsync(fs);
}
return true
}
Note that if the request returns an unsuccessful response code an exception will be thrown, so you may wish to wrap in a try..catch
and return false
in this case as in your original example.
Instead of GetAsync(Uri)
use the the GetAsync(Uri, HttpCompletionOption)
overload with the HttpCompletionOption.ResponseHeadersRead
value.
The same applies to SendAsync
and other methods of HttpClient
Sources:
- docs (see remarks) https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient.getasync?view=netcore-1.1#System_Net_Http_HttpClient_GetAsync_System_Uri_System_Net_Http_HttpCompletionOption_
The returned Task object will complete based on the completionOption parameter after the part or all of the response (including content) is read.
.NET Core implementation of
GetStreamAsync
that usesHttpCompletionOption.ResponseHeadersRead
https://github.com/dotnet/corefx/blob/release/1.1.0/src/System.Net.Http/src/System/Net/Http/HttpClient.cs#L163-L168HttpClient spike in memory usage with large response
- HttpClient.GetStreamAsync() with custom request? (don't mind the comment on response, the
ResponseHeadersRead
is what does the trick)