Using rest-client to download a file to disk without loading it all in memory first
Another way is to use raw_response
. This saves directly to a file, usually in /tmp
and handles redirects without a problem.
See Streaming Responses. Here's their example:
>> raw = RestClient::Request.execute(
method: :get,
url: 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-desktop-amd64.iso',
raw_response: true)
=> <RestClient::RawResponse @code=200, @file=#<Tempfile:/tmp/rest-client.20170522-5346-1pptjm1>, @request=<RestClient::Request @method="get", @url="http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-desktop-amd64.iso">>
>> raw.file.size
=> 1554186240
>> raw.file.path
=> "/tmp/rest-client.20170522-5346-1pptjm1"
My original answer promoted passing a block to RestClient::Request#execute
but this only passed data to the block once full response is read. Thus rendering the exercise worthless. This is how to properly do it:
File.open('/tmp/foo.iso', 'w') {|f|
block = proc { |response|
response.read_body do |chunk|
puts "Working on response"
f.write chunk
end
}
RestClient::Request.new(method: :get, url: 'http://mirror.pnl.gov/releases/xenial/ubuntu-16.04-server-amd64.iso', block_response: block).execute
}
It is from the related rest-client project issue.
Note: redirection does not work in this mode as well you lose HTTP exit status, cookies, headers, etc. Hope this is gonna be fixed some day.