How to download .gz files with requests in Python without decoding it?
As discussed in the comments above, this seems to have solved the issue:
From the docs for the requests
module:
Requests automatically decompresses gzip-encoded responses ... You can get direct access to the raw response (and even the socket), if needed as well.
Searching the docs for "raw responses" yields requests.Response.raw
, which gives a file
-like representation of the raw response stream.
import requests
r = requests.get(url, stream=True)
with open(local_filename, 'wb') as f:
for chunk in r.raw.stream(1024, decode_content=False):
if chunk:
f.write(chunk)
This way, you will avoid automatic decompress of gzip-encoded response, save it to file as it's received from web server, chunk by chunk.