Download and decompress gzipped file in memory?
For those using Python 3, the equivalent answer is:
import urllib.request
import io
import gzip
response = urllib.request.urlopen(FILE_URL)
compressed_file = io.BytesIO(response.read())
decompressed_file = gzip.GzipFile(fileobj=compressed_file)
with open(OUTFILE_PATH, 'wb') as outfile:
outfile.write(decompressed_file.read())
You need to seek to the beginning of compressedFile
after writing to it but before passing it to gzip.GzipFile()
. Otherwise it will be read from the end by gzip
module and will appear as an empty file to it. See below:
#! /usr/bin/env python
import urllib2
import StringIO
import gzip
baseURL = "https://www.kernel.org/pub/linux/docs/man-pages/"
filename = "man-pages-3.34.tar.gz"
outFilePath = "man-pages-3.34.tar"
response = urllib2.urlopen(baseURL + filename)
compressedFile = StringIO.StringIO()
compressedFile.write(response.read())
#
# Set the file's current position to the beginning
# of the file so that gzip.GzipFile can read
# its contents from the top.
#
compressedFile.seek(0)
decompressedFile = gzip.GzipFile(fileobj=compressedFile, mode='rb')
with open(outFilePath, 'w') as outfile:
outfile.write(decompressedFile.read())