How to download GZip file from S3?
I solved the issue using a Scanner
instead of an InputStream
.
The scanner takes the GZIPInputStream and reads the unzipped file line by line:
fileObj = s3Client.getObject(new GetObjectRequest(oSummary.getBucketName(), oSummary.getKey()));
fileIn = new Scanner(new GZIPInputStream(fileObj.getObjectContent()));
You have to use GZIPInputStream
to read GZIP file
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new ProfileCredentialsProvider())
.build();
String URL = downloadURL.getPrimitiveJavaObject(arg0[0].get());
S3Object fileObj = s3Client.getObject(getBucket(URL), getFile(URL));
byte[] buffer = new byte[1024];
int n;
FileOutputStream fileOuputStream = new FileOutputStream("temp.gz");
BufferedInputStream bufferedInputStream = new BufferedInputStream( new GZIPInputStream(fileObj.getObjectContent()));
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(fileOuputStream);
while ((n = bufferedInputStream.read(buffer)) != -1) {
gzipOutputStream.write(buffer);
}
gzipOutputStream.flush();
gzipOutputStream.close();
Please try this way to download GZip file from S3.