Read AWS s3 File to Java code
The 'File' class from Java doesn't understand that S3 exists. Here's an example of reading a file from the AWS documentation:
AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider());
S3Object object = s3Client.getObject(new GetObjectRequest(bucketName, key));
InputStream objectData = object.getObjectContent();
// Process the objectData stream.
objectData.close();
In 2019 there's a bit more optimal way to read a file from S3:
private final AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard().build();
private Collection<String> loadFileFromS3() {
try (final S3Object s3Object = amazonS3Client.getObject(BUCKET_NAME,
FILE_NAME);
final InputStreamReader streamReader = new InputStreamReader(s3Object.getObjectContent(), StandardCharsets.UTF_8);
final BufferedReader reader = new BufferedReader(streamReader)) {
return reader.lines().collect(Collectors.toSet());
} catch (final IOException e) {
log.error(e.getMessage(), e)
return Collections.emptySet();
}
}
If the file's content is a string, then you can use getObjectAsString. Otherwise, you can use IOUtils.toByteArray on getObjectContent()
to read the file's content into a byte array.
Obviously, these are best used on small-ish S3 objects that will easily fit into memory.
private final AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard().build();
private String loadStringFromS3() {
try {
return amazonS3Client.getObjectAsString(BUCKET_NAME, FILE_NAME);
} catch (final IOException e) {
log.error(e.getMessage(), e)
return null;
}
}
private byte[] loadDataFromS3() {
try (final S3Object s3Object = amazonS3Client.getObject(BUCKET_NAME, FILE_NAME)) {
return IOUtils.toByteArray(s3Object.getObjectContent());
} catch (final IOException e) {
log.error(e.getMessage(), e)
return null;
} finally {
IOUtils.closeQuietly(object, log);
}
}