Check file size on S3 without downloading?
Node.js example:
const AWS = require('aws-sdk');
const s3 = new AWS.S3();
function sizeOf(key, bucket) {
return s3.headObject({ Key: key, Bucket: bucket })
.promise()
.then(res => res.ContentLength);
}
// A test
sizeOf('ahihi.mp4', 'output').then(size => console.log(size));
Doc is here.
This is a solution for whoever is using Java and the S3 java library provided by Amazon. If you are using com.amazonaws.services.s3.AmazonS3
you can use a GetObjectMetadataRequest
request which allows you to query the object length.
The libraries you have to use are:
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3 -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.11.511</version>
</dependency>
Imports:
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.*;
And the code you need to get the content length:
GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest(bucketName, fileName);
final ObjectMetadata objectMetadata = s3Client.getObjectMetadata(metadataRequest);
long contentLength = objectMetadata.getContentLength();
Before you can execute the code above, you will need to build the S3 client. Here is some example code for that:
AWSCredentials credentials = new BasicAWSCredentials(
accessKey,
secretKey
);
s3Client = AmazonS3ClientBuilder.standard()
.withRegion(clientRegion)
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.build();
You can simply use the s3 ls
command:
aws s3 ls s3://mybucket --recursive --human-readable --summarize
Outputs
2013-09-02 21:37:53 10 Bytes a.txt
2013-09-02 21:37:53 2.9 MiB foo.zip
2013-09-02 21:32:57 23 Bytes foo/bar/.baz/a
2013-09-02 21:32:58 41 Bytes foo/bar/.baz/b
2013-09-02 21:32:57 281 Bytes foo/bar/.baz/c
2013-09-02 21:32:57 73 Bytes foo/bar/.baz/d
2013-09-02 21:32:57 452 Bytes foo/bar/.baz/e
2013-09-02 21:32:57 896 Bytes foo/bar/.baz/hooks/bar
2013-09-02 21:32:57 189 Bytes foo/bar/.baz/hooks/foo
2013-09-02 21:32:57 398 Bytes z.txt
Total Objects: 10
Total Size: 2.9 MiB
Reference: https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html
Send an HTTP HEAD request to the object. A HEAD request will retrieve the same HTTP headers as a GET request, but it will not retrieve the body of the object (saving you bandwidth). You can then parse out the Content-Length header value from the HTTP response headers.