Python boto, list contents of specific dir in bucket
By default, when you do a get_bucket
call in boto it tries to validate that you actually have access to that bucket by performing a HEAD
request on the bucket URL. In this case, you don't want boto to do that since you don't have access to the bucket itself. So, do this:
bucket = conn.get_bucket('my-bucket-url', validate=False)
and then you should be able to do something like this to list objects:
for key in bucket.list(prefix='dir-in-bucket'):
<do something>
If you still get a 403 Errror, try adding a slash at the end of the prefix.
for key in bucket.list(prefix='dir-in-bucket/'):
<do something>
Note: this answer was written about the boto version 2 module, which is obsolete by now. At the moment (2020), boto3 is the standard module for working with AWS. See this question for more info: What is the difference between the AWS boto and boto3
For boto3
import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('my_bucket_name')
for object_summary in my_bucket.objects.filter(Prefix="dir_name/"):
print(object_summary.key)