Get all s3 buckets given a prefix
The high level collection s3.buckets.filter(Filters=somefilter)
only work for ways that document under describe_tags
Filters (list). In such case, you MUST tag your bucket (s3.BucketTagging
) before you can use the very specific filtering method s3.buckets.filter(Filters=formatted_tag_filter)
(http://boto3.readthedocs.org/en/latest/reference/services/ec2.html#EC2.Client)
IMHO, tagging is a MUST if you plan to manage any resource inside AWS.
Currently, you can do this
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
if bucket.name.startswith("myapp-"):
print bucket.name
And following is example code given to filter out KEYS (not bucket) (http://boto3.readthedocs.org/en/latest/guide/collections.html)
# S3 list all keys with the prefix '/photos'
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
if bucket.name.startswith("myapp-") :
for obj in bucket.objects.filter(Prefix='/photos'):
print('{0}:{1}'.format(bucket.name, obj.key))
And there is a warning note using the above example :
Warning Behind the scenes, the above example will call
ListBuckets
,ListObjects
, andHeadObject
many times. If you have a large number of S3 objects then this could incur a significant cost.
When you retrieve a list of buckets from the S3 service, you're using the GET /
operation on the S3 service.
Docs: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTServiceGET.html
This function does not take any request parameters, so there is no filtering done server-side.
If you want to filter based on your desired prefix, you'll need to retrieve the entire list of buckets, then filter it yourself.