Boto3: grabbing only selected objects from the S3 resource

If we just need list of object-keys then, bucket.objects.filter is a better alternative to list_objects or list_object_v2, as those functions have limit of 1000 objects. Reference: list_objects_v2

Use the filter[1], [2] method of collections like bucket.

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
objs = bucket.objects.filter(Prefix='myprefix')
for obj in objs:
    pass

For folks using boto3.client('s3') rather than boto3.resource('s3'), you can use the 'Prefix' key to filter out objects in the s3 bucket

import boto3

s3 = boto3.client('s3')

params = {
    "Bucket": "HelloWorldBucket",
    "Prefix": "Happy"
}

happy_objects = s3.list_objects_v2(**params)

The above code snippet will fetch all files in the 'Happy' folder in the 'HelloWorldBucket'.

PS: folder in s3 is just a construct and is implemented as a prefix to the file/object name.