Is boto3.Bucket.upload_file blocking or non-blocking?
The current boto3 upload_file
is blocking. As mootmoot said, you should definitely implement some error handling to be safe if you delete the file.
Whether blocking or unblocking, you SHOULD NOT rely on the API alone when things went bad. You MUST add exception handling if the upload fail in the middle for any reason(e.g. admin decide to restart the router when you doing the upload).
bucket = session.Bucket(bucket_name)
try :
bucket.upload_file(Key=s3_key, Filename=source_path)
os.remove(source_path)
except :
raise
Another good practice to upload file to S3 is adding additional Metadata.
bucket.upload_file(
Key=s3_key,
Filename=source_path,
extra_args={'Metadata': {'source_path': source_path}}
)
Adding event to S3 Bucket to act on success PUT action also let you create cleanup process if there is success upload but failure on local file removal.(imagine the file is locked or the file is given Read-only access).