Throttling S3 commands with aws cli

The AWS CLI S3 transfer commands (which includes sync) have the following relevant configuration options:

• max_concurrent_requests -
  • Default: 10
  • The maximum number of concurrent requests.
• multipart_threshold -
  • Default: 8MB
  • The size threshold the CLI uses for multipart transfers of individual files.
• multipart_chunksize -
  • Default: 8MB
  • When using multipart transfers, this is the chunk size that the CLI uses for multipart transfers of individual files.

This isn't so granular as throttling packets per second, but it seems like setting a lower concurrent request value and lowering both multipart threshold and chunksize will help. If the values you pasted are close to average, I would start with these values and tweak until you're reliably not exceeding the limits anymore:

$ aws configure set default.s3.max_concurrent_requests 8
$ aws configure set default.s3.multipart_threshold 6MB
$ aws configure set default.s3.multipart_chunksize 6MB

As well as changing the max default connections and chunk size already mentions you can also set the max_bandwidth. This is very effective when uploading large single files.

aws configure set default.s3.max_bandwidth 50MB/s