Celery with Amazon SQS

I'm using Celery 3.0 and was getting deprecation warnings when launching the worker with the BROKER_USER / BROKER_PASSWORD settings.

I took a look at the SQS URL parsing in kombo.utils.url._parse_url and it is calling urllib.unquote on the username and password elements of the URL.

So, to workaround the issue of secret keys with forward slashes, I was able to successfully use the following for the BROKER_URL:

import urllib
BROKER_URL = 'sqs://%s:%s@' % (urllib.quote(AWS_ACCESS_KEY_ID, safe=''),
                               urllib.quote(AWS_SECRET_ACCESS_KEY, safe=''))

I'm not sure if access keys can ever have forward slashes in them but it doesn't hurt to quote it as well.

I ran into this question several times but still wasn't entirely sure how to setup Celery to work with SQS. It turns out that it is quite easy with the latest versions of Kombu and Celery. As an alternative to the BROKER_URL syntax mentioned in another answer, you can simply set the transport, options, user, and password like so:

BROKER_TRANSPORT = 'sqs'
BROKER_TRANSPORT_OPTIONS = {
    'region': 'us-east-1',
}
BROKER_USER = AWS_ACCESS_KEY_ID
BROKER_PASSWORD = AWS_SECRET_ACCESS_KEY

This gets around a purported issue with the URL parser that doesn't allow forward slashes in your API secret, which seems to be a fairly common occurrence with AWS. Since there didn't seem to be a wealth of information out there about the topic yet, I also wrote a short blog post on the topic here:

http://www.caktusgroup.com/blog/2011/12/19/using-django-and-celery-amazon-sqs/