Background mailing through sidekiq and devise-async, queue fills up but mails aren't processed

The answer is rather simple. You just need to tell sidekiq to use the mailer queue, by starting sidekiq with bundle exec sidekiq -q mailer. That way the mailer queue will be processed, without the option sidekiq will simply rely on the default queue.