Celery &Rabbitmq:WARNING/MainProcess] Received and deleted unknown message. Wrong destination?!?- a experiment on the GIT
It would be helpful to give the versions of celery and librabbitmq you are using. Since I had a very similar problem, I'll guess that you are using celery 4.0.2 and librabbitmq 1.6.1.
In such case, this is a known compatibility issue, you can refer to https://github.com/celery/celery/issues/3675 and https://github.com/celery/librabbitmq/issues/93.
The first link gives you recommendation to solve your problem namely:
uninstall librabbitmq
pip uninstall librabbitmq(you may have to call this command many times)change the occurrences of
amqptopyamqpin your borker urls. (Though not in your config file if your are using one. Doing that did not work for me).
To answer more precisely your other questions: you are right saying that there is a sender and a fetcher.
The sender role is assumed by the app created when you call Celery(...). One of its role is to act as a registry of tasks, and if you look at its implementation in app/base.py, you'll see that it implements a method send_task which is directly called by the method apply_async of the Task class. This method's role is to send a marshalled version of your task through the wire up to the broker so it can be fetched by a worker. The application protocol used to transmit the message is amqp, for which an implementation is librabbitmq.
On the other side of the wire, there is another instance, launched by the worker which does the fetching work. In celery's parlance, it is called a Consumer. You can find its implementation in worker/consumer/consumer.py. You will see that it implements a create_task_handler which in turns defines the on_task_received functions that raises the error you are seeing. It is the function called when a new task is fetched from the worker and next in line to by processed.
The solution suggested therefore consists in changing the implementation of the amqp protocol so that a TypeError is not raised in on_task_received (which it seems to me would be caused by an encoding issue).
I hope it answers all your questions and gives you a clearer view of how celery works. I should end by saying that to my knowledge a "conventional" use of Celery would never require you to tamper with those kind of internals, and that you can achieve 99% of what you may want by implementing custom task classes and custom backends for example.
Just so that the answer is located here as well. In the thread Anis refers to 23doors mentions that Celery 4's new default protocol does not play nice with librabbitmq:
Apparently librabbitmq issue is related to new default protocol in celery 4.x.
He also mentions that to resolve this issue you can make use of the older protocol Celery offers by setting (if you're using Django):
CELERY_TASK_PROTOCOL = 1
Otherwise you can set the following in your celeryconf.py file
app.conf.task_protocol = 1
All credit to 23doors :)