What are the consequences of disabling gossip, mingle and heartbeat for celery workers?

This is the base documentation which doesn't give us much info

heartbeat

Is related to communication between the worker and the broker (in your case the broker is CloudAMQP). See explanation

With the --without-heartbeat the worker won't send heartbeat events

mingle

It only asks for "logical clocks" and "revoked tasks" from other workers on startup.

Taken from whatsnew-3.1

The worker will now attempt to synchronize with other workers in the same cluster.

Synchronized data currently includes revoked tasks and logical clock.

This only happens at startup and causes a one second startup delay to collect broadcast responses from other workers.

You can disable this bootstep using the --without-mingle argument.

Also see docs

gossip

Workers send events to all other workers and this is currently used for "clock synchronization", but it's also possible to write your own handlers on events, such as on_node_join, See docs

Taken from whatsnew-3.1

Workers are now passively subscribing to worker related events like heartbeats.

This means that a worker knows what other workers are doing and can detect if they go offline. Currently this is only used for clock synchronization, but there are many possibilities for future additions and you can write extensions that take advantage of this already.

Some ideas include consensus protocols, reroute task to best worker (based on resource usage or data locality) or restarting workers when they crash.

We believe that although this is a small addition, it opens amazing possibilities.

You can disable this bootstep using the --without-gossip argument.


Celery workers started up with the --without-mingle option, as @ofirule mentioned above, will not receive synchronization data from other workers, particularly revoked tasks. So if you revoke a task, all workers currently running will receive that broadcast and store it in memory so that when one of them eventually picks up the task from the queue, it will not execute it:

https://docs.celeryproject.org/en/stable/userguide/workers.html#persistent-revokes

But if a new worker starts up before that task has been dequeued by a worker that received the broadcast, it doesn't know to revoke the task. If it eventually picks up the task, then the task is executed. You will see this behavior if you're running in an environment where you are dynamically scaling in and out celery workers constantly.