Celery does not release memory

It was this config option that made my worker does not release memory.

CELERYD_TASK_TIME_LIMIT = 600

refer to: https://github.com/celery/celery/issues/1427


There are two settings which can help you mitigate growing memory consumption of celery workers:

  • Max tasks per child setting (v2.0+):

    With this option you can configure the maximum number of tasks a worker can execute before it’s replaced by a new process. This is useful if you have memory leaks you have no control over for example from closed source C extensions.

  • Max memory per child setting (v4.0+):

    With this option you can configure the maximum amount of resident memory a worker can execute before it’s replaced by a new process. This is useful if you have memory leaks you have no control over for example from closed source C extensions.