How to cleanup celery_taskmeta?
I'm not using django-celery so my answer could be incomplete.
There is a method in the Backend that allow you to force cleanup of celery_taskmeta : in celery.backends.base.BaseBackend, you can find method cleanup()
So, when I need to force cleanup I have 3 ways :
- using the AsyncResult on producer side
- using the task on the consumer side
- celerybeat task
on producer side
# Launch a task
my_task = add_a_task() # use the Task.delay() method for example
# my_task is a celery.result.AsyncResult instance
# Retrieve backend and call cleanup
my_task.backend.cleanup()
on the consumer side
from celery import Task
from .celery_app import my_celery_app
# Class for a task
@my_celery_app.task()
class CopyTask(Task):
# A Task instance got a backend property
def run(self,**kwargs):
""" Running method of the task """
# Do something
pass
def force_cleanup(self):
""" Force cleanup. """
self.backend.cleanup()
Note that calling a cleanup in the task itself seems quiet awfull.
with celerybeat task
A third option, is using a celerybeat task that cleanup backend celery_taskmeta. This is the one I probably choose.
celery beat is basically a scheduler and you can run the task you want. So you can have a dedicated task, like this one :
from celery import Task
from .celery_app import my_celery_app
# Class for a task
@my_celery_app.task()
class CleanupTask(Task):
# A Task instance got a backend property
def run(self,**kwargs):
""" Running method of the task """
# Cleanup here
self.backend.cleanup()
Setting a celery beat, like a crontab job is well documented
I have a cronjob running a Django command that executes this SQL statement:
"truncate table celery_taskmeta"
deleted 50.000 records in 0.47ms.