How to run celery on windows?

There are two workarounds to make Celery 4 work on Windows:

  • use eventlet, gevent or solo concurrency pool (if your tasks as I/O and not CPU-bound)
  • set the environment variable FORKED_BY_MULTIPROCESSING=1 (this is what actually causes the underlying billiard package to to fail under Windows since version 4)

See https://www.distributedpython.com/2018/08/21/celery-4-windows for more details


yes:

celery -A your_application -l info

also note Celery have dropped support for Windows(since v4), so best to

pip install celery==3.1.25

3.1.25 was the last version that works on windows(just tested on my win10 machine). Didn't need to downgrade flower(browser monitor for celery) though.

See also the FAQ for Windows


Compile Celery with --pool=solo argument.

Example: celery -A your-application worker -l info --pool=solo


Celery 4.0+ does not officially support window already. But it still works on window for some development/test purpose.

Use eventlet instead as below:

pip install eventlet
celery -A <module> worker -l info -P eventlet

It works for me on window 10 + celery 4.1 + python 3.

This solution solved the following exception:

[2017-11-16 21:19:46,938: ERROR/MainProcess] Task handler raised error: ValueError('need more than 0 values to unpack',)
Traceback (most recent call last):
  File "c:\users\wchen8\work\venv\weinsta\lib\site-packages\billiard\pool.py", line 358, in workloop
    result = (True, prepare_result(fun(*args, **kwargs)))
  File "c:\users\wchen8\work\venv\weinsta\lib\site-packages\celery\app\trace.py", line 525, in _fast_trace_task
    tasks, accept, hostname = _loc
ValueError: need more than 0 values to unpack

===== update 2018-11 =====

Eventlet has an issue on subprocess.CalledProcessError:

https://github.com/celery/celery/issues/4063

https://github.com/eventlet/eventlet/issues/357

https://github.com/eventlet/eventlet/issues/413

So try gevent instead.

pip install gevent
celery -A <module> worker -l info -P gevent

This works for me on window 10 + celery 4.2 + python 3.6

Tags:

Python

Celery

Recent Posts