If you are going to run multiple workers and got some key error when doing async job in celery like (KeyError, Received unregistered task of type). 
This would be a solution.


Key statement

"Use different Queue name and run worker with that Queue name". reference


0. Structure

folder/

tasks/

some_tasks.py

the_other_tasks.py

scheduler.py 


1. scheduler.py

# import

from tasks.some_tasks import sum
from tasks.the_other_tasks import add

# use queue name, when you call task function
sum.apply_async(queue="some_tasks")
add.apply_async(queue="the_other_tasks")


2. tasks

A. some_tasks.py

app = Celery(..)
app.conf.task_default_queue = "some_tasks" 


B. the_other_tasks.py

app = Celery(..)
app.conf.task_default_queue = "the_other_tasks" 



3. running workers
$folder>celery -A tasks.some_tasks worker --loglevel=info --concurrency=1 -Q some_tasks
$folder>celery -A tasks.the_other_tasks worker --loglevel=info --concurrency=1 -Q the_other_tasks

*If you want to give name to worker, use -n option.
example) celery -A tasks.the_other_tasks worker --loglevel=info --concurrency=1 -n the_other_tasks -Q the_other_tasks

'celery' 카테고리의 다른 글

worker와 publisher가 init되는 것에 대한 오해가 있었다.  (0) 2015.05.11
celery 오해와 진실  (0) 2015.03.13
Revoking task  (0) 2014.11.24
예약 푸시  (0) 2014.11.20
블로그 이미지

시간을 거스르는자

ytkang86@gmail.com

,