Django celery異步任務實踐指南

 

 

 

最近項目中用到celery很多，Django快速接入celery，這里給份教程。

准備

pip安裝celery、flower、eventlet

快速接入

1.項目目錄的__init__文件

from __future__ import absolute_import

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celerypro import app as celery_app

2.celerypro.py文件

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'voice_quality_assurance_configure.settings')  #修改項目配置文件的地址
app = Celery('voice_quality_assurance_configure') #修改項目目錄名稱
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('voice_quality_assurance_configure.celeryconfig')  #修改celery配置文件的地址
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

3.celeryconfig.py文件,更多配置項，可以查看官方文檔。

from kombu import Queue
BROKER_URL = 'amqp://用戶名:密碼@ip:5672'# 指定 Broker
CELERY_RESULT_BACKEND = 'rpc://用戶名:密碼@ip:5672'# 指定 Backend
CELERY_TIMEZONE='Asia/Shanghai'# 指定時區，默認是 UTC
CELERY_TASK_SERIALIZER = 'pickle'
CELERY_RESULT_SERIALIZER = 'pickle'
CELERY_ACCEPT_CONTENT = ['pickle', 'json']
CELERY_IGNORE_RESULT = True
# CELERY_TIMEZONE='UTC'
CELERY_IMPORTS = (
    # 指定導入的任務模塊
    'apps.mission.tasks'
)

CELERY_QUEUES = (
    Queue('default', routing_key='default'), #聲明隊列和對應路由鍵
    Queue('worker_queue', routing_key='worker'), #聲明隊列和對應路由鍵
)
CELERY_ROUTES = {
    'apps.mission.tasks.createsingletask': {'queue': 'worker_queue', 'routing_key': 'worker'},
}

app代碼如何使用

app下新建tasks.py文件，名字一定要是tasks。（我這里是mission app下的tasks.py）

from celery import shared_task

@shared_task()
def createsingletask():
    print(test)

app下views調用如下：（我這里是mission app下的views.py）

from .tasks import createsingletask

createsingletask.apply_async(())

快速測試和監控

啟動多個celery worker，-A 指定項目目錄， -P 指定方式，我這里以協程方式運行， -n指定name

celery worker -A voice_quality_assurance_configure --loglevel=info -P eventlet -n worker1
celery worker -A voice_quality_assurance_configure --loglevel=info -P eventlet -n worker2
celery worker -A voice_quality_assurance_configure --loglevel=info -P eventlet -n worker3
celery worker -A voice_quality_assurance_configure --loglevel=info -P eventlet -n worker4
celery worker -A voice_quality_assurance_configure --loglevel=info -P eventlet -n worker5

啟動flower監控

celery flower --broker=amqp://用戶名:密碼@ip:5672 --broker-api=http://用戶名:密碼@ip:15672/api/

　

　查看監控,注意這里的監控數據是不持久化的。