win10下實現django+celery定時任務 保姆級別教程

\(\color{green}{友情提示：並沒有集成django項目來做，要是結合其實也很簡單，也就是一些函數調用，其他的不用改變}\)

年輕人要講碼德

系統環境：win10
python版本3.7
django==2.0.8
celery==4.3.0
django_celery==3.3.1
django_celery_beat==2.1.0
eventlet==0.25.2

eventlet是一個高性能協程庫，用來解決win10下celery進程的bug

\(\color{green}{項目結構目錄}\)

#config.py-------配置broker，result-backend,序列化等相關
#celery.py-------程序的入口文件
#__init__.py------告訴python這是一個包
#tasks.py----------任務文件

celery.py內容

from __future__ import absolute_import

from celery import Celery
#app是Celery類的實例，創建的時候添加了proj.tasks這個模塊，也就是包含了proj/tasks.py這個文件
app = Celery('proj',include=['proj.tasks'])

#另一種配置方式，這里就大概寫一下
#app = Celery('任務名',broker='xxx',backend='xxx',include=['xxx','xxx'])

#去找配置文件
app.config_from_object('proj.config')

config.py

from __future__ import absolute_import
#使用redis作為消息代理
BROKER_URL = 'redis://127.0.0.1:6379/1'
#把任務結果存放在redis2號庫
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379/2'

##以下內容可以不用配置，這些都是默認配置，除非自定義需要配置,我用 -- 框住
-------------------------------------------------
#任務序列化和反序列化使用msgpack方案
CELERY_TASK_SERIALIZER = 'msgpack'

#讀取任務結果一般性能要求不高，所以使用了可讀性更好的JSON
CELERY_RESULT_SERIALIZER = 'json'

#任務過期時間，不建議直接寫86400，應該讓這樣的magic數字表述更明顯
CELERY_TASK_RESULT_EXPIRES = 60*60*24 

#指定接受的內容類型
CELERY_ACCEPT_CONTENT = ['json','msgpack']

--------------------------------------------------


#定時任務配置
from datetime import timedelta
CELERY_TIMEZONE = 'Asia/Shanghai'
CELERY_ENABLE_UTC=True

#ps：友情提示 CELERYBEAT_SCHEDULE 千萬不要寫錯了，我之前寫成 CELERY_BEAT_SCHEDULE導致定時任務執行不了，慘痛的教訓！！！

CELERYBEAT_SCHEDULE = {
    #這個名字(print_test)隨便寫，無所謂
    'print_test': {                       
        'task': 'proj.tasks.print_test',      #綁定的定時任務的函數
        'schedule': timedelta(seconds=2),      #設置定時時間，這里是2秒執行一次
         'args':()       #用來給函數傳參

    }
}

tasks.py

from __future__ import absolute_import

from proj.celery import app

@app.task
def print_test():
    print("nict try")
    return 'hello'

\(\color{green}{我們就可以來做一個測試了}\)

啟動worker (# 執行celery 時要在proj的父目錄，這里是sfapp)

celery worker -A proj -l info -P eventlet

啟動定時任務的發送方： (# 執行celery 時要在proj的父目錄，這里是sfapp)

celery beat -A proj

(Django) D:\haha\sfapp>celery -A proj beat
c:\users\86177\django\lib\site-packages\celery\utils\imports.py:167: UserWarning: Cannot loa
d celery.commands extension 'flower.command:FlowerCommand': ModuleNotFoundError("No module n
amed '_contextvars'")
  namespace, class_name, exc))
celery beat v4.3.0 (rhubarb) is starting.
__    -    ... __   -        _
LocalTime -> 2020-12-22 10:23:58
Configuration ->
    . broker -> redis://127.0.0.1:6379/6
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%WARNING
    . maxinterval -> 5.00 minutes (300s)

(Django) D:\haha\sfapp>celery worker -A proj -l info -P eventlet
c:\users\86177\django\lib\site-packages\celery\utils\imports.py:167: UserWarning: Cannot loa
d celery.commands extension 'flower.command:FlowerCommand': ModuleNotFoundError("No module n
amed '_contextvars'")
  namespace, class_name, exc))

 -------------- celery@LAPTOP-15DN000A v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-10-10.0.19041-SP0 2020-12-22 10:23:36
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         proj:0x1f1f22d0cf8
- ** ---------- .> transport:   redis://127.0.0.1:6379/6
- ** ---------- .> results:     redis://127.0.0.1:6379/5
- *** --- * --- .> concurrency: 8 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . proj.tasks.print_test

[2020-12-22 10:23:36,559: INFO/MainProcess] Connected to redis://127.0.0.1:6379/6
[2020-12-22 10:23:36,569: INFO/MainProcess] mingle: searching for neighbors
[2020-12-22 10:23:37,591: INFO/MainProcess] mingle: all alone
[2020-12-22 10:23:37,617: INFO/MainProcess] celery@LAPTOP-15DN000A ready.
[2020-12-22 10:23:37,618: INFO/MainProcess] pidbox: Connected to redis://127.0.0.1:6379/6.
[2020-12-22 10:24:00,849: INFO/MainProcess] Received task: proj.tasks.print_test[ff4baf50-7c
bd-4b93-a958-5d9913edee6e]
[2020-12-22 10:24:00,850: WARNING/MainProcess] nict try
[2020-12-22 10:24:00,852: INFO/MainProcess] Task proj.tasks.print_test[ff4baf50-7cbd-4b93-a9
58-5d9913edee6e] succeeded in 0.0s: 'hello'
[2020-12-22 10:24:02,744: INFO/MainProcess] Received task: proj.tasks.print_test[0267d630-02
25-4928-abbf-3e65227f1c42]
[2020-12-22 10:24:02,745: WARNING/MainProcess] nict try
[2020-12-22 10:24:02,746: INFO/MainProcess] Task proj.tasks.print_test[0267d630-0225-4928-ab
bf-3e65227f1c42] succeeded in 0.0s: 'hello'

\(\color{red}{重大疑惑，綜上所述，能很明顯的注意到一個東西，就是下面這個}\)

\(\color{red}{具體什么原因我還沒弄清楚，不過，暫時不妨礙定時任務的執行}\)

\(\color{red}{如果那位大佬有解決方法，歡迎在文章下面留言，一定感激不盡}\)

c:\users\86177\django\lib\site-packages\celery\utils\imports.py:167: UserWarning: Cannot loa
d celery.commands extension 'flower.command:FlowerCommand': ModuleNotFoundError("No module n
amed '_contextvars'")