celery 快速入門教程 celery 定時器

當然首先得安裝celery和rabbitmq-server，如果有redis需要安裝redis

安裝Redis
$ yum install redis
啟動 Redis

$redis-server
檢查Redis是否在工作？

$redis-cli
這將打開一個Redis提示，如下圖所示：

redis 127.0.0.1:6379>
上面的提示127.0.0.1是本機的IP地址，6379為Redis服務器運行的端口。現在輸入PING命令，如下圖所示。

redis 127.0.0.1:6379> ping
PONG
這說明你已經成功地安裝Redis在您的機器上。

　　

tasks.py

from celery import Celery
from time import sleep

app = Celery('tasks', backend='amqp', broker='amqp://guest@localhost//')
#app = Celery('tasks', backend='redis://localhost', broker='amqp://guest@localhost//')
#app = Celery('tasks', backend='redis://localhost:6379/1', broker='amqp://guest@localhost//')
#app = Celery('tasks', backend='redis://localhost:6379/1', broker='redis://localhost:6379/0')
#app = Celery('taskwb', backend='amqp://guest@localhost:5672/0', broker='amqp://guest@localhost:5672/1')

@app.task
def add(x, y):
    sleep(5)
    return x + y

保存后執行celery -A tasks worker --loglevel=info，可以看到提示頁面

[wenbin celery]$ celery -A tasks worker --loglevel=info
[2016-03-07 15:22:56,443: WARNING/MainProcess] /usr/lib/python2.6/site-packages/celery/apps/worker.py:161: CDeprecationWarning: 
Starting from version 3.2 Celery will refuse to accept pickle by default.

The pickle serializer is a security concern as it may give attackers
the ability to execute any command.  It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.

If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::

    CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

You must only enable the serializers that you will actually use.


  warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
 
 -------------- celery@wb-test-multiple-portal v3.1.13 (Cipater)
---- **** ----- 
--- * ***  * -- Linux-2.6.32-431.el6.x86_64-x86_64-with-centos-6.6-Final
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x1071e50
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     amqp
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery

 

然后新開一個窗口進入python命令行：

>>>from tasks import add

>>>res = add.delay(4, 3)

可以看到之前啟動celery的那邊有信息輸出，並且過5s后執行了結果（因為我們add中sleep了5秒）

[2016-03-07 15:21:43,153: INFO/MainProcess] Received task: tasks.add[fd523296-8b99-4530-a77e-3fa56cc19a5d]
[2016-03-07 15:21:48,184: INFO/MainProcess] Task tasks.add[fd523296-8b99-4530-a77e-3fa56cc19a5d] succeeded in 5.02967603202s: 7

說明我們的demo已經成功了。

 

有一個小錯誤自己發現的，我之后又開了一個celery運行的窗口（其實是忘記關了之前的），發現在python命令行執行delay的時候只有一半成功了。。。這是個神奇的東西，能夠自動檢測到另一個的存在，然后分配任務；然后我又再運行一個celery，這樣發現原來3個celery都能輪流來進行任務的執行，簡直是個神奇的東西。。。

 

 

定時器功能

    使用定時器功能首先得配置一下schedule，剛剛我們是直接在Celery函數加入配置，現在我們專門用一個文件來放配置文件，schedule也會寫在這里面。

修改tasks.py

from celery import Celery
from time import sleep
import celeryconfig

app = Celery('tasks')#, backend='amqp', broker='amqp://guest@localhost//')
app.config_from_object('celeryconfig')

@app.task
def add(x, y):
    sleep(5)
    return x + y

增加配置文件celeryconfig.py

from celery.schedules import crontab

BROKER_URL = 'amqp://guest@localhost//'
CELERY_RESULT_BACKEND = 'amqp://'
CELERYBEAT_SCHEDULE={
        "every-1-minute": {
            'task': 'tasks.add',
            'schedule': crontab(minute='*/1'),
            'args': (5,6)
        }
}

表示一分鍾觸發一次add的函數，args是傳入的參數，表示一分鍾執行一次add(5,6)，注意如果再添加一個任務，不能與every-1-minute重復，不然只有最后一個生效了。

 

然后執行celery -A tasks worker -B --loglevel=info    就能夠增加觸發beat任務了，會在本地生成一個celerybeat-schedule文件。最好在-B后面加一個 -s /tmp/celerybeat-schedule  ，不然很可能導致當前目錄沒有寫權限而報permission refused

[wenbin@wb-test-multiple-portal celery]$ celery -A tasks worker -B --loglevel=info
[2016-04-18 17:35:30,047: WARNING/MainProcess] /usr/lib/python2.6/site-packages/celery/apps/worker.py:161: CDeprecationWarning: 
Starting from version 3.2 Celery will refuse to accept pickle by default.

The pickle serializer is a security concern as it may give attackers
the ability to execute any command.  It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.

If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::

    CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

You must only enable the serializers that you will actually use.


  warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
 
 -------------- celery@wb-test-multiple-portal v3.1.13 (Cipater)
---- **** ----- 
--- * ***  * -- Linux-2.6.32-431.el6.x86_64-x86_64-with-centos-6.6-Final
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x2a095d0
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     amqp://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery
                

[tasks]
  . tasks.add

[2016-04-18 17:35:30,060: INFO/Beat] beat: Starting...
[2016-04-18 17:35:30,069: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2016-04-18 17:35:30,081: INFO/MainProcess] mingle: searching for neighbors
[2016-04-18 17:35:31,104: INFO/MainProcess] mingle: all alone
[2016-04-18 17:35:31,118: WARNING/MainProcess] celery@wb-test-multiple-portal ready.
[2016-04-18 17:36:00,020: INFO/Beat] Scheduler: Sending due task every-1-minute (tasks.add)
[2016-04-18 17:36:00,027: INFO/MainProcess] Received task: tasks.add[a81182bd-9785-4d4a-b3cd-a81f7900a12b]
[2016-04-18 17:36:05,057: INFO/MainProcess] Task tasks.add[a81182bd-9785-4d4a-b3cd-a81f7900a12b] succeeded in 5.02911209001s: 11

 

當然也可以分開執行celery和beat，當然需要兩個窗口了

celery -A tasks worker --loglevel=info

celery -A tasks beat

 

參考文獻：

http://docs.jinkan.org/docs/celery/index.html

http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html

http://my.oschina.net/hochikong/blog/419191?p={{currentPage-1}}

http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html