celery使用的時候的坑


一、delay函數或者apply_async函數的傳參問題

1、通過delay或者apply_async傳參數給異步任務的時候不能傳實例,否則會報錯raised unexpected: EncodeError(TypeError('<ExtractCashRecord: ExtractCashRecord object> is not JSON serializable,因為我傳的參數有一個是ExtractCashRecord這個類的實例。

二、任務重復執行

celery使用redis作為broker的話,apply_async的異步任務如果執行之間是延遲超過1小時的話,celery會重復發布任務,會導致任務重復執行。這個在官方文檔中有說明celery文檔

if a task isn’t acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.This causes problems with ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout; in fact if that happens it will be executed again, and again in a loop.So you have to increase the visibility timeout to match the time of the longest ETA you’re planning to use.Note that Celery will redeliver messages at worker shutdown, so having a long visibility timeout will only delay the redelivery of ‘lost’ tasks in the event of a power failure or forcefully terminated workers.Periodic tasks won’t be affected by the visibility timeout, as this is a concept separate from ETA/countdown.You can increase this timeout by configuring a transport option with the same name:
app.conf.broker_transport_options = {'visibility_timeout': 43200}*
The value must be an int describing the number of seconds.

這個時候,我們就需要在配置文件里面增加一項,來增大visibility_timeout的時間
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 43200}
或者
app.conf.broker_transport_options = {'visibility_timeout': 43200}


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM