openstack中運行定時任務的兩種方法及源代碼分析


啟動一個進程,如要想要這個進程的某個方法定時得進行執行的話,在openstack有兩種方式: 一種是通過繼承 periodic_task.PeriodicTasks,另一種是使用loopingcall.py,針對兩種方式分別說一下實現原理。
 
(1) 繼承periodic_task.PeriodicTasks
         這種方式比較復雜,用到了python中的一些比較高級的特性,裝飾器和元類;首先看一下 periodic_task.py,在nova/openstack/common中,其他組件也有。
         看一下PeriodicTasks 這個類。
144 @six.add_metaclass(_PeriodicTasksMeta)
145 class PeriodicTasks(object):
146     def __init__(self):
147         super(PeriodicTasks, self).__init__()
148         self._periodic_last_run = {}
149         for name, task in self._periodic_tasks:
150             self._periodic_last_run[name] = task._periodic_last_run
151
152     def run_periodic_tasks(self, context, raise_on_error=False):
153         """Tasks to be run at a periodic interval."""
154         idle_for = DEFAULT_INTERVAL
155         for task_name, task in self._periodic_tasks:
156             full_task_name = '.'.join([self.__class__.__name__, task_name])
157
158             spacing = self._periodic_spacing[task_name]
159             last_run = self._periodic_last_run[task_name]
160
161             # If a periodic task is _nearly_ due, then we'll run it early
162             if spacing is not None:
163                 idle_for = min(idle_for, spacing)
164                 if last_run is not None:
165                     delta = last_run + spacing - time.time()
166                     if delta > 0.2:
167                         idle_for = min(idle_for, delta)
168                         continue
169
170             LOG.debug("Running periodic task %(full_task_name)s",
171                       {"full_task_name": full_task_name})
172             self._periodic_last_run[task_name] = time.time()
173
174             try:
175                 task(self, context)
176             except Exception as e:
177                 if raise_on_error:
178                     raise
179                 LOG.exception(_LE("Error during %(full_task_name)s: %(e)s"),
180                               {"full_task_name": full_task_name, "e": e})
181             time.sleep(0)
182
183         return idle_for
 
run_periodic_tasks 函數是用戶啟動各個 定時任務的,其中里面有幾個數據結構比較重要,self._periodic_tasks:記錄來每個task和每個task的函數句柄;self._periodic_spacing: 記錄每一個task的運行間隔時間。在__init__函數中,還有構造一個self._periodic_last_run 結構用來記錄每一個task上一次運行的時間;具體運行的時候會根據上次運行時間和間隔時間來確定是否運行,函數第162~168行;那具體的self._periodic_tasks和self._periodic_spacing是怎么得來的,是通過元類的方式來實現的;
 
元類可以干預一個類的實現形式,比方說在為一個類添加一個方法,或者為了一個類統一添加某種行為;上文的@six.add_metaclass(_PeriodicTasksMeta)語法就是添加了一個元類,我們看一下_PeriodicTasksMeta的實現;
 
100 class _PeriodicTasksMeta(type):
101     def __init__(cls, names, bases, dict_):
102         """Metaclass that allows us to collect decorated periodic tasks."""
103         super(_PeriodicTasksMeta, cls).__init__(names, bases, dict_)
104
105         # NOTE(sirp): if the attribute is not present then we must be the base
106         # class, so, go ahead an initialize it. If the attribute is present,
107         # then we're a subclass so make a copy of it so we don't step on our
108         # parent's toes.
109         try:
110             cls._periodic_tasks = cls._periodic_tasks[:]
111         except AttributeError:
112             cls._periodic_tasks = []
113
114         try:
115             cls._periodic_spacing = cls._periodic_spacing.copy()
116         except AttributeError:
117             cls._periodic_spacing = {}
118
119         for value in cls.__dict__.values():
120             if getattr(value, '_periodic_task', False):
121                 task = value
122                 name = task.__name__
123
124                 if task._periodic_spacing < 0:
125                     LOG.info(_LI('Skipping periodic task %(task)s because '
126                                  'its interval is negative'),
127                              {'task': name})
128                     continue
129                 if not task._periodic_enabled:
130                     LOG.info(_LI('Skipping periodic task %(task)s because '
131                                  'it is disabled'),
132                              {'task': name})
133                     continue
134
135                 # A periodic spacing of zero indicates that this task should
136                 # be run every pass
137                 if task._periodic_spacing == 0:
138                     task._periodic_spacing = None
139
140                 cls._periodic_tasks.append((name, task))
141                 cls._periodic_spacing[name] = task._periodic_spacing
其中109~117為類添加_periodic_tasks與_periodic_spacing兩個類變量, for value in cls.__dict__.values() 語句訪問類的各個成員,主要是函數成員;如果發現成員中有_periodic_task屬性,並且等於True,則構造_periodic_tasks與_periodic_spacing兩個數據結構;那么剩下就要弄清楚task的結構了,task就是類中的一個函數,它為什么具有_periodic_task屬性和_periodic_spacing的呢?這個活就是裝飾器做的事情了。
 
當你給一個函數設置 定時任務的裝飾器時,一般會這樣寫:
@periodic_task.periodic_task(spacing=…, run_immediately=...)
def f(args,kwargs):
…….
 
奧妙就在這個裝飾器里面了。
 42 def periodic_task(*args, **kwargs):
 43     """Decorator to indicate that a method is a periodic task.
 44
 45     This decorator can be used in two ways:
 46
 47         1. Without arguments '@periodic_task', this will be run on every cycle
 48            of the periodic scheduler.
 49
 50         2. With arguments:
 51            @periodic_task(spacing=N [, run_immediately=[True|False]])
 52            this will be run on approximately every N seconds. If this number is
 53            negative the periodic task will be disabled. If the run_immediately
 54            argument is provided and has a value of 'True', the first run of the
 55            task will be shortly after task scheduler starts.  If
 56            run_immediately is omitted or set to 'False', the first time the
 57            task runs will be approximately N seconds after the task scheduler
 58            starts.
 59     """
 60     def decorator(f):
 61         # Test for old style invocation
 62         if 'ticks_between_runs' in kwargs:
 63             raise InvalidPeriodicTaskArg(arg='ticks_between_runs')
 64
 65         # Control if run at all
 66         f._periodic_task = True
 67         f._periodic_external_ok = kwargs.pop('external_process_ok', False)
 68         if f._periodic_external_ok and not CONF.run_external_periodic_tasks:
 69             f._periodic_enabled = False
 70         else:
 71             f._periodic_enabled = kwargs.pop('enabled', True)
 72
 73         # Control frequency
 74         f._periodic_spacing = kwargs.pop('spacing', 0)
 75         f._periodic_immediate = kwargs.pop('run_immediately', False)
 76         if f._periodic_immediate:
 77             f._periodic_last_run = None
 78         else:
 79             f._periodic_last_run = time.time()
 80         return f
 81
 82     # NOTE(sirp): The `if` is necessary to allow the decorator to be used with
 83     # and without parents.
 84     #
 85     # In the 'with-parents' case (with kwargs present), this function needs to
 86     # return a decorator function since the interpreter will invoke it like:
 87     #
 88     #   periodic_task(*args, **kwargs)(f)
 89     #
 90     # In the 'without-parents' case, the original function will be passed
 91     # in as the first argument, like:
 92     #
 93     #   periodic_task(f)
 94     if kwargs:
 95         return decorator
 96     else:
 97         return decorator(args[0])
 
在裝飾器中,66~80行就是為一個函數設置_periodic_task屬性,並從裝飾器的kwargs中取得spacing參數,如果有run_immediately(啟動之后先立刻執行一次,然后再定時執行)會設置_periodic_last_run屬性。這樣定時函數運行需要的信息就都齊全了;
 
(2)另一個種方法是使用loopingcall.py ,也在nova/openstack/common中。
使用方法是:obj = loopingcall.FixedIntervalLoopingCall(f, args,kwargs),
                      obj.start(interval,initial_delay)
  代碼:
 62 class FixedIntervalLoopingCall(LoopingCallBase):
 63     """A fixed interval looping call."""
 64
 65     def start(self, interval, initial_delay=None):
 66         self._running = True
 67         done = event.Event()
 68
 69         def _inner():
 70             if initial_delay:
 71                 greenthread.sleep(initial_delay)
 72
 73             try:
 74                 while self._running:
 75                     start = timeutils.utcnow()
 76                     self.f(*self.args, **self.kw)
 77                     end = timeutils.utcnow()
 78                     if not self._running:
 79                         break
 80                     delay = interval - timeutils.delta_seconds(start, end)
 81                     if delay <= 0:
 82                         LOG.warn(_LW('task run outlasted interval by %s sec') %
 83                                  -delay)
 84                     greenthread.sleep(delay if delay > 0 else 0)
 85             except LoopingCallDone as e:
 86                 self.stop()
 87                 done.send(e.retvalue)
 88             except Exception:
 89                 LOG.exception(_LE('in fixed duration looping call'))
 90                 done.send_exception(*sys.exc_info())
 91                 return
 92             else:
 93                 done.send(True)
 94
 95         self.done = done
 96
 97         greenthread.spawn_n(_inner)
 98         return self.done
 
start()方法運行 定時任務,initial_delay表示是否有延時,75~84每次運行任務要記錄開始時間和結束時間,如果開始時間減去結束時間比interval還大的話,那么就不等待了,立刻運行:greenthread.sleep(delay if delay > 0 else 0)

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM