Python日志監控系統處理日志（pyinotify）

前言

最近項目中遇到一個用於監控日志文件的Python包pyinotify，結合自己的項目經驗和網上的一些資料總結一下，總的原理是利用pyinotify模塊監控日志文件夾，當日志到來的情況下，觸發相應的函數進行處理，處理完畢后刪除日志文件的過程，下面就着重介紹下pyinotify

pyinotify

Pyinotify是一個Python模塊，用來監測文件系統的變化。 Pyinotify依賴於Linux內核的功能—inotify（內核2.6.13合並）。 inotify的是一個事件驅動的通知器，其通知接口通過三個系統調用從內核空間到用戶空間。pyinotify結合這些系統調用，並提供一個頂級的抽象和一個通用的方式來處理這些功能。

• pyinotify 說百了就是通過 調用系統的inotify來實現通知的
• inotify 既可以監視文件，也可以監視目錄
• Inotify 使用系統調用而非 SIGIO 來通知文件系統事件。

Inotify 可以監視的文件系統事件包括：

Event Name	Is an Event	Description
IN_ACCESS	Yes	file was accessed.
IN_ATTRIB	Yes	metadata changed.
IN_CLOSE_NOWRITE	Yes	unwrittable file was closed.
IN_CLOSE_WRITE	Yes	writtable file was closed.
IN_CREATE	Yes	file/dir was created in watched directory.
IN_DELETE	Yes	file/dir was deleted in watched directory.
IN_DELETE_SELF	Yes	自刪除，即一個可執行文件在執行時刪除自己
IN_DONT_FOLLOW	No	don't follow a symlink (lk 2.6.15).
IN_IGNORED	Yes	raised on watched item removing. Probably useless for you, prefer instead IN_DELETE*.
IN_ISDIR	No	event occurred against directory. It is always piggybacked to an event. The Event structure automatically provide this information (via .is_dir)
IN_MASK_ADD	No	to update a mask without overwriting the previous value (lk 2.6.14). Useful when updating a watch.
IN_MODIFY	Yes	file was modified.
IN_MOVE_SELF	Yes	自移動，即一個可執行文件在執行時移動自己
IN_MOVED_FROM	Yes	file/dir in a watched dir was moved from X. Can trace the full move of an item when IN_MOVED_TO is available too, in this case if the moved item is itself watched, its path will be updated (see IN_MOVE_SELF).
IN_MOVED_TO	Yes	file/dir was moved to Y in a watched dir (see IN_MOVE_FROM).
IN_ONLYDIR	No	only watch the path if it is a directory (lk 2.6.15). Usable when calling .add_watch.
IN_OPEN	Yes	file was opened.
IN_Q_OVERFLOW	Yes	event queued overflowed. This event doesn't belongs to any particular watch.
IN_UNMOUNT	Yes	宿主文件系統被 umount

IN_ACCESS，即文件被訪問
IN_MODIFY，文件被write
IN_ATTRIB，文件屬性被修改，如chmod、chown、touch等
IN_CLOSE_WRITE，可寫文件被close
IN_CLOSE_NOWRITE，不可寫文件被close
IN_OPEN，文件被open
IN_MOVED_FROM，文件被移走,如mv
IN_MOVED_TO，文件被移來，如mv、cp
IN_CREATE，創建新文件
IN_DELETE，文件被刪除，如rm
IN_DELETE_SELF，自刪除，即一個可執行文件在執行時刪除自己
IN_MOVE_SELF，自移動，即一個可執行文件在執行時移動自己
IN_UNMOUNT，宿主文件系統被umount
IN_CLOSE，文件被關閉，等同於(IN_CLOSE_WRITE | IN_CLOSE_NOWRITE)
IN_MOVE，文件被移動，等同於(IN_MOVED_FROM | IN_MOVED_TO)

pyinotify使用例子

#!/usr/bin/python
# coding:utf-8

import os
from pyinotify import WatchManager, Notifier,ProcessEvent,IN_DELETE, IN_CREATE,IN_MODIFY
  
class EventHandler(ProcessEvent):
 """事件處理"""
 def process_IN_CREATE(self, event):
  print  "Create file: %s " %  os.path.join(event.path,event.name)
  
 def process_IN_DELETE(self, event):
  print  "Delete file: %s " %  os.path.join(event.path,event.name)
 
 def process_IN_MODIFY(self, event):
   print  "Modify file: %s " %  os.path.join(event.path,event.name)
  
def FSMonitor(path='.'):
  wm = WatchManager() 
  mask = IN_DELETE | IN_CREATE |IN_MODIFY
  notifier = Notifier(wm, EventHandler())
  wm.add_watch(path, mask,auto_add=True,rec=True)
  print 'now starting monitor %s'%(path)
  while True:
   try:
     notifier.process_events()
     if notifier.check_events():
       notifier.read_events()
   except KeyboardInterrupt:
     notifier.stop()
     break
  
if __name__ == "__main__":
 FSMonitor('/root/softpython/apk_url')