· Monit 簡介
Monit是一個輕量級(500KB)跨平台的用來監控Unix/linux系統的開源工具。部署簡單，並且不依賴任何第三方程序、插件或者庫。
Monit可以監控服務器進程、文件、文件系統、網絡狀態（HTTP/SMTP等協議）、遠程主機、服務器資源變化等等。 並且可以設定資源變化后需要做的動作，比如服務失敗后自動重啟，郵件告警等等。
Monit內置了WEB UI，可以一目了然地了解監控項的情況。Monit是監控本機服務的工具，M/Monit是其配套產品用以對Monit統一管理，但是M/Monit並不是開源的，需要購買。
本文我們介紹在CentOS7.3上部署Monit用來監控進程狀態，並實現服務失敗后自動重啟以及郵件告警通知的功能。監控其他資源的方法在默認配置文件中都有舉例這里不一一介紹。
Monit官網：https://mmonit.com/monit
Monit手冊：https://mmonit.com/monit/documentation/monit.html
· Monit 安裝
# rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# yum install -y monit
·Monit 常用命令
monit -t # 配置文件檢測 monit # 啟動monit daemon monit -c /var/monit/monitrc # 啟動monit daemon時指定配置文件 monit reload # 重新加載配置文件 monit status # 查看所有監控項務狀態 monit status nginx # 查看nginx服務狀態 monit stop all # 停止所有服務，這里需要注意的是，如果開啟了自動重啟功能，停止某個被監控的服務必須用monit stop xxx，若用系統命令停止服務，Monit會自動再把服務起來。 monit stop nginx # 停止nginx服務 monit start all # 啟動所有服務 monit start nginx # 啟動nginx服務 monit -V # 查看版本
·Monit 配置文件
/etc/monit.conf 主配置文件
/etc/monit.d/ 各項服務單獨配置文件路徑，在主配置文件中將其include進來。
/etc/monit.conf 舉例說明：
配置文件關鍵字：'if', 'and', 'with(in)', 'has', 'us(ing|e)', 'on(ly)', 'then', 'for', 'of'
############################################################################### ## Monit control file ############################################################################### ## ## Comments begin with a '#' and extend through the end of the line. Keywords ## are case insensitive. All path's MUST BE FULLY QUALIFIED, starting with '/'.
## ## Below you will find examples of some frequently used statements. For ## information about the control file and a complete list of statements and ## options, please have a look in the Monit manual. ## ## ############################################################################### ## Global section ############################################################################### ## ## Start Monit in the background (run as a daemon): #
# 設置檢測周期30s set daemon 30 # check services at 30 seconds intervals # with start delay 240 # optional: delay the first check by 4-minutes (by # # default Monit check immediately after Monit start) # # ## Set syslog logging. If you want to log to a standalone log file instead, ## specify the full path to the log file #
#設置log路徑，這里默認記錄到syslog set logfile syslog # # ## Set the location of the Monit lock file which stores the process id of the ## running Monit instance. By default this file is stored in $HOME/.monit.pid # # set pidfile /var/run/monit.pid # ## Set the location of the Monit id file which stores the unique id for the ## Monit instance. The id is generated and stored on first Monit start. By ## default the file is placed in $HOME/.monit.id. # # set idfile /var/.monit.id # ## Set the location of the Monit state file which saves monitoring states ## on each cycle. By default the file is placed in $HOME/.monit.state. If ## the state file is stored on a persistent filesystem, Monit will recover ## the monitoring state across reboots. If it is on temporary filesystem, the ## state will be lost on reboot which may be convenient in some situations. # # set statefile /var/.monit.state # ## Set the list of mail servers for alert delivery. Multiple servers may be ## specified using a comma separator. If the first mail server fails, Monit # will use the second mail server in the list and so on. By default Monit uses # port 25 - it is possible to override this with the PORT option. # # set mailserver mail.bar.baz, # primary mailserver # backup.bar.baz port 10025, # backup mailserver on port 10025 # localhost # fallback relay #
# 設置郵件服務器用來發送郵件告警通知 set mailserver mail.abcd.so # ## By default Monit will drop alert events if no mail servers are available. ## If you want to keep the alerts for later delivery retry, you can use the ## EVENTQUEUE statement. The base directory where undelivered alerts will be ## stored is specified by the BASEDIR option. You can limit the queue size ## by using the SLOTS option (if omitted, the queue is limited by space ## available in the back end filesystem). # # set eventqueue # basedir /var/monit # set the base directory where events will be stored # slots 100 # optionally limit the queue size # # ## Send status and events to M/Monit (for more informations about M/Monit ## see http://mmonit.com/). By default Monit registers credentials with
## M/Monit so M/Monit can smoothly communicate back to Monit and you don't
## have to register Monit credentials manually in M/Monit. It is possible to ## disable credential registration using the commented out option below. ## Though, if safety is a concern we recommend instead using https when ## communicating with M/Monit and send credentials encrypted. # # set mmonit http://monit:monit@192.168.1.10:8080/collector
# # and register without credentials # Don't register credentials
# # ## Monit by default uses the following format for alerts if the the mail-format ## statement is missing:: ## --8<--
# 設置郵件告警通知格式
set mail-format { from: monit@$HOST subject: monit alert -- $EVENT $SERVICE message: $EVENT Service $SERVICE Date: $DATE Action: $ACTION Host: $HOST Description: $DESCRIPTION Your faithful employee, Monit } ## --8<-- ## ## You can override this message format or parts of it, such as subject ## or sender using the MAIL-FORMAT statement. Macros such as $DATE, etc. ## are expanded at runtime. For example, to override the sender, use: # # set mail-format { from: monit@foo.bar } # # ## You can set alert recipients whom will receive alerts if/when a ## service defined in this file has errors. Alerts may be restricted on ## events by using a filter as in the second example below. #
# 設置郵件告警通知人，Monit默認會通知monit進程本身的變化情況，如果不想收到monit進程自身的通知，加上but not on {instance}配置 set alert weian@abcd.so but not on { instance } # receive all alerts # ## Do not alert when Monit starts, stops or performs a user initiated action. ## This filter is recommended to avoid getting alerts for trivial cases. # # set alert weian@abcd.so # # ## Monit has an embedded HTTP interface which can be used to view status of ## services monitored and manage services from a web interface. The HTTP ## interface is also required if you want to issue Monit commands from the ## command line, such as 'monit status' or 'monit restart service' The reason ## for this is that the Monit client uses the HTTP interface to send these ## commands to a running Monit daemon. See the Monit Wiki if you want to ## enable SSL for the HTTP interface. #
# 設置UI界面訪問信息 set httpd port 2812 and use address 10.2.2.28 # only accept connection from localhost # allow localhost # allow localhost to connect to the server and allow admin:monit # require user 'admin' with password 'monit' ############################################################################### ## Services ############################################################################### ## ## Check general system resources such as load average, cpu and memory ## usage. Each test specifies a resource, conditions and the action to be ## performed should a test fail. # # check system $HOST # if loadavg (1min) > 4 then alert # if loadavg (5min) > 2 then alert # if cpu usage > 95% for 10 cycles then alert # if memory usage > 75% then alert # if swap usage > 25% then alert # # ## Check if a file exists, checksum, permissions, uid and gid. In addition ## to alert recipients in the global section, customized alert can be sent to ## additional recipients by specifying a local alert handler. The service may ## be grouped using the GROUP option. More than one group can be specified by ## repeating the 'group name' statement. # # check file apache_bin with path /usr/local/apache/bin/httpd # if failed checksum and # expect the sum 8f7f419955cefa0b33a2ba316cba3659 then unmonitor # if failed permission 755 then unmonitor # if failed uid root then unmonitor # if failed gid root then unmonitor # alert security@foo.bar on { # checksum, permission, uid, gid, unmonitor # } with the mail-format { subject: Alarm! } # group server # # ## Check that a process is running, in this case Apache, and that it respond ## to HTTP and HTTPS requests. Check its resource usage such as cpu and memory, ## and number of children. If the process is not running, Monit will restart ## it by default. In case the service is restarted very often and the ## problem remains, it is possible to disable monitoring using the TIMEOUT ## statement. This service depends on another service (apache_bin) which ## is defined above. # # check process apache with pidfile /usr/local/apache/logs/httpd.pid # start program = "/etc/init.d/httpd start" with timeout 60 seconds # stop program = "/etc/init.d/httpd stop" # if cpu > 60% for 2 cycles then alert # if cpu > 80% for 5 cycles then restart # if totalmem > 200.0 MB for 5 cycles then restart # if children > 250 then restart # if loadavg(5min) greater than 10 for 8 cycles then stop # if failed host www.tildeslash.com port 80 protocol http # and request "/somefile.html" # then restart # if failed port 443 type tcpssl protocol http # with timeout 15 seconds # then restart # if 3 restarts within 5 cycles then unmonitor # depends on apache_bin # group server #
# 監控進程可以通過上面監控pid文件的方式，當沒有pid文件時，可以通過MATCHING正則表達式來匹配進程。
# 測試一個進程是否匹配來自命令行使用的模式monit procmatch "regex-pattern"，這將列出匹配或不匹配的所有進程，regex模式。
# 我們這里監控了包含shop-pad-server字段的進程，並指明了啟動以及停止的命令，這樣在進程因故斷掉后，Monit會自動重啟進程。
# 同時若進程ID變動，會發送郵件通知到之前指定的收件人。
check process shop-pad-server with MATCHING shop-pad-server start program = "/usr/bin/nohup /home/azureuser/pad-server/run.sh > /home/azureuser/pad-server/nohup.out 2>&1 &" stop program = "/usr/bin/ps -ef | /usr/bin/grep shop-pad| /usr/bin/grep -v grep | /usr/bin/awk '{print $2}' | xargs kill" if changed pid then alert
# ## Check filesystem permissions, uid, gid, space and inode usage. Other services, ## such as databases, may depend on this resource and an automatically graceful ## stop may be cascaded to them before the filesystem will become full and data ## lost. # # check filesystem datafs with path /dev/sdb1 # start program = "/bin/mount /data" # stop program = "/bin/umount /data" # if failed permission 660 then unmonitor # if failed uid root then unmonitor # if failed gid disk then unmonitor # if space usage > 80% for 5 times within 15 cycles then alert # if space usage > 99% then stop # if inode usage > 30000 then alert # if inode usage > 99% then stop # group server # # ## Check a file's timestamp. In this example, we test if a file is older ## than 15 minutes and assume something is wrong if its not updated. Also, ## if the file size exceed a given limit, execute a script # # check file database with path /data/mydatabase.db # if failed permission 700 then alert # if failed uid data then alert # if failed gid data then alert # if timestamp > 15 minutes then alert # if size > 100 MB then exec "/my/cleanup/script" as uid dba and gid dba # # ## Check directory permission, uid and gid. An event is triggered if the ## directory does not belong to the user with uid 0 and gid 0. In addition, ## the permissions have to match the octal description of 755 (see chmod(1)). # # check directory bin with path /bin # if failed permission 755 then unmonitor # if failed uid 0 then unmonitor # if failed gid 0 then unmonitor # # ## Check a remote host availability by issuing a ping test and check the ## content of a response from a web server. Up to three pings are sent and ## connection to a port and an application level network check is performed. # # check host myserver with address 192.168.1.1 # if failed ping then alert # if failed port 3306 protocol mysql with timeout 15 seconds then alert # if failed port 80 protocol http # and request /some/path with content = "a string" # then alert # # ## Check a network link status (up/down), link capacity changes, saturation ## and bandwidth usage. # # check network public with interface eth0 # if failed link then alert # if changed link then alert # if saturation > 90% then alert # if download > 10 MB/s then alert # if total upload > 1 GB in last hour then alert # # ## Check custom program status output. # # check program myscript with path /usr/local/bin/myscript.sh # if status != 0 then alert # # ############################################################################### ## Includes ############################################################################### ## ## It is possible to include additional configuration parts from other files or ## directories. # # include /etc/monit.d/* # # 導入其他單項服務的監控配置 # Include all files from /etc/monit.d/ include /etc/monit.d/*
· 通知周期設置
Monit默認情況下如果一個服務失敗只發送一個通知： alert foo@bar
如果您希望在服務保持處於失敗狀態時每十個周期通知一次，您可以使用： alert foo@bar with reminder on 10 cycles
同樣，如果您想在每個失敗的周期獲得通知，您可以使用： alert foo@bar with reminder on 1 cycle
要禁止某些用戶和服務的警報，可以在服務檢查的局部配置里添加語句： noalert mail-address
· 服務檢測周期設置
可以使用every語句修改服務檢查計划。 1.輪詢周期倍數 EVERY [number] CYCLES 2.Cron-style EVERY [cron] # [cron] # * * * * * # 分 時 日 月 周 3.與Cron-style相反（do-not-check） NOT EVERY [cron] 示例： 示例1：每兩個周期檢查一次 check process nginx with pidfile /var/run/nginx.pid every 2 cycles 示例2：在上午8點到下午7點之間檢查每個工作日 check program checkOracleDatabase with path /var/monit/programs/checkoracle.pl every "* 8-19 * * 1-5" 示例3：在星期日0AM到3AM之間不要在備份窗口中運行檢查，否則運行具有常規輪詢周期頻率的檢查。 check process mysqld with pidfile /var/run/mysqld.pid not every "* 0-3 * * 0" 注意不要使用特定的分鍾，因為Monit可能不會在那分鍾運行。
· WEB UI界面
參考：
http://www.cnblogs.com/52fhy/p/6412547.html
http://blog.csdn.net/senlin1202/article/details/54291176
https://mmonit.com/monit/documentation/monit.html