Elastalert安裝及使用

本文轉載自查看原文 2018-11-30 11:21 665 elastAlert

洮生關注

2018.02.08 10:21 字數 1497 閱讀 2982評論 1喜歡 3

簡介

Elastalert是用python2寫的一個報警框架(目前支持python2.6和2.7，不支持3.x)，github地址為 https://github.com/Yelp/elastalert。他提供不同場景的規則配置，若覺得規則、告警不滿足需求時，可以用python編寫插件Adding a New Rule Type、Adding a New Alerter。
Elastalert內置的告警方式：

Email
JIRA
OpsGenie
Commands
HipChat
MS Teams
Slack
Telegram
AWS SNS
VictorOps
PagerDuty
Exotel
Twilio
Gitter

前置條件

Elasticsearch
Python 2.7
pip

pip 是 Python 包管理工具，該工具提供了對Python 包的查找、下載、安裝、卸載的功能。
目前如果你在 python.org 下載最新版本的安裝包，則是已經自帶了該工具。
Python 2.7.9 + 或 Python 3.4+ 以上版本都自帶 pip 工具。
pip 官網：https://pypi.org/project/pip/
你可以通過以下命令來判斷是否已安裝：
pip --version
如果你還未安裝，則可以使用以下方法來安裝：
$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py # 下載安裝腳本 $ sudo python get-pip.py    # 運行安裝腳本
注意：用哪個版本的 Python 運行安裝腳本，pip 就被關聯到哪個版本，如果是 Python3 則執行以下命令：
$ sudo python3 get-pip.py    # 運行安裝腳本。
一般情況 pip 對應的是 Python 2.7，pip3 對應的是 Python 3.x。
部分 Linux 發行版可直接用包管理器安裝 pip，如 Debian 和 Ubuntu：
sudo apt-get install python-pip

sudo easy_install -U setuptools
sudo yum install python-devel
python-pip python-dev libffi-dev libssl-dev

安裝

直接使用pip安裝：

$ pip install elastalert  直接裝的官網發布的，不好用，還是下載源碼安裝

或者克隆源碼安裝：

$ git clone https://github.com/Yelp/elastalert.git

安裝模塊：

$ pip install "setuptools>=11.3" or sudo .... $ python setup.py install

根據不同的Elasticsearch版本安裝elasticsearch-py。
Elasticsearch 5.0+:

$ pip install "elasticsearch>=5.0.0"

Elasticsearch 2.X:

$ pip install "elasticsearch<3.0.0"

補充：
若出現如下問題：

error: six 1.9.0 is installed but six>=1.10.0 is required by set(['jira'])

解決辦法：

$ pip install --upgrade six

更新的位置不是 python lib找的位置：/usr/lib/python2.7/site-packages/pkg_resources            /root/.local/lib/python2.7/site-packages
1，
sudo pip install --upgrade six
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Requirement already up-to-date: six in /root/.local/lib/python2.7/site-packages (1.12.0)

2，
elastalert-create-index
Traceback (most recent call last):
  File "/usr/bin/elastalert-create-index", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module>
    @_call_aside
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)

解決方法：

pip uninstall  six

pip install --upgrade six

卸載后重新安裝就行。

然后繼續安裝即可。

如果在windows 64平台報錯：執行

pip install python-magic-bin==0.4.14
修復https://stackoverflow.com/questions/18374103/exception-valuefailed-to-find-libmagic-check-your-installation-in-windows-7

E:\elastalert\example_rules> python -m elastalert.elastalert --verbose --rule config.yaml
Traceback (most recent call last):
  File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "C:\Python27\lib\runpy.py", line 72, in _run_code exec code in run_globals File "C:\Python27\lib\site-packages\elastalert\elastalert.py", line 21, in <module> from alerts import DebugAlerter File "C:\Python27\lib\site-packages\elastalert\alerts.py", line 31, in <module> from thehive4py.api import TheHiveApi File "C:\Python27\lib\site-packages\thehive4py\api.py", line 7, in <module> import magic File "C:\Python27\lib\site-packages\magic.py", line 181, in <module> raise ImportError('failed to find libmagic. Check your installation') ImportError: failed to find libmagic. Check your installation

重命名配置文件：

$ cp config.yaml.example config.yaml

打開配置文件config.yaml：

$ vim config.yaml

設置es_host、es_host指向你的Elasticsearch，然后保存退出。

配置文件解析

此處的配置文件是指config.yaml文件。
rules_folder：ElastAlert將加載規則配置文件的地方，它將嘗試加載文件夾中的每個.yaml文件。
run_every：ElastAlert查詢Elasticsearch的頻率。
buffer_time：是查詢窗口的大小，從每個查詢運行的時間向后延伸。對於其中use_count_query或use_terms_query設置為true的規則，此值將被忽略。
es_host：是Elasticsearch集群的地址，ElastAlert將存儲有關其狀態、查詢運行、警報和錯誤的數據。每個規則也可以設置不同的elasticsearch主機進行查詢。
es_port：Elasticsearch對應的端口。
use_ssl：（可選的）是否使用TLS;連接到es_host；設置為True或False。
verify_certs：（可選的）是否驗證TLS證書; 設置為True或False，默認是True。
client_cert：（可選的）PEM證書的路徑。
client_key：（可選的）作為客戶端密鑰使用的私鑰文件的路徑。
ca_certs：（可選的）用於驗證SSL連接的CA證書的路徑。
es_username：（可選的）用於連接Elasticsearch的basic-auth用戶名。
es_password：（可選的）用於連接Elasticsearch的密碼。
es_url_prefix：（可選的） Elasticsearch端點的URL前綴。
es_send_get_body_as：（可選的）查詢Elasticsearch方法- GET，POST或source，默認是GET。
writeback_index：是ElastAlert將存儲數據的索引名稱。
alert_time_limit：是失敗警報的重試窗口。

創建Elasticsearch索引

elastalert-create-index這個命令會在elasticsearch創建索引，便於ElastAlert將有關其查詢及其警報的信息和元數據保存回Elasticsearch。這不是必須的步驟，但是強烈建議創建。因為對於審計，測試很有用，並且重啟elastalert不影響計數和發送alert。默認情況下，創建的索引叫 elastalert_status。

$ elastalert-create-index
New index name (Default elastalert_status) Name of existing index to copy (Default None) New index elastalert_status created Done!

創建成功之后，我們可以在Elasticsearch中看到所創建的索引elastalert_status：

$ curl 'localhost:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana _4Op-xRCSzyC5nI2wZSPvw 1 1 2 0 8.4kb 8.4kb yellow open elastalert_status ZTetmp8mTS6F28aIBJmJOg 5 1 0 0 810b 810b

elastalert_status
ElastAlert 根據elastalert_status去確定首次啟動的時候在什么時間范圍內去查詢，以避免重復查詢。對於每個規則，它將從最近的結束時間開始查詢。包括：

@timestamp：文件上傳到Elasticsearch的時間。這是在運行查詢並且已經處理結果之后。
rule_name：相應規則的名稱。
starttime：查詢的開始時間戳。
endtime：查詢結束時間戳。
hits：查詢結果的數量。
matches：處理命中后規則返回的匹配數。請注意，這並不一定意味着警報被觸發。
time_taken：此查詢運行所需的秒數。

測試Demo

本次elastalert安裝目錄為：/opt/elastalert。

配置SMTP

在/opt/elastalert目錄下創建文件smtp_auth_file.yaml，內容如下：

#郵箱用戶名 user: example@email.com #不是郵箱密碼，是設置的SMTP密碼 password: XXXXX

配置規則

在example_rules目錄下修改配置文件，如下：

$ cd /opt/elastalert/example_rules
$ vim example_frequency.yaml

修改后的配置文件：

# Alert when the rate of events exceeds a threshold # (Optional) # Elasticsearch host # es_host: elasticsearch.example.com # (Optional) # Elasticsearch port # es_port: 9200 # (OptionaL) Connect with SSL to Elasticsearch #use_ssl: True # (Optional) basic-auth username and password for Elasticsearch #es_username: someusername #es_password: somepassword # (Required) # Rule name, must be unique name: Example rule # (Required) # Type of alert. # the frequency rule type alerts when num_events events occur with timeframe time type: frequency # (Required) # Index to search, wildcard supported index: logstash-2017.08.28 # (Required, frequency specific) # Alert when this many documents matching the query occur within a timeframe num_events: 1 # (Required, frequency specific) # num_events must occur within this amount of time to trigger an alert timeframe: minutes: 1 # (Required) # A list of Elasticsearch filters used for find events # These filters are joined with AND and nested in a filtered query # For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html #匹配日志中field字段值為value的日志數據，若不想進行過濾可以寫成“filter: []”（這種方式可以用在日志數量的告警中） filter: - query: query_string: query: "field: value" #SMTP configration smtp_host: smtp.163.com smtp_port: 25 #SMTP auth smtp_auth_file: /opt/elastalert/smtp_auth_file.yaml email_reply_to: example@email.com from_addr: example@email.com # (Required) # The alert is use when a match is found alert: - "email" # (required, email specific) # a list of email addresses to send alerts to email: - "test@email.com"

保存修改並退出。
補充：
上述規則表示：在elastalert執行的一分鍾內，出現一條有field值為value日志，則觸發告警，並且告警通知將以email的形式從example@email.com郵箱發送給test@email.com。
規則配置解析：

es_host、es_port：應該指向我們要查詢的Elasticsearch集群。
name：是這個規則的唯一名稱。如果兩個規則共享相同的名稱，ElastAlert將不會啟動。
type：每個規則都有不同的類型，可能會采用不同的參數。該frequency類型表示“在timeframe時間內匹配成功次數超過num_events發出警報”。有關其他類型的信息，請參閱規則類型。
index：要查詢的索引的名稱。
num_events：此參數特定於frequency類型，是觸發警報時的閾值。
timeframe：timeframe是num_events必須發生的時間段。
filter：是用於過濾結果的Elasticsearch過濾器列表。有關
詳細信息，請參閱編寫過濾規則。
alert：警報。有關警報類型的更多信息，請參閱警報。電子郵件警報需要SMTP服務器才能發送郵件。默認情況下，它將嘗試使用localhost。這可以通過smtp_host選項更改。
email：是要發送警報的地址列表。

還有許多其他可選配置選項，請參閱常見配置選項。

myrule.yaml和config.yaml

# From example_rules/example_frequency.yaml
es_host: xxx.xxx
es_port: 9200
name: Example rule
type: frequency
index: myindex
timestamp_field: mytime
num_events: 1
timeframe:
  minutes: 50
filter:
- term:
    "carColor" : "blue"
alert:
- "email"
email:
- "elastalert@example.com"

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: example_rules

# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  minutes: 1

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15

# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
es_host: 211.100.75.204

# The Elasticsearch port
es_port: 9200

# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1

# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test

# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to Elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# Use SSL authentication with client certificates client_cert must be
# a pem file containing both cert and key for client
#verify_certs: True
#ca_certs: /path/to/cacert.pem
#client_cert: /path/to/client_cert.pem
#client_key: /path/to/client_key.key

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

更全的示例：

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
es_host: 110.10.21.77

# (Optional)
# Elasticsearch port
es_port: 9200

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# (Required)
# Rule name, must be unique
name:  system error 

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency

# (Required)
# Index to search, wildcard supported
index:  task*

attach_related: true
# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
num_events: 1

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
  hours: 1
timestamp_field: "@timestamp"
# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
filter:
- terms:
    "Level": ["fatal", "error"]

#use_kibana_dashboard: afAlertDashboard
use_kibana4_dashboard: https://nodejsgbl.italkbb.com/kibana/app/kibana#/dashboard/3bde48d0-9880-11e9-b5d5-2df46b09dea6
# (Required)
# The alert is use when a match is found
alert:

- "email"

# (required, email specific)
# a list of email addresses to send alerts to
email:
- "xxxx@net263.com"
- "xxxx@163.com"

smtp_host: smtp.263.net
smtp_port: 25
smtp_auth_file: ../smtp_auth_file.yaml
email_reply_to: No.reply@net263.com
from_addr: ElastAlert@net263.com
#cc: aaa@net263.com
email_format: html
#alert_text_type: exclude_fields #alert_text_only
alert_subject: "Alert: System {0} occurred {1} times."
alert_subject_args:
- Level #來自index里面json內容
- "num_hits"
alert_text: "<pre> You can access all of error log on Kibana: <a href={0}> Aijia server alert dashboard</a><br>"
alert_text_args:
- kibana_link

{"indexId":"1000001","mytime": "2019-01-14T17:40:17+08:00","carColor":"blue","@timestamp":"2018-12-18T07:48:17","cost":1001,"indexName":"default_log_index" }

{ "indexName" : "af-test",     "code" : "1",          "msg" : "testMsg",          "position" : "AHC.User.ReadService",          "eventType" : "alarm","eventTimestamp" : "2019-01-14T17:40:17+08:00",          "@timestamp" : "2019-01-14T17:44:24.6172215+08:00"}

key要區分大小寫，value 寫過濾條件時候，全部小寫，否則查詢不到。如 term: "msg":"testmsg" 才行

測試規則

命令如下：

$ elastalert-test-rule example_rules/example_frequency.yaml

能夠檢查規則是否正確，有關詳細信息測試部分。

運行

運行elastalert：

$ cd /opt/elastalert
$ python -m elastalert.elastalert --verbose --rule example_frequency.yaml

或者運行指定目錄下所有yaml文件：
 elastalert --verbose --config ./config.yaml  --start 2019-06-27T06:00:03
其中config.yaml里面存放着rules的目錄。仔細看好目錄名稱不要寫錯。

調試elasticsearch 的 查詢 語句是否有問題，可以加上參數 es_debug、es_debug_trace trace后面接文件名，里面會輸出查詢語句。

trace文件打印出查詢語句：

curl -XGET 'http://localhost:9200/?pretty' -d ''
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "must": [
                  "gt": "2019-07-03T09:30:38.664486Z",
                  "lte": "2019-07-03T09:32:03.508890Z"
                }
              }
            },
            {
              "query_string": {
                "query": "NOT AggregatedHealthState.keyword: Ok"
              }
            }
          ]
        }
      }
    }
  },
  "sort": [
    {
      "@timestamp": {
        "order": "asc"
      }
    }
  ]
}


給Elasticsearch添加測試數據：

$ curl -X POST "http://127.0.0.1:9200/logstash-2017.08.28/test" -d '{ "@timestamp": "2017-08-28T10:54:41.000Z", "field": "value" }'

注意：上述的@timestamp是UTC時間，也就是說這條數據是在2017-08-28 18:54:41產生的。

Alert

可在郵箱中看到如下圖所示的郵件：

收到郵件中會包含當前符合告警規則的日志數據。

參考資料

ElastAlert - Easy & Flexible Alerting With Elasticsearch
Rule Types and Configuration Options
Writing Filters For Rules
https://www.cnblogs.com/zhaishaomin/p/7417306.html

https://blog.csdn.net/pujiaolin/article/details/52252950?locationNum=3

https://blog.csdn.net/gamer_gyt/article/details/52917116

https://blog.csdn.net/davinciyxw/article/details/79104498

https://blog.csdn.net/vbaspdelphi/article/details/54311764

http://www.freebuf.com/sectool/164591.html

alert還有kibana插件,以及elasticalert還可以容器方式啟動

參考:
https://github.com/bitsensor/elastalert-kibana-plugin
插件下載慢:

https://git.bitsensor.io/front-end/elastalert-kibana-plugin/-/jobs/10874/artifacts/raw/artifact/elastalert-5.6.4-latest.zip 然后 ./bin/kibana-plugin install file:///usr/local/src/elastalert-5.6.4-latest.zip

安裝后效果:

微信報警：https://github.com/anjia0532/elastalert-wechat-plugin
釘釘報警：https://github.com/xuyaoqiang/elastalert-dingtalk-plugin

告警規則
告警模板
告警時間
告警方式
-----
參考:https://xizhibei.github.io/2017/11/19/alerting-with-elastalert/

https://github.com/chenryn/ELKstack-guide-cn/blob/master/elasticsearch/other/elastalert.md

安裝elastalert（按照官網的操作走）

先切換到自己的工作目錄

我這里是 /usr/local/dev/

git 克隆源碼

git clone https://github.com/Yelp/elastalert.git

cd elastalert

安裝

sudo python setup.py install

出錯了

Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2

The required version of setuptools (>=1.1.6) is not available,

and can’t be installed while this script is running. Please

install a more recent version first, using

‘easy_install -U setuptools’.

(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))

ok按照它提示的，，，執行：

sudo easy_install -U setuptools

再執行安裝（sudo python setup.py install），又出錯了

warning: no files found matching 'blist.rst'

blist/_blist.c:38:20: 致命錯誤：Python.h：沒有那個文件或目錄

#include<Python.h>

這其實是缺少開發包，執行命令：

sudo yum install python-devel

再執行安裝（sudo python setup.py install）

到這里基本能安裝成功。

有時候會報一些包找不到的錯誤，可能是網絡問題，重復執行安裝命令即可（我在阿里雲上遇到了）。

繼續執行安裝命令

pip install -r requirements.txt

找不到pip ，先安裝pip並更新

sudo yum -y install python-pip

sudo pip install --upgrade pip

安裝完繼續

sudo pip install -r requirements.txt

在elasticsearch中創建elastalert的日志索引

sudo elastalert-create-index

根據自己的情況，填入elasticsearch的相關信息，關於

elastalert_status部分直接回車默認的即可。

如下所示：

Enter elasticsearch host: log.example.com

Enter elasticsearch port: 9200

Use SSL? t/f: t

Enter optional basic-auth username (or leave blank): es_admin

Enter optional basic-auth password (or leave blank):

Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):

New index name? (Default elastalert_status)

Name of existing index to copy? (Default None)

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html

InsecureRequestWarning)

New index elastalert_status created

Done!

3.創建配置文件

sudo cp config.yaml.example config.yaml

sudo vi config.yaml

根據自己的具體情況進行修改

比如我這里的elasticsearch是ssl 並需要用戶登陸的，配置大致如下：

run_every:

minutes: 1

buffer_time:

minutes: 15

es_host: log.example.com

es_port: 9200

use_ssl: True

es_send_get_body_as: GET

es_username: es_admin

es_password: es_password

writeback_index: elastalert_status

alert_time_limit:

days: 2

以同樣的方式配置規則

cd example_rules/

sudo cp example_frequency.yaml my_rule.yaml

sudo vi my_rule.yaml

這里就是根據具體的elasticsearch的信息進行配置

es_host: log.example.com

es_port: 9200

use_ssl: True

es_username: es_admin

es_password: es_password

#name屬性要求唯一，這里最好能標示自己的產品

name: My-Product Exception Alert

#類型，我選擇任何匹配的條件都發送郵件警告

type: any

#需要監控的索引，支持通配

index: logstash-*

#下面兩個隨意配置

num_events: 50

timeframe:

hours: 4

#根據條件進行過濾查詢（這里我只要出現異常的日志，並且排除業務異常（自定義異常））

filter:

- query:

query_string:

query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"

#email的警告方式

alert:

- "email"

#增加郵件內容，這里我附加一個日志訪問路徑

alert_text: "Ref Log https://log.example.com:5601/app/kibana"

#SMTP協議的郵件服務器相關配置（我這里是騰訊企業郵箱）

smtp_host: smtp.exmail.qq.com

smtp_port: 25

#用戶認證文件，需要user和password兩個屬性

smtp_auth_file: smtp_auth_file.yaml

email_reply_to: no-reply@example.com

from_addr: no-reply@example.com

#需要接受郵件的郵箱地址列表

email:

- "user1@example.com"

接下來創建smtp_auth_file.yaml

sudo touch smtp_auth_file.yaml

sudo vi smtp_auth_file.yaml

##配置文件內容

user: "no-reply@example.com"

password: "password"

現在可以簡單測試一下配置文件是否正確

sudo elastalert-test-rule ./my_rule.yaml

如果有問題，就檢查配置文件。

然后啟動運行一下試試：

sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml

然后故意產生一下異常試試，比如

if(true){

throw new NullPointerException("測試日志異常監控服務");

}

執行一下，一般沒啥問題，郵件應該就能收到了，有問題就檢查配置文件。

4.我是要監控服務，所以要讓elastalert以服務的形式運行

首先創建elastalert的配置目錄和相關文件

sudo mkdir /etc/elastalert

cd /etc/elastalert

-- 復制配置文件

sudo cp /usr/local/dev/elastalert/config.yaml config.yaml

sudo mkdir rules

cd rules

-- 復制規則文件

sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml

-- 復制郵件用戶認證文件

sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml

接下來修改配置文件

修改 config.yaml 中

rules_folder: /etc/elastalert/rules

修改 my_rule.yaml中

smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml

接下來就是創建systemd服務了

cd /etc/systemd/system

sudo touch elastalert.service

sudo vi elastalert.service

elastalert.service 內容

[Unit]

Description=elastalert

After=elasticsearch.service

[Service]

Type=simple

User=root

Group=root

Restart=on-failure

WorkingDirectory=/usr/local/dev/elastalert

ExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml

[Install]

WantedBy=multi-user.target

保存退出。

啟動服務

sudo systemctl start elastalert

查看狀態

sudo systemctl status elastalert

不出意外的話，你將看到綠色的Active: active (running)

至此，日志的異常監控服務就搭建好了，好的，收工！

---------------------

作者：夢得溪

來源：CSDN

原文：https://blog.csdn.net/pujiaolin/article/details/52252950

安裝elastalert（按照官網的操作走）
先切換到自己的工作目錄我這里是 /usr/local/dev/ git 克隆源碼
git clone https://github.com/Yelp/elastalert.git
cd elastalert123安裝
sudo python setup.py install1出錯了 Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2 The required version of setuptools (>=1.1.6) is not available, and can’t be installed while this script is running. Please install a more recent version first, using ‘easy_install -U setuptools’.
(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))
ok按照它提示的，，，執行：
sudo easy_install -U setuptools1再執行安裝（sudo python setup.py install），又出錯了
warning: no files found matching 'blist.rst'blist/_blist.c:38:20: 致命錯誤：Python.h：沒有那個文件或目錄 #include<Python.h>123這其實是缺少開發包，執行命令：
sudo yum install python-devel1再執行安裝（sudo python setup.py install）到這里基本能安裝成功。有時候會報一些包找不到的錯誤，可能是網絡問題，重復執行安裝命令即可（我在阿里雲上遇到了）。
繼續執行安裝命令
pip install -r requirements.txt1找不到pip ，先安裝pip並更新
sudo yum -y install python-pipsudo pip install --upgrade pip12安裝完繼續
sudo pip install -r requirements.txt1在elasticsearch中創建elastalert的日志索引
sudo elastalert-create-index1根據自己的情況，填入elasticsearch的相關信息，關於 elastalert_status部分直接回車默認的即可。如下所示：
Enter elasticsearch host: log.example.comEnter elasticsearch port: 9200Use SSL? t/f: tEnter optional basic-auth username (or leave blank): es_adminEnter optional basic-auth password (or leave blank):Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):New index name? (Default elastalert_status)Name of existing index to copy? (Default None)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)New index elastalert_status createdDone!1234567891011121314151617181920212223243.創建配置文件
sudo cp config.yaml.example config.yaml
sudo vi config.yaml123根據自己的具體情況進行修改
比如我這里的elasticsearch是ssl 並需要用戶登陸的，配置大致如下：
run_every: minutes: 1
buffer_time: minutes: 15
es_host: log.example.com
es_port: 9200
use_ssl: True
es_send_get_body_as: GET
es_username: es_admin
es_password: es_password
writeback_index: elastalert_status
alert_time_limit: days: 212345678910111213141516171819202122以同樣的方式配置規則
cd example_rules/
sudo cp example_frequency.yaml my_rule.yaml
sudo vi my_rule.yaml12345這里就是根據具體的elasticsearch的信息進行配置
es_host: log.example.comes_port: 9200use_ssl: Truees_username: es_admines_password: es_password#name屬性要求唯一，這里最好能標示自己的產品name: My-Product Exception Alert#類型，我選擇任何匹配的條件都發送郵件警告type: any#需要監控的索引，支持通配index: logstash-*#下面兩個隨意配置num_events: 50timeframe: hours: 4#根據條件進行過濾查詢（這里我只要出現異常的日志，並且排除業務異常（自定義異常））filter:- query: query_string: query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"#email的警告方式alert:- "email"
#增加郵件內容，這里我附加一個日志訪問路徑alert_text: "Ref Log https://log.example.com:5601/app/kibana"#SMTP協議的郵件服務器相關配置（我這里是騰訊企業郵箱）smtp_host: smtp.exmail.qq.comsmtp_port: 25#用戶認證文件，需要user和password兩個屬性smtp_auth_file: smtp_auth_file.yamlemail_reply_to: no-reply@example.comfrom_addr: no-reply@example.com
#需要接受郵件的郵箱地址列表email:- "user1@example.com"- "user1@example.com"1234567891011121314151617181920212223242526272829303132333435363738接下來創建smtp_auth_file.yaml
sudo touch smtp_auth_file.yamlsudo vi smtp_auth_file.yaml
##配置文件內容user: "no-reply@example.com"password: "password"123456現在可以簡單測試一下配置文件是否正確
sudo elastalert-test-rule ./my_rule.yaml1如果有問題，就檢查配置文件。然后啟動運行一下試試：
sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml1然后故意產生一下異常試試，比如
if(true){ throw new NullPointerException("測試日志異常監控服務");}123執行一下，一般沒啥問題，郵件應該就能收到了，有問題就檢查配置文件。
4.我是要監控服務，所以要讓elastalert以服務的形式運行首先創建elastalert的配置目錄和相關文件
sudo mkdir /etc/elastalertcd /etc/elastalert-- 復制配置文件
sudo cp /usr/local/dev/elastalert/config.yaml config.yamlsudo mkdir rulescd rules
-- 復制規則文件sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml
-- 復制郵件用戶認證文件sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml12345678910111213接下來修改配置文件修改 config.yaml 中
rules_folder: /etc/elastalert/rules1修改 my_rule.yaml中
smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml1接下來就是創建systemd服務了
cd /etc/systemd/systemsudo touch elastalert.service
sudo vi elastalert.service1234elastalert.service 內容
[Unit]Description=elastalertAfter=elasticsearch.service
[Service]Type=simpleUser=rootGroup=rootRestart=on-failureWorkingDirectory=/usr/local/dev/elastalertExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml
[Install]WantedBy=multi-user.target1234567891011121314保存退出。啟動服務
sudo systemctl start elastalert1查看狀態
sudo systemctl status elastalert1不出意外的話，你將看到綠色的Active: active (running)
至此，日志的異常監控服務就搭建好了，好的，收工！--------------------- 作者：夢得溪來源：CSDN 原文：https://blog.csdn.net/pujiaolin/article/details/52252950 版權聲明：本文為博主原創文章，轉載請附上博文鏈接！

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 elastalert安裝使用安裝elastalert elastalert docker安裝 kibana 安裝 elastalert 插件 Elastic：使用ElastAlert發送通知 elasticsearch5之Elastalert 安裝使用配置郵件報警和微信報警使用ElastAlert+ELK實現日志監控釘釘告警 Elastalert 監控 ElastAlert規則 ELK7.11.2版本安裝部署及ElastAlert告警相關配置