1.ELK簡介
ELK是三個開源軟件的縮寫，分別表示：Elasticsearch , Logstash, Kibana , 它們都是開源軟件。新增了一個FileBeat，它是一個輕量級的日志收集處理工具(Agent)，Filebeat占用資源少，適合於在各個服務器上搜集日志后傳輸給Logstash，官方也推薦此工具。
Elasticsearch是個開源分布式搜索引擎，提供搜集、分析、存儲數據三大功能。它的特點有：分布式，零配置，自動發現，索引自動分片，索引副本機制，restful風格接口，多數據源，自動搜索負載等。
Logstash 主要是用來日志的搜集、分析、過濾日志的工具，支持大量的數據獲取方式。一般工作方式為c/s架構，client端安裝在需要收集日志的主機上，server端負責將收到的各節點日志進行過濾、修改等操作在一並發往elasticsearch上去。
Kibana 也是一個開源和免費的工具，Kibana可以為 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面，可以幫助匯總、分析和搜索重要數據日志。
Filebeat隸屬於Beats。目前Beats包含四種工具：
2.Topbeat（搜集系統、進程和文件系統級別的 CPU 和內存使用情況等數據）
3.Filebeat（搜集文件數據）
4.Winlogbeat（搜集 Windows 事件日志數據）
3.環境准備
| 名稱 | IP |
|---|---|
| es01 | 10.0.0.267 |
| es02 | 10.0.0.268 |
| web01 | 10.0.0.269 |
4.搭建elk
4.1關閉防火牆和selinux
systemctl stop firewalld systemctl disable firewalld sed -i 's/=enforcing/=disabled/g' /etc/selinux/config setenforce 0
[root@es01-10.0.0.237 ~]# cd /etc/yum.repos.d/ [root@es01-10.0.0.237 ~]# wget http://mirrors.aliyun.com/repo/Centos-7.repo
[root@es01-10.0.0.237 /elk_soft]# yum install java-1.8.0-openjdk -y
[root@es01-10.0.0.237 ~]# cd /elk_soft/ [root@es01-10.0.0.237 /elk_soft]# ls elasticsearch-6.4.1.rpm kibana-6.4.1-x86_64.rpm logstash-6.4.1.rpm [root@es01-10.0.0.237 /elk_soft]# yum localinstall elasticsearch-6.4.1.rpm -y
[root@es02-10.0.0.238 /elk_soft]# vim /etc/elasticsearch/elasticsearch.yml cluster.name: my-cluster #集群名稱 node.name: es02 #節點名稱 path.data: /var/lib/elasticsearch #存放數據的目錄（如果修改將目錄的屬組屬主改為elasticearch） path.logs: /var/log/elasticsearch #日志目錄（如果修改將目錄的屬組屬主改為elasticearch） network.host: 10.0.0.238 #本機IP地址 http.port: 9200 #默認端口 discovery.zen.ping.unicast.hosts: ["10.0.0.237", "10.0.0.238"] #集群中每個node的IP地址 #在配置文件倒數第二行加入，如果不添加是不能發現別的主機，只能發現自己 http.cors.enabled: true http.cors.allow-origin: "*"
[root@es01-10.0.0.237 /elk_soft]# systemctl start elasticsearch.service [root@es01-10.0.0.237 /elk_soft]# systemctl enable elasticsearch.service [root@es01-10.0.0.237 /elk_soft]# netstat -lntup Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6141/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6222/master tcp6 0 0 10.0.0.237:9200 :::* LISTEN 8998/java tcp6 0 0 10.0.0.237:9300 :::* LISTEN 8998/java
5.安裝head插件來管理集群
5.1在es01上安裝head插件，配置epel源
wget http://mirrors.aliyun.com/repo/epel-7.repo
[root@es01-10.0.0.237 ~]# yum install nodejs npm openssl screen -y
[root@es01-10.0.0.237 ~]# node -v v6.17.1 [root@es01-10.0.0.237 ~]# npm -v 3.10.10
[root@es01-10.0.0.237 ~]# npm install -g cnpm --registry=https://registry.npm.taobao.org
[root@es01-10.0.0.237 ~]# yum install git -y [root@es01-10.0.0.237 ~]# cd /opt/ [root@es01-10.0.0.237 /opt]# git clone git://github.com/mobz/elasticsearch-head.git [root@es01-10.0.0.237 /opt]# cd elasticsearch-head/ [root@es01-10.0.0.237 /opt/elasticsearch-head]# ls crx elasticsearch-head.sublime-project index.html plugin-descriptor.properties _site Dockerfile Gruntfile.js LICENCE proxy src Dockerfile-alpine grunt_fileSets.js package.json README.textile test [root@es01-10.0.0.237 /opt/elasticsearch-head]# cnmp install [root@es01-10.0.0.237 /opt/elasticsearch-head]# screen -S es-head #切屏指令 [root@es01-10.0.0.237 /opt/elasticsearch-head]# cnpm run start > elasticsearch-head@0.0.0 start /opt/elasticsearch-head > grunt server Running "connect:server" (connect) task Waiting forever... Started connect web server on http://localhost:9100 ###Ctrl+A+D #切換快捷鍵 后台運行程序
5.7集群的三種健康狀態
-
綠色：表示集群健康
-
黃色：表示亞健康狀態，勉強可以運行
-
紅色：表示已經很危險，有可能數據已經丟失
6.elasticearch數據庫操作
6.1查入三條數據
curl -XPUT '10.0.0.237:9200/megacorp/employee/1?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Li", "last_name": "si", "age" : 28, "about" : "I love cat", "interests": [ "swimming" ] } ' curl -XPUT '10.0.0.237:9200/megacorp/employee/2?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Zhang", "last_name": "san", "age" : 22, "about" : "I love dog", "interests": [ "swimming" ] } ' curl -XPUT '10.0.0.237:9200/megacorp/employee/3?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Wang", "last_name": "wu", "age" : 30, "about" : "I love beautiful girl", "interests": [ "swimming" ] } '
6.3索引介紹
| es數據庫 | |
|---|---|
| index（索引） | database（數據庫） |
| type （類型） | tables（數據庫中的表） |
| id （id號） | 對應表中的字段 |
6.4如何查詢一條數據
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XGET '10.0.0.237:9200/megacorp/employee/1?pretty' { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "first_name" : "Li", "last_name" : "si", "age" : 28, "about" : "I love cat", "interests" : [ "swimming" ] } }
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XDELETE '10.0.0.237:9200/megacorp/employee/1?pretty' { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 2, "result" : "deleted", "_shards" : { "total" : 2, "successful" : 2, "failed" : 0 }, "_seq_no" : 1, "_primary_term" : 1 }
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XPUT '10.0.0.237:9200/student_message?pretty' { "acknowledged" : true, "shards_acknowledged" : true, "index" : "student_message" }
curl -XPUT '10.0.0.237:9200/my_temp_index?pretty' -H 'Content-Type: application/json' -d' { "settings": { "number_of_shards" : 2, "number_of_replicas" : 1 } } '
7.kibana安裝配置
7.1在es01上安裝kibana
[root@es01-10.0.0.237 /elk_soft]# yum localinstall kibana-6.4.1-x86_64.rpm -y
[root@es01-10.0.0.237 /elk_soft]# vim /etc/kibana/kibana.yml server.port: 5601 #服務監聽端口 server.host: "10.0.0.237" #服務IP地址 elasticsearch.url: "http://10.0.0.237:9200" #elasticsearch的地址 elasticsearch.pingTimeout: 30000000 logging.dest: stdout #如果指定日志存儲路徑，需要修改文件的所有者及所屬組
[root@es01-10.0.0.237 /elk_soft]# systemctl start kibana.service [root@es01-10.0.0.237 /elk_soft]# systemctl enable kibana.service [root@es01-10.0.0.237 /elk_soft]# ss -lnt State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:9100 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 10.0.0.237:5601 *:*
8.在web01上安裝nginx+logstash
8.1配置nginx的yum源，安裝nginx
[root@web01-10.0.0.239 /etc/yum.repos.d]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 enabled=1 [root@web01-10.0.0.239 ~]# yum install nginx -y
[root@web01-10.0.0.239 ~]# vim /etc/nginx/conf.d/elk.conf server { listen 80; server_name elk.test.com; location / { proxy_pass http://10.0.0.237:5601; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } # access_log /var/log/nginx/elk_access.log main; }
[root@web01-10.0.0.239 ~]# vim /etc/nginx/nginx.conf log_format access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"url":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"status":"$status"}'; access_log /var/log/nginx/ access_json;
[root@web01-10.0.0.239 ~]# systemctl start nginx [root@web01-10.0.0.239 ~]# systemctl enable nginx
10.0.0.239 elk.tast.com
[root@web01-10.0.0.239 /elk_soft]# yum install java-1.8.0-openjdk -y
[root@web01-10.0.0.239 /elk_soft]# ls filebeat-6.4.1-x86_64.rpm logstash-6.4.1.rpm
[root@web01-10.0.0.239 /elk_soft]# vim /etc/logstash/logstash.yml path.data: /var/lib/logstash #數據存放目錄（如需修改記得修改屬主屬組） http.host: "10.0.0.239" #http的IP地址 http.port: 9600 #監聽端口 path.logs: /var/log/logstash #日志存放目錄（如需修改記得修改屬主屬組）
[root@web01-10.0.0.239 ~]# vim /etc/logstash/conf.d/nginx_access.conf input { file { path => "/var/log/nginx/access_json.log" start_position => "end" type => "nginx_access" codec => json } } output { elasticsearch { hosts => ["10.0.0.237:9200"] index => "nginx_access-%{+YYYY.MM.dd}" } }
[root@web01-10.0.0.239 /etc/logstash]# sudo /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd Using provided startup.options file: /etc/logstash/startup.options Manually creating startup for specified platform: systemd Successfully created system startup script for Logstash [root@web01-10.0.0.239 /etc/logstash]# systemctl start logstash [root@web01-10.0.0.239 /etc/logstash]# systemctl enable logstash
[root@es01-10.0.0.237 /elk_soft]# systemctl restart elasticsearch.service
9.2.kibana平台展示所收到的日志信息
9.2.1首先添加一個索引模式
9.2.2添加成功
9.2.3添加一個餅狀圖
9.2.4創建排班表
9.2.5通過dashboard進行詳細展示
9.通過rdis安裝部署新的架構
9.1停掉web01上的logstash服務
[root@web01-10.0.0.239 ~]# systemctl stop logstash.service
9.2在web01上安裝filebeat
[root@web01-10.0.0.239 /elk_soft]# yum localinstall filebeat-6.4.1-x86_64.rpm -y
9.3修改filebeat配置文件
24 enabled: true 27 paths: 28 - /var/log/nginx/access_json.log ##配置文件最后添加 output.redis: hosts: ["10.0.0.237"] #數據要發給的主機 port: 6379 #Redis的端口 key: "nginx-log" #日志類型 db: 0 #兩端要一樣 timeout: 5 #超時時間5秒
9.4啟動filebeat服務
[root@web01-10.0.0.239 ~]# systemctl start filebeat.service [root@web01-10.0.0.239 ~]# systemctl enable filebeat.service
9.5在es01上安裝Redis
[root@es01-10.0.0.237 ~]# yum install redis -y
9.6修改Redis配置文件
[root@es01-10.0.0.237 ~]# vim /etc/redis.conf 61 bind 10.0.0.237 #redis本機ip地址 port 6379 #redis默認端口 128 daemonize yes #開啟守護進程
9.7啟動Redis
[root@es01-10.0.0.237 ~]# systemctl start redis [root@es01-10.0.0.237 ~]# systemctl enable redis [root@es01-10.0.0.237 ~]# redis-cli -h 10.0.0.237 10.0.0.237:6379>
9.8檢查Redis是否取到日志
10.0.0.237:6379> keys * 1) "nginx-log" 10.0.0.237:6379> info keyspace # Keyspace db0:keys=1,expires=0,avg_ttl=0
9.8在es01上安裝logstash
[root@es01-10.0.0.237 /elk_soft]# yum localinstall logstash-6.4.1.rpm -y
10修改logstash的配置文件
http.host: "10.0.0.237" #http的IP地址 http.port: 9600 #監聽端口
11在/etc/logstash/conf.d/下新增redis_input.conf文件
[root@es01-10.0.0.237 /etc/logstash/conf.d]# vim redis_input.conf input { redis { host => "10.0.0.237" port => "6379" db => "0" data_type => "list" key => "nginx-log" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["10.0.0.237:9200"] index => "nginx-redis-%{+YYYY.MM.dd}" } }
12啟動logstash
[root@es01-10.0.0.237 /etc/logstash/conf.d]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd [root@es01-10.0.0.237 /etc/logstash/conf.d]# systemctl start logstash [root@es01-10.0.0.237 /etc/logstash/conf.d]# systemctl enable logstash
13kibana展示
