ELK6.6.0+filebeat6.6.0部署

elastic不能用root用戶去啟動，否則會報錯，所以創建elastic用戶ES集群部署

1.創建elastic用戶

$ useradd elastic
$ passwd elastic

2..部署JDK環境

$ tar xvf jdk-8u191-linux-x64.tar.gz -C /usr/local/
$ mv /usr/local/jdk1.8.0_191/ /usr/local/java
$ vim /etc/profile.d/elk.sh 
export JAVA_HOME=/usr/local/java
export PATH=$JAVA_HOME:$PATH
$ source /etc/profile

3.下載elastic源碼包

$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.tar.gz

4.解壓elastic

$ tar xvf elasticsearch-6.6.0.tar.gz -C /usr/local/
$ mv /usr/local/elasticsearch-6.6.0/ /usr/local/elastic
$ chown -R elastic /usr/local/elastic/

5.修改elastic內存配置

elasticsearch6.6.0默認內存需要1G，如果沒有1G內存可能會報錯，如果內存不夠則需要修改配置文件

$ vim /usr/local/elastic/config/jvm.options
-Xms512m
-Xmx512m

6.修改elastic配置文件

$ mkdir /data/es-data -p
$ mkdir /var/log/elastic/
$ vim /usr/local/elastic/config/elasticsearch.yml 
# 組名自定義，但是同一個組，組名必須一致
cluster.name: my-application
# 節點名稱，建議和主機名一致
node.name: elastic
# 數據存放目錄
path.data: /data/es-data
# 日志存放路徑
path.logs: /var/log/elastic
# 鎖住內存，bubei 使用到交換分區去
bootstrap.memory_lock: true
# 由於只部署兩個節點，因此設置為1，否則當master宕機，將無法重新選取master
discovery.zen.minimum_master_nodes: 1
​
# 網絡設置
network.host: 0.0.0.0
# 端口
http.port: 9200
​
# 從節點配置
# 關閉多播
discovery.zen.ping.unicast.enabled: false
# 發單播，ip地址是master和自己
discovery.zen.ping.unicast.hosts: ["192.168.1.131", "192.168.1.164"]

6.啟動elastic

$ chown -R elastic /data/
$ chown -R elastic /var/log/elastic/
$ su - elastic
$ /usr/local/elastic/bin/elasticsearch

7.測試

8.報錯

[1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]

#切換回root
$ vim /etc/security/limits.conf 
# elastic是用戶
elastic soft nofile 65536
elastic hard nofile 65536
# 登錄elastic查看
$ ulimit -Hn
65536

[2]: memory locking requested for elasticsearch process but memory is not locked

$ vim /etc/security/limits.conf 
elastic         -        memlock         unlimited

[3]: max number of threads [3802] for user [elastic] is too low, increase to at least [4096]

$ vim /etc/security/limits.d/20-nproc.conf
elastic    -       nproc     4096

[4]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

$ vim /etc/sysctl.conf
vm.max_map_count=655360
$ sysctl -p

插件安裝

elastic5.0之后，head插件需要獨立安裝

1.head插件

# 安裝NodeJS
$ wget https://npm.taobao.org/mirrors/node/latest-v4.x/node-v4.5.0-linux-x64.tar.gz
$ tar -zxvf node-v4.5.0-linux-x64.tar.gz -C /usr/local/
$ mv /usr/local/nodenode-v4.5.0-linux-x64 /usr/local/node
$ vim /etc/profile
    export NODE_HOME=/usr/local/node
    export PATH=$PATH:$NODE_HOME/bin/
    export NODE_PATH=$NODE_HOME/lib/node_modules
$ source /etc/profile
​
# 安裝npm
$ npm install -g cnpm --registry=https://registry.npm.taobao.org
​
# 安裝grunt
$ npm install -g grunt
$ npm install -g grunt-cli --registry=https://registry.npm.taobao.org --no-proxy
​
# 確認版本
node -v
v9.5.0
$ npm -v
5.6.0
$ grunt -version
grunt-cli v1.3.2
grunt v1.0.1
​
​
$ wget https://github.com/mobz/elasticsearch-head/archive/master.zip
$ unzip master.zip
$ cd elasticsearch-head-master/
# npm install -g cnpm --registry=https://registry.npm.taobao.org
$ npm install
​
# 修改es的配置文件
# head插件可以訪問es
$ vim /usr/local/elasticsearch-6.6.0/config/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
​
# 修改head插件配置文件
$ vim Gruntfile.js
# 增加一行hostname
               connect: {
                        server: {
                                options: {
                                        hostname: '0.0.0.0',
                                        port: 9100,
                                        base: '.',
                                        keepalive: true
                                }
                        }
                }
​
$ vim elasticsearch-head-master/_site/app.js
# 修改localhost為es的ip地址
# this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.1.126:9200";
# 重啟es
$ grunt server 

　　

 

 

Filebeat+Logstash部署

1.Filebeat部署

a.下載filebeat源碼包

$ wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.0-linux-x86_64.tar.gz

b.解壓源碼包

$ tar xvf filebeat-6.6.0-linux-x86_64.tar.gz -C /usr/local/
$ mv /usr/local/filebeat-6.6.0-linux-x86_64/ /usr/local/filebeat

c.修改配置文件

$ vim /usr/local/filebeat/filebeat.yml
#=========================== Filebeat inputs =============================
​
filebeat.inputs:
- type: log
  # Change to true to enable this input configuration.
  enabled: true
​
  # Paths that should be crawled and fetched. Glob based paths.
  # 指定讀取文件的位置
  paths:
    - /var/log/*.log
  # 只發送包含ERR,WARN字樣的日志
  # include_lines: ['^ERR', '^WARN']
  # 不發送包含OK字樣的日志
  # exclude_lines: ["^OK"]
  # 定義寫到ES時的type值
  # document_type: "test" 
​
# 輸出的位置，直接輸出到elastic的話，選第一個，輸出到logstash的話，選第二個 
  #output.elasticsearch:
    # Array of hosts to connect to.
    #hosts: ["localhost:9200"]
  
  output.logstash:
    # The Logstash hosts
    hosts: ["localhost:5044"]

　　

d.啟動filebeat

$ vim /etc/profile.d/elk.sh
export PATH=$PATH:/usr/local/filebeat/
$ source /etc/profile
$ filebeat -e -c /usr/local/filebeat/filebeat.yml

2.Logstash部署

a.部署JDK環境

$ tar xvf jdk-8u191-linux-x64.tar.gz -C /usr/local/
$ mv /usr/local/jdk1.8.0_191/ /usr/local/java
$ vim /etc/profile.d/elk.sh 
export JAVA_HOME=/usr/local/java
export PATH=$JAVA_HOME:$PATH:/usr/local/filebeat/
$ source /etc/profile

b.下載Logstash源碼包

$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.6.0.tar.gz

c.解壓源碼包

$ tar xvf logstash-6.6.0.tar.gz -C /usr/local/
$ mv /usr/local/logstash-6.6.0/ /usr/local/logstash

d.修改配置文件

input {
  beats {
    port => 5044
  }
}
output {
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => ["http://192.168.1.126:9200"]
    index => "test"
  }
}

Kibana部署

1. 下載Kibana

$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.0-linux-x86_64.tar.gz

2. 解壓源碼包

$ tar xvf kibana-6.6.0-linux-x86_64.tar.gz -C /usr/local/
$ mv /usr/local/kibana-6.6.0-linux-x86_64/ /usr/local/kibana

3. 修改配置文件

$ vim /usr/local/kibana/config/kibana.yml
server.host: "192.168.1.130"
elasticsearch.hosts: ["http://192.168.1.126:9200"]

4.啟動Kibana

$ /usr/local/kibana/bin/kibana

訪問 192.168.1.130:5601

 

PS：如有錯誤，歡迎指正