elasticsearch集群安裝+安全驗證+kibana安裝

准備環境

• 啟動4個centos容器, 並暴露相對應端口 （我的本機ip為172.16.1.236，以下涉及到的地方需要修改為自己的ip）

node_name	ip	http port	transport port
es01	docker宿主機ip	9205:9200	9305: 9300
es02	docker宿主機ip	9206:9200	9306: 9300
es03	docker宿主機ip	9207:9200	9307: 9300

kibana : 5601:5601

• 執行以下命令啟動centos容器並暴露相應端口（使用-p 來對外映射docker容器端口）

docker run -tid --name centos1 -p 9205:9200 -p9305:9300 --privileged=true centos:latest /sbin/init
docker run -tid --name centos2 -p 9206:9200 -p9306:9300 --privileged=true centos:latest /sbin/init
docker run -tid --name centos3 -p 9207:9200 -p9307:9300 --privileged=true centos:latest /sbin/init
#centos4用於kibana安裝
docker run -tid --name centos4 -p 5601:5601 --privileged=true centos:latest /sbin/init

• 注意：如果使用shell工具連接，增加 -p22:22 參數
• 參考es快速啟動准備es安裝包等數據

配置修改

修改system.conf配置 /etc/systemd/system.conf

sed -i "s/#DefaultLimitMEMLOCK=/DefaultLimitMEMLOCK=infinity/g" /etc/systemd/system.conf

修改limits.cong配置 /etc/security/limits.conf

• 修改如下

# nofile 最大打開文件描述符數
# nproc 最大進程數
# memlock 最大內存鎖定

echo "* soft nofile 65536
* hard nofile 131072
* soft nproc 4096
* hard nproc 4096
* soft memlock unlimited
* hard memlock unlimited" >> /etc/security/limits.conf

修改sysctl配置 /etc/sysctl.conf

• 修改如下

echo "vm.max_map_count = 262145" >> /etc/sysctl.conf

#生效
sysctl -p

修改es占用內存 /opt/es/config/jvm.options

sed -i "s/-Xms1g/-Xms"800m"/g" /opt/es/config/jvm.options
sed -i "s/-Xmx1g/-Xmx"800m"/g" /opt/es/config/jvm.options

開啟ES內存鎖定 /opt/es/config/elasticsearch.yml

• 增加如下配置

bootstrap.memory_lock: true

ES配置文件詳解

節點類型

• 主節點
  候選主節點的設置方法是設置node.mater為true，默認情況下，node.mater和node.data的值都為true，即該節點既可以做候選主節點也可以做數據節點。由於數據節點承載了數據的操作，負載通常都很高，所以隨着集群的擴大，建議將二者分離，設置專用的候選主節點。當我們設置node.data為false，就將節點設置為專用的候選主節點了。

node.master: true
node.data: false

• 數據節點
  數據節點負責數據的存儲和相關具體操作，比如CRUD、搜索、聚合。所以，數據節點對機器配置要求比較高，首先需要有足夠的磁盤空間來存儲數據，其次數據操作對系統CPU、Memory和IO的性能消耗都很大。通常隨着集群的擴大，需要增加更多的數據節點來提高可用性。

node.master: false
node.data: true

• client節點
  即不會被選作主節點，也不會存儲任何索引數據。該節點只能處理路由請求，處理搜索，分發索引操作等，從本質上來說該客戶節點表現為智能負載平衡器。

 node.master = false
 node.data = false

配置集群名稱

• 不同的節點配置同一個cluster.name可組成同一個集群，確保不同的集群使用不同的cluster.name
• 配置如下 :

cluster.name: es-cluster-test

配置ES節點名稱

• node.name表示集群節點的名稱，集群中節點進行區分，如果不配置則默認為主機名
• 配置如下

node.name: es01

配置ES節點監聽地址

• 如果不配置，默認是監聽在127.0.0.1 和 [::1]，同時以development的方式啟動。

#監聽在指定ip上
network.host : 172.17.0.1

#監聽在所有ip上
network.host : 0.0.0.0

日志數據路徑配置

• 配置方式

path.data: /opt/data/es
path.logs: /opt/log/es

path:
    data: /opt/data/es
    logs: /opt/log/es

• path.data, 可以設置多個目錄

path:  
    logs: /opt/log/es
    data:    
        - /opt/data/es-A    
        - /opt/data/es-B   
        - /opt/data/es-C

集群發現配置

• discovery.seed_hosts 配置方式如下
  用於多個集群節點進行發現，組成集群

discovery.seed_hosts: ["192.168.1.10:9300", "192.168.1.11", "seeds.mydomain.com"]

discovery.seed_hosts:
   - 192.168.1.10:9300
   - 192.168.1.11
   - seeds.mydomain.com
  

• cluster.initial_master_nodes 配置方式如下
  用於集群在第一次啟動時，指定可以參與選舉的主節點列表 （node.master: true）

cluster.initial_master_nodes: ["es01", "es02", "es03"]

cluster.initial_master_nodes:
    -es01
    -es02
    -es03

• discovery.seed_hosts 如果不配置，會自動監聽本地回環地址 將本地多個elasticsearch實例加入到集群中。

jvm配置

• /opt/es/config/jvm.options (一般配置為機器內存大小的一半)

sed -i "s/-Xms1g/-Xms"800m"/g" /opt/es/config/jvm.options
sed -i "s/-Xmx1g/-Xmx"800m"/g" /opt/es/config/jvm.options

配置Elasticsearch（本次測試配置）

es01

cluster.name: es-cluster-test
node.name: es01
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9306", "172.16.1.236:9307"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

es02

cluster.name: es-cluster-test
node.name: es02
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9305", "172.16.1.236:9307"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

es03

cluster.name: es-cluster-test
node.name: es03
path.logs: /opt/log/es
path.data: /opt/data/es
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["172.16.1.236:9305", "172.16.1.236:9306"]
cluster.initial_master_nodes: ["es01", "es02", "es03"]

啟動Elasticsearch

啟動

• 分別進入不同的容器啟動elasticsearch
• 命令如下

/opt/es/bin/elasticsearch -d

查看容器狀況（容器內）

• 查看集群是否啟動成功

[elasticsearch@813bf8515935 /]$ curl localhost:9200/_cat/nodes
172.17.0.7 28 91  9 0.58 0.66 0.90 dilm - es02
172.17.0.6 15 91  5 0.58 0.66 0.90 dilm - es01
172.17.0.8 35 91 13 0.58 0.66 0.90 dilm * es03

• 查看當前節點

[elasticsearch@813bf8515935 /]$ curl localhost:9200
{
  "name" : "es03",
  "cluster_name" : "es-cluster-test",
  "cluster_uuid" : "Syj18FUrR1GdnGzghBIacQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

• 查看節點健康狀況

[elasticsearch@813bf8515935 /]$ curl localhost:9200/_cat/health?v
epoch      timestamp cluster         status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1605859559 08:05:59  es-cluster-test green           3         3      0   0    0    0        0             0
      -                100.0%

查看容器狀況（宿主機）

• 由於使用docker容器安裝，因此也可以通過宿主機映射端口訪問，本次測試宿主機為windows
  

開啟集群安全驗證

修改配置

• 分別在不同的Es節點修改配置開啟安全驗證

echo "xpack.security.enabled: true" >> /opt/es/config/elasticsearch.yml
echo "xpack.security.transport.ssl.enabled: true" >> /opt/es/config/elasticsearch.yml

生成證書

• 選擇其中一個節點生成CA證書，默認情況下生成證書放在 /opt/es

#創建ca證書   /opt/es/bin/elasticsearch-certutil ca -v
[elasticsearch@c824e845075b es]$ /opt/es/bin/elasticsearch-certutil ca -v
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]:    #輸入回車，使用默認
Enter password for elastic-stack-ca.p12 :      #回車，暫不輸入密碼

• 與上一步驟同一個節點創建節點見認證用的證書

#創建節點間證書  /opt/es/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
[elasticsearch@c824e845075b es]$ /opt/es/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.

    * All certificates generated by this tool will be signed by a certificate authority (CA).
    * The tool can automatically generate a new CA for you, or you can provide your own with the
         -ca or -ca-cert command line options.

By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -keep-ca-key (retain generated CA key)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Enter password for CA (elastic-stack-ca.p12) :      #暫留空
Please enter the desired output file [elastic-certificates.p12]: #默認
Enter password for elastic-certificates.p12 :        #ca證書密碼，此次為空

Certificates written to /opt/es/elastic-certificates.p12

This file should be properly secured as it contains the private key for
your instance.

This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

• 配置ES節點使用這個證書

# 將生成證書拷貝到 /opt/es/config/certs目錄下
[elasticsearch@c824e845075b es]$ mkdir -p  /opt/es/config/certs
[elasticsearch@c824e845075b config]$ mv /opt/es/elastic-* /opt/es/config/certs/
# 將certs目錄拷貝到其它Es節點 （在宿主機進行該操作）
#拷貝certs目錄到本地
PS C:\Users\Administrator> docker cp centos2:/opt/es/config/certs C:\Users\Administrator\Desktop
#拷貝certs目錄到其它兩個節點
PS C:\Users\Administrator> docker cp C:\Users\Administrator\Desktop\certs centos1:/opt/es/config
PS C:\Users\Administrator> docker cp C:\Users\Administrator\Desktop\certs centos3:/opt/es/config
#在每個節點修改 /opt/es/config/elasticsearch.yml 配置
[elasticsearch@813bf8515935 /]$ echo "xpack.security.transport.ssl.verification_mode: certificate 
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12" >> /opt/es/config/elasticsearch.yml

• 重啟

#在每個容器殺死elastic進程並重啟
[elasticsearch@c824e845075b config]$ kill -9 $(ps -ef | grep 'elasticsearch' | grep '/bin/java' | grep -v grep | awk '{print $2}')
#啟動
[elasticsearch@c824e845075b config]$ /opt/es/bin/elasticsearch -d

• 任意一個節點生成密碼

[elasticsearch@6ebd0bc8cc5d certs]$ /opt/es/bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

• 測試

#此時直接訪問會報無權限異常，需要增加 -u elastic 參數進行訪問
[elasticsearch@6ebd0bc8cc5d certs]$ curl localhost:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}

#增加 -u elastic
[elasticsearch@6ebd0bc8cc5d certs]$ curl -u elastic localhost:9200
Enter host password for user 'elastic':
{
  "name" : "es01",
  "cluster_name" : "es-cluster-test",
  "cluster_uuid" : "Syj18FUrR1GdnGzghBIacQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

kibana的安裝配置

准備

下載地址

• kibana版本號與elasticsearch相同，本文使用7.6.2
• 將主備好的kibana壓縮包移動到容器內，並進入到容器

PS C:\Users\Administrator> docker cp C:\Users\Administrator\Downloads\kibana-7.6.2-linux-x86_64.tar.gz centos4:/opt

PS C:\Users\Administrator> docker exec -it centos4 /bin/bash

• 解壓安裝

[root@db0759d8c6c8 /]# useradd kibana
[root@db0759d8c6c8 /]# chown -R kibana /opt
[root@db0759d8c6c8 /]# su kibana
[kibana@db0759d8c6c8 /]$ cd /opt/
[kibana@db0759d8c6c8 opt]$ tar -zxvf /opt/kibana-7.6.2-linux-x86_64.tar.gz -C /opt && mv kibana-7.6.2-linux-x86_64 kibana

配置文件修改並啟動

• kibana配置文件修改

[kibana@db0759d8c6c8 opt]$ vi kibana/config/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
server.name: "mykibana"
elasticsearch.hosts: ["http://172.16.1.236:9205"]
kibana.index: ".kibana"
elasticsearch.username: "kibana"
elasticsearch.password: "123123"
i18n.locale: "zh-CN"

• 啟動

[kibana@db0759d8c6c8 opt]$ /opt/kibana/bin/kibana

• 在宿主訪問 127.0.0.1:5601 進行測試
  

可以查看集群狀態信息