Elasticsearch 7.1.1 集群 + 配置身份驗證

一、安裝Elasticsearch

1.1 環境說明

Centos7.6

Elasticsearch7.1.1

 

#掛載數據盤

fdisk /dev/vdb
n,p,1,回車，回車，wq
fdisk -l
mkfs.ext4 /dev/vdb1 
echo '/dev/vdb1 /opt ext4 defaults 0 0' >>/etc/fstab
mount -a
df -h

 

#時間同步

yum install -y ntp 
systemctl enable ntpd && systemctl start ntpd
timedatectl set-timezone Asia/Shanghai
timedatectl set-ntp yes
ntpq -p

 

1.2 操作系統調優

cat >> /etc/sysctl.conf <<EOF
fs.file-max=655360
vm.max_map_count = 262144
EOF

sysctl -p

vim /etc/security/limits.conf

* soft nproc 20480
* hard nproc 20480
* soft nofile 65536
* hard nofile 65536
* soft memlock unlimited
* hard memlock unlimited

 

vim /etc/security/limits.d/20-nproc.conf

* soft nproc 20480

 

 

1.3 安裝JDK

yum install -y java-1.8.0-openjdk*

vim /etc/profile

# set java environment 
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

source /etc/profile

echo "source /etc/profile" >> /etc/bashrc

 

1.4 安裝es

1）新建用戶

groupadd elsearch
useradd elsearch -g elsearch -p elasticsearch

 

2）下載
cd /opt
wget https://img.yiyao.cc/elasticsearch-7.1.1-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.1.1-linux-x86_64.tar.gz
mv elasticsearch-7.1.1 elasticsearch
chown -R elsearch.elsearch ./elasticsearch

3）JVM調優
物理內存一半
vim /opt/elasticsearch/config/jvm.options

-Xms8g
-Xmx8g

 

4）配置es，三個節點同時作為 master 和 data

vim /opt/elasticsearch/config/elasticsearch.yml

#節點1

cluster.name: wmqees
node.name: es-node1
node.master: true
node.data: true
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 172.16.2.141
http.port: 9200
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.hosts: ["172.16.2.141:9300","172.16.2.142:9300","172.16.2.143:9300"]
cluster.initial_master_nodes: ["es-node1", "es-node2", "es-node3"]
http.cors.enabled: true
http.cors.allow-origin: "*"

cluster.initial_master_nodes參數說明：es7 引用了 Bootstrapping a cluster 后，首次啟動Elasticsearch集群需要在集群中的一個或多個符合主節點的節點上顯式定義初始的符合主節點的節點集。這稱為群集自舉，這僅在群集首次啟動時才需要。

#節點2

cluster.name: wmqees
node.name: es-node2
node.master: true
node.data: true
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 172.16.2.142
http.port: 9200
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.hosts: ["172.16.2.141:9300","172.16.2.142:9300","172.16.2.143:9300"]
cluster.initial_master_nodes: ["es-node1", "es-node2", "es-node3"]
http.cors.enabled: true
http.cors.allow-origin: "*"

#節點3

cluster.name: wmqees
node.name: es-node3
node.master: true
node.data: true
path.data: /opt/elasticsearch/data
path.logs: /opt/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 172.16.2.143
http.port: 9200
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.hosts: ["172.16.2.141:9300","172.16.2.142:9300","172.16.2.143:9300"]
cluster.initial_master_nodes: ["es-node1", "es-node2", "es-node3"]
http.cors.enabled: true
http.cors.allow-origin: "*"

 

5）啟動

su - elsearch -c "/opt/elasticsearch/bin/elasticsearch -d"

6）驗證
 curl "172.16.2.143:9200/_xpack"

{"build":{"hash":"7a013de","date":"2019-05-23T14:05:50.009976Z"},"license":{"uid":"344f983f-9d20-4476-851a-4172fd669f12","type":"basic","mode":"basic","status":"active"},"features":{"ccr":{"description":"Cross Cluster Replication","available":false,"enabled":true},"graph":{"description":"Graph Data Exploration for the Elastic Stack","available":false,"enabled":true},"ilm":{"description":"Index lifecycle management for the Elastic Stack","available":true,"enabled":true},"logstash":{"description":"Logstash management component for X-Pack","available":false,"enabled":true},"ml":{"description":"Machine Learning for the Elastic Stack","available":false,"enabled":true,"native_code_info":{"version":"7.1.1","build_hash":"fd619a36eb77df"}},"monitoring":{"description":"Monitoring for the Elastic Stack","available":true,"enabled":true},"rollup":{"description":"Time series pre-aggregation and rollup","available":true,"enabled":true},"security":{"description":"Security for the Elastic Stack","available":true,"enabled":false},"sql":{"description":"SQL access to Elasticsearch","available":true,"enabled":true},"watcher":{"description":"Alerting, Notification and Automation for the Elastic Stack","available":false,"enabled":true}},"tagline":"You know, for X"}

說明：顯示 license 不為空則安裝成功。es7版本默認已經包含xpack認證，無需注冊。

 

1.5 開機自啟

有 systemd 和 service 兩種方式進行設置開機自啟，推薦 systemd 方式可以設置 es 異常掛起后能夠重啟。

1.5.1 Systemd 方式（推薦）

1）新建環境配置文件，指定Java路徑

vim /etc/sysconfig/elasticsearch

################################
# Elasticsearch
################################

# Elasticsearch home directory
#ES_HOME=/usr/share/elasticsearch

# Elasticsearch Java path
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b10-0.el7_6.x86_64
CLASSPATH=.:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b10-0.el7_6.x86_64/lib:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b10-0.el7_6.x86_64/jre/lib

# Elasticsearch configuration directory
#ES_PATH_CONF=${path.conf}

# Elasticsearch PID directory
#PID_DIR=/var/run/elasticsearch

# Additional Java OPTS
#ES_JAVA_OPTS=

# Configure restart on package upgrade (true, every other setting will lead to not restarting)
#RESTART_ON_UPGRADE=true

################################
# Elasticsearch service
################################

# SysV init.d
#
# The number of seconds to wait before checking if Elasticsearch started successfully as a daemon process
ES_STARTUP_SLEEP_TIME=5

################################
# System properties
################################

# Specifies the maximum file descriptor number that can be opened by this process
# When using Systemd, this setting is ignored and the LimitNOFILE defined in
# /usr/lib/systemd/system/elasticsearch.service takes precedence
#MAX_OPEN_FILES=65535

# The maximum number of bytes of memory that may be locked into RAM
# Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
# in elasticsearch.yml.
# When using systemd, LimitMEMLOCK must be set in a unit file such as
# /etc/systemd/system/elasticsearch.service.d/override.conf.
#MAX_LOCKED_MEMORY=unlimited

# Maximum number of VMA (Virtual Memory Areas) a process can own
# When using Systemd, this setting is ignored and the 'vm.max_map_count'
# property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf
#MAX_MAP_COUNT=262144

官網樣例：https://github.com/elastic/elasticsearch/blob/master/distribution/packages/src/common/env/elasticsearch

 

2）創建服務文件

vim /usr/lib/systemd/system/elasticsearch.service

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Restart=always
Type=simple
PrivateTmp=true
Environment=ES_HOME=/opt/elasticsearch
Environment=ES_PATH_CONF=/opt/elasticsearch/config
Environment=PID_DIR=/opt/elasticsearch
Environment=ES_SD_NOTIFY=true
EnvironmentFile=/etc/sysconfig/elasticsearch

WorkingDirectory=/opt/elasticsearch

User=elsearch
Group=elsearch

ExecStart=/opt/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /var/log/elasticsearch and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65535

# Specifies the maximum number of processes
LimitNPROC=20480

LimitMEMLOCK=infinity

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

官網樣例：https://github.com/elastic/elasticsearch/blob/master/distribution/packages/src/common/systemd/elasticsearch.service

 

3）啟動

systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

 可以 kill 掉 es 進程，es 會再次啟動。 

 

1.5.2 service 方式（不推薦）

1）創建啟動腳本

vim /etc/init.d/elasticsearch

#!/bin/sh
#chkconfig: 2345 80 05
#description: elasticsearch
#processname: elasticsearch-7.1.1

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64
export JAVA_BIN=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/bin
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export ES_HOME=/opt/elasticsearch

case $1 in
    start)
        su elsearch<<!
        cd $ES_HOME
        ./bin/elasticsearch -d -p pid
        exit
!
        echo "elasticsearch is started"
        ;;
    stop)
        pid=`cat $ES_HOME/pid`
        kill -9 $pid
        echo "elasticsearch is stopped"
        ;;
    restart)
        pid=`cat $ES_HOME/pid`
        kill -9 $pid
        echo "elasticsearch is stopped"
        sleep 1
        su elsearch<<!
        cd $ES_HOME
        ./bin/elasticsearch -d -p pid
        exit
!
        echo "elasticsearch is started"
        ;;
    *)
        echo "start|stop|restart"
        ;;  
esac
exit 0

說明：需指定JDK環境，要不然會默認使用es自帶的JDK，自帶的版本太新，去除了GC。

2）啟動 

# 添加到開機啟動任務
chmod +x /etc/init.d/elasticsearch
chkconfig --add elasticsearch

# 啟動
service elasticsearch start

 

二、配置 TLS 和身份驗證

2.1 創建證書文件

在一個master上執行即可

cd /opt/elasticsearch
./bin/elasticsearch-certutil ca
兩次回車
./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 
三次回車

賦予權限

mkdir config/certs
mv elastic-*.p12 config/certs/
chown -R elsearch:elsearch config/certs/

再把證書文件 elastic-certificates.p12 復制到其他master節點並賦予權限。 

 

2.2 修改配置

#所有主機配置文件添加ssl

cat >> config/elasticsearch.yml <<EOF
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
EOF

重啟 elasticsearch 

service elasticsearch restart

 

2.3 生成客戶端證書

cd /opt/elasticsearch
bin/elasticsearch-certutil cert --ca \
config/certs/elastic-stack-ca.p12 \
-name "CN=esuser,OU=dev,DC=weqhealth,DC=com"

回車
client.p12
回車

拆分證書

mv client.p12 config/certs/
cd config/certs/

openssl pkcs12 -in client.p12 -nocerts -nodes > client-key.pem
openssl pkcs12 -in client.p12 -clcerts -nokeys  > client.crt
openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.crt

chown elsearch:elsearch client*

 

2.4 設置默認密碼

bin/elasticsearch-setup-passwords interactive

y，分別設置 elastic、apm_system、kibana、logstash_system、beats_system、remote_monitoring_user賬號的密碼。

 

2.5 配置kibana

修改 kibana.yml 文件

elasticsearch.username: "kibana"
elasticsearch.password: "elasticxxxxxxx"

然后用超級管理員賬號 elastic 登入到 kibana。在kibana中設置角色和賬號，也可以修改賬號密碼。

 

2.6 驗證集群狀態

因為開啟了xpack驗證，需要指定賬號密碼

curl --user elastic:elasticxxxxxx -XGET '172.16.2.143:9200/_cat/health?v&pretty'

epoch      timestamp cluster   status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1564645243 07:40:43  wmqees green           3         3     14   7    0    0        0             0                  -                100.0%

 

參考：https://www.elastic.co/cn/blog/getting-started-with-elasticsearch-security

 

三、基本設置

3.1 設置分片數

es7默認主分片數和主分片副本數都為1，通過 default_template 指定分片數

PUT _template/default_template
{
  "index_patterns" : ["*"], 
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas" : 1
  }
}

number_of_shards：每個索引的主分片數，默認值是 1 不再是5。這個配置在索引創建后不能修改。

number_of_replicas：每個主分片的副本數，默認值是 1 。對於活動的索引庫，這個配置可以隨時修改。

 

3.2 集群最大分片數

1） 說明

基於集群中節點的數量，集群中的分片數量有一個軟限制（數據節點數 * 1000），如果群集中沒有數據節點，則不會執行該限制。

1、創建新索引，還原索引快照或打開關閉的索引會增加分片；關閉或刪除某些索引會減少分片。

2、副本數計入此限制，但封閉索引不計入。具有5個主要分片和2個副本的索引將計為15個分片。無論封閉索引包含多少個分片和副本，任何封閉索引都將計為0。

2）設置每個數據節點在集群中允許的分片數量

PUT _cluster/settings?pretty
{
  "persistent": {
    "cluster.max_shards_per_node": 3000
  }
}

3）查詢設置結果

GET _cluster/settings?pretty

# 結果如下
{
  "persistent" : {
    "cluster" : {
      "max_shards_per_node" : "3000"
    }
  },
  "transient" : { }
}

如果集群有 3 個數據節點，這樣集群總分片數為 9000 個。

官網解釋集群分配數限制

 

3.2 安全重啟es 

1）禁用分片分配

關閉節點時，分配過程將等待 index.unassigned.node_left.delayed_timeout 1分鍾（默認情況下為1分鍾），然后開始將該節點上的分片復制到集群中的其他節點，這可能涉及大量I/O。由於該節點不久將要重新啟動，因此該I/O是不必要的，通過在關閉節點之前禁用副本分配。

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}

設置成primaries，索引的主分片會均分到集群的各個node，副本分片處於unassigined狀態。

 

2）重啟es

依次挨個重啟一個節點，啟動好了后再重啟另外節點。

service elasticsearch restart

 

3）開啟分片分配

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": null
  }
}

官網參考：分配種類，重啟操作，分配規則