此部署過程以Elasticsearch-6.6.0版本為例, 后續的學習和演示也用此版本.
1 准備工作
1.1 安裝JDK
學習使用ES的前提是成功安裝JDK —— 很基礎的一項步驟, 這里省略.
此處學習演示所用的JDK版本為:
[root@localhost ~]# java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
1.2 下載安裝包
(1) 根據自己的系統版本下載相應的安裝包, 鏈接如下:
https://www.elastic.co/downloads/elasticsearch
這里下載MacOS/Linux系統下的版本
elasticsearch-6.6.0.tar.gz.
(2) 上傳並解壓:
# 工作路徑
mkdir -p /data/elk-6.6.0
# 上傳安裝包后解壓
tar -zxf elasticsearch-6.6.0.tar.gz
1.3 創建elastic用戶
Elasticsearch的啟動, 必須通過專用用戶elastic啟動, 否則將報錯.
# 創建用戶
useradd elastic -s /bin/bash
# 為該用戶賦予相關操作權限
chown -R elastic:elastic /data/elk-6.6.0
# 修改安裝目錄名稱
mv elasticsearch-6.6.0 es-node
2 啟動ES服務
2.1 修改配置文件
修改${ES_HOME}/config/elasticsearch.yml文件中關於網絡的配置:
# 大約17行, 修改集群名稱, 同一個集群中此名稱必須相同, 才能組成一個邏輯集群:
cluster.name: heal_es
# 大約23行, 修改節點名稱, 可以設置為與主機名稱相同:
node.name: es-1
# 大約55行, 指定可通過外部服務器訪問本地ES服務:
network.host: 0.0.0.0
# 並指定訪問的端口號, 默認是9200, 為了防止沖突, 這里修改為9301:
http.port: 9301
另外, 如果要考慮到后期的版本升級, 可以指定ES存儲索引和日志文件的路徑, 否則容易出現數據丟失的情況.
默認的路徑是Elasticsearch解壓包內的
data和logs.
# Path to directory where to store the data (separate multiple locations by comma):
#path.data: /data/elk-6.6.0/data
#
# Path to log files:
#path.logs: /data/elk-6.6.0/logs
2.2 啟動服務
# 切換用戶
su elastic
# 啟動服務, -d是指在后台啟動, 若不用此參數, ES將阻塞當前終端的命令輸入功能, 如果強制使用, 將導致ES服務終止
cd /data/elk-6.6.0/es-node
./bin/elasticsearch -d
注意事項:
Elasticsearch必須以 非root用戶啟動, 否則將拋出如下錯誤:
[2019-06-24T21:02:07,654][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es-1] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.0.jar:6.6.0] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.0.jar:6.6.0] Caused by: java.lang.RuntimeException: can not run elasticsearch as root at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:103) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.0.jar:6.6.0] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.0.jar:6.6.0] ... 6 more
3 驗證ES服務是否可用
(1) 通過jps命令查看當前系統中運行的所有Java進程, 前提是JDK的環境變量設置OK:
[elastic@localhost bin]$ jps
25810 Elasticsearch # Elasticsearch的進程號
25926 Jps
(2) 通過ps -ef或ps aux命令查看 - 可以查看到啟動ES時的JVM參數:
[elastic@gosearch-03 bin]$ ps aux | grep elasticsearch
elastic 24181 83.5 1.2 26269656 1701524 pts/0 Sl 13:51 2:03 /data/jdk1.8.0_151/bin/java
-Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m
-Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-8345885453572562051 -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/data/elk-6.6.0/es-node -Des.path.conf=/data/elk-6.6.0/es-node/config
-Des.distribution.flavor=default -Des.distribution.type=tar -cp /data/elk-6.6.0/es-node/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
elastic 28089 0.0 0.0 103248 844 pts/0 S+ 13:54 0:00 grep elasticsearch
(3) 在MacOS/Linux的終端(或Windows的命令行)下運行命令:
# MacOS/Linux下:
curl http://localhost:9301/
# Windows下:
Invoke-RestMethod http://localhost:9301
若能出現類似下述瀏覽器中的信息, 說明ES啟動成功.
(4) 或在瀏覽器中訪問 "http://localhost:9301/", 若能出現下述信息, 說明ES服務啟動成功:
(5) 啟動界面參數解釋:
{ "name" : "es-1", "cluster_name" : "heal_es", # 當前集群的名稱, 同一集群中要保證一致 "cluster_uuid" : "Rcgt8uy_T5uUAu4DsnXHdQ", "version" : { "number" : "6.6.0", # 當前運行的ES的版本號 "build_flavor" : "default", "build_type" : "tar", "build_hash" : "a9861f4", "build_date" : "2019-01-24T11:27:09.439740Z", "build_snapshot" : false, # 當前運行的版本是不是從源代碼構建而來 "lucene_version" : "7.6.0", # 當前ES底層的Lucene的版本號 "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }
4 關閉與重啟服務
4.1 關閉服務
Elasticsearch沒有直接關閉或重啟服務的命令, 關閉只能通過kill命令殺掉進程的方式, 如下:
[elastic@localhost bin]$ ps aux | grep elasticsearch # 查看ES進程的id
[elastic@localhost bin]$ kill -8 25810 # 通過進程id來殺掉服務
4.2 重啟服務
直接啟動服務:
[elastic@localhost bin]$ sh elasticsearch -d
當然可以編寫一個服務腳本, 來方便快捷地啟動或關閉Elasticsearch服務.
5 常見問題及解決方法
說明: 下述問題部分是在Elasticsearch 5.x版本中遇到的, 部分是在6.6.0版本中遇到的, 本篇文章已於2019-06-24日更新, 僅供參考.
5.1 使用ES專屬用戶登錄時出錯
(1) 問題描述:
使用ES專屬用戶啟動ES服務時, 終端拋出如下錯誤:
[elastic@localhost bin]$ 2018-11-05 04:26:38,466 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
......
SettingsException[Failed to load settings from /data/elk-6.6.0/es-node/config/elasticsearch.yml]; nested: AccessDeniedException[/data/elk-6.6.0/es-node/config/elasticsearch.yml];
......
Caused by: java.nio.file.AccessDeniedException: /data/elk-6.6.0/es-node/config/elasticsearch.yml
......
(2) 問題分析:
錯誤信息說明: 當前用戶無的訪問被拒絕, 可知ES專屬用戶無法執行當前應用.
(3) 解決方法:
為ES創建專屬用戶后, 對其賦予相應的讀寫權限.
# 為該用戶賦予相關操作權限
chown -R elastic:elastic /data/elk-6.6.0
5.2 syscall filter - 不能安裝
(1) 問題描述:
啟動ES時, 拋出如下錯誤信息:
[2018-11-06T03:12:35,812][WARN ][o.e.b.JNANatives ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:329) ~[elasticsearch-5.6.10.jar:5.6.10]
......
[2018-11-06T03:12:39,947][INFO ][o.e.n.Node ] initialized
[2018-11-06T03:12:39,947][INFO ][o.e.n.Node ] [jVSUBme] starting ...
[2018-11-06T03:12:40,131][INFO ][o.e.t.TransportService ] [jVSUBme] publish_address {10.0.20.50:9300}, bound_addresses {[::]:9300}
[2018-11-06T03:12:40,145][INFO ][o.e.b.BootstrapChecks ] [jVSUBme] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-11-06T03:12:40,148][ERROR][o.e.b.Bootstrap ] [jVSUBme] node validation exception
[1] bootstrap checks failed
[1]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2018-11-06T03:12:40,150][INFO ][o.e.n.Node ] [jVSUBme] stopping ...
[2018-11-06T03:12:40,186][INFO ][o.e.n.Node ] [jVSUBme] stopped
[2018-11-06T03:12:40,186][INFO ][o.e.n.Node ] [jVSUBme] closing ...
[2018-11-06T03:12:40,199][INFO ][o.e.n.Node ] [jVSUBme] closed
(2) 問題解決:
Centos 6.5不支持SecComp, 而ES 5.x版本起
bootstrap.system_call_filter的默認值是 true.
禁用: 在elasticsearch.yml中配置bootstrap.system_call_filter=false, 注意要在Memory配置項的下面添加:bootstrap.system_call_filter: false
5.3 memory is not locked - 內存沒有鎖定
(1) 問題描述:
啟動Elasticsearch時, 拋出如下錯誤信息:
[2018-11-06T03:18:53,221][WARN ][o.e.b.JNANatives ] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
[2018-11-06T03:18:53,232][WARN ][o.e.b.JNANatives ] This can result in part of the JVM being swapped out.
[2018-11-06T03:18:53,232][WARN ][o.e.b.JNANatives ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2018-11-06T03:18:53,233][WARN ][o.e.b.JNANatives ] These can be adjusted by modifying /etc/security/limits.conf, for example:
# allow user 'elastic' mlockall
elastic soft memlock unlimited
elastic hard memlock unlimited
[2018-11-06T03:18:53,233][WARN ][o.e.b.JNANatives ] If you are logged in interactively, you will have to re-login for the new limits to take effect.
......
[2018-11-06T03:18:57,644][ERROR][o.e.b.Bootstrap ] [jVSUBme] node validation exception
[1] bootstrap checks failed
[1]: memory locking requested for elasticsearch process but memory is not locked
[2018-11-06T03:18:57,646][INFO ][o.e.n.Node ] [jVSUBme] stopping ...
[2018-11-06T03:18:57,693][INFO ][o.e.n.Node ] [jVSUBme] stopped
[2018-11-06T03:18:57,693][INFO ][o.e.n.Node ] [jVSUBme] closing ...
[2018-11-06T03:18:57,707][INFO ][o.e.n.Node ] [jVSUBme] closed
(2) 問題分析:
Elasticsearch的配置文件中有如下選項: bootstrap.memory_lock: true, 意為在啟動Elasticsearch服務時, 鎖定JVM需要的內存, 避免OS層面的Swap交換 —— 降低ES服務性能.
此選項默認為false, 即不開啟鎖定.
(3) 問題解決:
① 在配置文件中用"#"注釋掉
bootstrap.memory_lock: true, 或修改其值為false;
② 或者修改系統/etc/security/limits.conf文件, 為ES專屬用戶elastic解除限制:# 在文件最后添加下述配置, 允許用戶'elastic'鎖定內存 elastic soft memlock unlimited elastic hard memlock unlimited
5.4 max virtual memory - 最大虛擬內存太小
(1) 問題描述:
啟動Elasticsearch時, 拋出如下錯誤信息:
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-06-24T21:05:12,355][INFO ][o.e.n.Node ] [es-1] stopping ...
[2019-06-24T21:05:12,425][INFO ][o.e.n.Node ] [es-1] stopped
[2019-06-24T21:05:12,426][INFO ][o.e.n.Node ] [es-1] closing ...
[2019-06-24T21:05:12,439][INFO ][o.e.n.Node ] [es-1] closed
[2019-06-24T21:05:12,442][INFO ][o.e.x.m.p.NativeController] [es-1] Native controller process has stopped - no new native processes can be started
(2) 問題分析:
Elasticsearch服務需要大量的虛擬內存支撐, 系統默認的最大虛擬內存是65530, 而ES至少需要262144.
(3) 問題解決:
① 切換到root用戶下, 修改配置文件sysctl.conf:
vim /etc/sysctl.conf
# 修改下述配置, 如果沒有就在文件末尾添加:
vm.max_map_count=655360
# 執行命令使修改生效:
sysctl -p
② 然后重新啟動Elasticsearch, 即可啟動成功.
5.5 max number of threads - 最大線程數太小
(1) 問題描述:
啟動Elasticsearch時, 拋出如下錯誤信息:
ERROR: [2] bootstrap checks failed
[1]: max number of threads [1024] for user [elastic] is too low, increase to at least [4096]
[2]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2019-06-24T21:51:04,810][INFO ][o.e.n.Node ] [es-2] stopping ...
[2019-06-24T21:51:04,847][INFO ][o.e.n.Node ] [es-2] stopped
[2019-06-24T21:51:04,847][INFO ][o.e.n.Node ] [es-2] closing ...
[2019-06-24T21:51:04,864][INFO ][o.e.n.Node ] [es-2] closed
[2019-06-24T21:51:04,867][INFO ][o.e.x.m.p.NativeController] [es-2] Native controller process has stopped - no new native processes can be started
(2) 問題分析:
Elasticsearch服務需要用到多線程以提高執行效率, Cent OS 6.5默認的單個用戶最大線程數是1024, 而ES用戶至少需要4096.
(3) 問題解決:
① 切換到root用戶下, 修改配置文件:
[elastic@localhost bin]$ su root
Password:
[root@localhost bin]# vim /etc/security/limits.d/90-nproc.conf
# 找到如下內容, 如果沒有就創建:
* soft nproc 1024
# 修改為8192, 其中*表示所有用戶:
* soft nproc 8192
② 保存, 退出, 然后重新登錄(可以打開新的會話終端), 最后啟動Elasticsearch, 即可啟動成功.
5.6 max file descriptores - 最大可創建文件數太小
(1) 問題描述:
啟動Elasticsearch時, 拋出如下錯誤信息:
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
[2019-06-24T22:06:04,810][INFO ][o.e.n.Node ] [es-2] stopping ...
[2019-06-24T22:06:04,847][INFO ][o.e.n.Node ] [es-2] stopped
[2019-06-24T22:06:04,847][INFO ][o.e.n.Node ] [es-2] closing ...
[2019-06-24T22:06:04,864][INFO ][o.e.n.Node ] [es-2] closed
[2019-06-24T22:06:04,867][INFO ][o.e.x.m.p.NativeController] [es-2] Native controller process has stopped - no new native processes can be started
(2) 問題分析:
Elasticsearch服務在運行期間需要創建大量的本地文件, Cent OS 6.5默認的單個用戶最多可操作文件數為4096個, 而ES要求至少需要65536.
(3) 問題解決:
① 切換到root用戶下, 修改配置文件:
[elastic@localhost bin]$ su root
Password:
[root@localhost bin]# vim /etc/security/limits.conf
# 找到如下內容, 如果沒有就創建:
* soft nofile 4096
# 修改為65536, 其中*表示所有用戶:
* soft nproc 65536
② 保存, 退出, 然后重新登錄(可以打開新的會話終端), 最后啟動Elasticsearch, 即可啟動成功.
版權聲明
出處: 博客園 瘦風的博客(https://www.cnblogs.com/shoufeng)
感謝閱讀, 如果文章有幫助或啟發到你, 點個[好文要頂👆] 或 [推薦👍] 吧😜
本文版權歸博主所有, 歡迎轉載, 但 [必須在文章頁面明顯位置標明原文鏈接], 否則博主保留追究相關人員法律責任的權利.