最近項目中要求對文章進行全文檢索,所以選擇了elasticsearch作為全文檢索引擎。在安裝以及使用的過程中也是碰到了很多的坑,從最開始的7.3.2換到6.4.3再換到5.5.0,最后終於可以
使用了,太不容易了,所以開篇帖子記錄下,希望能幫助到剛入門es的朋友們。
下面介紹如何在Centos7上安裝es,以及中文分詞工具ik。因Elasticsearch是基於Java寫的,所以它的運行環境中需要java的支持,在Linux下執行命令:java -version,檢查JDK是否安裝。如果沒有安裝,請安裝JDK1.8以上版本。
一、es安裝
1.下載elasticsearch壓縮包
可前往 https://learnku.com/articles/33403 這個網址下載。選擇5.5.0版本elasticsearch-5.5.0.zip即可。
2.解壓elasticsearch壓縮包
將壓縮包拷貝之/usr/local/ 目錄下,執行解壓名稱。
cp elasticsearch-5.5.0.zip /usr/local/ cd /usr/local unzip elasticsearch-5.5.0.zip
3.修改elasticsearch.yml配置文件
vi /usr/local/elasticsearch-5.5.0/config/elasticsearch.yml //插入如下內容#集群的名稱 cluster.name: knowgram # 節點的名稱 node.name: knowgram_node1 #配置你存數據的目錄 path.data: /usr/local/elasticsearch-5.5.0/data # 配置你存日志的目錄 path.logs: /usr/local/elasticsearch-5.5.0/logs # Lock the memory on startup: 這兩個配置用於解決后面報錯問題,否則會報錯 bootstrap.memory_lock: false bootstrap.system_call_filter: false # Set the bind address to a specific IP (IPv4 or IPv6): #需要配置成本面的ip,否則外部無法訪問 network.host: 0.0.0.0 transport.tcp.port: 9300 #設置跨域以及端口 http.port: 9200 http.cors.enabled: true http.cors.allow-origin: "*"
4.創建一個elasticsearch組和賬戶
因為Elasticsearch5.0之后,不能使用root賬戶啟動,所以我們要創建專門的elasticsearch用戶組並授予文件夾權限。
groupadd elasticsearch useradd elasticsearch -g elasticsearch -p elasticsearch chown -R elasticsearch:elasticsearch /usr/local/elasticsearch-5.5.0
5.切換到elasticsearch用戶並啟動elasticsearch
cd /usr/local/elasticsearch5.5.0/ su elasticsearch ./bin/elasticsearch //或用以下命令 su - elasticsearch -c "/usr/local/elasticsearch-5.5.0/bin/elasticsearch -d"
若看到輸出started表示啟動成功,若沒有啟動成功參考后面常見問題
[2019-11-05T12:34:09,961][INFO ][o.e.n.Node ] [node_1] initializing ... [2019-11-05T12:34:10,113][INFO ][o.e.e.NodeEnvironment ] [node_1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [46gb], net total_space [59.9gb], spins? [unknown], types [rootfs] [2019-11-05T12:34:10,116][INFO ][o.e.e.NodeEnvironment ] [node_1] heap size [1.9gb], compressed ordinary object pointers [true] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] node name [node_1], node ID [b7PKnpL8TBqfX_j6N0ydyQ] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] version[5.5.0], pid[17571], build[260387d/2017-06-30T23:16:05.735Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.5.0] [2019-11-05T12:34:14,197][INFO ][o.e.p.PluginsService ] [node_1] loaded module [aggs-matrix-stats] [2019-11-05T12:34:14,198][INFO ][o.e.p.PluginsService ] [node_1] loaded module [ingest-common] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-expression] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-groovy] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-mustache] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-painless] [2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService ] [node_1] loaded module [parent-join] [2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService ] [node_1] loaded module [percolator] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [reindex] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [transport-netty3] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [transport-netty4] [2019-11-05T12:34:14,203][INFO ][o.e.p.PluginsService ] [node_1] loaded plugin [analysis-ik] [2019-11-05T12:34:21,588][INFO ][o.e.d.DiscoveryModule ] [node_1] using discovery type [zen] [2019-11-05T12:34:24,056][INFO ][o.e.n.Node ] [node_1] initialized [2019-11-05T12:34:24,057][INFO ][o.e.n.Node ] [node_1] starting ... [2019-11-05T12:34:24,930][INFO ][o.e.t.TransportService ] [node_1] publish_address {192.168.16.152:9300}, bound_addresses {[::]:9300} [2019-11-05T12:34:25,035][INFO ][o.e.b.BootstrapChecks ] [node_1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks [2019-11-05T12:34:28,268][INFO ][o.e.c.s.ClusterService ] [node_1] new_master {node_1}{b7PKnpL8TBqfX_j6N0ydyQ}{8rQez1gDSturfNgSHluIyw}{192.168.16.152}{192.168.16.152:9300}, reason: zen-disco-elected-as-master ([0] nodes joined) [2019-11-05T12:34:28,651][INFO ][o.e.h.n.Netty4HttpServerTransport] [node_1] publish_address {192.168.16.152:9200}, bound_addresses {[::]:9200} [2019-11-05T12:34:28,651][INFO ][o.e.n.Node ] [node_1] started //看到這條信息表示啟動成功 [2019-11-05T12:34:28,676][INFO ][o.e.g.GatewayService ] [node_1] recovered [0] indices into cluster_state
6.常見問題處理
問題一:
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [es] likely too low, increase to at least [2048]
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [es] likely too low, increase to at least [2048]
解決方式:更換到root用戶,編輯limits.conf 配置文件
vi /etc/security/limits.conf //添加如下內容: * soft nofile 65536 * hard nofile 131072 * soft nproc 2048 * hard nproc 4096
問題二:
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解決方式:修改 /etc/sysctl.conf 配置文件,增加如下配置
vi /etc/sysctl.conf
//追加如下內容
vm.max_map_count=655360
二、安裝elasticsearch-analysis-ik
1.下載對應的版本IK
前往github下載對應elasticsearch對應的ik版本,我安裝的是elasticsearch5.5.0,所以選擇對應5.5.0版本
2.在elasticsearch的plugins文件夾下新建ik目錄
mkdir /usr/local/elasticsearch-5.5.0/plugins/ik
3.將壓縮包復制到ik目錄下並解壓
cp elasticsearch-analysis-ik-5.5.0.zip /usr/local/elasticsearch-5.5.0/plugins/ik cd /usr/local/elasticsearch-5.5.0/plugins/ik unzip elasticsearch-analysis-ik-5.5.0.zip
最后重啟elasticsearch,看到loaded plugin [analysis-ik]標識插件加載完成。
[2019-11-05T12:34:09,961][INFO ][o.e.n.Node ] [node_1] initializing ... [2019-11-05T12:34:10,113][INFO ][o.e.e.NodeEnvironment ] [node_1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [46gb], net total_space [59.9gb], spins? [unknown], types [rootfs] [2019-11-05T12:34:10,116][INFO ][o.e.e.NodeEnvironment ] [node_1] heap size [1.9gb], compressed ordinary object pointers [true] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] node name [node_1], node ID [b7PKnpL8TBqfX_j6N0ydyQ] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] version[5.5.0], pid[17571], build[260387d/2017-06-30T23:16:05.735Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09] [2019-11-05T12:34:10,125][INFO ][o.e.n.Node ] [node_1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.5.0] [2019-11-05T12:34:14,197][INFO ][o.e.p.PluginsService ] [node_1] loaded module [aggs-matrix-stats] [2019-11-05T12:34:14,198][INFO ][o.e.p.PluginsService ] [node_1] loaded module [ingest-common] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-expression] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-groovy] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-mustache] [2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService ] [node_1] loaded module [lang-painless] [2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService ] [node_1] loaded module [parent-join] [2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService ] [node_1] loaded module [percolator] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [reindex] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [transport-netty3] [2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService ] [node_1] loaded module [transport-netty4] [2019-11-05T12:34:14,203][INFO ][o.e.p.PluginsService ] [node_1] loaded plugin [analysis-ik] //看到這個標識插件加載成功 [2019-11-05T12:34:21,588][INFO ][o.e.d.DiscoveryModule ] [node_1] using discovery type [zen] [2019-11-05T12:34:24,056][INFO ][o.e.n.Node ] [node_1] initialized [2019-11-05T12:34:24,057][INFO ][o.e.n.Node ] [node_1] starting ... [2019-11-05T12:34:24,930][INFO ][o.e.t.TransportService ] [node_1] publish_address {192.168.16.152:9300}, bound_addresses {[::]:9300} [2019-11-05T12:34:25,035][INFO ][o.e.b.BootstrapChecks ] [node_1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks [2019-11-05T12:34:28,268][INFO ][o.e.c.s.ClusterService ] [node_1] new_master {node_1}{b7PKnpL8TBqfX_j6N0ydyQ}{8rQez1gDSturfNgSHluIyw}{192.168.16.152}{192.168.16.152:9300}, reason: zen-disco-elected-as-master ([0] nodes joined) [2019-11-05T12:34:28,651][INFO ][o.e.h.n.Netty4HttpServerTransport] [node_1] publish_address {192.168.16.152:9200}, bound_addresses {[::]:9200} [2019-11-05T12:34:28,651][INFO ][o.e.n.Node ] [node_1] started [2019-11-05T12:34:28,676][INFO ][o.e.g.GatewayService ] [node_1] recovered [0] indices into cluster_state
至此,elasticsearch+ik分詞工具安裝完成!