Elasticsearch-2.4.3的3節點安裝(多種方式圖文詳解)(含 head、kopf、marvel、shield和watcher插件安裝和使用)


 

 

 

 

 

 

 

前提:

Elasticsearch-2.4.3的下載(圖文詳解)

Elasticsearch-2.4.3的單節點安裝(多種方式圖文詳解)

 

 

 

 

  我這里,以192.168.80.10(HadoopMaster)、192.168.80.11(HadoopSlave1)、192.168.80.12(HadoopSlave2)三台機器,來安裝ES的3節點集群部署。

  具體的,3集群搭建安裝基礎,請移步,先看

hadoop-2.6.0.tar.gz的集群搭建(3節點)

hadoop-2.6.0-cdh5.4.5.tar.gz(CDH)的3節點集群搭建

  更多,請移步,擴展看

http://www.cnblogs.com/zlslch/category/913076.html

 

 

 

 

 

ElasticSearch 規划-集群規划

      

 

 

 

 ElasticSearch 規划-集群規划

        

 

 

 

 

 

 

ElasticSearch 規划-用戶規划

 

 

 

 

 

 

ElasticSearch 規划-目錄規划

   軟件目錄:/home/hadoop/app/

       數據目錄:/home/hadoop/data/elasticsearch-2.4.3/data

      日志目錄:/home/hadoop/data/elasticsearch-2.4.3/datalog

   進程目錄:/home/hadoop/data/elasticsearch-2.4.3/pid

    三台均要去做。






 

 

 

 

ElasticSearch 集群環境檢查-時鍾同步

1.設置本地時間

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

2.集群時間日期同步NTP

yum install ntp

ntpdate pool.ntp.org

ElasticSearch 集群環境檢查-hosts文件檢查

 

 

配置ElasticSearch集群ip與hostname之間的對應關系

vi /etc/hosts

192.168.80.10   HadoopMaster

192.168.80.11   HadoopSlave1

192.168.80.12   HadoopSlave2

 

 

ElasticSearch 集群環境檢查-禁用防火牆

查看防火牆狀態

service iptables status

關閉防火牆

chkconfig iptables off //永久關閉防火牆

service iptables stop //臨時關閉防火牆

 

 

 

 

 

ElasticSearch jdk安裝

下載解壓

配置環境變量

這個很簡單,不多贅述

 

 

 

 

 

 

Elasticsearch-2.4.3的3節點安裝的前提步驟

1、HadoopMaster

[hadoop@HadoopMaster ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:EE:5E:5B 
inet addr:192.168.80.10 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feee:5e5b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:258 errors:0 dropped:0 overruns:0 frame:0
TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:24027 (23.4 KiB) TX bytes:7192 (7.0 KiB)

lo Link encap:Local Loopback 
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)

[hadoop@HadoopMaster ~]$ ssh HadoopSlave1
Last login: Wed Feb 22 00:34:08 2017 from 192.168.80.1
[hadoop@HadoopSlave1 ~]$ ssh HadoopSlave2
Last login: Wed Feb 22 00:34:09 2017 from 192.168.80.1
[hadoop@HadoopSlave2 ~]$

   證明,ssh免密碼登錄是可以的。

 

 

 

2、HadoopSlave1

[hadoop@HadoopSlave1 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:D8:AA:6A 
inet addr:192.168.80.11 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fed8:aa6a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:249 errors:0 dropped:0 overruns:0 frame:0
TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:23321 (22.7 KiB) TX bytes:5876 (5.7 KiB)

lo Link encap:Local Loopback 
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)

[hadoop@HadoopSlave1 ~]$ ssh HadoopMaster
Last login: Wed Feb 22 00:34:06 2017 from 192.168.80.1
[hadoop@HadoopMaster ~]$ ssh HadoopSlave2
Last login: Wed Feb 22 00:34:45 2017 from hadoopslave1
[hadoop@HadoopSlave2 ~]$

 

 

 

 

 3、HadoopSlave2 

[hadoop@HadoopSlave2 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:FA:65:4A 
inet addr:192.168.80.12 Bcast:192.168.80.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fefa:654a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:199 errors:0 dropped:0 overruns:0 frame:0
TX packets:119 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:23401 (22.8 KiB) TX bytes:17008 (16.6 KiB)

lo Link encap:Local Loopback 
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)

[hadoop@HadoopSlave2 ~]$ ssh HadoopMaster
Last login: Wed Feb 22 00:38:54 2017 from hadoopslave1
[hadoop@HadoopMaster ~]$ ssh HadoopSlave1
Last login: Wed Feb 22 00:34:33 2017 from hadoopmaster
[hadoop@HadoopSlave1 ~]$

 

 

 

 

 

 

 

 

Elasticsearch-2.4.3的3節點安裝的過程步驟

  1、上傳elasticsearch-2.4.3.tar.gz壓縮包

[hadoop@HadoopMaster app]$ pwd
/home/hadoop/app
[hadoop@HadoopMaster app]$ ll
total 20
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rz

[hadoop@HadoopMaster app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

 

 

 

 

 

[hadoop@HadoopSlave1 app]$ pwd
/home/hadoop/app
[hadoop@HadoopSlave1 app]$ ll
total 20
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ rz

[hadoop@HadoopSlave1 app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$

 

 

 

 

 

 

[hadoop@HadoopSlave2 app]$ pwd
/home/hadoop/app
[hadoop@HadoopSlave2 app]$ ll
total 16
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ rz 
[hadoop@HadoopSlave2 app]$ ll
total 26720
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$

 

 

 

 

 

  2、解壓縮elasticsearch-2.4.3.tar.gz壓縮包

[hadoop@HadoopMaster app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

 

 

 

 

 

[hadoop@HadoopSlave1 app]$ ll
total 26724
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

 

 

 

 

 

[hadoop@HadoopSlave2 app]$ ll
total 26720
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ tar -zxvf elasticsearch-2.4.3.tar.gz

 

 

 

 

  3、刪除elasticsearch-2.4.3.tar.gz壓縮包

 

[hadoop@HadoopMaster app]$ ll
total 26728
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rm elasticsearch-2.4.3.tar.gz 
[hadoop@HadoopMaster app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

 

 

 

 

 

[hadoop@HadoopSlave1 app]$ ll
total 26728
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$ rm elasticsearch-2.4.3.tar.gz 
[hadoop@HadoopSlave1 app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:31 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxrwxr-x. 9 hadoop hadoop 4096 Nov 4 04:19 hive-1.2.1
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave1 app]$

 

 

 

 

[hadoop@HadoopSlave2 app]$ ll
total 26724
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:38 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 27342693 Jan 10 11:38 elasticsearch-2.4.3.tar.gz
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$ rm elasticsearch-2.4.3.tar.gz 
[hadoop@HadoopSlave2 app]$ ll
total 20
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:38 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:19 hadoop-2.6.0
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 2 05:28 zookeeper-3.4.6
[hadoop@HadoopSlave2 app]$

 

 

 

  4、解讀認識ES的目錄結構

  這里,以HadoopMaster為例

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ ll
total 44
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 bin
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 config
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 lib
-rw-rw-r--. 1 hadoop hadoop 11358 Aug 24 00:46 LICENSE.txt
drwxrwxr-x. 5 hadoop hadoop 4096 Dec 8 00:41 modules
-rw-rw-r--. 1 hadoop hadoop 150 Aug 24 00:46 NOTICE.txt
-rw-rw-r--. 1 hadoop hadoop 8700 Aug 24 00:46 README.textile
[hadoop@HadoopMaster elasticsearch-2.4.3]$

 

 

  5、修改配置文件

  進入$ES_HOME/config

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ cd config/
[hadoop@HadoopMaster config]$ pwd
/home/hadoop/app/elasticsearch-2.4.3/config
[hadoop@HadoopMaster config]$ ll
total 8
-rw-rw-r--. 1 hadoop hadoop 3192 Aug 24 00:46 elasticsearch.yml
-rw-rw-r--. 1 hadoop hadoop 2571 Aug 24 00:46 logging.yml
[hadoop@HadoopMaster config]$ vim elasticsearch.yml

   當然,這個配置文件,直接可以下載到本地,用Notepad++等軟件來打開。

 

 

 

 

 

 

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs


改為 network.host:
0.0.0.0 (當然這里,可以寫為192.168.80.10)
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 120s
client.transport.ping_timeout: 60s discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"]
cluster.name: escluster
node.name: node-0
path.data: /home/hadoop/data/elasticsearch-2.4.3/data (當然若為了更專業,比如實現負載均衡,掛載不同盤上,則可以用多個目錄,用逗號隔開句好,這里為了學習只寫一個)
path.logs: /home/hadoop/data/elasticsearch-2.4.3/datalog


其實,這里自行去設置。這里牽扯到單播和多播

   注意,這些一定要頂格。具體,更多詳細,見

elasticsearch-.yml(中文配置詳解)

 

 

 

 

 

   對其他兩台也要進行同樣的配置文件修改工作。

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs




改為

network.host: 0.0.0.0 (當然,這里可以寫為192.168.80.11) discovery.zen.ping.multicast.enabled: false discovery.zen.ping_timeout: 120s client.transport.ping_timeout: 60s discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"] cluster.name: escluster node.name: node-1 path.data: /home/hadoop/data/elasticsearch-2.4.3/data (當然若為了更專業,比如實現負載均衡,掛載不同盤上,則可以用多個目錄,用逗號隔開句好,這里為了學習只寫一個) path.logs: /home/hadoop/data/elasticsearch-2.4.3/datalog


其實,這里自行去設置。這里牽扯到單播和多播

 

 

 

 

 

# network.host: 192.168.0.1

# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# cluster.name: my-application
# node.name: node-1
# path.data: /path/to/data
# path.logs: /path/to/logs




改為
 network.host: 0.0.0.0 (當然這里,可以寫為192.168.80.12) discovery.zen.ping.multicast.enabled: false discovery.zen.ping_timeout: 120s client.transport.ping_timeout: 60s discovery.zen.ping.unicast.hosts: ["192.168.80.10","192.168.80.11", "192.168.80.12"] cluster.name: escluster node.name: node-2 path.data: /home/hadoop/data/elasticsearch-2.4.3/data (當然若為了更專業,比如實現負載均衡,掛載不同盤上,則可以用多個目錄,用逗號隔開句好,這里為了學習只寫一個) path.logs: /home/hadoop/data/elasticsearch-2.4.3/datalog


其實,這里自行去設置。這里牽扯到單播和多播

 

  或者用scp命令,進行分發,這個很簡單!

 

 

 

 

 

 

 

 

ElasticSearch 插件安裝(含 head、kopf、marvel、shield和watcher插件安裝)

  安裝marvel插件(所有節點都需要安裝)(在線安裝)

    bin/plugin install license

    bin/plugin install marvel-agent

  安裝head插件(選擇一個節點安裝即可)(在線安裝)(其實安裝一個節點就可以看到整個集群的了)(我這里就3個節點都安裝了)

    bin/plugin install mobz/elasticsearch-head

 

 

 

 

 

  6下載並安裝插件(含 head、kopf、marvel、shield和watcher插件安裝)

Elasticsearch之插件介紹及安裝

Elasticsearch之head插件安裝之后的瀏覽詳解

Elasticsearch之kopf插件安裝之后的瀏覽詳解

  我這里,以離線方式來安裝es插件之Head Plugin Kopf Plugin。

      以在線方式來安裝es插件之和marvel插件。

 

 

安裝 head和kopf插件:

[hadoop@HadoopMaster app]$ pwd
/home/hadoop/app
[hadoop@HadoopMaster app]$ ll
total 24
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$ rz

[hadoop@HadoopMaster app]$ rz

[hadoop@HadoopMaster app]$ ll
total 3092
drwxrwxr-x. 6 hadoop hadoop 4096 Feb 22 01:37 elasticsearch-2.4.3
-rw-r--r--. 1 hadoop hadoop 908862 Jan 10 11:38 elasticsearch-head-master.zip -rw-r--r--. 1 hadoop hadoop 2228252 Jan 10 11:38 elasticsearch-kopf-master.zip
drwxr-xr-x. 10 hadoop hadoop 4096 Oct 31 17:15 hadoop-2.6.0
drwxr-xr-x. 15 hadoop hadoop 4096 Nov 14 2014 hadoop-2.6.0-src
drwxrwxr-x. 8 hadoop hadoop 4096 Nov 2 18:20 hbase-1.2.3
drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79
drwxr-xr-x. 10 hadoop hadoop 4096 Nov 1 23:39 zookeeper-3.4.6
[hadoop@HadoopMaster app]$

 

  

 

 

 

 

 

 

 

 

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ ll
total 44
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 bin
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 02:08 config
drwxrwxr-x. 2 hadoop hadoop 4096 Feb 22 01:37 lib
-rw-rw-r--. 1 hadoop hadoop 11358 Aug 24 00:46 LICENSE.txt
drwxrwxr-x. 5 hadoop hadoop 4096 Dec 8 00:41 modules
-rw-rw-r--. 1 hadoop hadoop 150 Aug 24 00:46 NOTICE.txt
-rw-rw-r--. 1 hadoop hadoop 8700 Aug 24 00:46 README.textile
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/elasticsearch-head-master.zip
-> Installing from file:/home/hadoop/app/elasticsearch-head-master.zip...
Plugins directory [/home/hadoop/app/elasticsearch-2.4.3/plugins] does not exist. Creating...
Trying file:/home/hadoop/app/elasticsearch-head-master.zip ...
Downloading .........DONE
Verifying file:/home/hadoop/app/elasticsearch-head-master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed head into /home/hadoop/app/elasticsearch-2.4.3/plugins/head
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/elasticsearch-kopf-master.zip
-> Installing from file:/home/hadoop/app/elasticsearch-kopf-master.zip...
Trying file:/home/hadoop/app/elasticsearch-kopf-master.zip ...
Downloading ......................DONE
Verifying file:/home/hadoop/app/elasticsearch-kopf-master.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed kopf into /home/hadoop/app/elasticsearch-2.4.3/plugins/kopf
[hadoop@HadoopMaster elasticsearch-2.4.3]$

 

  同理,HadoopSlave1和HadoopSlave2也如上述步驟。這里,不多贅述。

 

 

 

  安裝marvel插件:

  (所有節點都需要安裝)(HadoopMaster、HadoopSlave1和Hadoopslave2)

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license -> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ...
Downloading .DONE
Installed license into /home/hadoop/app/elasticsearch-2.4.3/plugins/license
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install marvel-agent -> Installing marvel-agent...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.3/marvel-agent-2.4.3.zip ...
Downloading ..........DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.3/marvel-agent-2.4.3.zip checksums if available ...
Downloading .DONE
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission setFactory
* javax.net.ssl.SSLPermission setHostnameVerifier
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
Installed marvel-agent into /home/hadoop/app/elasticsearch-2.4.3/plugins/marvel-agent
[hadoop@HadoopMaster elasticsearch-2.4.3]$ 

     同理,HadoopSlave1和HadoopSlave2也如上述步驟。這里,不多贅述。

 

 

 

  安裝shield插件:

(所有節點都需要安裝)(HadoopMaster、HadoopSlave1和Hadoopslave2)

 

  默認情況下elasticsearch可以不受限制訪問,如果在外網的情況下就不安全了,所以,elastic官方提供了shield插件,可以實現權限控制,但是不好的地方就是這個插件是收費的,可以免費試用一個月。那我就帶你來試試吧! 

  我告訴你一個小技巧:等到30天之后,卸載 shield,再重新安裝就是。該如何卸載,后面我也有寫。

  Shield 插件必須安裝在集群中的每一個節點,並且每一個節點在安裝完成之后必須要重啟才能生效。

   我這里,是3節點集群,所以HadoopMaster、HadoopSlave1和HadoopSlave2都安裝shield插件。

  注意:你必須保證Shield的版本和你運行的Elasticsearch的版本保持一致。

Java 7 or later
Elasticsearch 2.4.3
Elasticsearch License 2.4.3 plugin

 

 

 

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3 [hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license -> Installing license... Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ... Downloading .......DONE Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ... Downloading .DONE ERROR: plugin directory /home/hadoop/app/elasticsearch-2.4.3/plugins/license already exists. To update the plugin, uninstall it first using 'remove license' command [hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install shield

 

 

    當然,若你的網速不好的話,也可以選擇離線來安裝。即,先下載好。

https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip

https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/shield/2.4.3/shield-2.4.3.zip

 

 

 

 

 

 

 

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/shield-2.4.3.zip 
-> Installing from file:/home/hadoop/app/shield-2.4.3.zip...
Trying file:/home/hadoop/app/shield-2.4.3.zip ...
Downloading .......................DONE
Verifying file:/home/hadoop/app/shield-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
Installed shield into /home/hadoop/app/elasticsearch-2.4.3/plugins/shield
[hadoop@HadoopMaster elasticsearch-2.4.3]$ 

 同理,HadoopSlave1和HadoopSlave2也如上述步驟。

 

 

 

 

   然后,默認情況下es並沒有禁用自動創建索引庫功能。

  假設,若你在以后時,如果你在es中禁用了自動創建索引庫的功能。那么你需要在elasticsearch.yml中配置action.auto_create_index 以便允許Shield去創建.security 索引庫。action.auto_create_index: .security

  這里,我不贅述。

 

  然后,啟動/重啟es  

 

 

 

 

安裝shield插件之后,如何來使用呢?(HadoopMaster、HadoopSlave1、HadoopSlave2)

 

  創建用戶:

    用戶級別分為以下三種: 

      (1)admin:可以對就所有集群或者索引庫執行操作

      (2)power_user:可以監控集群並且對所有索引庫執行操作

      (3)user:可以對所有索引庫執行read操作

 

 

 

 

  創建用戶命令:【執行之后會提示用戶設置密碼】

bin/shield/esusers useradd es_admin -r admin

 

解釋:
useradd:表示添加用戶
es_admin:表示要新建的用戶名稱
-r:指定權限信息
admin:給新建的用戶賦予admin權限

   注意:當提示輸入密碼的時候,給新用戶設置一個新密碼,密碼必須至少包含6個字符。比如我這里設置為123456

  這也是,為什么后續,編程java api時,需要用到的地方。見如下:

使用java api操作es需要注意的地方(Elasticsearch開啟shield插件之后)

 

 

 

   訪問es:-u es_admin 表示使用用戶es_admin進行訪問

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.10:9200/'
Enter host password for user 'es_admin':
{
  "name" : "node-0",
  "cluster_name" : "escluster",
  "cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
  "version" : {
    "number" : "2.4.3",
    "build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
    "build_timestamp" : "2016-12-07T16:28:56Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$ 

 

curl -u es_admin -XGET 'http://localhost:9200/'

curl -u es_admin -XGET 'http://192.168.80.10:9200/'

 

 

 

 

 

 

 

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ bin/shield/esusers useradd es_admin -r admin
Enter new password: 
Retype new password: 
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.11:9200/'
Enter host password for user 'es_admin':
{
  "name" : "node-1",
  "cluster_name" : "escluster",
  "cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
  "version" : {
    "number" : "2.4.3",
    "build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
    "build_timestamp" : "2016-12-07T16:28:56Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ 

 

 

 

 

 

 

 

 

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/shield/esusers useradd es_admin -r admin
Enter new password: 
Retype new password: 
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ curl -u es_admin -XGET 'http://192.168.80.12:9200/'
Enter host password for user 'es_admin':
{
  "name" : "node-2",
  "cluster_name" : "escluster",
  "cluster_uuid" : "gBwdRD7iRRyC6TRji7ohFw",
  "version" : {
    "number" : "2.4.3",
    "build_hash" : "d38a34e7b75af4e17ead16f156feffa432b22be3",
    "build_timestamp" : "2016-12-07T16:28:56Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ 

 

 

 

 

  訪問的會提示輸入剛才設置的密碼。(我剛才輸入的密碼是123456,用戶名是es_admin)。

  這樣以后再訪問es的時候就需要輸入用戶名和密碼了。

 

 

 

kibana中也能使用shield

  具體見,https://www.elastic.co/guide/en/shield/current/kibana.html

   以后寫上,

 

 

 

如果您的安全要求更復雜,你可以這樣做:

  1、定義和使用細粒度訪問控制的自定義角色

  2、使用LDAP或Active Directory集成,或使用證書的認證。

  3、使用IP過濾來自特定IP地址或地址范圍的請求

  具體,見https://www.elastic.co/guide/en/shield/current/ip-filtering.html

 

 

 

目前是試用期30天(商業),如何卸載shield?

  如果不想使用shield,需要移除掉。

    第一步:停止es

    第二步:從es中移除shield插件

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/plugin remove shield

    第三步:啟動es. 

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ jps
2394 Jps
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/elasticsearch -d
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ jps
2428 Jps
2415 Elasticsearch

  這樣做的目的是,為了再次重新使用30天,目前,只能這樣個傻瓜辦法了。

 

 

 

 

 

 

 

 安裝watcher插件:

  (所有節點都需要安裝)(HadoopMaster、HadoopSlave1和Hadoopslave2)

   watcher插件可以提供elasticsearch集群的監控提醒功能。

   你需要在你的集群中所有節點安裝license和watcher插件。

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install license
-> Installing license...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip ...
Downloading .......DONE
Verifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.3/license-2.4.3.zip checksums if available ...
Downloading .DONE
ERROR: plugin directory /home/hadoop/app/elasticsearch-2.4.3/plugins/license already exists. To update the plugin, uninstall it first using 'remove license' command
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install watcher
-> Installing watcher...
Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/watcher/2.4.3/watcher-2.4.3.zip ...

 

   可以通過上述網址,去下載到。

 

 

 

  當然,照樣也是,可以離線方式來安裝的。

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip 
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopMaster elasticsearch-2.4.3]$ 

 

 

 

 

 

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip 
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ 

 

 

 

 

 

 

 

 

 

 

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ bin/plugin install file:///home/hadoop/app/watcher-2.4.3.zip 
-> Installing from file:/home/hadoop/app/watcher-2.4.3.zip...
Trying file:/home/hadoop/app/watcher-2.4.3.zip ...
Downloading ................DONE
Verifying file:/home/hadoop/app/watcher-2.4.3.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
Installed watcher into /home/hadoop/app/elasticsearch-2.4.3/plugins/watcher
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ 

 

   默認這一步不需要操作,因為默認情況下es並沒有禁用自動創建索引庫功能。

  如果你在es中禁用了自動創建索引庫的功能。那么你需要在elasticsearch.yml中配置action.auto_create_index 以便允許watcher去創建.watches, .triggered_watches, and .watcher-history* 索引庫。

  如果你已經安裝了shield,你必須允許shield創建.security索引庫

  這里,我不贅述。

 

  然后,啟動/重啟es  

 

 

 

  驗證watcher插件后的效果

curl -XGET 'http://localhost:9200/_watcher/stats?pretty'



curl -XGET 'http://192.168.80.10:9200/_watcher/stats?pretty'

 

 

[hadoop@HadoopMaster elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.10:9200/_watcher/stats?pretty'
{
  "error" : {
    "root_cause" : [ {
      "type" : "security_exception",
      "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
      "header" : {
        "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
      }
    } ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$ 

 

 

 

 

 

 

 

 

[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.11:9200/_watcher/stats?pretty'
{
  "error" : {
    "root_cause" : [ {
      "type" : "security_exception",
      "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
      "header" : {
        "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
      }
    } ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}
[hadoop@HadoopSlave1 elasticsearch-2.4.3]$ 

 

 

 

 

 

 

[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ pwd
/home/hadoop/app/elasticsearch-2.4.3
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ curl -XGET 'http://192.168.80.12:9200/_watcher/stats?pretty'
{
  "error" : {
    "root_cause" : [ {
      "type" : "security_exception",
      "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
      "header" : {
        "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
      }
    } ],
    "type" : "security_exception",
    "reason" : "missing authentication token for REST request [/_watcher/stats?pretty]",
    "header" : {
      "WWW-Authenticate" : "Basic realm=\"shield\" charset=\"UTF-8\""
    }
  },
  "status" : 401
}
[hadoop@HadoopSlave2 elasticsearch-2.4.3]$ 

 

 

 暫時,放這里。

 

 

 

監控功能:

  1:監控錯誤日志

    具體見,https://www.elastic.co/guide/en/watcher/current/watch-log-data.html

  2:監控集群監控狀態

    具體見,https://www.elastic.co/guide/en/watcher/current/watch-cluster-status.html

 

 

 

 

面以監控集群狀態為例演示:

  1:創建一個每10秒運行一次的調度器

curl -XPUT 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
}
}'

 

  2:給這個調度器增加一個input,把集群的健康狀態添加到watcher中

curl -XPUT 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
}
}'

 

   3:給這個wacher添加一個條件,判斷集群的狀態是否為red

curl -XPUT 'http://192.168.80.100:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
},
"condition" : {
"compare" : {
"ctx.payload.status" : { "eq" : "red" }
}
}
}'

 

  4:給watcher設置觸發任務,當滿足條件的時候,執行。

  一般都是發送郵件。

  所以需要先配置郵箱信息。修改es的配置文件。

  vi conf/elasticsearch.yml

 

watcher.actions.email.service.account:
work:
profile: qqmail
smtp:
auth: true
starttls.enable: true
host: smtp.qq.com
port: 587
user: 12345677@qq.com
password: 12345678

 

 

  添加觸發任務【手動把某一個索引庫的分片刪除一部分可以模擬分片丟失,集群狀態會變為red,此任務就會被觸發,發送郵件。】

curl -XPUT 'http://192.168.80.100:9200/_watcher/watch/cluster_health_watch' -d '{
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "192.168.80.100",
"port" : 9200,
"path" : "/_cluster/health"
}
}
},
"condition" : {
"compare" : {
"ctx.payload.status" : { "eq" : "yellow" }
}
},
"actions" : {
"send_email" : {
"email" : {
"to" : "87654321@qq.com",
"subject" : "Cluster Status Warning",
"body" : "Cluster status is RED"
}
}
}
}'

 

 

  驗證集群狀態,如果集群狀態為red,則剛才配置的watcher任務就會執行,發送郵件提醒。

curl -XGET 'http://192.168.80.10:9200/_cluster/health?pretty'

 

 

  5:實驗結束,可以移除掉這個watcher

curl -XDELETE 'http://192.168.80.10:9200/_watcher/watch/cluster_health_watch'

 

 

 

 

  目前是試用期30天(商業),如何卸載watcher?

    第一步:停止Elasticsearch.

    第二步:從Elasticsearch中移除watcher:

bin/plugin remove watcher

    第三步:啟動Elasticsearch.

 

 

 

 

 

 

 

 

 

 

  7、啟動,驗證

[hadoop@HadoopMaster elasticsearch-2.4.3]$ jps
2020 Jps
[hadoop@HadoopMaster elasticsearch-2.4.3]$ bin/elasticsearch -d
[hadoop@HadoopMaster elasticsearch-2.4.3]$ jps
2054 Jps
2041 Elasticsearch
[hadoop@HadoopMaster elasticsearch-2.4.3]$

 

   其實啊,或者通過es的這種后台來啟動也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

 

 

 

 

 

 

  輸入,http://192.168.80.10:9200/

   以上是默認的。

 

 

  以上是自己修改了es的配置文件得到的。

 

 

 

 

 

 其實啊,或者通過es的這種后台來啟動也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

  以上是默認的。

 

  以上是自己修改了es的配置文件得到的。

 

 

 

 

 

 

 

 

 

 其實啊,或者通過es的這種后台來啟動也是可以的。

bin/elasticsearch  -d  -p /home/hadoop/data/elasticsearch-2.4.3/pid

   以上是默認的。

 

 

  以上是自己修改了es的配置文件得到的。

 

 

 

  三個節點的整個。

 

 

 

 

 

 

 

 

 

   啟動head插件界面

  注意:每次啟動會隨機生成一個節點名稱 。

   默認情況下,三台機器,內部默認選舉,選出哪台是主節點,但是對於我們而言,也可以自己限制出哪台是主節點。

 

 

 

  在這里,要說明一個問題:

  出現了 no_route_to_host_exception: No route to host ,最后查博客得知,是防火牆沒關閉,建議作永久關閉,即 chkconfig iptables off 。

 

 節點和節點之間通信 是走9300端口    
 我們通過http請求連接es是走9200端口 
 
 
 
   插件kopf啟動界面
 http://192.168.80.12:9200/_plugin/kopf/#!/cluster

   對應,

  這里,不多贅述。

 

   至此,ES的3台機器組件的集群,成功安裝!

 

 

 

 

 

擴展學習

Elasticsearch集群如何擴容機器?

實例展示elasticsearch集群生態,分片以及水平擴展.

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM