目前時間是：2017-01-24
本文不涉及activemq的安裝
需求
activemq實時傳遞數據至服務 elasticsearch做索引 對外開放查詢接口 完成全文檢索
環境
jdk：1.8
spirng boot：1.4.3.RELEASE
elasticsearch：2.4.3
activemq：5.13.2
ES插件
head：版本好像無太大差別 能查數據就行
analysis-ik：1.10.3
search-guard-2：2.4.3.9
search-guard-ssl：2.4.3.19
注意
作者遇到的最大問題就是版本兼容 網上資料少且版本較低 故列出以下版本矩陣 方便查閱
spring boot與elasticsearch：
| Spring Boot Version (x) | Spring Data Elasticsearch Version (y) | Elasticsearch Version (z) |
| x <= 1.3.5 | y <= 1.3.4 | z <= 1.7.2* |
| x >= 1.4.x | 2.0.0 <=y < 5.0.0** | 2.0.0 <= z < 5.0.0** |
矩陣來源以及更多版本兼容：https://github.com/spring-projects/spring-data-elasticsearch
elasticsearch與ik：我的ES版本為2.x 對應如下
| IK version | ES version |
|---|---|
| master | 2.4.0 -> master |
| 1.10.3 | 2.4.3 |
| 1.9.5 | 2.3.5 |
| 1.9.4 | 2.3.4 |
| 1.9.3 | 2.3.3 |
| 1.9.0 | 2.3.0 |
| 1.8.1 | 2.2.1 |
| 1.7.0 | 2.1.1 |
| 1.5.0 | 2.0.0 |
| 1.2.6 | 1.0.0 |
| 1.2.5 | 0.90.x |
| 1.1.3 | 0.20.x |
| 1.0.0 | 0.16.2 -> 0.19.0 |
矩陣來源以及更多版本兼容：https://github.com/medcl/elasticsearch-analysis-ik
elasticsearch與search-guard以及search-guard-ssl：
| Elasticsearch Version | Latest Search Guard Version | Search Guard SSL Version | Commercial support available |
|---|---|---|---|
| 1.x.y | not available | - | - |
| 2.0.x | not available | - | - |
| 2.1.x | not available | - | - |
| 2.2.0 | 2.2.0.7 | 2.2.0.16 | Yes |
| 2.3.1 | available upon request | - | - |
| 2.3.2 | available upon request | - | - |
| 2.3.3 | 2.3.3.10 | 2.3.3.19 | YES |
| 2.3.4 | 2.3.4.10 | 2.3.4.19 | YES |
| 2.3.5 | 2.3.5.10 | 2.3.5.19 | YES |
| 2.4.0 | 2.4.0.10 | 2.4.0.19 | YES |
| 2.4.1 | 2.4.1.10 | 2.4.1.19 | YES |
| 2.4.2 | 2.4.2.10 | 2.4.2.19 | YES |
| 2.4.3 | 2.4.3.10 | 2.4.3.19 | YES |
| 2.4.4 | 2.4.4.10 | 2.4.4.19 | YES |
| 5.0.0 | 5.0.0-10 | (comes bundled since SG 5) | YES |
| 5.0.1 | 5.0.1-10 | (comes bundled since SG 5) | YES |
| 5.0.2 | 5.0.2-10 | (comes bundled since SG 5) | YES |
| 5.1.1 | 5.1.1-10 | (comes bundled since SG 5) | YES |
| 5.1.2 | 5.1.2-10 | (comes bundled since SG 5) | YES |
矩陣來源以及更多版本兼容：https://github.com/floragunncom/search-guard/wiki
開始
安裝elasticsearch
我的安裝目錄：/usr/local
注意：elasticsearch不能用root用戶運行 所以創建你的用戶組和用戶 切換到新用戶再安裝 如何創建切換 請自行搜索
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.3/elasticsearch-2.4.3.tar.gz
tar -zxvf elasticsearch-2.4.3.tar.gz
cd elasticsearch-2.4.3/config/
vim elasticsearch.yml
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please see the documentation for further information on configuration options: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html> # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: 你的集群名稱 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # # node.name: node-1 # # Add custom attributes to the node: # # node.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # # path.data: /path/to/data # # Path to log files: # # path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # # bootstrap.memory_lock: true # # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory # available on the system and that the owner of the process is allowed to use this limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # # http.port: 9200 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html> # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # # discovery.zen.ping.unicast.hosts: ["host1", "host2"] # # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1): # # discovery.zen.minimum_master_nodes: 3 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html> # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # # gateway.recover_after_nodes: 3 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html> # # ---------------------------------- Various ----------------------------------- # # Disable starting multiple nodes on a single system: # # node.max_local_storage_nodes: 1 # # Require explicit names when deleting indices: # # action.destructive_requires_name: true
我修改了兩個地方 cluster.name和network.host
cd ../bin/
./elasticsearch
[2017-01-24 10:02:49,627][INFO ][node ] [Ariel] version[2.4.3], pid[23274], build[d38a34e/2016-12-07T16:28:56Z] [2017-01-24 10:02:49,628][INFO ][node ] [Ariel] initializing ... [2017-01-24 10:02:50,259][INFO ][plugins ] [Ariel] modules [reindex, lang-expression, lang-groovy], plugins [], sites [] [2017-01-24 10:02:50,279][INFO ][env ] [Ariel] using [1] data paths, mounts [[/ (overlay)]], net usable_space [75.3gb], net total_space [113.9gb], spins? [possibly], types [overlay] [2017-01-24 10:02:50,279][INFO ][env ] [Ariel] heap size [990.7mb], compressed ordinary object pointers [true] [2017-01-24 10:02:52,051][INFO ][node ] [Ariel] initialized [2017-01-24 10:02:52,051][INFO ][node ] [Ariel] starting ... [2017-01-24 10:02:52,110][INFO ][transport ] [Ariel] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} [2017-01-24 10:02:52,116][INFO ][discovery ] [Ariel] elasticsearch/MI21JVBWSbKfj9nC1V6N9w [2017-01-24 10:02:55,166][INFO ][cluster.service ] [Ariel] new_master {Ariel}{MI21JVBWSbKfj9nC1V6N9w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received) [2017-01-24 10:02:55,197][INFO ][http ] [Ariel] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200} [2017-01-24 10:02:55,197][INFO ][node ] [Ariel] started
出現類似這樣的信息 說明安裝成功
安裝head插件
elasticsearch根目錄執行
bin/plugin install mobz/elasticsearch-head
注意：2.0以下版本應該是 -install
訪問地址：http://ip:9200/_plugin/head/
安裝analysis-ik插件
下載地址：https://github.com/medcl/elasticsearch-analysis-ik/tree/v1.10.3
使用maven打包：mvn clean package
生成的zip包在target/releases下
在elasticsearch-2.4.3/plugins下創建ik目錄
將zip包放到該目錄下並解壓 解壓出如下文件
編輯elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下內容
index: analysis: analyzer: ik: alias: [ik_analyzer] type: org.elasticsearch.index.analysis.IkAnalyzerProvider ik_max_word: type: ik use_smart: false ik_smart: type: ik use_smart: true
或者
index.analysis.analyzer.ik.type : “ik”
測試：http://ip:9200/_analyze?analyzer=ik&pretty=true&text=我是中國人
安裝searchguard
elasticsearch根目錄執行
bin/plugin install -b com.floragunn/search-guard-2/2.4.3.9
bin/plugin install -b com.floragunn/search-guard-ssl/2.4.3.19
下載 searchguard-ssl 的包，里面包含自動創建證書的腳本：
wget https://github.com/floragunncom/search-guard-ssl/archive/v2.4.3.19.zip
unzip v2.4.3.19.zip
cd search-guard-ssl-2.4.3.19/example-pki-scripts/
有三個腳本
gen_client_node_cert.sh 創建客戶端證書
gen_node_cert.sh 創建節點證書
gen_root_ca.sh 創建根證書
編輯腳本 vim example.sh
#!/bin/bash set -e ./clean.sh ./gen_root_ca.sh password password ./gen_node_cert.sh 0 password password ./gen_node_cert.sh 1 password password ./gen_client_node_cert.sh admin password password cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.4.3/config/ cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.4.3/plugins/search-guard-2/sgconfig/
./example.sh
可以發現 最后兩句就是將證書cp到相應目錄
編輯elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下內容
############################################################################################# # SEARCH GUARD # # Configuration # ############################################################################################# security.manager.enabled: false searchguard.authcz.admin_dn: - "CN=admin, OU=client, O=client, L=Test, C=DE" ############################################################################################# # SEARCH GUARD SSL # # Configuration # ############################################################################################# ############################################################################################# # Transport layer SSL # # # ############################################################################################# # Enable or disable node-to-node ssl encryption (default: true) # searchguard.ssl.transport.enabled: true # JKS or PKCS12 (default: JKS) #searchguard.ssl.transport.keystore_type: PKCS12 # Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks # Alias name (default: first alias which could be found) #searchguard.ssl.transport.keystore_alias: my_alias # Keystore password (default: changeit) searchguard.ssl.transport.keystore_password: password # JKS or PKCS12 (default: JKS) #searchguard.ssl.transport.truststore_type: PKCS12 # Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir searchguard.ssl.transport.truststore_filepath: truststore.jks # Alias name (default: first alias which could be found) #searchguard.ssl.transport.truststore_alias: my_alias # Truststore password (default: changeit) searchguard.ssl.transport.truststore_password: password # Enforce hostname verification (default: true) # searchguard.ssl.transport.enforce_hostname_verification: true # If hostname verification specify if hostname should be resolved (default: true) # searchguard.ssl.transport.resolve_hostname: true # Use native Open SSL instead of JDK SSL if available (default: true) # searchguard.ssl.transport.enable_openssl_if_available: false
在elasticsearch根目錄 執行命令 將配置插入
./plugins/search-guard-2/tools/sgadmin.sh -cn 集群名稱 -h hostname -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv
注意：elasticsearch的服務必須是運行狀態
elasticsearch-2.4.3/plugins/search-guard-2/sgconfig下的配置文件是管理用戶角色的
安裝配置成功后 任何客戶端訪問elasticsearch 需提供用戶名及密碼
至此服務端安裝結束
客戶端將以源碼方式提供 為公司信息安全着想 僅提供關鍵性代碼供參考 無法運行