1、安裝elasticsearch
先要安裝jdk8 （配置jdk8的環境變量）
下載地址：
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.5.1-linux-x86_64.tar.gz 解壓
修改配置文件：
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#
cluster.name: myes
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.0.117
xpack.ml.enabled: false
#
# Set a custom port for HTTP:
#
http.port: 9200
transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
cluster.name: myes 這個是集群節點的名稱
node.name: node-1 節點的名稱
path.data: /path/to/data 數據存儲的位置
path.logs: /path/to/logs 日志
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 192.168.0.117 這個是服務器的地址
xpack.ml.enabled: false
http.port: 9200 這個是http訪問的地址就是頁面訪問的地址
transport.tcp.port: 9300 這個是連接時的端口 就是代碼連接時候用的
cluster.initial_master_nodes: ["node-1"] 指定master 節點
vi /etc/sysctl.conf
添加
vm.max_map_count=655360
保存后執行
sysctl -p
vi /etc/security/limits.conf
在最后一行加入下面的代碼
* hard nofile 131072
* soft nofile 65536
* soft nproc 4096
* hard nproc 4096
執行時可能會遇到的錯誤
JVM is using the client VM [Java HotSpot(TM) Client VM] but should be using a server VM for the best performance JVM正在使用客戶機VM [Java HotSpot(TM)客戶機VM]，但是為了獲得最佳性能，應該使用服務器VM
如果有這個錯
將VM設置成 Server VM： 找到 jre安裝目錄 /lib /i386 /jvm.cfg 文件，JVM默認是client版本 ：如圖所示，第一行和第二行互換位置即可，誰在上面就是誰。目前是Server VM
進入 es的 bin 目錄下執行./elasticsearch -d 后台 啟動
下載kibana
https://artifacts.elastic.co/downloads/kibana/kibana-7.5.1-linux-x86_64.tar.gz
進入kibana下面的config目錄kibana.yml這個文件
server.port: 5601
server.host: "192.168.0.117" 這個是當前機器的ip
elasticsearch.hosts: ["http://192.168.0.117:9200"] 這個是es的址
kibana不支付root用戶啟動所以要新建用戶來啟動
groupadd es 創建分組
useradd liuchao -g es -p 123456 創建用戶並指定分組
chown -R liuchao:es elasticsearch 將es的目錄整個授權
chown -R liuchao:es kibana 將 kibana的目錄整個授權給新建用戶
su liuchao 切換用戶
nohup ../bin/kibana & 后台啟動 kibana
logstash的安裝配置
下載地址：
https://artifacts.elastic.co/downloads/logstash/logstash-7.5.1.tar.gz
直接解壓
進入bin 目錄下面
vim /usr/local/logstash-5.5.2/bin/mysqltoes.conf
input {
stdin { }
jdbc {
#需要同步的數據庫
jdbc_connection_string => "jdbc:mysql://192.168.0.23:3306/demo"
jdbc_user => "root"
jdbc_password => "123456"
#本地jar包
jdbc_driver_library => "/usr/local/es/mysql-connector-java-3.1.12-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
#獲取到記錄的SQL查詢語句
statement => "SELECT * FROM user"
#定時字段 各字段含義（由左至右）分、時、天、月、年，全部為*默認含義為每分鍾都更新
schedule => "* * * * *"
}
}
output {
stdout {
codec => json_lines
}
elasticsearch {
#ESIP地址與端口
hosts => ["192.168.0.117:9200"]
#ES索引名稱（自己定義的）
index => "logdemo"
#文檔類型
document_type => "user"
#文檔類型id，%{userid}意思是取查詢出來的userid的值，並將其映射到es中_id字段中
document_id => "%{id}"
}
}
啟動：./logstash -f mysqltoes.conf