ElasticSearch + Logstash + Kibana 搭建筆記

ElasticSearch 安裝

1、下載 ElasticSearch，本文使用的版本為 5.5.1。

2、配置

path.data: /data/es #數據路徑
path.logs: /data/logs/es #日志路徑
network.host: 本機地址 #服務器地址
http.port: 9200 #端口

如果不修改配置的話，默認的數據和日志都位於elasticsearch文件夾下。

默認地址會使用 192.168.0.1 的地址，此時ElasticSearch運行於開發模式，只能從本機訪問。如果修改為生產地址，就會進入生產模式，並且運行 bootstrap check 。

3、啟動

./bin/elasticsearch

注意，elasticsearch 不能使用 root 用戶啟動，使用其他用戶啟動，要注意有文件夾的讀寫權限。

我在安裝過程中還出現了下面幾個警告信息

[2017-08-07T09:13:59,951][WARN ][o.e.b.JNANatives         ] unable to install syscall filter: 
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
    at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:350) ~[elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:638) ~[elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:245) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:194) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.5.1.jar:5.5.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.5.1.jar:5.5.1]
[2017-08-01T14:10:57,843][WARN ][o.e.b.BootstrapChecks    ] [VAfWGGZ] max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]
[2017-08-01T14:10:57,844][WARN ][o.e.b.BootstrapChecks    ] [VAfWGGZ] max number of threads [1024] for user [maserati] is too low, increase to at least [2048]
[2017-08-01T14:10:57,844][WARN ][o.e.b.BootstrapChecks    ] [VAfWGGZ] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-08-01T14:10:57,844][WARN ][o.e.b.BootstrapChecks    ] [VAfWGGZ] system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

針對文件描述符，調成 65536 ulimit -n 65536，如果提示沒有權限，則可以在用戶的 .bash_profile 中增加一行，退出用戶重新登陸就可以。

針對 max number of threads 問題，修改 /etc/security/limits.d/90-nproc.conf 。

*          soft    nproc     2048
root       soft    nproc     unlimited

針對 max virtual memory areas ，修改 /etc/sysctl.conf。如果沒有，就新增一行。

vm.max_map_count = 262144

針對 system_call_filter 可以，通過修改配置文件（elasticsearch.yml）關掉這個參數。

bootstrap.system_call_filter: false 

4、訪問，出現下面的結果表示啟動成功。

[root@iZ627x15h6pZ cloud]# curl http://localhost:9200
{
  "name" : "VAfWGGZ",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "J9Tm5R2zRt2PkOSwtXj5Wg",
  "version" : {
    "number" : "5.5.1",
    "build_hash" : "19c13d0",
    "build_date" : "2017-07-18T20:44:24.823Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

Logstash 安裝

1、下載並解壓 Logstash，本文用的 Logstash-5.5.1 版本

2、創建一個簡單的配置文件 logstash_test.conf

input { stdin { } }
output {
  stdout { codec => rubydebug }
}

3、啟動 logstash

./bin/logstash -f logstash_test.conf 

出現這些信息，表示啟動成功了。

[2017-08-01T13:58:38,437][INFO ][logstash.pipeline        ] Pipeline main started
The stdin plugin is now waiting for input:
[2017-08-01T13:58:38,532][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

4、與ElasticSearch配合。

Kibana 安裝

1、下載 Kibana

2、修改配置

//啟動端口 因為端口受限 所以變更了默認端口
server.port: 5601
//啟動服務的ip
server.host: "本機ip"
//elasticsearch地址
elasticsearch.url: "http://localhost:9200”

3、啟動程序

./bin/kibana

4、訪問查看Kibana啟動是否成功，並檢索查看數據

http://localhost:5601

參考資料：
1、Download Logstash
2、ElasticSearch Download
3、ElasticSearch 5.0啟動出現的錯誤
4、Ulimit詳解
5、ELK+Filebeat 安裝配置入門