Elasticsearch配置文件說明

 

一、Cluster  setting

1. Cluster

indices.ttl.interval  允許設置多久過期的文件會被自動刪除。默認值是60秒。

indices.cache.filter.size  ES的filter cache有兩種，一種是node級別的cache（filter cache默認類型），一種是index級別的filter cache。Node級別的cache被整個node共享，並且可以使用百分比設置，對應的屬性為index.cache.filter.size，這個屬性的值可以使百分比，也可以是具體的大小。Index級別的cache，顧名思義，就是針對單個索引的大小。ES官方並不推薦使用這種設置，因為誰也無法預測索引級別的緩存到底有多大（可能非常大，超過了node的內存），一個索引可能分布在多個node上面，而多個node的結果如果匯總到一個node上，其結果可想而知。默認值是10%。

discovery.zen.minimum_master_nodes        避免腦裂現象（由於某些節點的失效，部分節點的網絡連接會斷開，並形成一個與原集群一樣的集群，這種情況稱為集群腦裂現象）discovery.zen.minimum_master_nodes參數決定了要選舉一個master需要多少個節點（最少候選節點數）。默認值是1。根據一般經驗這個一般設置成N/2+1（向下取整），N是集群中節點的數量。

1. routing

node_initial_primaries_recoveries  每個初選節點允許控制初始修復的具體數量

cluster_concurrent_rebalance  控制集群寬度允許的分片平衡可以並發多少

awareness.attributes  集群配置意識允許配置分片和副本分配在與節點相關聯的通用屬性

node_concurrent_recoveries  每個節點上允許多少個並發修復，默認是2

disable_allocation   允許禁用主要分配

disable_replica_allocation   允許禁用副本分配

1. recovery

concurrent_streams  設置當從對等中恢復碎片時限制打開的並發流的數量 默認5

file_chunk_size  文件塊大小  默認512kb

translog_ops  

translog_size

max_bytes_per_sec   設置恢復時每分鍾油門的吞吐量  默認20mb

compress  啟用壓縮為所有節點間的通信   默認禁用

 

二、Elasticsearch Configuration Example

#####################Elasticsearch Configuration Example #####################

 

# This file contains an overview of various configuration settings,

#本文件針對操作人員包含了對不同配置設置的概述

# targeted at operations staff. Application developers should

# consult the guide at <http://elasticsearch.org/guide>.

#應用程序開發人員應該參考http://elasticsearch.org/guide指南

# The installation procedure is covered at

# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html>.

#

#安裝過程在<http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html>

# Elasticsearch comes with reasonable defaults for most settings,

# so you can try it out without bothering with configuration.

#                                                  

#ela本身附帶最合理的默認值設置，所以你可以不用修改配置文件直接使用

# Most of the time, these defaults are just fine for running a production

# cluster. If you're fine-tuning your cluster, or wondering about the

#  effect of certain configuration option, please _do ask_ on the

# mailing list or IRC channel [http://elasticsearch.org/community].

#大多時候，這些默認設置可以很好的運行生產環境集群。

#如果你微調你的集群或者考慮到某些配置選項的效果，請在郵件列表或者IRC頻道[http://elasticsearch.org/community]詢問。

# Any element in the configuration can be replaced with environment variables

# by placing them in ${...} notation. For example:

#

#任何組成的配置可以使用環境變量${…}替換他們，例如：

#node.rack: ${RACK_ENV_VAR}

 

# For information on supported formats and syntax for the config file, see

#<http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html>

#支持的格式和語法信息的配置文件，請參考<http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html>

 

三、Cluster

################################### Cluster ###################################

################################## 集群設置###################################

# Cluster name identifies your cluster for auto-discovery. If you're running

# multiple clusters on the same network, make sure you're using unique names.

#                           

#集群名稱識別集群自動發現。如果在同一網段運行多個集群，要確保他們使用唯一的名稱。

 

cluster.name: log-center-it-test

#集群名稱

cluster.routing.allocation.disk.watermark.low: "90%"

cluster.routing.allocation.disk.watermark.high: "96%"

 

indices.fielddata.cache.size: "30%"

索引字段數據緩存大小

 

四、Node

#################################### Node #####################################

################################## 節點設置###################################

 

# Node names are generated dynamically on startup, so you're relieved

# from configuring them manually. You can tie this node to a specific name:

#

#節點名稱是在啟動的時候動態生成的，免去了手動配置。你也可以為這個節點取一個特定的名字:

node.name: "ip"

 

# Every node can be configured to allow or deny being eligible as the master,

# and to allow or deny to store the data.

#

#每個節點都可以被配置為允許或者拒絕成為主節點，並且允許或拒絕存儲數據。

# Allow this node to be eligible as a master node (enabled by default):

#

#允許這個節點成為主節點（默認是允許的）

#node.master: true

#

# Allow this node to store data (enabled by default):

#

#允許這個節點存儲數據（默認是允許的）

#node.data: true

 

# You can exploit these settings to design advanced cluster topologies.

#

#你可以利用這些設置設計先進的集群拓撲。

# 1. You want this node to never become a master node, only to hold data.

#    This will be the "workhorse" of your cluster.

#

#1.不想這個節點成為主節點，只作為存儲數據節點。它會成為集群的馱馬（重負荷機器）

node.master: false

node.data: true

#

# 2. You want this node to only serve as a master: to not store any data and

#    to have free resources. This will be the "coordinator" of your cluster.

#

#2.想要這個節點只作為主節點，不存儲任何數據和有免費資源。他會成為集群的協調員

#node.master: true

#node.data: false

#

# 3. You want this node to be neither master nor data node, but

#    to act as a "search load balancer" (fetching data from nodes,

#    aggregating results, etc.)

#

#3.想要這個節點既不作為主節點也不作為數據節點，只作為搜索負載平衡器（從節點獲取數據，聚合結果，等等）

#node.master: false

#node.data: false

 

# Use the Cluster Health API [http://localhost:9200/_cluster/health], the

# Node Info API [http://localhost:9200/_nodes] or GUI tools

# such as <http://www.elasticsearch.org/overview/marvel/>,

# <http://github.com/karmi/elasticsearch-paramedic>,

# <http://github.com/lukas-vlcek/bigdesk> and

# <http://mobz.github.com/elasticsearch-head> to inspect the cluster state.

 

#使用集群健康API，節點信息API或者GUI工具，如。。、。。和。。檢查集群的狀態。

# A node can have generic attributes associated with it, which can later be used

# for customized shard allocation filtering, or allocation awareness. An attribute

# is a simple key value pair, similar to node.key: value, here is an example:

#

#一個節點可以有與后來被用於定制的碎片分配過濾或分配意識相關聯的通用的屬性，一個屬性是一個簡單的鍵值對，類似於節點。

#node.rack: rack314

 

# By default, multiple nodes are allowed to start from the same installation location

# to disable it, set the following:

#默認的，多個節點可以從相同的安裝位置來禁用它，設置如下:

#node.max_local_storage_nodes: 1

 

五、Index

#################################### Index ####################################

 

# You can set a number of options (such as shard/replica options, mapping

# or analyzer definitions, translog settings, ...) for indices globally,

# in this file.

#

#可以在這個文件中為全局索引設置一些選項（如分片/副本操作，映像或分析儀定義，translog設置）

# Note, that it makes more sense to configure index settings specifically for

# a certain index, either when creating it or by using the index templates API.

#

#注意，當創建或者通過使用API索引模板，為某一個特定的索引配置索引會是更有意義的配置。

#See<http://elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules.html> and

#<http://elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html>

# for more information.

 

# Set the number of shards (splits) of an index (5 by default):

#

#設置一個索引的分片數（默認是5）：

#index.number_of_shards: 5

 

# Set the number of replicas (additional copies) of an index (1 by default):

#

#設置一個索引的副本數（默認是1）：

#index.number_of_replicas: 1

 

# Note, that for development on a local machine, with small indices, it usually

# makes sense to "disable" the distributed features:

#

#注意，在本地機器上開發，一個小的索引通常會有意義的禁用分布式的特性（通俗易懂的說就是不需要副本了，因為索引太小，沒必要建副本。）

#index.number_of_shards: 1

#index.number_of_replicas: 0

 

# These settings directly affect the performance of index and search operations

# in your cluster. Assuming you have enough machines to hold shards and

# replicas, the rule of thumb is replicas, the rule of thumb is:

#

#這些設置直接影響集群中索引和搜索操作的性能。假如你有足夠機器來存儲分片和副本，拇指律（作為一項經驗法則）是副本，拇指律是：

# 1. Having more *shards* enhances the _indexing_ performance and allows to

#    _distribute_ a big index across machines.

# 2. Having more *replicas* enhances the _search_ performance and improves the

#    cluster _availability_.

#

#1.有很多的分片提高索引的性能並且允許在機器上分配更大的索引。

#2.有更多的副本提高搜索的性能並且提高集群的可利用性。

# The "number_of_shards" is a one-time setting for an index.

#

# "number_of_shards"對一個索引的一次性設置

# The "number_of_replicas" can be increased or decreased anytime,

# by using the Index Update Settings API.

#

#"number_of_replicas"可以在任何時間增加或者取消，通過使用更新設置索引API

# Elasticsearch takes care about load balancing, relocating, gathering the

# results from nodes, etc. Experiment with different settings to fine-tune

# your setup.

#ela關注的是負載均衡，遷移，來自節點收集結果，等。嘗試不同的設置來調整你的設置。

         

# Use the Index Status API (<http://localhost:9200/A/_status>) to inspect

# the index status.

#用索引狀態API來檢測索引狀態。

 

六、Paths

################################### Paths ####################################

################################# 路徑設置##################################

 

# Path to directory containing configuration (this file and logging.yml):

#

#配置文件的路徑

#path.conf: /path/to/conf

 

# Path to directory where to store index data allocated for this node.

#

#存儲為該節點分配的索引數據的路徑

#path.data: /path/to/data

#

# Can optionally include more than one location, causing data to be striped across

# the locations (a la RAID 0) on a file level, favouring locations with most free

# space on creation. For example:

#

#能夠包含多個位置，導致數據在整個范圍在文件級被剝離，支持在創新的時候可以有最自由的空間（並不懂什么意思）

#path.data: /path/to/data1,/path/to/data2

 

# Path to temporary files:

#

#臨時文件的路徑

#path.work: /path/to/work

 

# Path to log files:

#

#日志文件的路徑

#path.logs: /path/to/logs

 

# Path to where plugins are installed:

#

#安裝插件的路徑

#path.plugins: /path/to/plugins

 

 

七、Plugin

#################################### Plugin ###################################

#################################### 插件 ###################################

# If a plugin listed here is not installed for current node, the node will not start.

#

#如果這里列出的插件沒有安裝在當前節點，該節點將無法啟動

#plugin.mandatory: mapper-attachments,lang-groovy

 

 

八、Memory

################################### Memory ####################################

################################### 內存 ####################################

 

# Elasticsearch performs poorly when JVM starts swapping: you should ensure that

# it _never_ swaps.

#

#當JVM開始啟動時，ela執行很差：應該確保它是it _never_ swaps

# Set this property to true to lock the memory:

#

#將這個屬性設置為true來鎖定內存

#bootstrap.mlockall: true

bootstrap.mlockall: true

 

# Make sure that the ES_MIN_MEM and ES_MAX_MEM environment variables are set

# to the same value, and that the machine has enough memory to allocate

#  for Elasticsearch, leaving enough memory for the operating system itself.

#

#確保ES_MIN_MEM和ES_MAX_MEM環境變量設置為相同的值，而機器為ela有足夠的內存分配，為操作系統本身留下足夠的內存。

# You should also make sure that the Elasticsearch process is allowed to lock

# the memory, eg. by using `ulimit -l unlimited`.

#應該確保ela過程允許鎖定內存，如通過使用`ulimit -l unlimited`

 

 

九、Network And HTTP

############################## Network And HTTP ###############################

##############################網絡和HTTP設置 ###############################

 

# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens

# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node

# communication. (the range means that if the port is busy, it will automatically

# try the next port).

#ela默認情況下，結合本身的0.0.0.0地址和端口上偵聽http傳輸（9200-9300）端口和（9300-9400）為節點到節點通信傳輸端口。（這意味着如果端口繁忙，他會自動嘗試下一個端口）。

 

# Set the bind address specifically (IPv4 or IPv6):

#

#設置綁定地址（IPv4或者IPv6）：

#network.bind_host: 192.168.0.1

 

# Set the address other nodes will use to communicate with this node. If not

# set, it is automatically derived. It must point to an actual IP address.

#

#設置地址其他節點將使用此地址與這個節點通信，如果沒有設置，他會自動派生。必須指向一個實際的IP地址。

#network.publish_host: 192.168.0.1

 

# Set both 'bind_host' and 'publish_host':

#

#設置綁定主機和發布主機：

#network.host: 192.168.0.1

 

# Set a custom port for the node to node communication (9300 by default):

#

#為節點到節點通信設置自定義端口（默認是9300）

#transport.tcp.port: 9300

 

# Enable compression for all communication between nodes (disabled by default):

#

#啟用壓縮為所有節點之間通信（默認情況下禁用）：

#transport.tcp.compress: true

 

# Set a custom port to listen for HTTP traffic:

#

#設置HTTP傳輸端口

#http.port: 9200

 

# Set a custom allowed content length:

#

#設置一個自定義允許內容長度

#http.max_content_length: 100mb

 

# Disable HTTP completely:

#

#禁用HTTP完全

#http.enabled: false

 

 

十、Gateway

################################### Gateway ###################################

 

# The gateway allows for persisting the cluster state between full cluster

# restarts. Every change to the state (such as adding an index) will be stored

# in the gateway, and when the cluster starts up for the first time,

# it will read its state from the gateway.

#網關允許堅持全面重啟集群之間的集群狀態。每一個變化的狀態（例如添加索引）將存儲在網關中，當集群首次啟動時，它將從網關讀取它的狀態。

 

# There are several types of gateway implementations. For more information, see

# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html>.

 

# The default gateway type is the "local" gateway (recommended):

#

#有幾種類型的網關的實現。更多的信息請參見。。

#默認的網關類型是本地網關（推薦）：

#gateway.type: local

 

# Settings below control how and when to start the initial recovery process on

# a full cluster restart (to reuse as much local data as possible when using shared

# gateway).

#設置在集群重啟時控制如何什么時間開始最初的恢復進程（當使用分享網關時盡可能的重用本地數據）

 

# Allow recovery process after N nodes in a cluster are up:

#

#集群中的N個節點啟動之后允許恢復進程：

#gateway.recover_after_nodes: 1

 

# Set the timeout to initiate the recovery process, once the N nodes

# from previous setting are up (accepts time value):

#

#為啟動恢復過程設置超時，一旦先前設置的n個節點開啟（接受時間價值）：

#gateway.recover_after_time: 5m

 

# Set how many nodes are expected in this cluster. Once these N nodes

# are up (and recover_after_nodes is met), begin recovery process immediately

# (without waiting for recover_after_time to expire):

#

#設置集群中期望有多少個結節點，一旦這些節點啟動，立即開始恢復過程。

#gateway.expected_nodes: 2

 

 

十一、Recovery Throttling

############################# Recovery Throttling #############################

############################# 恢復限流 #############################

 

# These settings allow to control the process of shards allocation between

# nodes during initial recovery, replica allocation, rebalancing,

# or when adding and removing nodes.

#這些設置允許在最初恢復、副本分配、平衡或者添加刪除節點時在節點之間控制分片的過程，

 

# Set the number of concurrent recoveries happening on a node:

#

#設置一個節點上並發復蘇的數量

# 1. During the initial recovery

#

#1.最初恢復

#cluster.routing.allocation.node_initial_primaries_recoveries: 4

#

# 2. During adding/removing nodes, rebalancing, etc

#

#2.在添加、刪除節點，再平衡，等等

#cluster.routing.allocation.node_concurrent_recoveries: 2

 

# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):

#

#當恢復的時候設置油門吞吐量（例如100mb，默認20mb）

#indices.recovery.max_bytes_per_sec: 20mb

 

# Set to limit the number of open concurrent streams when

# recovering a shard from a peer:

#

#當分片從同位體中恢復時設置限制打開並發流的數量

#indices.recovery.concurrent_streams: 5

 

 

十二、Discovery

################################## Discovery ##################################

################################## 發現 ##################################

 

# Discovery infrastructure ensures nodes can be found within a cluster

# and master node is elected. Multicast discovery is the default.

#發現基礎設施確保節點能夠在集群中被發現並且主節點被選出。多播發現是默認的。

 

# Set to ensure a node sees N other master eligible nodes to be considered

# operational within the cluster. This should be set to a quorum/majority of

# the master-eligible nodes in the cluster.

#

#確保一個節點在集群中和在N個其他有資格的節點中被認為是操作者。#discovery.zen.minimum_master_nodes: 1

 

# Set the time to wait for ping responses from other nodes when discovering.

# Set this option to a higher value on a slow or congested network

# to minimize discovery failures:

#

#當發現節點的時候設置等待來自其他節點的ping的回應的時間

#設置這個操作在緩慢和擁擠的網絡中達到一個更高的值：

#discovery.zen.ping.timeout: 3s

discovery.zen.ping.timeout: 120s

 

# For more information, see

# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html>

 

# Unicast discovery allows to explicitly control which nodes will be used

# to discover the cluster. It can be used when multicast is not present,

# or to restrict the cluster communication-wise.

#

#單播發現允許顯示的控制哪些節點將被用來發現集群。它可以用在當多播不生效或者限制集群的通信

# 1. Disable multicast discovery (enabled by default):

#

#1.禁止多播發現（默認啟用）

#discovery.zen.ping.multicast.enabled: false

#

# 2. Configure an initial list of master nodes in the cluster

#    to perform discovery when new nodes (master or data) are started:

#

#2.在集群中配置一個初始主節點列表來執行發現，當新的節點（主或者數據）開啟的時候

#discovery.zen.ping.unicast.hosts: ["host1", "host2:port"]

 

# EC2 discovery allows to use AWS EC2 API in order to perform discovery.

#

#EC2發現允許使用AWS WC2 API來執行發現

# You have to install the cloud-aws plugin for enabling the EC2 discovery.

#

#必須安裝cloud-aws插件啟用EC2的發現

# For more information, see

# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html>

#

# See <http://elasticsearch.org/tutorials/elasticsearch-on-ec2/>

# for a step-by-step tutorial.

 

# GCE discovery allows to use Google Compute Engine API in order to perform discovery.

#

#GCE發現允許使用谷歌計算引擎API來執行發現

# You have to install the cloud-gce plugin for enabling the GCE discovery.

#

#必須安裝cloud-gce插件啟用GCE發現

# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-gce>.

 

# Azure discovery allows to use Azure API in order to perform discovery.

#

#Azure發現允許使用Azure API來執行發現

# You have to install the cloud-azure plugin for enabling the Azure discovery.

#

#必須安裝cloud-azure插件啟用Azure的發現

# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-azure>.

 

 

十三、Slow Log

################################## Slow Log ##################################

############################ 日志 ##################################

 

# Shard level query and fetch threshold logging.

 

#index.search.slowlog.threshold.query.warn: 10s

#index.search.slowlog.threshold.query.info: 5s

#index.search.slowlog.threshold.query.debug: 2s

#index.search.slowlog.threshold.query.trace: 500ms

 

#index.search.slowlog.threshold.fetch.warn: 1s

#index.search.slowlog.threshold.fetch.info: 800ms

#index.search.slowlog.threshold.fetch.debug: 500ms

#index.search.slowlog.threshold.fetch.trace: 200ms

 

#index.indexing.slowlog.threshold.index.warn: 10s

#index.indexing.slowlog.threshold.index.info: 5s

#index.indexing.slowlog.threshold.index.debug: 2s

#index.indexing.slowlog.threshold.index.trace: 500ms

 

################################## GC Logging ################################

 

#monitor.jvm.gc.young.warn: 1000ms

#monitor.jvm.gc.young.info: 700ms

#monitor.jvm.gc.young.debug: 400ms

 

#monitor.jvm.gc.old.warn: 10s

#monitor.jvm.gc.old.info: 5s

#monitor.jvm.gc.old.debug: 2s

 

################################## Security ################################

 

# Uncomment if you want to enable JSONP as a valid return transport on the

# http server. With this enabled, it may pose a security risk, so disabling

# it unless you need it is recommended (it is disabled by default).

#

#http.jsonp.enable: true