Elasticsearch之常見問題

　　一. 聚合操作時，報Fielddata is disabled on text fields by default

GET /megacorp/employee/_search
{
  "aggs": {
    "all_interests": {
      "terms": {"field": "interests" }
    }
  }
}


{ "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [interests] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "megacorp", "node": "sNvWT__lQl6p0dMTRaAOAg", "reason": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [interests] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead." } } ], "caused_by": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [interests] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.", "caused_by": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [interests] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead." } } }, "status": 400 }

　　

text類型的字段在查詢時使用的是在內存中的稱為fielddata的數據結構。這種數據結構是在第一次將字段用於聚合/排序/腳本時基於需求建立的。

它通過讀取磁盤上每個segmet上所有的倒排索引來構建，反轉term和document的關系(倒排)，並將結果存在Java堆上(內存中)。(因此會耗費很多的堆空間，特別是在加載很高基數的text字段時)。一旦fielddata被加載到堆中，它在segment中的生命周期還是存在的。

因此，加載fielddata是一個非常消耗資源的過程，甚至能導致用戶體驗到延遲.這就是為什么 fielddata 默認關閉。

 

PUT megacorp/_mapping/employee/
{
  "properties": {
    "interests": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

 

　　二.Too many dynamic script compilations within, max: [75/5m]

需要設置索引允許最大編譯速度

curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{ "transient": { "script.max_compilations_rate": "100000/1m"}}' 

 

　　三. max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

在root用戶下

　　臨時修改:

sysctl -w vm.max_map_count=262144

sysctl -p  
# 重啟恢復原值

 

　　永久修改:

　

echo "vm.max_map_count=262144" > /etc/sysctl.conf
sysctl -p

 

　　四. the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

　　解決:

cluster.name: "docker-cluster"
network.host: 0.0.0.0


# custom config
node.name: "node-1"
discovery.seed_hosts: ["127.0.0.1", "[::1]"]
cluster.initial_master_nodes: ["node-1"]
# 開啟跨域訪問支持，默認為false
http.cors.enabled: true
# 跨域訪問允許的域名地址，(允許所有域名)以上使用正則
http.cors.allow-origin: /.*/ 


重點: node.name 和 cluster.initial_master_nodes 設置

 

　　五. 有時候新啟動的es服務無法組成集群,只能單機運行

　　解決:

　　　　需要將之前的數據進行刪除,並重新啟動服務

rm -rf /elasticsearch/data/*

 

 　　六.  

同一個index新增type報錯 Rejecting mapping update to [website] as the final mapping would have more than 1 type: [blog2, blog]

　　7之后, 已經不推薦使用type,所以在添加數據的時候可以不指定type即可　　

 

 　七.

 

 原因: 磁盤空間不足, 超過95%, 則開啟只讀模式, 可以進行數據刪除  ( df  -h 查看)

{
  "index": {
    "blocks": {
      "read_only_allow_delete": "false"
    }
  }
}

 

 　　八. Can't update non dynamic settings [[index.analysis.filter

場景: 在為索引添加分詞器的時候, 發生報錯

解決方案: 先關閉索引, 再進行設置, 設置完成之后, 重新打開索引即可

# 關閉索引
POST mp_account2/_close

# 設置分詞器
PUT mp_account2/_settings
{
  "index": {
    "analysis": {
      "analyzer": {
        "ik_pinyin_analyzer": {
          "type": "custom",
          "tokenizer": "ik_smart",
          "filter": "pinyin_filter"
        }
      },
      "filter": {
        "pinyin_filter": {
          "type": "pinyin",
          "keep_first_letter": false
        }
      }
    }
  }
}

# 開啟索引
POST mp_account2/_open