一、前言
es可以用來存儲日志，一般日志存儲只是短期保存，超過一定時間日志要是能自動刪除最好，這樣保證索引文檔不會過多，查詢時效性也能得到保證。本文參考的官網地址是：https://www.elastic.co/guide/en/elasticsearch/reference/7.5/getting-started-index-lifecycle-management.html 。索引的生命周期分為四個階段：HOT->WARM->COLD->DELETE。
上面除了HOT為必須的階段外，其他為非必須階段，可以任意選擇配置。因為日志索引只要滿足自己刪除功能，所以下文只配置了HOT與DELETE階段。
二、配置
配置自己刪除功能，必須配置策略(policy)、索引模版(template)、索引(index)。上述配置缺一不可。
1）配置策略
PUT _ilm/policy/datastream_policy { "policy": { "phases": { "hot": { "actions": { "rollover":{ "max_docx":1 } } }, "delete": { "min_age": "30s", "actions": { "delete": {} } } } } }
采用PUT方式向es創建一個名為“datastream_policy“的策略,"_ilm/policy"為固定寫法。如果是使用postman，選擇PUT請求方式，地址為：http://ip:port/_ilm/policy/datastream_policy ，HOT階段：當日志索引超過1條時就會發生rollover，此處設置1條主要是為了測試方便。DELETE階段：超過30秒時執行刪除。
hot->actions->rollover還支持其他維度的控制，比如：
"max_size": "50GB" "max_age": "30d"
2）配置索引模版
PUT _template/datastream_template { "index_patterns": ["datastream-*"], "settings": { "number_of_shards": 1, "number_of_replicas": 1, "index.lifecycle.name": "datastream_policy", "index.lifecycle.rollover_alias": "datastream" } }
創建一個索引模版，索引以datastream-開頭的自動采用settings的配置。"index.lifecycle.name"意為采用”datastream_policy“的策略，"index.lifecycle.rollover_alias"意為創建使用該模版創建的索引統一用”datastream“的別名進行管理。
3）創建索引
PUT datastream-000001 { "aliases": { "datastream": { "is_write_index": true } } }
創建一個開始的索引，並設置索引可寫入。
4）配置lifecycle檢測時間
PUT /_cluster/settings
{ "transient": { "indices.lifecycle.poll_interval": "10s" } }
默認為十分鍾，為了測試看效果，改為10秒鍾。
5）驗證功能
向別名為"datastream"的索引中進行寫入多條數據
PUT /datastream/_doc
{
"message":"hello-01"
}
索引采用別名索引。
寫入后，es根據配置的lifecycle時間進行rollover。用以下命令查看索引rollover進度
GET datastream-*/_ilm/explain
等待30秒后，返回的數據如下：
{ "indices": { "datastream-000002": { "index": "datastream-000002", "managed": true, "policy": "datastream_policy", "lifecycle_date_millis": 1587608411208, "age": "1.08m", "phase": "hot", "phase_time_millis": 1587608411441, "action": "rollover", "action_time_millis": 1587608471383, "step": "check-rollover-ready", "step_time_millis": 1587608471383, "phase_execution": { "policy": "datastream_policy", "phase_definition": { "min_age": "0ms", "actions": { "rollover": { "max_docs": 1 } } }, "version": 1, "modified_date_in_millis": 1587608224989 } }, "datastream-000001": { "index": "datastream-000001", "managed": true, "policy": "datastream_policy", "lifecycle_date_millis": 1587608411268, "age": "1.08m", "phase": "delete", "phase_time_millis": 1587608471345, "action": "delete", "action_time_millis": 1587608471345, "step": "wait-for-shard-history-leases", "step_time_millis": 1587608471345, "phase_execution": { "policy": "datastream_policy", "phase_definition": { "min_age": "60s", "actions": { "delete": {} } }, "version": 1, "modified_date_in_millis": 1587608224989 } } } }
索引自動創建了一個新的"datastream-000002"，舊的索引”datastream-000001“ action變成DELETE。
再過一段時間再次查看索引
GET datastream-*/_ilm/explain
查看結果：
{ "indices": { "datastream-000002": { "index": "datastream-000002", "managed": true, "policy": "datastream_policy", "lifecycle_date_millis": 1587608411208, "age": "2.8m", "phase": "hot", "phase_time_millis": 1587608411441, "action": "rollover", "action_time_millis": 1587608471383, "step": "check-rollover-ready", "step_time_millis": 1587608471383, "phase_execution": { "policy": "datastream_policy", "phase_definition": { "min_age": "0ms", "actions": { "rollover": { "max_docs": 1 } } }, "version": 1, "modified_date_in_millis": 1587608224989 } } } }
舊的索引進行了刪除。在做日志查詢時，我們只需要使用別名”datastream“進行日志查詢即可，es便可無感知完成日志存儲刪除動作。