1、业务报错
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]
初步判断索引变成只读了。
2、查找原因
cluster.routing.allocation.disk.watermark.flood_stage
原文:
Controls the flood stage watermark. It defaults to 95%, meaning that Elasticsearch enforces a read-only index block(index.blocks.read_only_allow_delete)on every index that has one or more shards allocated on the node that has at least one disk exceeding the flood stage. This is a last resort to prevent nodes from running out of disk space. The index block is automatically released once the disk utilization falls below the high watermark.
翻译:
控制洪泛水位线。默认为95%,这意味着Elasticsearch在每个索引上强制执行只读索引块(“index.blocks.read_only_allow_delete”),该索引在至少有一个磁盘超过洪泛阶段的节点上分配了一个或多个碎片。这是防止节点耗尽磁盘空间的最后手段。一旦磁盘利用率低于高水位线,索引块就会自动释放。
3、解决方案
以上是可以在 elasticsearch.yml 配置文件中配置的设置,也可以使用 cluster-update-settings API 在活动集群上动态更新设置。
1)先对磁盘进行扩容,或者删除不用的索引释放空间
2)重置该只读索引块
PUT /_all/_settings { "index.blocks.read_only_allow_delete": null }