背景
由於需要存儲風控審核相關的數據,審核數據原數據是保存在MySQL,數據量太大,按月分表,分表后導致后台無法跨月查詢,因此將數據同步到ElasticSearch中
為了合理的划分索引的大小,需要將對應的ElasticSearch索引也按月創建索引,將對應的MySQL月份表數據同步到ES月份索引中,所有的ElasticSearch 按月創建的索引使用同一個別名,后台查詢統一使用索引別名查詢,這樣就解決了MySQL按月分表后無法進行跨月查詢的問題
測試代碼
為需要按月創建的索引,創建指定的索引模板
1.創建模板 test_template,指定索引別名為test,以及對應的mapping
template屬性表示 在創建以 “test”開頭的索引時,會自動使用該模板
# 模板 PUT /_template/test_template { "template": "test*", "settings": { "number_of_shards": 10 }, "mappings": { "data": { "_source": { "enabled": false }, "properties": { "name": { "type": "keyword" }, "id": { "type": "long" } } } }, "aliases": { "test": {} } }
模板創建完成后,可以使用API查詢模板詳情
GET /_template/test_template
2.創建索引 test2和test1
PUT test1
PUT test2
由於索引test1和索引test2匹配到模板test_template,會自動套用該模板屬性
因此查詢test1和test2索引詳情可以看到,與模板屬性一致
test1索引詳情
{ "test1" : { "aliases" : { "test" : { } }, "mappings" : { "data" : { "_source" : { "enabled" : false }, "properties" : { "id" : { "type" : "long" }, "name" : { "type" : "keyword" } } } }, "settings" : { "index" : { "creation_date" : "1646624507395", "number_of_shards" : "10", "number_of_replicas" : "1", "uuid" : "5qbPh-QsQ2SNzKqNGL401Q", "version" : { "created" : "6080799" }, "provided_name" : "test1" } } } }
test2索引詳情
{ "test2" : { "aliases" : { "test" : { } }, "mappings" : { "data" : { "_source" : { "enabled" : false }, "properties" : { "id" : { "type" : "long" }, "name" : { "type" : "keyword" } } } }, "settings" : { "index" : { "creation_date" : "1646624657372", "number_of_shards" : "10", "number_of_replicas" : "1", "uuid" : "t0irKTFQT9qU_F5m_v7RlA", "version" : { "created" : "6080799" }, "provided_name" : "test2" } } } }
3.為test1和test2添加數據
POST test1/_doc/1 { "id":1, "name":"test1-aaaa" } POST test1/_doc/2 { "id":2, "name":"test1-bbbb" } POST test2/_doc/1 { "id":1, "name":"test2-aaaa" } POST test2/_doc/2 { "id":2, "name":"test2-bbbb" }
4.通過別名test查詢,可以查詢test1和test2全部數據
GET test/_search
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 40, "successful" : 40, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 4, "max_score" : 1.0, "hits" : [ { "_index" : "test1", "_type" : "data", "_id" : "1", "_score" : 1.0 }, { "_index" : "test2", "_type" : "data", "_id" : "1", "_score" : 1.0 }, { "_index" : "test1", "_type" : "data", "_id" : "2", "_score" : 1.0 }, { "_index" : "test2", "_type" : "data", "_id" : "2", "_score" : 1.0 } ] } }