java連接ES操作入門

 

如何建立數據模型？

1：建立索引

DELETE /hd_insurance_data
​
PUT hd_insurance_data
{
  "settings": {
    "number_of_replicas": 1
  }, 
  "mappings" : {
    "properties" : {
      "age" : {
         "type" : "integer"
      },
      "channel_code" : {
        "type" : "keyword"
      },
      "channel_name" : {
        "type" : "keyword"
      }
    }
  }
}

　　

2：寫入數據

2.1:手動寫入數據

put /hd_insurance_data/_doc/1 
{
"id": "1",
"department_name": "大項目部",
"department_code": "",
"policy_number": "be16985732a40212345690ed30d1db"
}

 

2.2:通過使用kibana進行數據批量導入

這里不做介紹，導入數據嘗試幾次都失敗了，不太好用

 

2.3:命令行導入數據

curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"

搜索java的api了解

1：啟動集群elasticsearch 和kibana

bin/elasticsearch -E node.name=node1 -E cluster.name=health -E path.data=node1_data -d
​
bin/elasticsearch -E node.name=node2 -E cluster.name=health -E path.data=node2_data -d
​
bin/elasticsearch -E node.name=node3 -E cluster.name=health -E path.data=node3_data -d

bin/kibana

2：建立索引導入數據

3：Java連接es

導入es配置

  <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
  </dependency>

spring.elasticsearch.rest.uris=http://localhost:9200

在項目中注入RestHighLevelClient直接就可以使用

 @Autowired
 private ElasticsearchRestTemplate client;
 @Autowired
 private RestHighLevelClient highLevelClient;

4：建議

1、做簡單的bool查詢獲取狀態是否正常

GET /hd_insurance_data/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "policy_status": {
              "value": "1"
            }
          }
        }
      ]
    }
  }
}

 

2、對年齡每隔5歲進行分桶，並將分桶的年齡分男女進行分桶

GET /hd_insurance_data/_search
{
  "size": 0,
  "aggs": {
    "age_histrogram":{
      "histogram": {
        "field": "age",
        "interval": 5
      },
      "aggs": {
        "sex_count": {
          "terms": {
            "field": "sex",
            "size": 10
          }
        }
      }
    }
  }
}

返回結果，是每個5歲一個分類，同時獲取這個分桶中男女的數量

{
"key" : 20.0,
"doc_count" : 1,
"sex_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 1
}
]
}
},
{
"key" : 25.0,
"doc_count" : 5,
"sex_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 3
},
{
"key" : "2",
"doc_count" : 2
}
]
}
}

 

3、將兩個步驟結合起來，獲取想要的結果

GET /hd_insurance_data/_search
{
  "size": 0, 
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "policy_status": {
              "value": "1"
            }
          }
        }
      ], 
      "filter": [
        {
          "range": {
            "insurance_time": {
              "gte": "2017-01-01 15:09:13",
              "lte": "2021-10-01 15:09:13"
            }
          }
        }
      ]
    }
  }, 
  "aggs": {
    "age_histrogram":{
      "histogram": {
        "field": "age",
        "interval": 5
      }
    },
    "aggs":{
      "sex_count":{
        "terms": {
          "field": "sex",
          "size": 10
        }
      }
    }
  }
}

 

 

4、將上面的查詢通過api翻譯為java代碼

 public void searchByConditions() throws IOException {
    SearchRequest searchRequest = new SearchRequest("hd_insurance_data");
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
​
    // 設定搜索條件
    BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
        .must(QueryBuilders.termQuery("policy_status", "1"))
        .must(QueryBuilders.termQuery("department_name", "目部"))
        .must(QueryBuilders.termQuery("channel_name", "保險代理"))
        .must(QueryBuilders.termQuery("sales_plan_code", "S1230445"))
        .must(QueryBuilders.termQuery("quota_plan_code", "10123123101"))
        .filter(QueryBuilders.rangeQuery("insurance_time").gte("2019-01-01 15:09:13").lte("2019-10-01 15:09:13"));
​
    // 聚類分析，並子聚類
    AggregationBuilder aggregationBuilder = AggregationBuilders.dateHistogram("age_histrogram").field("age").interval(5);
    AggregationBuilder subAggregationBuilder = AggregationBuilders.terms("sex_count").field("sex");
    aggregationBuilder.subAggregation(subAggregationBuilder);
​
    searchSourceBuilder.size(0);
    searchSourceBuilder.query(boolQueryBuilder);
    searchSourceBuilder.aggregation(aggregationBuilder);
    searchRequest.source(searchSourceBuilder);
​
    SearchResponse searchResponse = highLevelClient.search(searchRequest, RequestOptions.DEFAULT);
    log.info("searchResponse {}", searchResponse.toString());
//    log.info("searchResponse getAggregations {}", Strings.toString(searchResponse.getAggregations()));
  }

 

 

總結：

1、es使用非常簡單，但是各種查詢有點多，還需要進一步梳理

本文是對工作中使用es的一個簡單調研，剛學習，還請多多指教。

參考

1:elasticsearch的搜索api

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/java-rest-high-query-builders.html