elasticsearch筆記(5) java操作es的查詢_04深分頁scroll查詢

一. 為什么用深度分頁scroll查詢

   分頁可以用terms查詢的from to來實現;但是from to二者之和大於一萬后就效率低下. 原因是

es查詢的方式:

1. 將用戶指定的關鍵字進行分詞,
2. 將詞匯去分詞庫中去檢索. 得到多個文檔的id
3. 去各個分片中去拉取指定數據; 這步最慢
4. 將數據根據匹配度score排序; 耗時長
5. 將查詢的數據舍棄一部分: 如from5to10, 就把不是5-10條的數據舍棄
6. 返回結果

       scroll查詢的方式:         

1. 將用戶指定的關鍵字進行分詞,
2. 將詞匯去分詞庫中去檢索. 得到多個文檔的id,
3. 將id存在一個es的上下文中
4. 根據size鍵es中檢索指定的數據, 拿到數據的文檔id, 會從上下文中移出,
5. 如果需要下一頁數據, 直接到es的上下文中找到后續的內容.
6. 循環4  5 查詢

 

 在java中用scroll查詢

   @Test
    public void scrollQueryTest() throws IOException {
        //        1. 創建查詢對象
        String index = "sms-logs-index";
        String type = "sms-logs-type";
        SearchRequest searchRequest = new SearchRequest(index);//指定索引
        searchRequest.types(type);//指定類型
        searchRequest.scroll(TimeValue.timeValueMinutes(1l));//指定存在內存的時長為1分鍾
//    2. 封裝查詢條件
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.sort("fee", SortOrder.DESC);
        searchSourceBuilder.size(2);
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        searchRequest.source(searchSourceBuilder);


        //        3.執行查詢
        // client執行
        HttpHost httpHost = new HttpHost("192.168.43.30", 9200);
        RestClientBuilder restClientBuilder = RestClient.builder(httpHost);
        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(restClientBuilder);

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        String scrollId = searchResponse.getScrollId();
        System.out.println(scrollId);//獲取scorllId


//        4.獲取數據
        SearchHit[] hits = searchResponse.getHits().getHits();
        for(SearchHit searchHit : hits){
            System.out.println(searchHit);
        }

        //獲取全部的下一頁, 不過我不知道這種有什么用?????
        while (true){
            //創建SearchScrollRequest對象
            SearchScrollRequest searchScrollRequest = new SearchScrollRequest(scrollId);
            searchScrollRequest.scroll(TimeValue.timeValueMinutes(1l));//設置1分鍾
            SearchResponse scroll = restHighLevelClient.scroll(searchScrollRequest, RequestOptions.DEFAULT);
            SearchHit[] hits1 = scroll.getHits().getHits();
            if(hits1 != null && hits1.length > 0){
                System.out.println("------------下一頁--------------");
                for(SearchHit searchHit : hits1){
                    System.out.println(searchHit);
                }

            }else {
                System.out.println("------------結束--------------");
                break;
            }
        }
        
        //刪除ScrollId
        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        System.out.println("刪除scroll"  + clearScrollResponse);
    }