Elasticsearch Java API 的使用(8)—Scroll (游標)API詳解


####滾動查詢
Elasticsearch中進行大數據量查詢時,往往因為設備、網絡傳輸問題影響查詢數據的效率;Elasticsearch中提供了 Scroll(游標)的方式對數據進行少量多批次的滾動查詢,來提高查詢效率。

public class Scroll {

    public static void main(String[] args) {

        try{
            long startTime = System.currentTimeMillis();
            /*創建客戶端*/
            //client startup
            //設置集群名稱
            Settings settings = Settings.builder()
                    .put("cluster.name", "elsearch")
                    .put("client.transport.sniff", true)
                    .build();
            //創建client
            TransportClient client = new PreBuiltTransportClient(settings)
                    .addTransportAddress(new InetSocketTransportAddress(
                    InetAddress.getByName("54.223.232.95"),9300));

            List<String> result = new ArrayList<>();

            String scrollId = "";

            //第一次請求
            SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();


            //TODO: 設置查詢條件
            RangeQueryBuilder rangequerybuilder = QueryBuilders
                .rangeQuery("inputtime")
                .from("2016-12-14 02:00:00").to("2016-12-14 07:59:59");
            sourceBuilder.query(QueryBuilders.boolQuery()
                .must(QueryBuilders
                        .matchPhraseQuery("pointid","W3.UNIT1.10HFC01CT013"))
                    .must(rangequerybuilder))
                    .size(100)//如果開啟游標,則滾動獲取
                    .sort("inputtime", SortOrder.ASC);
            //查詢
            SearchRequest request = Requests.searchRequest("pointdata");
                request.scroll("2m");
                request.source(sourceBuilder);
            SearchResponse response = client.search(request).actionGet();
            //TODO:處理數據
            SearchHits hits = response.getHits();
            for(int i = 0; i < hits.getHits().length; i++) {
                //System.out.println(hits.getHits()[i].getSourceAsString());
                result.add(hits.getHits()[i].getSourceAsString());
            }
            //記錄滾動ID
            scrollId = response.getScrollId();


            while(true){
                //后續的請求
                //scrollId = query.getScollId();
                SearchScrollRequestBuilder searchScrollRequestBuilder = client
                    .prepareSearchScroll(scrollId);            
                // 重新設定滾動時間            
                //TimeValue timeValue = new TimeValue(30000);
                searchScrollRequestBuilder.setScroll("2m");
                // 請求            
                SearchResponse response1 = searchScrollRequestBuilder.get();

                //TODO:處理數據
                SearchHits hits2 = response1.getHits();
                if(hits2.getHits().length == 0){
                    break;
                }
                for(int i = 0; i < hits2.getHits().length; i++) {
                    result.add(hits2.getHits()[i].getSourceAsString());
                }
                //下一批處理
                scrollId = response1.getScrollId();
            }

            System.out.println(result.size());
            long endTime = System.currentTimeMillis();
            System.out.println("Java程序運行時間:" + (endTime - startTime) + "ms");
        }catch(Exception e){
            e.printStackTrace();
        }

    }

Scroll原理查看Elasticsearch 使用scroll滾動技術實現大數據量搜索

轉載於:https://blog.csdn.net/zx711166/article/details/77847134


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM