ElasticSearch java API - 聚合查詢

    "mappings": {
        "player": {
            "properties": {
                "name": {
                    "index": "not_analyzed",
                    "type": "string"
                },
                "age": {
                    "type": "integer"
                },
                "salary": {
                    "type": "integer"
                },
                "team": {
                    "index": "not_analyzed",
                    "type": "string"
                },
                "position": {
                    "index": "not_analyzed",
                    "type": "string"
                }
            },
            "_all": {
                "enabled": false
            }
        }
    }

 

索引中的全部數據：

 

 

 首先，初始化Builder：

 

SearchRequestBuilder sbuilder = client.prepareSearch("player").setTypes("player");

 

接下來舉例說明各種聚合操作的實現方法，因為在es的api中，多字段上的聚合操作需要用到子聚合(subAggregation)，初學者可能找不到方法（網上資料比較少，筆者在這個問題上折騰了兩天，最后度了源碼才徹底搞清楚T_T），后邊會特意說明多字段聚合的實現方法。另外，聚合后的排序也會單獨說明。

• group by/count

例如要計算每個球隊的球員數，如果使用SQL語句，應表達如下：

select team, count(*) as player_count from player group by team;

ES的java api：

 

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");
sbuilder.addAggregation(teamAgg);
SearchResponse response = sbuilder.execute().actionGet();

 

 

• group by多個field

例如要計算每個球隊每個位置的球員數，如果使用SQL語句，應表達如下：

select team, position, count(*) as pos_count from player group by team, position;

 

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");
TermsBuilder posAgg= AggregationBuilders.terms("pos_count").field("position");
sbuilder.addAggregation(teamAgg.subAggregation(posAgg));
SearchResponse response = sbuilder.execute().actionGet();

 

• max/min/sum/avg

例如要計算每個球隊年齡最大/最小/總/平均的球員年齡，如果使用SQL語句，應表達如下：

select team, max(age) as max_age from player group by team;

ES的java api：

 

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");
MaxBuilder ageAgg= AggregationBuilders.max("max_age").field("age");
sbuilder.addAggregation(teamAgg.subAggregation(ageAgg));
SearchResponse response = sbuilder.execute().actionGet();

 

• 對多個field求max/min/sum/avg

例如要計算每個球隊球員的平均年齡，同時又要計算總年薪，如果使用SQL語句，應表達如下：

 

select team, avg(age)as avg_age, sum(salary) as total_salary from player group by team;

 

ES的java api：

 

TermsBuilder teamAgg= AggregationBuilders.terms("team");
AvgBuilder ageAgg= AggregationBuilders.avg("avg_age").field("age");
SumBuilder salaryAgg= AggregationBuilders.avg("total_salary ").field("salary");
sbuilder.addAggregation(teamAgg.subAggregation(ageAgg).subAggregation(salaryAgg));
SearchResponse response = sbuilder.execute().actionGet();

 

• 聚合后對Aggregation結果排序

例如要計算每個球隊總年薪，並按照總年薪倒序排列，如果使用SQL語句，應表達如下：

select team, sum(salary) as total_salary from player group by team order by total_salary desc;

ES的java api：

 

TermsBuilder teamAgg= AggregationBuilders.terms("team").order(Order.aggregation("total_salary ", false);
SumBuilder salaryAgg= AggregationBuilders.avg("total_salary ").field("salary");
sbuilder.addAggregation(teamAgg.subAggregation(salaryAgg));
SearchResponse response = sbuilder.execute().actionGet();

 

需要特別注意的是，排序是在TermAggregation處執行的，Order.aggregation函數的第一個參數是aggregation的名字，第二個參數是boolean型，true表示正序，false表示倒序。

 

• Aggregation結果條數的問題

默認情況下，search執行后，僅返回10條聚合結果，如果想反悔更多的結果，需要在構建TermsBuilder 時指定size：

TermsBuilder teamAgg= AggregationBuilders.terms("team").size(15);

• Aggregation結果的解析/輸出

得到response后：

 

Map<String, Aggregation> aggMap = response.getAggregations().asMap();
StringTerms teamAgg= (StringTerms) aggMap.get("keywordAgg");
Iterator<Bucket> teamBucketIt = teamAgg.getBuckets().iterator();
while (teamBucketIt .hasNext()) {
Bucket buck = teamBucketIt .next();
//球隊名
String team = buck.getKey();
//記錄數
long count = buck.getDocCount();
//得到所有子聚合
Map subaggmap = buck.getAggregations().asMap();
//avg值獲取方法
double avg_age= ((InternalAvg) subaggmap.get("avg_age")).getValue();
//sum值獲取方法
double total_salary = ((InternalSum) subaggmap.get("total_salary")).getValue();
//...
//max/min以此類推
}

 

• 總結

綜上，聚合操作主要是調用了SearchRequestBuilder的addAggregation方法，通常是傳入一個TermsBuilder，子聚合調用TermsBuilder的subAggregation方法，可以添加的子聚合有TermsBuilder、SumBuilder、AvgBuilder、MaxBuilder、MinBuilder等常見的聚合操作。
 
從實現上來講，SearchRequestBuilder在內部保持了一個私有的 SearchSourceBuilder實例， SearchSourceBuilder內部包含一個List<AbstractAggregationBuilder>，每次調用addAggregation時會調用 SearchSourceBuilder實例，添加一個AggregationBuilder。
同樣的，TermsBuilder也在內部保持了一個List<AbstractAggregationBuilder>，調用addAggregation方法（來自父類addAggregation）時會添加一個AggregationBuilder。有興趣的讀者也可以閱讀源碼的實現。