以球員信息為例,player索引的player type包含5個字段,姓名,年齡,薪水,球隊,場上位置。
index的mapping為:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
"mappings"
: {
"player"
: {
"properties"
: {
"name"
: {
"index"
:
"not_analyzed"
,
"type"
:
"string"
},
"age"
: {
"type"
:
"integer"
},
"salary"
: {
"type"
:
"integer"
},
"team"
: {
"index"
:
"not_analyzed"
,
"type"
:
"string"
},
"position"
: {
"index"
:
"not_analyzed"
,
"type"
:
"string"
}
},
"_all"
: {
"enabled"
:
false
}
}
}
|
索引中的全部數據:
首先,初始化Builder:
1
|
SearchRequestBuilder sbuilder = client.prepareSearch(
"player"
).setTypes(
"player"
);
|
接下來舉例說明各種聚合操作的實現方法,因為在es的api中,多字段上的聚合操作需要用到子聚合(subAggregation),初學者可能找不到方法(網上資料比較少,筆者在這個問題上折騰了兩天,最后度了源碼才徹底搞清楚T_T),后邊會特意說明多字段聚合的實現方法。另外,聚合后的排序也會單獨說明。
- group by/count
例如要計算每個球隊的球員數,如果使用SQL語句,應表達如下:
select team, count(*) as player_count from player group by team;
ES的java api:
1
2
3
|
TermsBuilder teamAgg= AggregationBuilders.terms(
"player_count "
).field(
"team"
);
sbuilder.addAggregation(teamAgg);
SearchResponse response = sbuilder.execute().actionGet();
|
- group by多個field
例如要計算每個球隊每個位置的球員數,如果使用SQL語句,應表達如下:
select team, position, count(*) as pos_count from player group by team, position;
ES的java api:
1
2
3
4
|
TermsBuilder teamAgg= AggregationBuilders.terms(
"player_count "
).field(
"team"
);
TermsBuilder posAgg= AggregationBuilders.terms(
"pos_count"
).field(
"position"
);
sbuilder.addAggregation(teamAgg.subAggregation(posAgg));
SearchResponse response = sbuilder.execute().actionGet();
|
- max/min/sum/avg
例如要計算每個球隊年齡最大/最小/總/平均的球員年齡,如果使用SQL語句,應表達如下:
select team, max(age) as max_age from player group by team;
ES的java api:
1
2
3
4
|
TermsBuilder teamAgg= AggregationBuilders.terms(
"player_count "
).field(
"team"
);
MaxBuilder ageAgg= AggregationBuilders.max(
"max_age"
).field(
"age"
);
sbuilder.addAggregation(teamAgg.subAggregation(ageAgg));
SearchResponse response = sbuilder.execute().actionGet();
|
- 對多個field求max/min/sum/avg
例如要計算每個球隊球員的平均年齡,同時又要計算總年薪,如果使用SQL語句,應表達如下:
select team, avg(age)as avg_age, sum(salary) as total_salary from player group by team;
ES的java api:
1
2
3
4
5
6
|
TermsBuilder teamAgg= AggregationBuilders.terms(
"team"
);
AvgBuilder ageAgg= AggregationBuilders.avg(
"avg_age"
).field(
"age"
);
SumBuilder salaryAgg= AggregationBuilders.avg(
"total_salary "
).field(
"salary"
);
sbuilder.addAggregation(teamAgg.subAggregation(ageAgg).subAggregation(salaryAgg));
SearchResponse response = sbuilder.execute().actionGet();
|
- 聚合后對Aggregation結果排序
例如要計算每個球隊總年薪,並按照總年薪倒序排列,如果使用SQL語句,應表達如下:
select team, sum(salary) as total_salary from player group by team order by total_salary desc;
ES的java api:
1
2
3
4
|
TermsBuilder teamAgg= AggregationBuilders.terms(
"team"
).order(Order.aggregation(
"total_salary "
,
false
);
SumBuilder salaryAgg= AggregationBuilders.avg(
"total_salary "
).field(
"salary"
);
sbuilder.addAggregation(teamAgg.subAggregation(salaryAgg));
SearchResponse response = sbuilder.execute().actionGet();
|
需要特別注意的是,排序是在TermAggregation處執行的,Order.aggregation函數的第一個參數是aggregation的名字,第二個參數是boolean型,true表示正序,false表示倒序。
- Aggregation結果條數的問題
默認情況下,search執行后,僅返回10條聚合結果,如果想反悔更多的結果,需要在構建TermsBuilder 時指定size:
TermsBuilder teamAgg= AggregationBuilders.terms("team").size(15);
- Aggregation結果的解析/輸出
得到response后:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
Map<String, Aggregation> aggMap = response.getAggregations().asMap();
StringTerms teamAgg= (StringTerms) aggMap.get(
"keywordAgg"
);
Iterator<Bucket> teamBucketIt = teamAgg.getBuckets().iterator();
while
(teamBucketIt .hasNext()) {
Bucket buck = teamBucketIt .next();
//球隊名
String team = buck.getKey();
//記錄數
long
count = buck.getDocCount();
//得到所有子聚合
Map subaggmap = buck.getAggregations().asMap();
//avg值獲取方法
double
avg_age= ((InternalAvg) subaggmap.get(
"avg_age"
)).getValue();
//sum值獲取方法
double
total_salary = ((InternalSum) subaggmap.get(
"total_salary"
)).getValue();
//...
//max/min以此類推
}
|
- 總結
綜上,聚合操作主要是調用了SearchRequestBuilder的addAggregation方法,通常是傳入一個TermsBuilder,子聚合調用TermsBuilder的subAggregation方法,可以添加的子聚合有TermsBuilder、SumBuilder、AvgBuilder、MaxBuilder、MinBuilder等常見的聚合操作。
從實現上來講,SearchRequestBuilder在內部保持了一個私有的 SearchSourceBuilder實例, SearchSourceBuilder內部包含一個List<AbstractAggregationBuilder>,每次調用addAggregation時會調用 SearchSourceBuilder實例,添加一個AggregationBuilder。
同樣的,TermsBuilder也在內部保持了一個List<AbstractAggregationBuilder>,調用addAggregation方法(來自父類addAggregation)時會添加一個AggregationBuilder。有興趣的讀者也可以閱讀源碼的實現。