一、ElasticSearch概述
ElasticSearch是一個分布式,RESTFUL風格的搜索和數據分析引擎
二、Elasticsearch 安裝
下載地址:
https://www.elastic.co/cn/downloads/past-releases#elasticsearch
1、選擇windows7.8.0版本解壓到本地
2、解壓后,進入 bin 文件目錄,點擊 elasticsearch.bat 文件啟動 ES 服務
ES目錄結構詳解
注意:9300 端口為 Elasticsearch 集群間組件的通信端口,9200 端口為瀏覽器訪問的 http 協議 RESTful 端口。
3、打開瀏覽器(推薦使用谷歌瀏覽器),輸入地址:http://localhost:9200,測試結果
三、Elasticsearch 基本操作
1、索引操作
1)創建索引
PUT http://127.0.0.1:9200/shopping
重復創建索引會報錯
【注意】創建索引只能使用PUT請求
2)查看所有索引
GET http://127.0.0.1:9200/_cat/indices?v
3)查看單個索引
GET http://127.0.0.1:9200/shopping
4)刪除索引
2、文檔操作
1)創建文檔
POST請求和PUT請求都可以創建文檔
創建文檔時,POST可以攜帶ID也可以不攜帶ID,但是PUT請求必須要攜帶ID
創建文檔時,POST請求攜帶ID,如果ES中有相同的文檔,則為修改,但是version一直增加
創建文檔時,PUT請求攜帶ID,如果ES中有相同的文檔,則為修改,但是version不會增加
2)查看文檔(根據ID查看文檔)
3)修改文檔(全量覆蓋)
4)修改文檔字段 (局部字段更新)
注意requestbody格式
5)刪除文檔
6)根據條件刪除文檔
3、映射操作
1)創建映射
PUT http://127.0.0.1:9200/student/_mapping { "properties": { "name":{ "type": "text", "index": true }, "sex":{ "type": "text", "index": false }, "age":{ "type": "long", "index": false } } }
映射字段說明:
字段名:任意填寫,下面指定許多屬性,例如:title、subtitle、images、price
type:類型,Elasticsearch 中支持的數據類型非常豐富,說幾個關鍵的:
String 類型,又分兩種:
text:可分詞
keyword:不可分詞,數據會作為完整字段進行匹配
Numerical:數值類型,分兩類
基本數據類型:long、integer、short、byte、double、float、half_float
浮點數的高精度類型:scaled_float
Date:日期類型
Array:數組類型
Object:對象
index:是否索引,默認為 true,也就是說你不進行任何配置,所有字段都會被索引。
true:字段會被索引,則可以用來進行搜索
false:字段不會被索引,不能用來搜索
store:是否將數據進行獨立存儲,
默認為 false 原始的文本會存儲在_source 里面,默認情況下其他提取出來的字段都不是獨立存 儲 的,是從_source 里面提取出來的。當然你也可以獨立的存儲某個字段,只要設置 "store": true 即可,獲取獨立存儲的字段要比從_source 中解析快得多,但是也會占用 更多的空間,所以要根據實際業務需求來設置。
analyzer:分詞器,這里的 ik_max_word 即使用 ik 分詞器
2)查看映射
GET http://127.0.0.1:9200/student/_mapping
4、高級查詢
1)查詢所有文檔 match_all
http://127.0.0.1:9200/shopping/_search { "query":{ "match_all":{} } }
2)匹配查詢 match
http://127.0.0.1:9200/shopping/_search { "query":{ "match":{ "title":"手機" } } }
3)多字段匹配查詢
http://127.0.0.1:9200/shopping/_search { "query": { "multi_match": { "query": "手機", "fields": ["title","category"] } } }
4)關鍵字精確查詢 term
term 查詢,精確的關鍵詞匹配查詢,不對查詢條件進行分詞。
http://127.0.0.1:9200/shopping/_search { "query": { "term": { "title":{ "value":"小米手機" } } } }
5)多關鍵字精確查詢 term
terms 查詢和 term 查詢一樣,但它允許你指定多值進行匹配。 如果這個字段包含了指定值中的任何一個值,那么這個文檔滿足條件
http://127.0.0.1:9200/shopping/_search { "query": { "term": { "title":["小米","華為"] } } }
6)指定查詢字段 _source
http://127.0.0.1:9200/shopping/_search { "_source": ["name","nickname"], "query": { "terms": { "title": ["小米"] } } }
也可以通過
includes:來指定想要顯示的字段
excludes:來指定不想要顯示的字段
http://127.0.0.1:9200/shopping/_search { "_source": { "includes": ["name","nickname"] }, "query": { "terms": { "nickname": ["zhangsan"] } } }
7)組合查詢 bool
8)范圍查詢 range
http://127.0.0.1:9200/shopping/_search { "query": { "range": { "age": { "gte": 30, "lte": 35 } } } }
9)模糊查詢 fuzzy
返回包含與搜索字詞相似的字詞的文檔。
為了找到相似的術語,fuzzy 查詢會在指定的編輯距離內創建一組搜索詞的所有可能的變體 或擴展。然后查詢返回每個擴展的完全匹配
http://127.0.0.1:9200/shopping/_search { "query": { "fuzzy": { "title": { "value": "zhangsan" } } } }
10)字段排序 sort
http://127.0.0.1:9200/shopping/_search { "query":{ "match":{ "name":"zhangsan" } }, "sort":[ { "age":{ "order":"desc" } } ] }
11)高亮查詢 highlight
在進行關鍵字搜索時,搜索出的內容中的關鍵字會顯示不同的顏色,稱之為高亮
Elasticsearch 可以對查詢內容中的關鍵字部分,進行標簽和樣式(高亮)的設置。 在使用 match 查詢的同時,加上一個 highlight 屬性:
pre_tags:前置標簽
post_tags:后置標簽
fields:需要高亮的字段
12)分頁查詢
from:當前頁的起始索引,默認從 0 開始。 from = (pageNum - 1) * size
size:每頁顯示多少條
http://127.0.0.1:9200/shopping/_search { "query": { "match_all": {} }, "from": 0, "size": 2 }
13)聚合查詢
①、求最大值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "max_age":{ "max":{"field":"age"} } } }
②、求最小值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "min_age":{ "min":{"field":"age"} } } }
③、對字段求和
http://127.0.0.1:9200/shopping/_search { "aggs":{ "sum_age":{ "sum":{"field":"age"} } } }
④、求平均值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "avg_age":{ "avg":{"field":"age"} } } }
⑤、State 聚合
stats 聚合,對某個字段一次性返回 count,max,min,avg 和 sum 五個指標
http://127.0.0.1:9200/shopping/_search { "aggs":{ "stats_age":{ "stats":{"field":"price"} } } }
⑥、桶聚合查詢
桶聚和相當於 sql 中的 group by 語句
terms聚合,分組統計
http://127.0.0.1:9200/shopping/_search { "aggs":{ "price_agg":{ "terms":{"field":"price"} } } }
四、Java操作 ElasticSearch
1、引入pom文件
<dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>7.8.0</version> </dependency> <!-- elasticsearch 的客戶端 --> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.8.0</version> </dependency> <!-- elasticsearch 依賴 2.x 的 log4j --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.8.2</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.8.2</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.9.9</version> </dependency> <!-- junit 單元測試 --> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> </dependency> </dependencies>
2、創建es客戶端
public class EsTestClient { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.2", 9200, "http"))); System.out.println(client); //關閉Es客戶端 client.close(); } }
3、索引操作
1)創建索引
public class EsTestClient { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、創建索引 CreateIndexRequest createIndexRequest = new CreateIndexRequest("user"); CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequest, RequestOptions.DEFAULT); boolean acknowledged = createIndexResponse.isAcknowledged(); System.out.println("創建索引"+ (acknowledged ? "成功":"失敗")); //關閉Es客戶端 client.close(); } }
在postman中查詢
2)查看索引
//2、查看索引 GetIndexRequest getIndexResponse = new GetIndexRequest("user"); GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest, RequestOptions.DEFAULT);
3)刪除索引
//3、刪除索引 DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("user"); AcknowledgedResponse deleteResponse = client.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
4、文檔操作
1)插入文檔
public class EsTestDocInsert { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、插入文檔 IndexRequest indexRequest = new IndexRequest(); indexRequest.index("user").id("00001"); User user = new User(); user.setName("張三"); user.setAge(15); user.setSex("男"); ObjectMapper mapper = new ObjectMapper(); String userStr = mapper.writeValueAsString(user); indexRequest.source(userStr, XContentType.JSON); IndexResponse reponse = client.index(indexRequest, RequestOptions.DEFAULT); } }
2)修改文檔
public class EsTestDocUpdate { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、修改文檔 UpdateRequest updateRequest = new UpdateRequest(); updateRequest.index("user").id("00001"); updateRequest.doc(XContentType.JSON,"sex","女的"); UpdateResponse updateResponse = client.update(updateRequest, RequestOptions.DEFAULT); System.out.println(updateResponse.getResult()); } }
3)查詢文檔(根據ID進行查詢)
public class EsTestDocGet { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //根據ID查詢文檔 GetRequest getRequest = new GetRequest(); getRequest.index("user").id("00001"); GetResponse res = client.get(getRequest, RequestOptions.DEFAULT); System.out.println(res.getSourceAsString()); } }
4)刪除文檔
public class EsTestDocDelete { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //根據ID查詢文檔 DeleteRequest deleteRequest = new DeleteRequest(); deleteRequest.index("user").id("00001"); DeleteResponse res = client.delete(deleteRequest, RequestOptions.DEFAULT); System.out.println(res.getResult()); } }
5)批量新增
public class EsTestDocInsertBatch { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、批量插入文檔 BulkRequest bulkRequest = new BulkRequest(); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","張三")); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","李四")); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","王五")); BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT); System.out.println(response.getItems()); client.close(); } }
5、高級插敘
1)match_all 查詢
public class EsTestDocQuery { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); SearchHit[] hits1 = hits.getHits(); for (SearchHit documentFields : hits1) { System.out.println(documentFields); } client.close(); } }
2)分頁查詢
public class EsTestDocPageQuery { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); // 分頁查詢 // 當前頁其實索引(第一條數據的順序號),from sourceBuilder.from(0); // 每頁顯示多少條 size sourceBuilder.size(2); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //輸出每條查詢的結果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
3)數據排序
public class EsTestDocSortQuery { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); // 排序 sourceBuilder.sort("age", SortOrder.ASC); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); SearchHit[] hits1 = hits.getHits(); for (SearchHit documentFields : hits1) { System.out.println(documentFields); } client.close(); } }
4)過濾字段
public class EsTestDocFilterQuery { public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 創建搜索請求對象 SearchRequest request = new SearchRequest(); request.indices("student"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); //查詢字段過濾 String[] excludes = {}; String[] includes = {"name", "age"}; sourceBuilder.fetchSource(includes, excludes); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //輸出每條查詢的結果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
5)bool查詢
public class EsTestDocFilterQuery { public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 創建搜索請求對象 SearchRequest request = new SearchRequest(); request.indices("student"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); // 必須包含 boolQueryBuilder.must(QueryBuilders.matchQuery("age", "30")); // 一定不含 boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name", "zhangsan")); // 可能包含 boolQueryBuilder.should(QueryBuilders.matchQuery("sex", "男")); sourceBuilder.query(boolQueryBuilder); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //輸出每條查詢的結果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
6)范圍查詢
public class EsTestDocRangeQuery { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 創建搜索請求對象 SearchRequest request = new SearchRequest(); request.indices("student"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age"); // 大於等於 rangeQuery.gte("30"); // 小於等於 rangeQuery.lte("40"); sourceBuilder.query(rangeQuery); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //輸出每條查詢的結果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
7)模糊查詢
// 創建搜索請求對象 SearchRequest request = new SearchRequest(); request.indices("student"); // 構建查詢的請求體 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.fuzzyQuery("name","zhangsan").fuzziness(Fu zziness.ONE)); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查詢匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //輸出每條查詢的結果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========");
8)高亮查詢
// 高亮查詢 SearchRequest request = new SearchRequest().indices("student"); //2.創建查詢請求體構建器 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); //構建查詢方式:高亮查詢 TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("name","zhangsan"); //設置查詢方式 sourceBuilder.query(termsQueryBuilder) //構建高亮字段 HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.preTags("<font color='red'>");//設置標簽前綴 highlightBuilder.postTags("</font>");//設置標簽后綴 highlightBuilder.field("name");//設置高亮字段 //設置高亮構建對象 sourceBuilder.highlighter(highlightBuilder); //設置請求體 request.source(sourceBuilder); //3.客戶端發送請求,獲取響應對象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印響應結果 SearchHits hits = response.getHits(); System.out.println("took::"+response.getTook()); System.out.println("time_out::"+response.isTimedOut()); System.out.println("total::"+hits.getTotalHits()); System.out.println("max_score::"+hits.getMaxScore()); System.out.println("hits::::>>"); for (SearchHit hit : hits) { String sourceAsString = hit.getSourceAsString(); System.out.println(sourceAsString); //打印高亮結果 Map<String, HighlightField> highlightFields = hit.getHighlightFields(); System.out.println(highlightFields); } System.out.println("<<::::");
9)聚合查詢
① 最大值
public class EsTestDocMaxAggregation { public static void main(String[] args) throws IOException { //創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest request = new SearchRequest(); request.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.max("maxAge").field("age")); //設置請求體 request.source(sourceBuilder); //3.客戶端發送請求,獲取響應對象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印響應結果 SearchHits hits = response.getHits(); System.out.println(response); client.close(); } }
② 分組聚合
public class EsTestDocMaxAggregation { public static void main(String[] args) throws IOException { //1創建Es客戶端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest request = new SearchRequest(); request.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.terms("age_groupby").field("age")); //設置請求體 request.source(sourceBuilder); //3.客戶端發送請求,獲取響應對象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印響應結果 SearchHits hits = response.getHits(); System.out.println(response); client.close(); } }