Elasticsearch7.8.0教程（一）
一. 前序
Elasticsearch是一個基於
Apache Lucene(TM)的開源搜索引擎。無論在開源還是專有領域，Lucene 可以被認為是迄今為止最先進、性能最好的、功能最全的搜索引擎庫。但是，Lucene只是一個庫。想要 使用它，你必須使用Java來作為開發語言並將其直接集成到你的 應用中，更糟糕的是，Lucene非常復雜，你需要深入了解檢索的相關知識來理解它是如何工作的。
Elasticsearch也使用Java開發並使用Lucene作為其核心來實現所有索引和搜索的功能，但是它的目 的是通過簡單的
RESTful API來隱藏Lucene的復雜性，從而讓全文搜索變得簡單。Elasticsearch的中文網址：https://www.elastic.co/cn/products/elasticsearch
1.1 正向索引和倒排索引
正向索引與倒排索引，這是在搜索領域中非常重要的兩個名詞，正向索引通常用於數據庫中，在搜 索引擎領域使用的最多的就是倒排索引，我們根據如下兩個網頁來對這兩個概念進行闡述：
html1
我愛我的祖國，我愛編程
html2
我愛編程，我是個快樂的小碼農
1.1.1 正向索引
假設我們使用mysql的全文檢索，會對如上兩句話分別進行分詞處理，那么預計得到的結果如下：
我 愛 愛我 祖國 我的祖國 編程 愛編程 我愛編程
我 我愛 愛 編程 愛編程 我愛編程 快樂 碼農 小碼農
假設我們現在使用正向索引搜索 編程 這個詞，那么會到第一句話中去查找是否包含有
編程這 個關鍵詞，如果有則加入到結果集中；第二句話也是如此。假設現在有成千上百個網頁，每個網頁 非常非常多的分詞，那么搜索的效率將會非常非常低些。
1.1.2 倒排索引
倒排索引是按照分詞與文檔進行映射，我們來看看如果按照倒排索引的效果：
| 關鍵詞 | 文檔名 |
|---|---|
| 我 | html1,html2,html3 |
| 愛 | html1,html2 |
| 愛我 | html1 |
| 我愛 | html2 |
| 祖國 | html1 |
| 我的祖國 | html1 |
| 編程 | html1,html2 |
| 我愛編程 | html1,html2 |
| 愛編程 | html1,html2 |
| 快樂 | html2 |
| 碼農 | html2 |
| 小碼農 | html2 |
如果采用倒排索引的方式搜索
編程這個詞，那么會直接找到關鍵詞中查找到編程，然后查找 到對應的文檔，這就是所謂的倒排索引。
二. 軟件簡介以及啟動
2.1 相關軟件下載地址(ELK)
| 軟件名 | 下載地址 |
|---|---|
| Elasticsearch | https://www.elastic.co/cn/start |
| Logstash | https://www.elastic.co/cn/downloads/logstash |
| Kibana | https://www.elastic.co/cn/start |
2.2 Elasticsearch安裝
進入到
elasticsearch解壓目錄下的 bin 目錄下，雙擊elasticsearch.bat即可啟動。 在瀏覽器地址欄輸入: http://localhost:9200/ ，如果出現如下頁面表示 elasticsearch 啟動成功
2.3 Kibana
2.3.1 Kibana簡介
Kibana是世界上最受歡迎的開源日志分析平台ELK Stack中的“K” ，它為用戶提供了一個工具，用於 在存儲於Elasticsearch集群中的日志數據進行檢索，可視化和構建儀表板。
Kibana的核心功能是數據查詢和分析。使用各種方法，用戶可以搜索Elasticsearch中索引的數據， 以查找其數據中的特定事件或字符串，以進行根本原因分析和診斷。基於這些查詢，用戶可以使用 Kibana的可視化功能，允許用戶使用圖表，表格，地理圖和其他類型的可視化以各種不同的方式可 視化數據。
2.3.2 Kibana的啟動
進入到 kibana 解壓目錄下的 bin 目錄下，雙擊 kibana.bat 即可啟動 kibana. 在瀏覽器地址欄輸入：http://localhost:5601，出現如下頁面代表 kibana 啟動成功。
2.4 Logstash
2.4.1 logstash簡介
Logstash是一個開源的服務器端數據處理管道，可以同時從多個數據源獲取數據，並對其進行轉 換，然后將其發送到你最喜歡的“存儲”。創建於2009年，於2013年被elasticsearch收購。
2.4.2 logstash導入數據
雖然
kibana提供了一些數據集供我們使用，為了加深對logstash的理解，我們movielens的電影數據集。movielens 數據集的下載地址為：http://files.grouplens.org/datasets/movielens，進入該網頁只 用下載 ml-latest.zip 數據即可，如下圖所示：
將
ml-latest.zip加壓文件中的movies.csv文件拷貝到logstash的家目錄下； 再將logstash的config目錄下新建名為logstash.conf的文件，文件內容如下:
input {
file {
# 引號的的內容為 movies.csv 的實際路徑，根據實際情況而定
path => "D:/environment/logstash-7.8.0/movies.csv"
start_position => "beginning"
sincedb_path => "D:/environment/logstash-7.8.0/db_path.log"
}
}
filter {
csv {
separator => ","
columns => ["id","content","genre"]
}
mutate {
split => { "genre" => "|" }
remove_field => ["path", "host","@timestamp","message"]
}
mutate {
split => ["content", "("]
add_field => { "title" => "%{[content][0]}"}
add_field => { "year" => "%{[content][1]}"}
}
mutate {
convert => {
"year" => "integer"
}
strip => ["title"]
remove_field => ["path", "host","@timestamp","message","content"]
}
}
output {
elasticsearch {
# 雙引號中的內容為ES的地址，視實際情況而定
hosts => "http://localhost:9200"
index => "movies"
document_id => "%{id}"
}
stdout {}
}
movies.csv數據
打開dos命令行，進入到
logstash的bin目錄下，執行如下命令導入movielens的數據集。備注：如果要重新導入，需先刪除
db_path.log文件
logstash.bat -f D:\environment\logstash-7.8.0\logstash.conf
2.4.3 驗證
進入到 kibana 的命令行頁面，執行
GET _cat/indices驗證數據是否成功導入
三. Elasticsearch的基本概念
3.1 索引
Elasticsearch中的索引有多層的意思：
a. 某一類文檔的集合就構成了一個索引，類比到數據庫就是一個數據庫(或者數據庫表);
b.它還描述了一個動作，就是將某個文檔保存在elasticsearch的過程也叫索引;
c. 倒排索引。
3.2 文檔
具體的一條數據，類比到數據庫就是一條記錄。
GET movies/_search
3.4 mapping
mapping是ES每一個文檔的約束信息，例如屬性的類型，是否能被索引等。
3.5 DSL
DSL 是 ES 的查詢語言。
3.6 類比
我們通過大家比較熟悉的
DBMS與ES的基本概念進行類比，加深大家的理解。
| DBMS | Elasticsearch |
|---|---|
| database | Index |
| table | type(在7.0之后type為固定值_doc) |
| Row | Document |
| Column | Field |
| Schema | Mapping |
| SQL | DSL(Descriptor Structure Language) |
在7.0之前，一個
Index可以創建多個類型，從7.0開始，一個索引只能創建一個類型，也就是_doc
四. RestAPI
4.1 基本CRUD
4.1.1 查詢movies的數據
GET movies/_search
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32225",
"_score" : 1.0,
"_source" : {
"title" : "Project Grizzly",
"genre" : [
"Documentary"
],
"year" : 1996,
"id" : "32225",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32230",
"_score" : 1.0,
"_source" : {
"title" : "Snow Queen, The",
"genre" : [
"Children",
"Fantasy"
],
"year" : 0,
"id" : "32230",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32234",
"_score" : 1.0,
"_source" : {
"title" : "Julia",
"genre" : [
"Drama"
],
"year" : 1977,
"id" : "32234",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32239",
"_score" : 1.0,
"_source" : {
"title" : "Save the Green Planet!",
"genre" : [
"Comedy",
"Drama",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 0,
"id" : "32239",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32243",
"_score" : 1.0,
"_source" : {
"title" : "Stealing Rembrandt",
"genre" : [
"Action",
"Comedy",
"Crime"
],
"year" : 0,
"id" : "32243",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32250",
"_score" : 1.0,
"_source" : {
"title" : "Snake of June, A",
"genre" : [
"Drama",
"Mystery"
],
"year" : 0,
"id" : "32250",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32261",
"_score" : 1.0,
"_source" : {
"title" : "Thirty Seconds Over Tokyo",
"genre" : [
"Drama",
"War"
],
"year" : 1944,
"id" : "32261",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32263",
"_score" : 1.0,
"_source" : {
"title" : "Vares: Private Eye",
"genre" : [
"Action",
"Comedy",
"Crime",
"Thriller"
],
"year" : 0,
"id" : "32263",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32277",
"_score" : 1.0,
"_source" : {
"title" : "Girl Crazy",
"genre" : [
"Comedy",
"Musical",
"Romance"
],
"year" : 1943,
"id" : "32277",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32280",
"_score" : 1.0,
"_source" : {
"title" : "The 3 Penny Opera",
"genre" : [
"Comedy",
"Drama",
"Musical"
],
"year" : 1931,
"id" : "32280",
"@version" : "1"
}
}
]
}
}
4.1.2 查詢movies的總數
GET movies/_count
{
"count" : 58099,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
4.1.3 查看所有的索引
GET _cat/indices
green open .kibana-event-log-7.8.0-000001 IPfj1sfGQWCv3wiBM5nfXA 1 0 2 0 10.4kb 10.4kb
yellow open movies BdLadipoRa2MfeKeFkkjiw 1 1 58099 0 7mb 7mb
green open .kibana-event-log-7.8.0-000002 mtvXKAziSWGWwtxTMHCnmw 1 0 1 0 5.3kb 5.3kb
yellow open product 9Hovl5eKR0Gg1qET5rlDcw 3 1 0 0 624b 624b
green open .apm-custom-link j-margrYSpylvIEI8pVa-w 1 0 0 0 208b 208b
green open .kibana_task_manager_1 aDRY7wE3TJqArR7VSzyHGQ 1 0 5 8 73.7kb 73.7kb
green open kibana_sample_data_ecommerce j84eHNbtRkGeA8UDcGurkg 1 0 4675 0 4.5mb 4.5mb
green open .apm-agent-configuration DKoxjNaMSFa78gU_t2c_9g 1 0 0 0 208b 208b
green open kibana_sample_data_logs AYGZxvG5R7K7R6L4g0KT1w 1 0 14074 0 11.3mb 11.3mb
green open .kibana_1 r7H-MJHtSbWB-nyYTio9-w 1 0 232 0 1.3mb 1.3mb
green open kibana_sample_data_flights aZEbUyVwTUeinyzpRPZ4Ng 1 0 13059 0 6.2mb 6.2mb
yellow open users 1RU70EgyTKinRJbcG-QigQ 1 1 2 0 8kb 8kb
yellow open shopping TXu8AmJCS-2nj1oHdljATw 1 1 6 0 16.1kb 16.1kb
4.1.4 查詢id為24的數據
GET movies/_doc/24
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "24",
"_version" : 1,
"_seq_no" : 249,
"_primary_term" : 1,
"found" : true,
"_source" : {
"title" : "Powder",
"genre" : [
"Drama",
"Sci-Fi"
],
"year" : 1995,
"id" : "24",
"@version" : "1"
}
}
4.1.5 加一條數據到索引users
POST users/_doc
{
"firstname": "will",
"lastname": "smith"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
}
]
}
}
4.1.6 添加指定id為1的文檔
POST users/_doc/1
{
"firstname": "jack",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 3,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1
}
GET users/_search
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.7 添加指定相同id為1的文檔，會把之前的數據覆蓋(比較危險，盡量少用)
POST users/_doc/1
{
"firstname": "rod",
"lastname": "johnson"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 4,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1
}
GET users/_search
{
"took" : 1006,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
}
]
}
}
4.1.8創建id為1的文檔，如果索引中已存在相同id，會報錯(可以替代4.1.7命令)
POST users/_create/1
{
"firstname": "jack",
"lastname": "ma"
}
{
"error" : {
"root_cause" : [
{
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [4])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
}
],
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [4])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
},
"status" : 409
}
POST users/_create/2
{
"firstname": "jack",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 5,
"_primary_term" : 1
}
GET users/_search
{
"took" : 215,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.9 添加一個屬性age(會覆蓋原有屬性)
POST users/_doc/2
{
"age":53
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 6,
"_primary_term" : 1
}
GET users/_search
{
"took" : 994,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53
}
}
]
}
}
4.1.10 添加一個屬性age(不會覆蓋原有屬性)
POST users/_update/2
{
"doc": {
"firstname": "jack",
"lastname": "ma"
}
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 7,
"_primary_term" : 1
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "rod",
"lastname" : "johnson"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
}
]
}
}
4.1.11 PUT添加一條數據(用法與POST相同)
PUT users/_doc/1
{
"firstname": "pony",
"lastname": "ma"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 5,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 8,
"_primary_term" : 1
}
GET users/_search
{
"took" : 771,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
}
]
}
}
PUT users/_create/2
{
"firstname": "harry",
"lastname": "potter"
}
{
"error" : {
"root_cause" : [
{
"type" : "version_conflict_engine_exception",
"reason" : "[2]: version conflict, document already exists (current version [3])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
}
],
"type" : "version_conflict_engine_exception",
"reason" : "[2]: version conflict, document already exists (current version [3])",
"index_uuid" : "rskHwG5VSQS3mZVmba6sKg",
"shard" : "0",
"index" : "users"
},
"status" : 409
}
PUT users/_create/3
{
"firstname": "harry",
"lastname": "potter"
}
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 9,
"_primary_term" : 1
}
4.1.12 刪除指定id為3的數據
GET users/_search
{
"took" : 308,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"firstname" : "harry",
"lastname" : "potter"
}
}
]
}
}
DELETE users/_doc/3
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 2,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 10,
"_primary_term" : 1
}
GET users/_search
{
"took" : 530,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "bn-oSnwBXvXiB6zYgyQf",
"_score" : 1.0,
"_source" : {
"firstname" : "will",
"lastname" : "smith"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"age" : 53,
"firstname" : "jack",
"lastname" : "ma"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "pony",
"lastname" : "ma"
}
}
]
}
}
4.1.13 刪除整個users索引
DELETE users
{
"acknowledged" : true
}
GET _cat/indices
green open .kibana-event-log-7.8.0-000001 IPfj1sfGQWCv3wiBM5nfXA 1 0 2 0 10.4kb 10.4kb
yellow open movies BdLadipoRa2MfeKeFkkjiw 1 1 58099 0 7mb 7mb
green open .kibana-event-log-7.8.0-000002 mtvXKAziSWGWwtxTMHCnmw 1 0 1 0 5.3kb 5.3kb
yellow open product 9Hovl5eKR0Gg1qET5rlDcw 3 1 0 0 624b 624b
green open .apm-custom-link j-margrYSpylvIEI8pVa-w 1 0 0 0 208b 208b
green open kibana_sample_data_ecommerce j84eHNbtRkGeA8UDcGurkg 1 0 4675 0 4.5mb 4.5mb
green open .kibana_task_manager_1 aDRY7wE3TJqArR7VSzyHGQ 1 0 5 8 73.7kb 73.7kb
green open .apm-agent-configuration DKoxjNaMSFa78gU_t2c_9g 1 0 0 0 208b 208b
green open kibana_sample_data_logs AYGZxvG5R7K7R6L4g0KT1w 1 0 14074 0 11.3mb 11.3mb
green open .kibana_1 r7H-MJHtSbWB-nyYTio9-w 1 0 254 20 1.3mb 1.3mb
yellow open user -U8L42soTjiCygg8Cr3M9g 1 1 1 0 4.2kb 4.2kb
green open kibana_sample_data_flights aZEbUyVwTUeinyzpRPZ4Ng 1 0 13059 0 6.2mb 6.2mb
yellow open shopping TXu8AmJCS-2nj1oHdljATw 1 1 6 0 16.1kb 16.1kb
4.1.14 批量添加數據
POST users/_bulk
{"index":{"_id":1}}
{"firstname": "a", "lastname": "A"}
{"index": {"_id": 2}}
{"firstname": "x", "lastname": "X"}
{"index": {}}
{"firstname": "y", "lastname": "Y"}
{"index": {"_id": 3}}
{"firstname": "z", "lastname": "Z"}
{
"took" : 560,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "cX_dSnwBXvXiB6zYeylY",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1,
"status" : 201
}
}
]
}
GET users/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "cX_dSnwBXvXiB6zYeylY",
"_score" : 1.0,
"_source" : {
"firstname" : "y",
"lastname" : "Y"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"firstname" : "z",
"lastname" : "Z"
}
}
]
}
}
4.1.15 批量查詢多個指定的id的數據，也可以批量查詢
GET _mget
{
"docs": [
{"_index": "users", "_id": 1},
{"_index": "users", "_id": 2},
{"_index": "users", "_id": 3}
]
}
{
"docs" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"_seq_no" : 3,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "z",
"lastname" : "Z"
}
}
]
}
GET _mget
{
"docs": [
{"_index": "users", "_id": 1},
{"_index": "users", "_id": 2},
{"_index": "movies", "_id": 38701}
]
}
{
"docs" : [
{
"_index" : "users",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "a",
"lastname" : "A"
}
},
{
"_index" : "users",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"firstname" : "x",
"lastname" : "X"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "38701",
"_version" : 1,
"_seq_no" : 10500,
"_primary_term" : 1,
"found" : true,
"_source" : {
"title" : "Don't Come Knocking",
"genre" : [
"Drama",
"Western"
],
"year" : 2005,
"id" : "38701",
"@version" : "1"
}
}
]
}
4.2 URI查詢
4.2.1 查詢所有的屬性中只要包含2012的所有的數據，泛查詢(沒有指定特定字段的查找)
GET movies/_search?q=2012
{
"took" : 29,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1867,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : 11.507438,
"_source" : {
"title" : "2012",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
],
"year" : 2009,
"id" : "72378",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "2012",
"_score" : 10.564456,
"_source" : {
"title" : "Back to the Future Part III",
"genre" : [
"Adventure",
"Comedy",
"Sci-Fi",
"Western"
],
"year" : 1990,
"id" : "2012",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "80505",
"_score" : 9.692035,
"_source" : {
"title" : "2012: Supernova",
"genre" : [
"Action",
"Sci-Fi"
],
"year" : 2009,
"id" : "80505",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "102848",
"_score" : 9.692035,
"_source" : {
"title" : "Armageddon 2012",
"genre" : [
"Sci-Fi"
],
"year" : 2012,
"id" : "102848",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "84017",
"_score" : 7.367466,
"_source" : {
"title" : "2012: Time for Change",
"genre" : [
"Animation",
"Documentary"
],
"year" : 2010,
"id" : "84017",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "189451",
"_score" : 7.367466,
"_source" : {
"title" : "Macross: Flash Back 2012",
"genre" : [
"Animation",
"Sci-Fi"
],
"year" : 1987,
"id" : "189451",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "144456",
"_score" : 6.578554,
"_source" : {
"title" : "All's Well, Ends Well 2012",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2012,
"id" : "144456",
"@version" : "1"
}
}
]
}
}
4.2.2 查詢title中包含2012的所有的電影，df(default field)
GET movies/_search?q=2012&df=title
GET movies/_search?q=title:2012
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 13,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : 11.507438,
"_source" : {
"title" : "2012",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
],
"year" : 2009,
"id" : "72378",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "80505",
"_score" : 9.692035,
"_source" : {
"title" : "2012: Supernova",
"genre" : [
"Action",
"Sci-Fi"
],
"year" : 2009,
"id" : "80505",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "102848",
"_score" : 9.692035,
"_source" : {
"title" : "Armageddon 2012",
"genre" : [
"Sci-Fi"
],
"year" : 2012,
"id" : "102848",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "84017",
"_score" : 7.367466,
"_source" : {
"title" : "2012: Time for Change",
"genre" : [
"Animation",
"Documentary"
],
"year" : 2010,
"id" : "84017",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "189451",
"_score" : 7.367466,
"_source" : {
"title" : "Macross: Flash Back 2012",
"genre" : [
"Animation",
"Sci-Fi"
],
"year" : 1987,
"id" : "189451",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "144456",
"_score" : 6.578554,
"_source" : {
"title" : "All's Well, Ends Well 2012",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2012,
"id" : "144456",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "160329",
"_score" : 6.578554,
"_source" : {
"title" : "I Love Hong Kong 2012",
"genre" : [
"Comedy"
],
"year" : 2012,
"id" : "160329",
"@version" : "1"
}
}
]
}
}
4.2.3 查詢title中包含Beautiful或者Mind的所有的數據
GET movies/_search?q=title:Beautiful Mind
GET movies/_search?q=title:(Beautiful Mind)
GET movies/_search?q=title:(+Beautiful +Mind)
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 145,
"relation" : "eq"
},
"max_score" : 13.474831,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474831,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "74064",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2009,
"id" : "74064",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "113760",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "113760",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "3912",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2000,
"id" : "3912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "47404",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Game",
"genre" : [
"Adventure",
"Animation",
"Comedy",
"Fantasy",
"Romance",
"Sci-Fi"
],
"year" : 2004,
"id" : "47404",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "124129",
"_score" : 8.210646,
"_source" : {
"title" : "Menno's Mind",
"genre" : [
"Sci-Fi",
"Thriller"
],
"year" : 1997,
"id" : "124129",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "150048",
"_score" : 8.210646,
"_source" : {
"title" : "Dirty Mind",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2009,
"id" : "150048",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "153272",
"_score" : 8.210646,
"_source" : {
"title" : "Mirrored Mind",
"genre" : [
"(no genres listed)"
],
"year" : 2006,
"id" : "153272",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "175563",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Blown",
"genre" : [
"Action",
"Sci-Fi",
"Thriller"
],
"year" : 2016,
"id" : "175563",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "188239",
"_score" : 8.210646,
"_source" : {
"title" : "Mind Ripper",
"genre" : [
"Horror",
"Sci-Fi"
],
"year" : 1995,
"id" : "188239",
"@version" : "1"
}
}
]
}
}
4.2.4 查詢title中包含Beautiful但是不包含mind的所 有的數據
GET movies/_search?q=title:(Beautiful NOT Mind)
GET movies/_search?q=title:(Beautiful -Mind)
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 97,
"relation" : "eq"
},
"max_score" : 8.774164,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "74064",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2009,
"id" : "74064",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "113760",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "113760",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "3912",
"_score" : 8.774164,
"_source" : {
"title" : "Beautiful",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2000,
"id" : "3912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "66701",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Ohio",
"genre" : [
"Comedy",
"Drama"
],
"year" : 2006,
"id" : "66701",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "89449",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Lies",
"genre" : [
"Comedy",
"Drama",
"Romance"
],
"year" : 0,
"id" : "89449",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "90353",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Boy",
"genre" : [
"Drama"
],
"year" : 2010,
"id" : "90353",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "110123",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Darling",
"genre" : [
"Documentary"
],
"year" : 2010,
"id" : "110123",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "114126",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Losers",
"genre" : [
"Documentary"
],
"year" : 2008,
"id" : "114126",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "117899",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Girl",
"genre" : [
"Drama"
],
"year" : 2014,
"id" : "117899",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "27232",
"_score" : 7.3899603,
"_source" : {
"title" : "Beautiful Joe",
"genre" : [
"Comedy",
"Drama",
"Romance"
],
"year" : 2000,
"id" : "27232",
"@version" : "1"
}
}
]
}
}
4.2.5 查詢title中既包含Mind又包含Beautiful的所有 的數據，與順序沒有關系
GET movies/_search?q=title:(mind AND beautiful)
GET movies/_search?q=title:(beautiful AND mind)
#如果and變小寫這表示搜索包含mind或and或beautiful的數據
#GET movies/_search?q=title:(mind and beautiful)
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 13.474831,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474831,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
}
]
}
}
4.2.6 查詢title中包含 "Beautiful Mind"這個短語的所 有的數據
GET movies/_search?q=title:"beautiful mind"
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 13.474829,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "4995",
"_score" : 13.474829,
"_source" : {
"title" : "Beautiful Mind, A",
"genre" : [
"Drama",
"Romance"
],
"year" : 2001,
"id" : "4995",
"@version" : "1"
}
}
]
}
}
GET movies/_search?q=title:"beautiful AND mind"
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
4.2.7 查詢title中包含2012，每頁3條從第二頁開始，查詢3條 數據
GET movies/_search?q=title:2012&from=3&size=3
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 13,
"relation" : "eq"
},
"max_score" : 11.507438,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "158731",
"_score" : 9.692035,
"_source" : {
"title" : "Kony 2012",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "158731",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "173613",
"_score" : 9.692035,
"_source" : {
"title" : "2012 Doomsday",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Horror",
"Sci-Fi",
"Thriller"
],
"year" : 2008,
"id" : "173613",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "125874",
"_score" : 8.371374,
"_source" : {
"title" : "2012: Ice Age",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2011,
"id" : "125874",
"@version" : "1"
}
}
]
}
}
4.2.8 查詢2018年之后上映的電影
GET movies/_search?q=year:>=2018
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 859,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122910",
"_score" : 1.0,
"_source" : {
"title" : "Captain Marvel",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2018,
"id" : "122910",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122912",
"_score" : 1.0,
"_source" : {
"title" : "Avengers: Infinity War - Part I",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2018,
"id" : "122912",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "135448",
"_score" : 1.0,
"_source" : {
"title" : "Avatar 4",
"genre" : [
"Action",
"Adventure",
"Fantasy",
"Sci-Fi"
],
"year" : 2018,
"id" : "135448",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "143347",
"_score" : 1.0,
"_source" : {
"title" : "Aquaman",
"genre" : [
"Action",
"Fantasy",
"Sci-Fi"
],
"year" : 2018,
"id" : "143347",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182369",
"_score" : 1.0,
"_source" : {
"title" : "Journey's End",
"genre" : [
"War"
],
"year" : 2018,
"id" : "182369",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182715",
"_score" : 1.0,
"_source" : {
"title" : "Annihilation",
"genre" : [
"Adventure",
"Mystery",
"Sci-Fi",
"Thriller"
],
"year" : 2018,
"id" : "182715",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "182835",
"_score" : 1.0,
"_source" : {
"title" : "The 15:17 to Paris",
"genre" : [
"Drama",
"Thriller"
],
"year" : 2018,
"id" : "182835",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183011",
"_score" : 1.0,
"_source" : {
"title" : "The Commuter",
"genre" : [
"Crime",
"Drama",
"Mystery",
"Thriller"
],
"year" : 2018,
"id" : "183011",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183055",
"_score" : 1.0,
"_source" : {
"title" : "The Beyond",
"genre" : [
"Horror",
"Sci-Fi"
],
"year" : 2018,
"id" : "183055",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "183155",
"_score" : 1.0,
"_source" : {
"title" : "Lean on Pete",
"genre" : [
"Adventure",
"Drama"
],
"year" : 2018,
"id" : "183155",
"@version" : "1"
}
}
]
}
}
4.2.9查詢在2012到2017年上映的電影
GET movies/_search?q=year:(>=2012 AND <2018)
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 2.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "89745",
"_score" : 2.0,
"_source" : {
"title" : "Avengers, The",
"genre" : [
"Action",
"Adventure",
"Sci-Fi",
"IMAX"
],
"year" : 2012,
"id" : "89745",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121124",
"_score" : 2.0,
"_source" : {
"title" : "StreetDance 2",
"genre" : [
"Drama",
"Romance"
],
"year" : 2012,
"id" : "121124",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121129",
"_score" : 2.0,
"_source" : {
"title" : "The Hungover Games",
"genre" : [
"Comedy"
],
"year" : 2014,
"id" : "121129",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121135",
"_score" : 2.0,
"_source" : {
"title" : "Nurse 3D",
"genre" : [
"Horror",
"Thriller"
],
"year" : 2013,
"id" : "121135",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121137",
"_score" : 2.0,
"_source" : {
"title" : "Wrong Turn 5: Bloodlines",
"genre" : [
"Horror",
"Thriller"
],
"year" : 2012,
"id" : "121137",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121143",
"_score" : 2.0,
"_source" : {
"title" : "Flu",
"genre" : [
"Action",
"Drama",
"Sci-Fi"
],
"year" : 2013,
"id" : "121143",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121145",
"_score" : 2.0,
"_source" : {
"title" : "McCullin",
"genre" : [
"Documentary"
],
"year" : 2012,
"id" : "121145",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121152",
"_score" : 2.0,
"_source" : {
"title" : "The White Haired Witch of Lunar Kingdom",
"genre" : [
"Action",
"Fantasy"
],
"year" : 2014,
"id" : "121152",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121155",
"_score" : 2.0,
"_source" : {
"title" : "Kevin Hart: Let Me Explain",
"genre" : [
"Comedy",
"Documentary"
],
"year" : 2013,
"id" : "121155",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "121157",
"_score" : 2.0,
"_source" : {
"title" : "Tad, the Lost Explorer",
"genre" : [
"Adventure",
"Animation",
"Children",
"Comedy"
],
"year" : 2012,
"id" : "121157",
"@version" : "1"
}
}
]
}
}
4.2.10 查詢title中包含beautiful或mind，並且電影上映年份在[1990,1992]的所有的數據
GET movies/_search?q=year:(>=1=990 AND <=1992) AND title:beautiful mind
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
}
}
4.2.11 查詢2016年到2017年上映的電影，必須以 ] 結尾
#前開后閉 大於2015，小於等於2017
GET movies/_search?q=year:{2015 TO 2017]
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4099,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122888",
"_score" : 1.0,
"_source" : {
"title" : "Ben-hur",
"genre" : [
"(no genres listed)"
],
"year" : 2016,
"id" : "122888",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122890",
"_score" : 1.0,
"_source" : {
"title" : "Warcraft",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2016,
"id" : "122890",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122894",
"_score" : 1.0,
"_source" : {
"title" : "Avatar 2",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2016,
"id" : "122894",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122896",
"_score" : 1.0,
"_source" : {
"title" : "Pirates of the Caribbean: Dead Men Tell No Tales",
"genre" : [
"(no genres listed)"
],
"year" : 2017,
"id" : "122896",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122898",
"_score" : 1.0,
"_source" : {
"title" : "Justice League",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122898",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122904",
"_score" : 1.0,
"_source" : {
"title" : "Deadpool",
"genre" : [
"Action",
"Adventure",
"Comedy",
"Sci-Fi"
],
"year" : 2016,
"id" : "122904",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122906",
"_score" : 1.0,
"_source" : {
"title" : "Black Panther",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122906",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122916",
"_score" : 1.0,
"_source" : {
"title" : "Thor: Ragnarok",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122916",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122918",
"_score" : 1.0,
"_source" : {
"title" : "Guardians of the Galaxy 2",
"genre" : [
"Action",
"Adventure",
"Sci-Fi"
],
"year" : 2017,
"id" : "122918",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "122920",
"_score" : 1.0,
"_source" : {
"title" : "Captain America: Civil War",
"genre" : [
"Action",
"Sci-Fi",
"Thriller"
],
"year" : 2016,
"id" : "122920",
"@version" : "1"
}
}
]
}
}
4.2.12 占位符搜索
#查詢title中包含以 Min開頭的字母的電影
GET movies/_search?q=title:Min*
#?代表一個字母
GET movies/_search?q=title:Min?
{
"took" : 6,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 248,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32653",
"_score" : 1.0,
"_source" : {
"title" : "This Land Is Mine",
"genre" : [
"Drama",
"War"
],
"year" : 1943,
"id" : "32653",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "32830",
"_score" : 1.0,
"_source" : {
"title" : "44 Minutes: The North Hollywood Shoot-Out",
"genre" : [
"Action",
"Crime"
],
"year" : 2003,
"id" : "32830",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "38824",
"_score" : 1.0,
"_source" : {
"title" : "Mother of Mine",
"genre" : [
"Drama"
],
"year" : 0,
"id" : "38824",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "40962",
"_score" : 1.0,
"_source" : {
"title" : "Yours, Mine and Ours",
"genre" : [
"Comedy",
"Romance"
],
"year" : 2005,
"id" : "40962",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "46115",
"_score" : 1.0,
"_source" : {
"title" : "Forever Mine",
"genre" : [
"Crime",
"Drama",
"Romance",
"Thriller"
],
"year" : 1999,
"id" : "46115",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "47404",
"_score" : 1.0,
"_source" : {
"title" : "Mind Game",
"genre" : [
"Adventure",
"Animation",
"Comedy",
"Fantasy",
"Romance",
"Sci-Fi"
],
"year" : 2004,
"id" : "47404",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "49371",
"_score" : 1.0,
"_source" : {
"title" : "Min and Bill",
"genre" : [
"Comedy",
"Drama"
],
"year" : 1930,
"id" : "49371",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "49642",
"_score" : 1.0,
"_source" : {
"title" : "Unaccompanied Minors",
"genre" : [
"Children",
"Comedy"
],
"year" : 2006,
"id" : "49642",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "53138",
"_score" : 1.0,
"_source" : {
"title" : "Librarian: Return to King Solomon's Mines, The",
"genre" : [
"Action",
"Adventure",
"Fantasy"
],
"year" : 2006,
"id" : "53138",
"@version" : "1"
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "53207",
"_score" : 1.0,
"_source" : {
"title" : "88 Minutes",
"genre" : [
"Crime",
"Drama",
"Mystery",
"Thriller"
],
"year" : 2008,
"id" : "53207",
"@version" : "1"
}
}
]
}
}
五. Analysis
analysis(只是一個概念)，文本分析是將全文本轉換為一系列單詞的過程，也叫分詞。analysis是通 過analyzer(分詞器)來實現的，可以使用Elasticsearch內置的分詞器，也可以自己去定制一些分詞 器。 除了在數據寫入的時候進行分詞處理，也會在查詢的時候也可以使用分析器對查詢語句進行分詞。 anaylzer是由三部分組成，例如有：
<p>Hello a World, the world is beautifu</p>
- Character Filter: 將文本中html標簽剔除掉。
- Tokenizer: 按照規則進行分詞，在英文中按照空格分詞。
- Token Filter: 去掉stop world(停頓詞，a, an, the, is, are等)，然后轉換小寫
5.1 內置分詞器
| 分詞器名稱 | 處理過程 |
|---|---|
| Standard Analyzer | 默認的分詞器，按詞切分，小寫處理 |
| Simple Analyzer | 按照非字母切分(符號被過濾)，小寫處理 |
| Stop Analyzer | 小寫處理，停用詞過濾(the, a, this) |
| Whitespace Analyzer | 按照空格切分，不轉小寫 |
| Keyword Analyzer | 不分詞，直接將輸入當做輸出 |
| Pattern Analyzer | 正則表達式，默認是\W+(非字符串分隔) |
5.2 內置分詞器示例
5.2.1 Standard Analyzer
GET _analyze
{
"analyzer": "standard",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "<NUM>",
"position" : 0
},
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "<ALPHANUM>",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "<ALPHANUM>",
"position" : 2
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "<ALPHANUM>",
"position" : 3
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "<ALPHANUM>",
"position" : 4
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "<ALPHANUM>",
"position" : 5
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "<ALPHANUM>",
"position" : 6
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "<ALPHANUM>",
"position" : 7
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "<ALPHANUM>",
"position" : 8
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "<ALPHANUM>",
"position" : 9
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "<ALPHANUM>",
"position" : 10
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "<ALPHANUM>",
"position" : 11
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "<ALPHANUM>",
"position" : 12
}
]
}
5.2.2 Simple Analyzer
GET _analyze
{
"analyzer": "simple",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 0
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 1
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 2
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 8
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 9
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.3 Stop Analyzer
GET _analyze
{
"analyzer": "stop",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 0
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 1
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 2
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.4 Whitespace Analyzer
GET _analyze
{
"analyzer": "whitespace",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "word",
"position" : 0
},
{
"token" : "Running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 2
},
{
"token" : "brown-foxes",
"start_offset" : 16,
"end_offset" : 27,
"type" : "word",
"position" : 3
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 4
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 5
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 6
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 7
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 8
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 9
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 10
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 11
}
]
}
5.2.5 Keyword Analyzer
GET _analyze
{
"analyzer": "keyword",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2 Running quick brown-foxes leap over lazy dog in the summer evening",
"start_offset" : 0,
"end_offset" : 68,
"type" : "word",
"position" : 0
}
]
}
5.2.6 Pattern Analyzer
GET _analyze
{
"analyzer": "pattern",
"text": "2 Running quick brown-foxes leap over lazy dog in the summer evening"
}
{
"tokens" : [
{
"token" : "2",
"start_offset" : 0,
"end_offset" : 1,
"type" : "word",
"position" : 0
},
{
"token" : "running",
"start_offset" : 2,
"end_offset" : 9,
"type" : "word",
"position" : 1
},
{
"token" : "quick",
"start_offset" : 10,
"end_offset" : 15,
"type" : "word",
"position" : 2
},
{
"token" : "brown",
"start_offset" : 16,
"end_offset" : 21,
"type" : "word",
"position" : 3
},
{
"token" : "foxes",
"start_offset" : 22,
"end_offset" : 27,
"type" : "word",
"position" : 4
},
{
"token" : "leap",
"start_offset" : 28,
"end_offset" : 32,
"type" : "word",
"position" : 5
},
{
"token" : "over",
"start_offset" : 33,
"end_offset" : 37,
"type" : "word",
"position" : 6
},
{
"token" : "lazy",
"start_offset" : 38,
"end_offset" : 42,
"type" : "word",
"position" : 7
},
{
"token" : "dog",
"start_offset" : 43,
"end_offset" : 46,
"type" : "word",
"position" : 8
},
{
"token" : "in",
"start_offset" : 47,
"end_offset" : 49,
"type" : "word",
"position" : 9
},
{
"token" : "the",
"start_offset" : 50,
"end_offset" : 53,
"type" : "word",
"position" : 10
},
{
"token" : "summer",
"start_offset" : 54,
"end_offset" : 60,
"type" : "word",
"position" : 11
},
{
"token" : "evening",
"start_offset" : 61,
"end_offset" : 68,
"type" : "word",
"position" : 12
}
]
}