使用SQL查詢ES有一定的局限性,沒有原生的Query DSL那么強大,對於嵌套屬性和某些函數的支持並不怎么好,但是平時用來查詢下數據基本夠用了。
在線sql轉dsl:http://www.atotoa.com/
elasticsearch-sql插件
注:本人安裝的是es 7.x(7.10.2),默認是支持SQL的,所以不用安裝elasticsearch-sql插件。
1、下載 https://github.com/NLPchina/elasticsearch-sql
2、下載elasticsearch-sql,修改名字為sql。放在es解壓目錄的plugins文件中即可。重啟es則sql安裝完畢
支持的SQL語法
首先我們創建一個索引:
{
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 2
}
},
"mappings": {
"properties": {
"name": {
"type": "text",
"fields":{"keyword": {"type": "keyword","ignore_above": 256}} //可以對一個字段提供多種索引模式,同一個字段的值,一個分詞,一個不分詞
},
"author": {
"type": "text",
"fields":{"keyword": {"type": "keyword","ignore_above": 256}}
},
"release_date": {
"type": "date",
"format":"yyyy-MM-dd"
},
"page_count": {
"type": "integer"
}
}
}
}
這里要特別注意:如果要用到like這種模糊操作,但是type是text的,那么就還需要為該字段設置不分詞的索引方式,即type=keyword,如上面的name和author字段。
批量插入數據(使用Postman)
PUT /book_info/_bulk
{ "index" : { "_id" : "101" } }
{ "name" : "hello","author":"tom","release_date":"2020-01-26","page_count": 500 }
{ "index" : { "_id" : "102" } }
{ "name" : "world","author":"Marry","release_date":"2018-01-26","page_count": 352 }
{ "index" : { "_id" : "103" } }
{ "name" : "test","author":"test","release_date":"2019-03-26","page_count": 436 }
{ "index" : { "_id" : "104" } }
{ "name" : "demo","author":"demo","release_date":"2020-06-26","page_count": 337 }
{ "index" : { "_id" : "105" } }
{ "name" : "zhangsan","author":"zhangsan","release_date":"2015-05-21","page_count": 562 }
{ "index" : { "_id" : "106" } }
{ "name" : "zhangsan02","author":"zhangsan","release_date":"2015-05-21","page_count": 562 }
#這里需要換行

語法格式
語法格式:/_sql?format=txt
format 返回格式,不設置則默認返回JSON (支持,csv,txt,json,yaml。
請求體body參數:
- query:要查詢的sql參數,如:"select * from user"
- fetch_size:一個響應中要返回的最大行(或條目)數,默認是1000
- filter:可選的Elasticsearch查詢DSL,以進行其他過濾。
- request_timeout:請求失敗之前的超時,默認是90s
- page_timeout:分頁請求失敗之前的超時,默認是45s
- columnar:以列方式而不是基於行的方式返回結果。有效期為json,yaml,cbor和smile。默認是false。
- field_multi_value_leniency:當遇到一個字段的多個值(默認值)或寬大處理時拋出異常,並從列表中返回第一個值(不保證將要返回的值-通常以自然的升序方式第一個)。默認是false。
- index_include_frozen:是否在查詢執行中包括凍結索引(默認值)。默認是false。
條件查詢
1、單條件查詢
POST /_sql?format=txt
{
"query": "SELECT * FROM book_info where name = 'hello'"
}
2、多條件查詢
POST /_sql?format=txt
{
"query": "SELECT * FROM book_info where name = 'hello' and author='tom'"
}
排序+限制行數
POST /_sql?format=txt
{
"query": "SELECT * FROM book_info ORDER BY release_date DESC LIMIT 2"
}
排序+限制返回字段
POST /_sql?format=txt
{
"query": "SELECT name,release_date,page_count FROM book_info ORDER BY release_date DESC LIMIT 3"
}
時間比較查詢
POST /_sql?format=txt
{
"query": "SELECT * FROM book_info WHERE release_date < '2020-01-01' LIMIT 3"
}
聚合查詢
POST /_sql?format=txt
{
"query": "SELECT sum(page_count) FROM book_info"
}
POST /_sql?format=txt
{
"query": "SELECT sum(page_count),name FROM book_info group by name"
}
POST /_sql?format=txt
{
"query": "SELECT avg(page_count) FROM book_info"
}
POST /_sql?format=txt
{
"query": "SELECT min(page_count) FROM book_info"
}
POST /_sql?format=txt
{
"query": "SELECT max(page_count) FROM book_info"
}
count查詢
POST /_sql?format=txt
{
"query": "SELECT count(*),count(distinct author) FROM book_info "
}
分組查詢
POST /_sql?format=txt
{
"query": "SELECT author,count(*) as count FROM book_info group by author"
}
SQL轉DSL
普通sql
POST /_sql/translate
{
"query": "SELECT name,author FROM book_info where name ='hello'",
"fetch_size": 1
}
右模糊
POST /_sql/translate
{
"query": "SELECT name,author FROM book_info where name like 'hello%'",
"fetch_size": 10
}
時間區間
POST /_sql/translate
{
"query": "SELECT name,author,release_date FROM book_info WHERE release_date >'2019-01-01' and release_date < '2020-01-01' LIMIT 3"
}
常用的SQL總結
在ES中使用SQL查詢的語法與在數據庫中使用基本一致,具體格式如下:
SELECT select_expr [, ...]
[ FROM table_name ]
[ WHERE condition ]
[ GROUP BY grouping_element [, ...] ]
[ HAVING condition]
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
[ LIMIT [ count ] ]
[ PIVOT ( aggregation_expr FOR column IN ( value [ [ AS ] alias ] [, ...] ) ) ]
WHERE
可以使用WHERE語句設置查詢條件,比如查詢state字段為VA的記錄,查詢語句如下:
POST /_sql?format=txt
{
"query": "SELECT account_number,address,age,balance,state FROM account WHERE state='VA' LIMIT 10 "
}
GROUP BY
我們可以使用GROUP BY語句對數據進行分組,統計出分組記錄數量,最大age和平均balance等信息,查詢語句如下:
POST /_sql?format=txt
{
"query": "SELECT state,COUNT(*),MAX(age),AVG(balance) FROM account GROUP BY state LIMIT 10"
}
HAVING
我們可以使用HAVING語句對分組數據進行二次篩選,比如篩選分組記錄數量大於15的信息,查詢語句如下:
POST /_sql?format=txt
{
"query": "SELECT state,COUNT(*),MAX(age),AVG(balance) FROM account GROUP BY state HAVING COUNT(*)>15 LIMIT 10"
}
ORDER BY
我們可以使用ORDER BY語句對數據進行排序,比如按照balance字段從高到低排序,查詢語句如下:
POST /_sql?format=txt
{
"query": "SELECT account_number,address,age,balance,state FROM account ORDER BY balance DESC LIMIT 10 "
}
DESCRIBE
我們可以使用DESCRIBE語句查看表(ES中為索引)中有哪些字段,比如查看account表的字段,查詢語句如下:
POST /_sql?format=txt
{
"query": "DESCRIBE account"
}
SHOW TABLES
我們可以使用SHOW TABLES查看所有的表(ES中為索引)。
POST /_sql?format=txt
{
"query": "SHOW TABLES"
}
支持的函數
使用SQL查詢ES中的數據,不僅可以使用一些SQL中的函數,還可以使用一些ES中特有的函數。
1、我們可以使用SHOW FUNCTIONS語句查看所有支持的函數,比如搜索所有帶有DATE字段的函數可以使用如下語句:
POST /_sql?format=txt
{
"query": "SHOW FUNCTIONS LIKE '%DATE%'"
}
2、全文搜索函數是ES中特有的,當使用MATCH或QUERY函數時,會啟用全文搜索功能,SCORE函數可以用來統計搜索評分。
(1)使用MATCH函數查詢address中包含Street的記錄。
POST /_sql?format=txt
{
"query": "SELECT account_number,address,age,balance,SCORE() FROM account WHERE MATCH(address,'Street') LIMIT 10"
}
(2)使用QUERY函數查詢address中包含Street的記錄。
POST /_sql?format=txt
{
"query": "SELECT account_number,address,age,balance,SCORE() FROM account WHERE QUERY('address:Street') LIMIT 10"
}
