Elasticsearch的SQL查詢


使用SQL查詢ES有一定的局限性,沒有原生的Query DSL那么強大,對於嵌套屬性和某些函數的支持並不怎么好,但是平時用來查詢下數據基本夠用了。

在線sql轉dsl:http://www.atotoa.com/

elasticsearch-sql插件

注:本人安裝的是es 7.x(7.10.2),默認是支持SQL的,所以不用安裝elasticsearch-sql插件。

1、下載 https://github.com/NLPchina/elasticsearch-sql

2、下載elasticsearch-sql,修改名字為sql。放在es解壓目錄的plugins文件中即可。重啟es則sql安裝完畢

支持的SQL語法

首先我們創建一個索引:

{
    "settings": {
        "index": {
            "number_of_shards": 3,
            "number_of_replicas": 2
        }
    },
    "mappings": {
        "properties": {
            "name": {
                "type": "text",
		"fields":{"keyword": {"type": "keyword","ignore_above": 256}} //可以對一個字段提供多種索引模式,同一個字段的值,一個分詞,一個不分詞  
            },
            "author": {
                "type": "text",
		"fields":{"keyword": {"type": "keyword","ignore_above": 256}} 
            },
            "release_date": {
                "type": "date",
		"format":"yyyy-MM-dd"
				
            },
            "page_count": {
                "type": "integer"
            }
        }
    }
}

這里要特別注意:如果要用到like這種模糊操作,但是type是text的,那么就還需要為該字段設置不分詞的索引方式,即type=keyword,如上面的name和author字段。

批量插入數據(使用Postman)

PUT /book_info/_bulk
{ "index" : {  "_id" : "101" } }
{ "name" : "hello","author":"tom","release_date":"2020-01-26","page_count": 500 }

{ "index" : {  "_id" : "102" } }
{ "name" : "world","author":"Marry","release_date":"2018-01-26","page_count": 352 }

{ "index" : {  "_id" : "103" } }
{ "name" : "test","author":"test","release_date":"2019-03-26","page_count": 436 }

{ "index" : {  "_id" : "104" } }
{ "name" : "demo","author":"demo","release_date":"2020-06-26","page_count": 337 }

{ "index" : {  "_id" : "105" } }
{ "name" : "zhangsan","author":"zhangsan","release_date":"2015-05-21","page_count": 562 }

{ "index" : {  "_id" : "106" } }
{ "name" : "zhangsan02","author":"zhangsan","release_date":"2015-05-21","page_count": 562 }
#這里需要換行

 

語法格式

語法格式:/_sql?format=txt

format 返回格式,不設置則默認返回JSON (支持,csv,txt,json,yaml。

請求體body參數:

  • query:要查詢的sql參數,如:"select * from user"
  • fetch_size:一個響應中要返回的最大行(或條目)數,默認是1000
  • filter:可選的Elasticsearch查詢DSL,以進行其他過濾。
  • request_timeout:請求失敗之前的超時,默認是90s
  • page_timeout:分頁請求失敗之前的超時,默認是45s
  • columnar:以列方式而不是基於行的方式返回結果。有效期為json,yaml,cbor和smile。默認是false。
  • field_multi_value_leniency:當遇到一個字段的多個值(默認值)或寬大處理時拋出異常,並從列表中返回第一個值(不保證將要返回的值-通常以自然的升序方式第一個)。默認是false。
  • index_include_frozen:是否在查詢執行中包括凍結索引(默認值)。默認是false。

條件查詢

1、單條件查詢

POST /_sql?format=txt
{
  "query": "SELECT * FROM book_info where name = 'hello'"
}

2、多條件查詢

POST /_sql?format=txt
{
  "query": "SELECT * FROM book_info where name = 'hello' and author='tom'"
}

排序+限制行數

POST /_sql?format=txt
{
  "query": "SELECT * FROM book_info ORDER BY release_date DESC LIMIT 2"
}

排序+限制返回字段

POST /_sql?format=txt
{
  "query": "SELECT name,release_date,page_count FROM book_info ORDER BY release_date DESC  LIMIT 3"
}

時間比較查詢

POST /_sql?format=txt
{
  "query": "SELECT * FROM book_info WHERE release_date < '2020-01-01'  LIMIT 3"
}

聚合查詢

POST /_sql?format=txt
{
  "query": "SELECT sum(page_count) FROM book_info"
}

POST /_sql?format=txt
{
  "query": "SELECT sum(page_count),name FROM book_info group by name"
}

POST /_sql?format=txt
{
  "query": "SELECT avg(page_count) FROM book_info"
}

POST /_sql?format=txt
{
  "query": "SELECT min(page_count) FROM book_info"
}

POST /_sql?format=txt
{
  "query": "SELECT max(page_count) FROM book_info"
}

count查詢

POST /_sql?format=txt
{
  "query": "SELECT count(*),count(distinct author) FROM book_info "
}

分組查詢

POST /_sql?format=txt
{
  "query": "SELECT author,count(*) as count FROM book_info group by author"
}

SQL轉DSL

普通sql

POST /_sql/translate 
{ 
  "query": "SELECT name,author FROM book_info where name ='hello'", 
  "fetch_size": 1
} 

右模糊

POST /_sql/translate 
{ 
  "query": "SELECT name,author FROM book_info where name like 'hello%'", 
  "fetch_size": 10 
} 

時間區間

POST /_sql/translate 
{
  "query": "SELECT name,author,release_date FROM book_info WHERE release_date >'2019-01-01' and release_date < '2020-01-01'  LIMIT 3"
}

常用的SQL總結

在ES中使用SQL查詢的語法與在數據庫中使用基本一致,具體格式如下:

SELECT select_expr [, ...]
[ FROM table_name ]
[ WHERE condition ]
[ GROUP BY grouping_element [, ...] ]
[ HAVING condition]
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
[ LIMIT [ count ] ]
[ PIVOT ( aggregation_expr FOR column IN ( value [ [ AS ] alias ] [, ...] ) ) ]

WHERE

可以使用WHERE語句設置查詢條件,比如查詢state字段為VA的記錄,查詢語句如下:

POST /_sql?format=txt
{
  "query": "SELECT account_number,address,age,balance,state FROM account WHERE state='VA' LIMIT 10 "
}

GROUP BY

我們可以使用GROUP BY語句對數據進行分組,統計出分組記錄數量,最大age和平均balance等信息,查詢語句如下:

POST /_sql?format=txt
{
  "query": "SELECT state,COUNT(*),MAX(age),AVG(balance) FROM account GROUP BY state LIMIT 10"
}

HAVING

我們可以使用HAVING語句對分組數據進行二次篩選,比如篩選分組記錄數量大於15的信息,查詢語句如下:

POST /_sql?format=txt
{
  "query": "SELECT state,COUNT(*),MAX(age),AVG(balance) FROM account GROUP BY state HAVING COUNT(*)>15 LIMIT 10"
}

ORDER BY

我們可以使用ORDER BY語句對數據進行排序,比如按照balance字段從高到低排序,查詢語句如下:

POST /_sql?format=txt
{
  "query": "SELECT account_number,address,age,balance,state FROM account ORDER BY balance DESC LIMIT 10 "
}

DESCRIBE

我們可以使用DESCRIBE語句查看表(ES中為索引)中有哪些字段,比如查看account表的字段,查詢語句如下:

POST /_sql?format=txt
{
  "query": "DESCRIBE account"
}

SHOW TABLES

我們可以使用SHOW TABLES查看所有的表(ES中為索引)。

POST /_sql?format=txt
{
  "query": "SHOW TABLES"
}

支持的函數

使用SQL查詢ES中的數據,不僅可以使用一些SQL中的函數,還可以使用一些ES中特有的函數。

1、我們可以使用SHOW FUNCTIONS語句查看所有支持的函數,比如搜索所有帶有DATE字段的函數可以使用如下語句:

POST /_sql?format=txt
{
  "query": "SHOW FUNCTIONS LIKE '%DATE%'"
}

2、全文搜索函數是ES中特有的,當使用MATCH或QUERY函數時,會啟用全文搜索功能,SCORE函數可以用來統計搜索評分。

(1)使用MATCH函數查詢address中包含Street的記錄。

POST /_sql?format=txt
{
  "query": "SELECT account_number,address,age,balance,SCORE() FROM account WHERE MATCH(address,'Street') LIMIT 10"
}

(2)使用QUERY函數查詢address中包含Street的記錄。

POST /_sql?format=txt
{
  "query": "SELECT account_number,address,age,balance,SCORE() FROM account WHERE QUERY('address:Street') LIMIT 10"
}

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM