Elasticsearch-sql 用SQL查詢Elasticsearch

Elasticsearch的查詢語言(DSL)真是不好寫，偏偏查詢的功能千奇百怪，filter/query/match/agg/geo各種各樣，不管你是通過封裝JSON還是通過python/java的api進行封裝，都非常不方便。

最近發現了一個插件，Elasticsearch-SQL可以用sql查詢Elasticsearch，感覺這個輪子造的真是好。

Elasticsearch-sql的項目地址：https://github.com/NLPchina/elasticsearch-sql

1、簡介

Elasticsearch-sql實現的功能：

1)插件式的安裝

2)SQL查詢

3)超越SQL之外的查詢

4)對JDBC方式的支持

 

2、插件式的安裝

安裝方法和elasticsearch-head的安裝方法類似：

我們使用的es版本是2.1.1，如果你用的是不同的版本，可以在https://github.com/NLPchina/elasticsearch-sql找到支持。

$ cd ~/elasticsearch-2.1.1
$./bin/plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/2.1.1.1/elasticsearch-sql-2.1.1.1.zip 

如果成功，命令行打印如下東東：

[bigdata-dw@bigdata-arch-client10 es2.1.1]$ ./bin/plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/2.1.1.1/elasticsearch-sql-2.1.1.1.zip 
-> Installing from https://github.com/NLPchina/elasticsearch-sql/releases/download/2.1.1.1/elasticsearch-sql-2.1.1.1.zip...
Trying https://github.com/NLPchina/elasticsearch-sql/releases/download/2.1.1.1/elasticsearch-sql-2.1.1.1.zip ...
Downloading .................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE
Verifying https://github.com/NLPchina/elasticsearch-sql/releases/download/2.1.1.1/elasticsearch-sql-2.1.1.1.zip checksums if available ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
Installed sql into /home/bigdata-dw/es2.1.1/plugins/sql

 

3、SQL查詢

安裝成功以后我們就可以通過sql查詢ES了。

es-sql還提供了web頁面，訪問方式是http://10.93.18.34:9200/_plugin/sql/（如果你使用head，那么你的head訪問應該是http://10.93.18.34:9200/_plugin/head/）

這里的ip和port是你安裝es的主機和http端口。

訪問到的頁面是這樣的

那么你現在有兩種方式可以執行你的SQL：

1）在搜索框里直接輸入你的sql了。（我的版本行尾不要寫“；”否則會解析不了SQL）

2）通過http請求如

curl -XPOST http://10.93.18.34:8049/_sql -d 'SELECT * FROM audit where dDelay=-2053867461'

你會收到一個json格式的返回

{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":12.549262,"hits":[{"_index":"audit","_type":"kafka","_id":"AVzzK-h_V9seINxbZ2Ox","_score":12.549262,"_source":{"timestamp":"1498726500000","dCount":680008,"dDelay":-2053867461,"cDelay":0,"clanName":"DJ_elk_common","checkTime":1498728360063,"cCount":0,"pCount":680008,"topicName":"DJ_elk_common_clean","pDelay":370356423}}]}}

下面我們簡單說4種類型的sql的書寫方式：

1）query

SELECT * FROM bank WHERE age >30 AND gender = 'm'

2)aggregation

select COUNT(*),SUM(age),MIN(age) as m, MAX(age),AVG(age)
  FROM bank GROUP BY gender ORDER BY SUM(age), m DESC

3)delete

DELETE FROM bank WHERE age >30 AND gender = 'm'

4)geo

SELECT * FROM locations WHERE GEO_BOUNDING_BOX(fieldname,100.0,1.0,101,0.0)

5)需要指定index+type

  SELECT * FROM indexName/type

6)如何指定路由

select /*! ROUTINGS(salary) */ sum(count)  from index where type="salary"

 

4、對JDBC的支持

上述查詢方式不管是直接在web上輸入sql還是通過http請求。elasticsearch-sql還支持通過jdbc進行編程。

這個還沒有研究，抽空研究一下再回來。