Elasticsearch的能力雖然已經非常強大,但是它也提供了基於插件的擴展功能,基於此我們可以擴展查詢、分詞、監控、腳本等能力。
這是學習Elasticsearch插件的第一篇,主要是閱讀官方文檔的筆記,介紹官方的一些插件和優秀的社區插件;后面一篇主要是通過源碼來深入學習Elasticsearch插件的開發,並通過實戰開發一個自定義的插件。
Plugin Management
- Plugins are a way to enhance the core Elasticsearch functionality in a custom manner. They range from adding custom mapping types, custom analyzers, native scripts, custom discovery and more.
- Site plugins and mixed plugins are deprecated and will be removed in 5.0.0. Instead, site plugins should either be migrated to Kibana or should use a standalone web server.
- The plugin script is used to install, list, and remove plugins.
sudo bin/plugin install [plugin_name] #Core Elasticsearch plugins
sudo bin/plugin install [org]/[user|component]/[version] #Community and non-core plugins
sudo bin/plugin install [url] #Custom URL or file system
sudo bin/plugin list #Listing plugins
sudo bin/plugin remove [plugin_name] #Removing plugins
API Extension Plugins
API extension plugins add new functionality to Elasticsearch by adding new APIs or features, usually to do with search or mapping.
- [Core]Delete By Query Plugin:Elasticsearch 1.0版本有一個delete-by-query的API,由於存在兼容性、一致性和可靠性的問題而被移除,該插件使用scroll獲取文檔ID和版本然后使用bulk進行批量刪除;
- carrot2 Plugin:可以自動地將相似的文檔組織起來,並且給每個文檔的群組分類貼上相應的較為用戶可以理解的標簽。這樣的聚類也可以看做是一種動態的針對每個搜索和命中結果集合的動態 facet。可以在Carrot2 demo page體驗一下這個工具。
- Elasticsearch Image Plugin:基於LIRE的相似圖片搜索插件;
- Entity Resolution Plugin:基於貝葉斯概率模型去除重復數據的插件;
- SQL language Plugin:支持采用SQL查詢的ES插件;
- Elasticsearch Taste Plugin:基於用戶和內容推薦的ES插件;
Analysis Plugins
Analysis plugins extend Elasticsearch by adding new analyzers, tokenizers, token filters, or character filters to Elasticsearch.
- [Core]ICU:使用ICU實現的一個針對亞洲語言的分詞器插件;
- [Core]SmartCN:官方提供的一個基於概率的針對中文或中英混合的分詞器;
- Combo Analysis Plugin:通常一個分析器里只能配置一個分詞器,該插件支持能配置多個分詞器組合;
- IK Analysis Plugin:一個非常流行的中文分析器插件,遷移自Lucene的IK分析器;
- Mmseg Analysis Plugin:基於MMSEG算法的中文分析器,在中英混合時分詞效果較差;
- Pinyin Analysis Plugin:將中文轉換為拼音的分析器,支持首字母和連接符配置;
Discovery Pluginsedit
Discovery plugins extend Elasticsearch by adding new discovery mechanisms that can be used instead of Zen Discovery.
- [Core]AWS Cloud/Azure Cloud/GCE Cloud:官方提供的基於各種雲服務的插件;
- ZooKeeper Discovery Plugin:基於ZooKeeper的Elasticsearch集群發現插件;
- Kubernetes Discovery Plugin:使用K8 API單播發現插件;
Security Plugins
- Kerberos/SPNEGO Realm:基於Kerberos/SPNEGO驗證HTTP和傳輸請求。
- Readonly REST:只對外暴露查詢相關的操作,拒絕刪除和更新操作。
Integrations
Integrations are not plugins, but are external tools or modules that make it easier to work with Elasticsearch.
- JDBC importer:通過JDBC將數據庫的數據導入到Elasticsearch中;
- Kafka Standalone Consumer(Indexer):讀取kafka消息並處理,最終批量寫入到Elasticsearch中;
- Mongolastic:將MongoDB的數據遷移到Elasticsearch中;
- Scrutineer:比較Elasticsearch和數據庫中數據的一致性;
Help for plugin authors
插件描述文件plugin-descriptor.properties可以參考:https://github.com/elastic/elasticsearch/blob/2.4/dev-tools/src/main/resources/plugin-metadata/plugin-descriptor.properties