使用logstash同步mysql數據到elasticsearch

下載 logstash

tar -zxvf https://artifacts.elastic.co/downloads/logstash/logstash-6.3.2.tar.gz

tar -zxvf logstash-6.3.2.tar.gz

安裝 jdbc 和 elasticsearch 插件

bin/logstash-plugin install logstash-input-jdbc
bin/logstash-plugin install logstash-output-elasticsearch

獲取 jdbc mysql 驅動

wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.46.zip
unzip mysql-connector-java-5.1.46.zip

在config下添加配置文件sync_tabperson.conf名字隨意

input {
    stdin { }
    jdbc {
        jdbc_driver_library => "/home/zl/mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://192.168.1.180:3306/es"
        jdbc_user => "zl"
        jdbc_password => "1234"
        jdbc_paging_enabled => "true"
        jdbc_page_size => "50000"
        statement => "SELECT * FROM person"
        schedule => "* * * * *"
    }
 }

 output {
    elasticsearch {
        hosts => "localhost:9200"
        index => "es"
        document_type => "person"
        document_id => "%{id}"
    }
}

啟動

bin/logstash -f config/sync_tabperson.conf

效果