項目細節也就不說了
kafka + spark streaming >>
kafka版本1.0，spark streaming也是依照kafka1.0開發的
這個kfk集群叫kfk_1.0吧
包括kafka在內的集群資源都是個人維護的
目前有一個情況是要從另一個kafka集群版本0.8.1,叫kfk_0.8.1里，取數計算寫入kfk_1.0
比較坑爹的是
https://spark.apache.org/docs/latest/streaming-kafka-0-8-integration.html
spark-streaming 要求的最低kfk版本是 0.8.2.1
不兼容kfk_A
正在維護人員升級或部分topic遷移
還有周轉的辦法，是用中間導數程序將0.8.1 導入 kfk_1.0
以個人經驗來說，先考慮應用logstash
但logstash對kafka也有版本的兼容問題
https://www.elastic.co/guide/en/logstash/6.5/plugins-inputs-kafka.html
This plugin uses Kafka Client 2.0.0.
https://www.elastic.co/guide/en/logstash/6.3/plugins-inputs-kafka.html
This plugin uses Kafka Client 1.1.0
https://www.elastic.co/guide/en/logstash/5.0/plugins-inputs-kafka.html
This plugin uses Kafka Client 0.10.0.1
https://www.elastic.co/guide/en/logstash/2.4/plugins-inputs-kafka.html
This plugin uses Kafka Client 0.8.2.2
https://www.elastic.co/guide/en/logstash/2.3/plugins-inputs-kafka.html
```
Kafka Client Version Logstash Version Plugin Version Security Features Why?
0.8 2.0.0 - 2.x.x<3.0.0 Legacy, 0.8 is still popular
0.9 2.0.0 - 2.3.x 3.x.x Basic Auth, SSL
```
經測試 logstash 2.0 兼容kafka 0.8.1
看遷移的進度，如不支持遷移
考慮以kafka2.0讀取kafka 0.8.1 寫入kafka 1.0.0
kafka2.0寫入kafka 1.0.0還未測試