使用golang kafka sarama 包時,遇到如下問題:
高並發情況下使用同步sync producer,偶爾遇到crash:
panic: interface conversion: interface {} is nil, not chan *sarama.ProducerError
goroutine 413 [running]:
github.com/Shopify/sarama.(*syncProducer).handleSuccesses(0xc420384840)
/home/lanyang/github.com/Shopify/sarama/sync_producer.go:147 +0x139
github.com/Shopify/sarama.handleSuccesses)-fm()
/home/lanyang/src/github.com/Shopify/sarama/sync_producer.go:76 +0x2f
github.com/Shopify/sarama.withRecover(0xc4201d0750)
/home/lanyang/src/github.com/Shopify/sarama/utils.go:43 +0x45
created by github.com/Shopify/sarama.newSyncProducerFromAsyncProducer
/home/lanyang/src/github.com/Shopify/sarama/sync_producer.go:76 +0x112
問題不容易復現,github上已經留言:
https://github.com/Shopify/sarama/issues/785
download
https://github.com/Shopify/sarama
https://shopify.github.io/sarama/
doc
https://godoc.org/github.com/Shopify/sarama
依賴
github.com/davecgh/go-spew
github.com/eapache/go-resiliency
github.com/eapache/go-xerial-snappy
github.com/eapache/queue
github.com/pierrec/lz4
github.com/pierrec/xxHash
github.com/rcrowley/go-metrics
github.com/golang/snappy
如果kafka down掉,又恢復,client 可自動連接上。
異步方式,網絡斷開后,仍然會從queue,放入producer的channel中,重連后,這些內容丟失。
例子參考
sync
http://legendtkl.com/2016/06/30/kafka-in-action/
https://studygolang.com/articles/7102
