初始化的主要代碼如下:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForAll // Wait for all in-sync replicas to ack the message
config.Producer.Retry.Max = 10 // Retry up to 10 times to produce the message
config.Producer.Return.Successes = true
config.Producer.Compression = sarama.CompressionSnappy // Compress messages
config.Producer.Partitioner = sarama.NewRandomPartitioner
config.Producer.Return.Errors = true
producer, err := sarama.NewSyncProducer(brokerList, config)
if err != nil {
log.Printf("sarama.NewSyncProducer fail:%+v\n", err)
return err
}
生產消息的主要代碼如下:
msg := sarama.ProducerMessage{
Topic: monitorInstance.topic,
Value: sarama.StringEncoder(resportString),
//Key: sarama.StringEncoder("monitor_data"),
}
partition, offset, err := (*producer).SendMessage(&msg)
if err != nil {
log.Printf("kafka SendMessage fail:%+v\n", err)
return
}
log.Printf("Your data is stored with unique identifier important/%d/%d\n", partition, offset)
嘗試了以下辦法都不行:
1.把producer從async修改為sync
2.增加msg中的key
3.嘗試更換topic
最后發現是這個問題:
config.Producer.Compression = sarama.CompressionSnappy
注釋掉這一行后正常。
難道就不能加壓縮?問了騰訊雲的工程師,得到了解決:
config.Version = sarama.V2_1_0_0
config.Producer.Compression = sarama.CompressionSnappy