【解決了一個問題】騰訊雲中使用ckafka生產消息時出現“kafka server: Message contents does not match its CRC.”錯誤


初始化的主要代碼如下:

	config := sarama.NewConfig()
	config.Producer.RequiredAcks = sarama.WaitForAll // Wait for all in-sync replicas to ack the message
	config.Producer.Retry.Max = 10                   // Retry up to 10 times to produce the message
	config.Producer.Return.Successes = true
	config.Producer.Compression = sarama.CompressionSnappy // Compress messages
	config.Producer.Partitioner = sarama.NewRandomPartitioner
	config.Producer.Return.Errors = true
	producer, err := sarama.NewSyncProducer(brokerList, config)
	if err != nil {
		log.Printf("sarama.NewSyncProducer fail:%+v\n", err)
		return err
	}

生產消息的主要代碼如下:

	msg := sarama.ProducerMessage{
		Topic: monitorInstance.topic,
		Value: sarama.StringEncoder(resportString),
		//Key: sarama.StringEncoder("monitor_data"),
	}
	partition, offset, err := (*producer).SendMessage(&msg)
	if err != nil {
		log.Printf("kafka SendMessage fail:%+v\n", err)
		return
	}
	log.Printf("Your data is stored with unique identifier important/%d/%d\n", partition, offset)

嘗試了以下辦法都不行:
1.把producer從async修改為sync
2.增加msg中的key
3.嘗試更換topic

最后發現是這個問題:
config.Producer.Compression = sarama.CompressionSnappy
注釋掉這一行后正常。

難道就不能加壓縮?問了騰訊雲的工程師,得到了解決:
config.Version = sarama.V2_1_0_0
config.Producer.Compression = sarama.CompressionSnappy


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM