springcloudstream配置使用kafka案例

一、Spring cloud stream概述

Spring Cloud Stream是構建消息驅動的微服務應用程序框架。提供統一的接收發送管道以連接到消息代理。通過@EnableBinding注解開啟SpringCloudStream的支持。通過@StreamListener注解，使其接收流處理的時間。

二、引入包依賴

        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.6.4</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>

　　

三、自定義信息通道

官方提供了Sink（輸入通道）、Source（輸出通道）、Processor（集成Sink和Source通道），我們也可以自定義我們自己的信息通道。
@Input注解標識一個輸入通道
@Output注解標識一個輸出通道
通道名稱作為參數，如果未提供參數，默認使用方法名稱作為通道名稱。
如下我們自定義信息通道ExamFinishChannel

public interface ExamFinishChannel {
    String EXAM_FINISH_OUTPUT = "exam-finish-output";
    String EXAM_FINISH_INPUT = "exam-finish-input";

    @Output(EXAM_FINISH_OUTPUT)
    MessageChannel sendExamFinishEvent();

    @Input(EXAM_FINISH_INPUT)
    SubscribableChannel receiveExamFinishEvent();
}

　　

四、SpringCloudStream及kafka配置

spring:
    cloud:
        stream:
            kafka:
                binder:
                    brokers: ${kafka.brokers:127.0.0.1:9092}
            bindings:
                exam-finish-output:
                    destination: ${kafka.exam-finish-event:spacer_tiangong_exam_finish_event_dev}
                    content-type: application/json
                exam-finish-input:
                    destination: ${kafka.exam-finish-event:spacer_tiangong_exam_finish_event_dev}
                    content-type: application/json
                    group: ${kafka.exam-finish-consumer-group:spacer_tiangong_exam_group}

　　

從上面配置可以看出
1、定義了通道名稱及分組，binder代表綁定實現的標識名稱（如kafka或者rabbit），與3中的定義名稱相對應。
2、定義了入站消費者的並發性，指在一個實例內的並發性，不同實例之間本身就是並發的，默認值為1
spring.cloud.stream.bindings.<channelName>.consumer.concurrency=1
3、定義了kafka連接信息
如果未配置autoCommitOffset,默認自動提交偏移量

詳細參數配置可參考 官網

五、發送消息到輸出通道

@Slf4j
@EnableBinding(ExamFinishChannel.class)
public class ExamFinishProducer {

    @Autowired
    private ExamFinishChannel examFinishChannel;

    public void sendExamFinishEventMsg(ExamFinishEventMessage payload) {
        examFinishChannel.sendExamFinishEvent().send(MessageBuilder.withPayload(payload).build());
        log.info("send msg success: {}", payload);
    }
}

　　

注入先前定義的通道ExamFinishChannel自定義的發送方法，可將消息發送到通道中，每個通道對應一個kafka的主題

 

六、從輸入通道訂閱消息

@Slf4j
@EnableBinding(ExamFinishChannel.class)
public class ExamFinishConsumer {

    @StreamListener(ExamFinishChannel.EXAM_FINISH_INPUT)
    public void receive(ExamFinishEventMessage payload) {
        log.info("start....");
        System.out.println(payload);
        try {
            Thread.sleep(1000*10);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        log.info("receive success end:{}", payload);
    }
}

　　

七、這樣完整的消息系統就搭建好了，定義Controller發送消息測試

 @Autowired
    private ExamFinishProducer examFinishProducer;

    @GetMapping("/test")
    public void test() {
        ExamFinishEventMessage examFinishEventMessage = new ExamFinishEventMessage();
        examFinishEventMessage.setName("mzq");
        examFinishProducer.sendExamFinishEventMsg(examFinishEventMessage);
        examFinishProducer.sendExamFinishEventMsg(examFinishEventMessage);
        examFinishProducer.sendExamFinishEventMsg(examFinishEventMessage);
        examFinishProducer.sendExamFinishEventMsg(examFinishEventMessage);
    }

　　

八、並發性測試

如七中所示，一次發送4條消息到缺省消息通道中,

在並發性配置為1的情況下，即spring.cloud.stream.bindings.exam-finish-input.consumer.concurrency=1

 

start....
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
start....
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
start....
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
start....
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)

　　

如果將concurrency修改為2，即spring.cloud.stream.bindings.exam-finish-input.consumer.concurrency=2

start...
start...
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
start...
ExamFinishEventMessage(name=mzq)
start...
ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)
receive success end:ExamFinishEventMessage(name=mzq)

從日志可以看出，實現了兩個線程的並發消費。