reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response報錯

后端使用的jetty服務

報錯

2022-01-11 17:20:24.362  WARN 4784 --- [ctor-http-nio-3] r.netty.http.client.HttpClientConnect    : [id: 0x3eaaea4e, L:0.0.0.0/0.0.0.0:58790 ! R:localhost/127.0.0.1:8090] The connection observed an error

reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response

2022-01-11 17:20:24.362 ERROR 4784 --- [ctor-http-nio-3] a.w.r.e.AbstractErrorWebExceptionHandler : [31e53fe7-21772]  500 Server Error for HTTP GET "/status"

reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
	|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ HTTP GET "/status" [ExceptionHandlingWebHandler]
Stack trace:

說明

• 通過網關服務Spring Cloud Gateway調用后端服務Jetty報異常

原因

• 后端服務socket超時之后自動斷開連接，網關服務從HttpClient連接池中取出了已經斷開的連接進行請求

復現

• Spring Cloud GateWay : 2.2.1.RELEASE
• Spring Cloud : Hoxton.SR1
• Spring Boot : 2.3.4.RELEASE
• Server使用SpringBoot模擬

GateWay

spring:
  application:
    name: gateway
  cloud:
    gateway:
      httpclient:
        pool:
          max-idle-time: 60000ms
      routes:
      - id: dev
        uri: ${service-url.dev-service}/status
        predicates:
        - Path=/status
service-url:
  dev-service: http://localhost:8080
server:
  port: 8098

Server

spring:
  application:
    name: dev
server:
  tomcat:
    connection-timeout: 100ms

Jmeter

Threads:300
period seconds:5
Loop Count:20

說明：

• max-idle-time：最大空閑連接時間
• connection-timeout：socket read()超時時間。SpringBoot : server.tomcat.connection-timeout配置解析
• GateWay的max-idle-time：60000ms之后才釋放連接，Server的connection-timeout在100ms之后就斷開連接，此時GateWay會從連接池取已經斷開的連接請求，以致報錯

解決

• 因為Server是Provider, GateWay是Consumer，盡可能保證Consumer先於Provider斷開連接，設置max-idle-time的時間不大於connection-timeout
• Jetty服務
  • Jetty修改jetty.http.idleTimeout或http.timeout。默認30000ms

參考

reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response解決方案

Gateway網關使用不規范，同事加班淚兩行~

【排障手記】WebClient調用拋出異常：PrematureCloseException: Connection prematurely closed BEFORE response

https://github.com/reactor/reactor-netty/pull/1442

https://github.com/reactor/reactor-netty/issues/1639

https://github.com/reactor/reactor-netty/issues/796

https://github.com/reactor/reactor-netty/issues/1092

spring-boot中內嵌jetty9配置連接超時時延