最近在測試環境遇到一次jdbc連接池占滿的問題。背景如下:
有一個批量操作,分頁去查表數據然后進行后續處理,該查詢跨表並且需要返回自定義的字段。
spring-data-jpa提供了方便使用的JpaRepository
接口,依次繼承PagingAndSortingRepository
、CrudRepository
、Repository
。
自定義查詢一般步驟如下:
- 定義包含自定義查詢的接口
public interface XxxRepositoryCustom {
Page<XxxRespVo> findXxx(XxxReqVo, Pageable pageable);
}
其中`findXxx`為自定義的查詢接口
- 定義表對應的接口,同時繼承
JpaRepository
和XxxRepositoryCustom
。
@Repository
public interface XxxRepository extends JpaRepository<Xxx, Long>, XxxRepositoryCustom, JpaSpecificationExecutor<Xxx> {
...
}
注:這里同時繼承了`JpaSpecificationExecutor<Xxx>`接口,便於同時能使用`Specification`查詢功能。
- 實現
XxxDao
接口,實現自定義的查詢方法
@Repository
public class XxxRepositoryImpl implements XxxRepositoryCustom {
@PersistenceContext
private EntityManager em;
@Override
public Page<XxxRespVo> findSimpleDepotProductList(XxxReqVo reqVo, Pageable pageable) {
...
StringBuilder selectSql = new StringBuilder();
selectSql.append("select xx.aa,yy.bb,zz.cc")
.append(" from xx join yy join zz ...")
.append(" where param1=:param1")
.append(" where param2=:param2")
.append(" where param3=:param3")
.append("...");
Query selectQuery = em.createNativeQuery(selectSql.toString());
selectQuery.setParameter("xx", xx);
List<XxxRespVo> list = selectQuery.unwrap(SQLQuery.class)
.addScalar("param1", StandardBasicTypes.STRING)
.addScalar("param2", StandardBasicTypes.STRING)
.addScalar("param3", StandardBasicTypes.LONG)
.setResultTransformer(Transformers.aliasToBean(XxxRespVo.class))
.setFirstResult(pageable.getPageNumber() * pageable.getPageSize())
.setMaxResults(pageable.getPageSize())
.list();
return new PageImpl<>(list, pageable, total);
}
}
用單元測試調該查詢方法成功,在測試環境跑,結果一會兒前端界面調該服務的其它都變慢了,查看日志發現了大量異常:
|ERROR|2020-10-27 16:30:21.811|DubboServerHandler-192.168.x.x:20001-thread-461|c.a.d.r.f.ExceptionFilter:85
[DUBBO] Got unchecked and undeclared exception which called by 192.168.x.x. service: com.xxx.XxxService, method: findXxx, exception: org.springframework.orm.jpa.JpaSystemException: Unable to acquire JDBC Connection; nested exception is org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection, dubbo version: 2.6.2, current host: 192.168.x.x
org.springframework.orm.jpa.JpaSystemException: Unable to acquire JDBC Connection; nested exception is org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:314)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:225)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:417)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
...
Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
...
Caused by: com.alibaba.druid.pool.GetConnectionTimeoutException: wait millis 60000, active 50, maxActive 50
at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1139)
at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:960)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:940)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:930)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:102)
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:386)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:84)
... 85 common frames omitted
注意到Unable to acquire JDBC Connection
和GetConnectionTimeoutException
,獲取數據庫連接池超時了。
測試環境的jdbc連接池配置如下:
<jdbc.initialSize>2</jdbc.initialSize>
<jdbc.minIdle>2</jdbc.minIdle>
<jdbc.maxActive>50</jdbc.maxActive>
<jdbc.maxWait>60000</jdbc.maxWait>
最大活躍鏈接數50,等待時間60秒,跟日志中的異常信息對應。
推測可能是連接沒有關閉,改小maxActive值,用單元測試循環去調,很快復現出了測試環境的報錯異常。
檢查發現,這里使用了自定義查詢,但缺少事務的注解,連接沒有自動關閉。
解決方法:加上spring的事務注解,因為是查詢,設置為只讀能一定程度提高性能。
@Transactional(readOnly = true)
@Override
public Page<XxxRespVo> findSimpleDepotProductList(XxxReqVo reqVo, Pageable pageable) {
...
}
參考:
spring-data-jpa官方文檔:https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.custom-implementations