spring-data-jpa自定義查詢導致jdbc連接池占滿


最近在測試環境遇到一次jdbc連接池占滿的問題。背景如下:
有一個批量操作,分頁去查表數據然后進行后續處理,該查詢跨表並且需要返回自定義的字段。
spring-data-jpa提供了方便使用的JpaRepository接口,依次繼承PagingAndSortingRepositoryCrudRepositoryRepository
自定義查詢一般步驟如下:

  1. 定義包含自定義查詢的接口
public interface XxxRepositoryCustom {
    Page<XxxRespVo> findXxx(XxxReqVo, Pageable pageable);
}
其中`findXxx`為自定義的查詢接口
  1. 定義表對應的接口,同時繼承JpaRepositoryXxxRepositoryCustom
@Repository
public interface XxxRepository extends JpaRepository<Xxx, Long>, XxxRepositoryCustom, JpaSpecificationExecutor<Xxx> {
    ...
}

注:這里同時繼承了`JpaSpecificationExecutor<Xxx>`接口,便於同時能使用`Specification`查詢功能。
  1. 實現XxxDao接口,實現自定義的查詢方法
@Repository
public class XxxRepositoryImpl implements XxxRepositoryCustom {

    @PersistenceContext
    private EntityManager em;
    
    @Override
    public Page<XxxRespVo> findSimpleDepotProductList(XxxReqVo reqVo, Pageable pageable) {
         ...
        StringBuilder selectSql = new StringBuilder();
        selectSql.append("select xx.aa,yy.bb,zz.cc")
                        .append(" from xx join yy join zz ...")
                        .append(" where param1=:param1")
                        .append(" where param2=:param2")
                        .append(" where param3=:param3")
                        .append("...");

        Query selectQuery = em.createNativeQuery(selectSql.toString());
        selectQuery.setParameter("xx", xx);

        List<XxxRespVo> list = selectQuery.unwrap(SQLQuery.class)
                .addScalar("param1", StandardBasicTypes.STRING)
                .addScalar("param2", StandardBasicTypes.STRING)
                .addScalar("param3", StandardBasicTypes.LONG)
                .setResultTransformer(Transformers.aliasToBean(XxxRespVo.class))
                .setFirstResult(pageable.getPageNumber() * pageable.getPageSize())
                .setMaxResults(pageable.getPageSize())
                .list();
        return new PageImpl<>(list, pageable, total);
    }
}

用單元測試調該查詢方法成功,在測試環境跑,結果一會兒前端界面調該服務的其它都變慢了,查看日志發現了大量異常:

|ERROR|2020-10-27 16:30:21.811|DubboServerHandler-192.168.x.x:20001-thread-461|c.a.d.r.f.ExceptionFilter:85
      [DUBBO] Got unchecked and undeclared exception which called by 192.168.x.x. service: com.xxx.XxxService, method: findXxx, exception: org.springframework.orm.jpa.JpaSystemException: Unable to acquire JDBC Connection; nested exception is org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection, dubbo version: 2.6.2, current host: 192.168.x.x

org.springframework.orm.jpa.JpaSystemException: Unable to acquire JDBC Connection; nested exception is org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:314)
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:225)
        at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:417)
        at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59)
        at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
       ...

Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
        at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
       ...

Caused by: com.alibaba.druid.pool.GetConnectionTimeoutException: wait millis 60000, active 50, maxActive 50
        at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1139)
        at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:960)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:940)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:930)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:102)
        at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
        at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:386)
        at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:84)
        ... 85 common frames omitted  

注意到Unable to acquire JDBC ConnectionGetConnectionTimeoutException,獲取數據庫連接池超時了。
測試環境的jdbc連接池配置如下:

<jdbc.initialSize>2</jdbc.initialSize>
<jdbc.minIdle>2</jdbc.minIdle>
<jdbc.maxActive>50</jdbc.maxActive>
<jdbc.maxWait>60000</jdbc.maxWait>

最大活躍鏈接數50,等待時間60秒,跟日志中的異常信息對應。

推測可能是連接沒有關閉,改小maxActive值,用單元測試循環去調,很快復現出了測試環境的報錯異常。

檢查發現,這里使用了自定義查詢,但缺少事務的注解,連接沒有自動關閉。

解決方法:加上spring的事務注解,因為是查詢,設置為只讀能一定程度提高性能。

@Transactional(readOnly = true)
@Override
public Page<XxxRespVo> findSimpleDepotProductList(XxxReqVo reqVo, Pageable pageable) {
...
}   

參考:
spring-data-jpa官方文檔:https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.custom-implementations


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM