Oracle死鎖問題及解決辦法

死鎖通常是2個及以上線程共同競爭同一資源而造成的一種互相等待的僵局。

我們看下圖所示場景： 線程1執行的事務先更新資源1，然后更新資源2；而線程2涉及到的事務先更新資源2，然后更新資源1。

這種情況下，很容易出現你等我我等你，導致死鎖。

 

我用Oracle數據庫來模擬這種場景的死鎖。

●service類

如下PayAccountServiceMock類， up方法和up2方法，這2個方法使用了spring事務，邏輯是根據賬戶id來更新兩條賬戶的金額。不過，兩個方法更新兩條賬戶記錄的順序是相反的。我們用后面的testcase很容易就能模擬出Oracle死鎖。

package com.xxx.accounting;

import org.springframework.transaction.annotation.Transactional;

@Service
@Slf4j
public class PayAccountServiceMock {
    @Autowired
    private TAccTransService tAccTransService;

    @Transactional
    public void up() throws InterruptedException {
        tAccTransService.updateBalance("89900000426016346075");

        Thread.sleep(RandomUtils.nextInt(100, 300));
        select("89900000426016346075");

        tAccTransService.updateBalance("PF00060");
    }

    @Transactional
    public void up2(TAccTrans at4) throws InterruptedException {
        tAccTransService.updateBalance("PF00060");

        Thread.sleep(550);

        tAccTransService.updateBalance("89900000426016346075");
    }

    @Transactional
    public void select(String id) {

        tAccTransService.selectByPrimaryKey(id);
        try {
            Thread.sleep(1100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

 

●testcase類

如下Junit測試類，使用倒計數門栓（CountDownLatch，就是JUC包下倒計時門栓，個人覺得用“倒計數門栓”感覺更合適~）來保證多線程同時執行，達到並行處理的效果。

package com.xxx.accounting;


@Slf4j
public class PayAccountingServiceTest extends BaseTest {
    @Autowired
    private PayAccountServiceMock payAccountingServiceMock;
    
    @Test
    public void testDeadlock() throws InterruptedException {
        CountDownLatch latch = new CountDownLatch(1);
        ExecutorService pool = Executors.newFixedThreadPool(3);

        // first
        pool.execute(() -> {
                    try {
                        latch.await();
                        log.info("thread begin"); 
                        
                        payAccountingServiceMock.up();
                    } catch (Exception e) {
                        log.error("-----------異常：", e);
                    }
                }
        );
        // second
        pool.execute(() -> {
                    try {
                        latch.await();
                        log.info("thread begin"); 
                        
                        payAccountingServiceMock.up2();
                    } catch (Exception e) {
                        log.error("-----------異常：", e);
                    }
                }
        );
        // third
        pool.execute(() -> {
                    try {
                        latch.await();
                        log.info("thread begin"); 
                        
                        payAccountingServiceMock.select();
                    } catch (Exception e) {
                        log.error("-----------異常：", e);
                    }
                }
        );

        Thread.sleep(100);
        latch.countDown();
        pool.awaitTermination(5, TimeUnit.SECONDS);
    }
}

 

●運行testcase

接下來，運行testcase，出現“ORA-00060: 等待資源時檢測到死鎖”。

org.springframework.dao.DeadlockLoserDataAccessException:
### Error updating database. Cause: java.sql.SQLException: ORA-00060: 等待資源時檢測到死鎖

具體日志如下：

13:50:40,297 [pool_5_thread_1] [com.xxx.accounting.PayAccountingServiceTest:114] thread await
13:50:40,297 [pool_5_thread_3] [com.xxx.accounting.PayAccountingServiceTest:160] thread await
13:50:40,297 [pool_5_thread_2] [com.xxx.accounting.PayAccountingServiceTest:141] thread await
13:50:40,482 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] ==>  Preparing: select * from T_ACC_TRANS where ID = ? 
13:50:40,483 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] ==> Parameters: PF00060(String)
13:50:40,525 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] <==      Total: 1
13:50:40,636 [pool_5_thread_1] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==>  Preparing: update T_ACC_TRANS set ...  where ID = ? ... 
13:50:40,638 [pool_5_thread_1] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==> Parameters: ... , 89900000386316297067(String), ... 
13:50:40,698 [pool_5_thread_1] [com.xxx.dao.TAccTransDAO.updateBalance:145] <==    Updates: 1
13:50:41,658 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==>  Preparing: update T_ACC_TRANS set ...  where ID = ? ... 
13:50:41,660 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==> Parameters: ... , PF00060(String), ... 
13:50:41,668 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] <==    Updates: 1
13:50:45,705 [pool_5_thread_1] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==>  Preparing: update T_ACC_TRANS set ...  where ID = ? ... 
13:50:45,707 [pool_5_thread_1] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==> Parameters: ... , PF00060(String), ... 
13:50:46,680 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==>  Preparing: update T_ACC_TRANS set ...  where ID = ? ... 
13:50:46,681 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==> Parameters: ... , 89900000386316297067(String), ... 
13:50:49,194 [pool_5_thread_1] [org.springframework.beans.factory.xml.XmlBeanDefinitionReader:317] Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml]
13:50:49,247 [pool_5_thread_1] [org.springframework.jdbc.support.SQLErrorCodesFactory:126] SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase, Hana]
13:50:49,262 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] <==    Updates: 1
13:50:49,272 [pool_5_thread_1] [com.xxx.accounting.PayAccountingServiceTest:121] -----------異常：
org.springframework.dao.DeadlockLoserDataAccessException: 
### Error updating database.  Cause: java.sql.SQLException: ORA-00060: 等待資源時檢測到死鎖

### The error may involve com.xxx.dao.TAccTransDAO.updateBalance-Inline
### The error occurred while setting parameters
### SQL: update T_ACC_TRANS set CASH_AMT =CASH_AMT + ?,     CASH_FREEZE =CASH_FREEZE+ ?,     MANUAL_FREEZE=MANUAL_FREEZE + ?,     SEQ = SEQ+1,     UPDATE_TIME = sysdate,     mac=MD5(ID||(CASH_AMT+?)||(CASH_FREEZE + ?)|| (MANUAL_FREEZE+ ?)||TO_CHAR(sysdate,'YYYY-MM-DD HH24:MI:SS'))     where ID = ?     and (MAC=MD5(ID||CASH_AMT||CASH_FREEZE||MANUAL_FREEZE||TO_CHAR(UPDATE_TIME,'YYYY-MM-DD HH24:MI:SS')) or MAC IS NULL)     and CASH_AMT + ? >= 0     and CASH_FREEZE + ? >= 0     and MANUAL_FREEZE + ? >= 0     and CASH_AMT >=CASH_FREEZE+?     and STATE in (0, 2) and ACCOUNT_TYPE in(0,1,2)             and (BANK_ID = ' ' or BANK_ID = ?)
### Cause: java.sql.SQLException: ORA-00060: 等待資源時檢測到死鎖

; SQL []; ORA-00060: 等待資源時檢測到死鎖
; nested exception is java.sql.SQLException: ORA-00060: 等待資源時檢測到死鎖

    at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:263)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
    at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:75)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:447)
    at com.sun.proxy.$Proxy30.update(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.update(SqlSessionTemplate.java:295)
    at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:62)
    at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
    at com.sun.proxy.$Proxy38.updateBalance(Unknown Source)
    at com.xxx.accounting.PayAccountingService.up(PayAccountingService.java:669)
    at com.xxx.accounting.PayAccountingService$$FastClassBySpringCGLIB$$7c2d7604.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:280)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
    at com.xxx.accounting.PayAccountingService$$EnhancerBySpringCGLIB$$a9c6994a.up(<generated>)
    at com.xxx.accounting.PayAccountingServiceTest.lambda$pTest$3(PayAccountingServiceTest.java:116)
    at com.xxx.accounting.PayAccountingServiceTest$$Lambda$77/1402979793.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: ORA-00060: 等待資源時檢測到死鎖

    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
    at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059)
    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)
    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
    at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
    at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:225)
    at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:53)
    at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:943)
    at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1150)
    at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:4798)
    at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:4901)
    at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1385)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:198)
    at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:198)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.ibatis.logging.jdbc.PreparedStatementLogger.invoke(PreparedStatementLogger.java:59)
    at com.sun.proxy.$Proxy78.execute(Unknown Source)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:46)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
    at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
    at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
    at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
    ... 21 more
13:50:49,275 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] ==>  Preparing: select * from T_ACC_TRANS where ID = ? 
13:50:49,276 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] ==> Parameters: PF00060(String)
13:50:49,283 [pool_5_thread_3] [com.xxx.dao.TAccTransDAO.selectByPrimaryKey:145] <==      Total: 1
13:50:49,389 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==>  Preparing: update T_ACC_TRANS set ...  where ID = ? ... 
13:50:49,390 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] ==> Parameters: ... , PF00060(String), ... 
13:50:49,396 [pool_5_thread_2] [com.xxx.dao.TAccTransDAO.updateBalance:145] <==    Updates: 1

 

 死鎖解決辦法

1. 很顯然，調整各事務所執行的資源操作的順序，讓各操作按照相同的順序執行。

2. 實際情況中，就拿我們的系統來說，系統業務比較復雜，並不像上面service里那樣簡單明了，一眼就可以看到問題。而是許多業務（充值、付款請求、調賬、付款完成）都操作原子性的動賬方法，這時，梳理起來也是比較耗費時間和精力的。此時呢，我們采用了利用redis分布式鎖來保證線程（進程）同步。具體來說，就是同時只有一個線程來更改同一賬戶的數據記錄，此時其他線程將等待，直到分布式鎖得到釋放。