關於JDBC的批量操作executeBatch()所引發sql語句異常

java.sql.BatchUpdateException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('1512144017', 'quqiang01' at line 1
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.Util.getInstance(Util.java:408) at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1162) at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1587) at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1253) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970) Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('1512144017', 'quqiang01' at line 1
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.Util.getInstance(Util.java:408) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:943) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2487) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2079) at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2013) at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5104) at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1548) ... 5 more

 

最近在使用JDBC的時候,一個比較坑的細節,就是關於他里面使用PreparedStatement或者Statement 的 addBatch()/executeBatch()的具體實現問題;

不要手賤在你傳入的sql語句沒末尾加上分號;

 

具體是這樣:

//以一個批量提交的實現工具舉例
// List<List<Object>> lists: 所有數據的list
// List<Object>:單條數據"值"的list

public void batchExecutePstamt(String sql, List < List < Object >> lists) {
    try {
        connection.setAutoCommit(false);
        pstmt = connection.prepareStatement(sql);
        if (lists != null && !lists.isEmpty()) {
            for (List < Object > cList: lists) {
                if (cList == null || cList.isEmpty())
                    continue;

                for (int i = 0; i < cList.size(); i++) {
                    pstmt.setObject(i + 1, cList.get(i));
                }
                pstmt.addBatch();
            }
            log.info(pstmt.toString());
            pstmt.executeBatch();
            connection.commit();
        }
    } catch (SQLException e) {
        e.printStackTrace();
    }
}

在這里,我們傳入大量的需要插入的對象的List<List<Object>>,里面的 List<Object> 就是某一條具體的記錄的值;

 

如果你調用的時候, 在傳入sql的時候, 傳入了類似於  insert into `tablename` (name,age) values (?,?); 的sql語句
那么他每次addBatch的時候就會在
com.mysql.jdbc.PreparedStatement.addBatch()的創建com.mysql.jdbc.PreparedStatement.BatchParams 然后放進batchedArgs 靜態列表里面

 

 public void addBatch() throws SQLException {
        synchronized (checkClosed().getConnectionMutex()) {
            if (this.batchedArgs == null) {
                this.batchedArgs = new ArrayList<Object>();
            }

            for (int i = 0; i < this.parameterValues.length; i++) {
                checkAllParametersSet(this.parameterValues[i], this.parameterStreams[i], i);
            }

            this.batchedArgs.add(new BatchParams(this.parameterValues, this.parameterStreams, this.isStream, this.streamLengths, this.isNull));
        }
    }

 

 public class BatchParams {
        public boolean[] isNull = null;

        public boolean[] isStream = null;

        public InputStream[] parameterStreams = null;

        public byte[][] parameterStrings = null;

        public int[] streamLengths = null;

        BatchParams(byte[][] strings, InputStream[] streams, boolean[] isStreamFlags, int[] lengths, boolean[] isNullFlags) {
            //
            // Make copies
            //
            this.parameterStrings = new byte[strings.length][];
            this.parameterStreams = new InputStream[streams.length];
            this.isStream = new boolean[isStreamFlags.length];
            this.streamLengths = new int[lengths.length];
            this.isNull = new boolean[isNullFlags.length];
            System.arraycopy(strings, 0, this.parameterStrings, 0, strings.length);
            System.arraycopy(streams, 0, this.parameterStreams, 0, streams.length);
            System.arraycopy(isStreamFlags, 0, this.isStream, 0, isStreamFlags.length);
            System.arraycopy(lengths, 0, this.streamLengths, 0, lengths.length);
            System.arraycopy(isNullFlags, 0, this.isNull, 0, isNullFlags.length);
        }
    }

PreparedStatement batchedStatement = null;

 

然后會在最后使用executeBatch()的時候處理 batchedArgs中的 數據

處理的步驟為:
>> com.mysql.jdbc.StatementImpl.executeBatch()
>> com.mysql.jdbc.PreparedStatement.executeBatchInternal()
這里回去判斷你設置的一些參數, 比如connection.getRewriteBatchedStatements(), 這個有助於批量數據處理的參數, 需要在jdbc.url連接中設置 rewriteBatchedStatements = true, 不過好像只針對於某個版本(5.2 ?不確定 )以后的數據操作性能有幫助.
在這里會進入一個將多個PreparedStatement轉化為 BLUK 模式的一條語句;(bluk模式不知道的請自行百度)

 

>> com.mysql.jdbc.PreparedStatement.executeBatchedInserts(int)

這里面有一段源碼:

PreparedStatement batchedStatement = null;

int batchedParamIndex = 1;
long updateCountRunningTotal = 0;
int numberToExecuteAsMultiValue = 0;
int batchCounter = 0;
CancelTask timeoutTask = null;
SQLException sqlEx = null;

long[]updateCounts = new long[numBatchedArgs];
//上面不多說

try {
    //這句會把初始化我們傳進來的sql
    //我們傳進來的是類似於 insert into  tablename (columns…) values (?,?...)這種
    //到這里相當於初始化成 INSERT INTO tablename` (`name`, `age`) values (** NOT SPECIFIED **, ** NOT SPECIFIED **);
    //然后他會根據順序傳進我們上面所傳進的多個已經賦值的PreparedStatement,
    batchedStatement = /* FIXME -if we ever care about folks proxying our MySQLConnection */
        prepareBatchedInsertSQL(locallyScopedConn, numValuesPerBatch);

    if (locallyScopedConn.getEnableQueryTimeouts() && batchTimeout != 0 && locallyScopedConn.versionMeetsMinimum(5, 0, 0)) {
        timeoutTask = new CancelTask(batchedStatement);
        locallyScopedConn.getCancelTimer().schedule(timeoutTask, batchTimeout);
    }

    if (numBatchedArgs < numValuesPerBatch) {
        numberToExecuteAsMultiValue = numBatchedArgs;
    } else {
        numberToExecuteAsMultiValue = numBatchedArgs / numValuesPerBatch;
    }

    int numberArgsToExecute = numberToExecuteAsMultiValue * numValuesPerBatch;

    for (int i = 0; i < numberArgsToExecute; i++) {
        if (i != 0 && i % numValuesPerBatch == 0) {
            try {
                updateCountRunningTotal += batchedStatement.executeLargeUpdate();
            } catch (SQLException ex) {
                sqlEx = handleExceptionForBatch(batchCounter - 1, numValuesPerBatch, updateCounts, ex);
            }

            getBatchedGeneratedKeys(batchedStatement);
            batchedStatement.clearParameters();
            batchedParamIndex = 1;

        }
        //@see 這里是關鍵 48         //@see 如果還有后續的PreparedStatement,根據后面的一個參數,那么他就會將batchedStatement設置為:
        // INSERT INTO tablename` (`name`, `age`) values("名字1",'23'), (** NOT SPECIFIED **, ** NOT SPECIFIED **); 50 
        //@see 問題就在這里 52         //@see 如果你傳入的sql :" insert into  tablename (columns…) values (?,?...);"是這樣的,那么生成的單條語句以及單個PreparedStatement沒有任何問題,你拿到數據庫執行去也OK
        //@see 但是他現在要給你轉為BLUK模式的,如果你結尾帶了分號,語句就會變成:
        // INSERT INTO tablename` (`name`, `age`) values("名字1",'23');, (** NOT SPECIFIED **, ** NOT SPECIFIED **);然后等待下一次填充參數
        //@see 多的這個分號就是導致sql語句錯誤的原因 56 

        //@see 有興趣可以進源碼的這個方法看看
        batchedParamIndex = setOneBatchedParameterSet(batchedStatement, batchedParamIndex, this.batchedArgs.get(batchCounter++));
    }

    try {
        updateCountRunningTotal += batchedStatement.executeLargeUpdate();
    } catch (SQLException ex) {
        sqlEx = handleExceptionForBatch(batchCounter - 1, numValuesPerBatch, updateCounts, ex);
    }

    getBatchedGeneratedKeys(batchedStatement);

    numValuesPerBatch = numBatchedArgs - batchCounter;
}
finally {
    if (batchedStatement != null) {
        batchedStatement.close();
        batchedStatement = null;
    }
}