关于 JDBC 的批量操作 executeBatch() 所引发 sql 语句异常

关于 JDBC 的批量操作 executeBatch() 所引发 sql 语句异常

java.sql.BatchUpdateException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('1512144017', 'quqiang01' at line 1
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.Util.getInstance(Util.java:408)
    at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1162)
    at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1587)
    at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1253)
    at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '('1512144017', 'quqiang01' at line 1
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.Util.getInstance(Util.java:408)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:943)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2487)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858)
    at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2079)
    at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2013)
    at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5104)
    at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1548)
    ... 5 more

最近在使用 JDBC 的时候, 一个比较坑的细节, 就是关于他里面使用 PreparedStatement 或者 Statement 的 addBatch()/executeBatch() 的具体实现问题;

不要手贱在你传入的 sql 语句没末尾加上分号;

具体是这样:

1 //以一个批量提交的实现工具举例
 2 // List<List<Object>> lists: 所有数据的list
 3 // List<Object>:单条数据"值"的list
 5 public void batchExecutePstamt(String sql, List < List < Object >> lists) {
 6     try {
 7         connection.setAutoCommit(false);
 8         pstmt = connection.prepareStatement(sql);
 9         if (lists != null && !lists.isEmpty()) {
10             for (List < Object > cList: lists) {
11                 if (cList == null || cList.isEmpty())
12                     continue;
14                 for (int i = 0; i < cList.size(); i++) {
15                     pstmt.setObject(i + 1, cList.get(i));
16                 }
17                 pstmt.addBatch();
18             }
19             log.info(pstmt.toString());
20             pstmt.executeBatch();
21             connection.commit();
22         }
23     } catch (SQLException e) {
24         e.printStackTrace();
25     }
26 }

在这里, 我们传入大量的需要插入的对象的 List<List<Object>>, 里面的 List<Object> 就是某一条具体的记录的值;

如果你调用的时候, 在传入 sql 的时候, 传入了类似于  insert into `tablename` (name,age) values (?,?); 的 sql 语句
那么他每次 addBatch 的时候就会在
com.mysql.jdbc.PreparedStatement.addBatch() 的创建 com.mysql.jdbc.PreparedStatement.BatchParams 然后放进 batchedArgs 静态列表里面

1  public void addBatch() throws SQLException {
 2         synchronized (checkClosed().getConnectionMutex()) {
 3             if (this.batchedArgs == null) {
 4                 this.batchedArgs = new ArrayList<Object>();
 5             }
 7             for (int i = 0; i < this.parameterValues.length; i++) {
 8                 checkAllParametersSet(this.parameterValues[i], this.parameterStreams[i], i);
 9             }
11             this.batchedArgs.add(new BatchParams(this.parameterValues, this.parameterStreams, this.isStream, this.streamLengths, this.isNull));
12         }
13     }

addBatch()

1  public class BatchParams {
 2         public boolean[] isNull = null;
 4         public boolean[] isStream = null;
 6         public InputStream[] parameterStreams = null;
 8         public byte[][] parameterStrings = null;
10         public int[] streamLengths = null;
12         BatchParams(byte[][] strings, InputStream[] streams, boolean[] isStreamFlags, int[] lengths, boolean[] isNullFlags) {
13             //
14             // Make copies
15             //
16             this.parameterStrings = new byte[strings.length][];
17             this.parameterStreams = new InputStream[streams.length];
18             this.isStream = new boolean[isStreamFlags.length];
19             this.streamLengths = new int[lengths.length];
20             this.isNull = new boolean[isNullFlags.length];
21             System.arraycopy(strings, 0, this.parameterStrings, 0, strings.length);
22             System.arraycopy(streams, 0, this.parameterStreams, 0, streams.length);
23             System.arraycopy(isStreamFlags, 0, this.isStream, 0, isStreamFlags.length);
24             System.arraycopy(lengths, 0, this.streamLengths, 0, lengths.length);
25             System.arraycopy(isNullFlags, 0, this.isNull, 0, isNullFlags.length);
26         }
27     }
29 PreparedStatement batchedStatement = null;

BatchParams

然后会在最后使用 executeBatch() 的时候处理 batchedArgs 中的 数据

处理的步骤为:
>> com.mysql.jdbc.StatementImpl.executeBatch()
>> com.mysql.jdbc.PreparedStatement.executeBatchInternal()
这里回去判断你设置的一些参数, 比如 connection.getRewriteBatchedStatements(), 这个有助于批量数据处理的参数, 需要在 jdbc.url 连接中设置 rewriteBatchedStatements = true, 不过好像只针对于某个版本 (5.2 ? 不确定) 以后的数据操作性能有帮助.
在这里会进入一个将多个 PreparedStatement 转化为 BLUK 模式的一条语句;(bluk 模式不知道的请自行百度)

>> com.mysql.jdbc.PreparedStatement.executeBatchedInserts(int)

这里面有一段源码:

1 PreparedStatement batchedStatement = null;
 3 int batchedParamIndex = 1;
 4 long updateCountRunningTotal = 0;
 5 int numberToExecuteAsMultiValue = 0;
 6 int batchCounter = 0;
 7 CancelTask timeoutTask = null;
 8 SQLException sqlEx = null;
10 long[]updateCounts = new long[numBatchedArgs];
11 //上面不多说
13 try {
14     //这句会把初始化我们传进来的sql
15     //我们传进来的是类似于 insert into  tablename (columns…) values (?,?...)这种
16     //到这里相当于初始化成 INSERT INTO tablename` (`name`, `age`) values (** NOT SPECIFIED **, ** NOT SPECIFIED **);
17     //然后他会根据顺序传进我们上面所传进的多个已经赋值的PreparedStatement,
18     batchedStatement = /* FIXME -if we ever care about folks proxying our MySQLConnection */
19         prepareBatchedInsertSQL(locallyScopedConn, numValuesPerBatch);
21     if (locallyScopedConn.getEnableQueryTimeouts() && batchTimeout != 0 && locallyScopedConn.versionMeetsMinimum(5, 0, 0)) {
22         timeoutTask = new CancelTask(batchedStatement);
23         locallyScopedConn.getCancelTimer().schedule(timeoutTask, batchTimeout);
24     }
26     if (numBatchedArgs < numValuesPerBatch) {
27         numberToExecuteAsMultiValue = numBatchedArgs;
28     } else {
29         numberToExecuteAsMultiValue = numBatchedArgs / numValuesPerBatch;
30     }
32     int numberArgsToExecute = numberToExecuteAsMultiValue * numValuesPerBatch;
34     for (int i = 0; i < numberArgsToExecute; i++) {
35         if (i != 0 && i % numValuesPerBatch == 0) {
36             try {
37                 updateCountRunningTotal += batchedStatement.executeLargeUpdate();
38             } catch (SQLException ex) {
39                 sqlEx = handleExceptionForBatch(batchCounter - 1, numValuesPerBatch, updateCounts, ex);
40             }
42             getBatchedGeneratedKeys(batchedStatement);
43             batchedStatement.clearParameters();
44             batchedParamIndex = 1;
46         }
47         //@see 这里是关键
48         //@see 如果还有后续的PreparedStatement,根据后面的一个参数,那么他就会将batchedStatement设置为:
49         // INSERT INTO tablename` (`name`, `age`) values("名字1",'23'), (** NOT SPECIFIED **, ** NOT SPECIFIED **);
51         //@see 问题就在这里
52         //@see 如果你传入的sql :" insert into  tablename (columns…) values (?,?...);"是这样的,那么生成的单条语句以及单个PreparedStatement没有任何问题,你拿到数据库执行去也OK
53         //@see 但是他现在要给你转为BLUK模式的,如果你结尾带了分号,语句就会变成:
54         // INSERT INTO tablename` (`name`, `age`) values("名字1",'23');, (** NOT SPECIFIED **, ** NOT SPECIFIED **);然后等待下一次填充参数
55         //@see 多的这个分号就是导致sql语句错误的原因
58         //@see 有兴趣可以进源码的这个方法看看
59         batchedParamIndex = setOneBatchedParameterSet(batchedStatement, batchedParamIndex, this.batchedArgs.get(batchCounter++));
60     }
62     try {
63         updateCountRunningTotal += batchedStatement.executeLargeUpdate();
64     } catch (SQLException ex) {
65         sqlEx = handleExceptionForBatch(batchCounter - 1, numValuesPerBatch, updateCounts, ex);
66     }
68     getBatchedGeneratedKeys(batchedStatement);
70     numValuesPerBatch = numBatchedArgs - batchCounter;
71 }
72 finally {
73     if (batchedStatement != null) {
74         batchedStatement.close();
75         batchedStatement = null;
76     }
77 }

全文完

本文由 简悦 SimpRead 转码，用以提升阅读体验， 原文地址