JPA中以HibernatePersistence為provider的批量插入問題

為什么要批量插入

要插入10000條數據，如果不批量插入的話，那么我們執行的sql語句將是10000條insert

insert into member (group_id, user_id, role, extend) values (101, 100, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 101, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 102, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 103, 3, NULL)

......

如果是批量插入，一批次插入50條，那么10000條數據將生成200條insert語句

insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)

insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)

好了，以上是jdbc級別的批量插入，換言之，我們需要JPA提供批量插入功能

 

JPA批量插入

搜索了Stackoverflow，似乎JPA本身並沒有對批量插入的支持

http://stackoverflow.com/questions/7440397/how-to-do-the-batch-insert-in-jpa

 

Hibernate批量插入

因為使用Hibernate作為JPA的provider實現，因此我們可以使用Hibernate提供的批量插入功能

查閱hibernate文檔

http://docs.jboss.org/hibernate/core/3.3/reference/en/html/batch.html

(1) 首先要在hibernate配置文件中配置屬性

<property name="hibernate.jdbc.batch_size" value="20" /> <!-- value文檔建議在10-50間 -->

(2) 批量插入代碼示例

for ( int i=0; i<100000; i++ ) {
    Customer customer = new Customer(.....);
    session.save(customer);
    if ( i % 20 == 0 ) { //20, same as the JDBC batch size
        //flush a batch of inserts and release memory:
        session.flush();
        session.clear();
    }
}

 

注意

如果批量插入的實體主鍵為@GeneratedValue(strategy=IDENTITY)類型，則hibernate會透明關閉在jdbc級別的批量插入；因為實際上，每次插入該實體的一個實例，hibernate都需要搜索該實例的主鍵；想想在代碼中，我們persist實例之后，就能夠通過get方法獲取實例的主鍵；

實驗結果也說明了這點

實體定義

@Entity
@Table (name="member")
public class GroupUser {
　　@Id
　　@GeneratedValue(strategy=IDENTITY)
　　@Column(name="id", nullable=true, unique=true)
　　private Integer id;

　　......

}

 

實驗代碼

Session session = (Session)em.getDelegate();
session.setFlushMode(FlushMode.MANUAL);

for (int i = 0; i < userIds.size(); ++i) {
　　session.save(new GroupUser(groupId, userIds.get(i), role));
　　if (i % 20 == 0) {
　　　　session.flush();
　　　　session.clear();
　　}
}

 

日志輸出

Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
......