cursor 與 insert ...select 對比：
cursor：安全，不會造成死鎖，可以在服務運行階段跑,比較穩定。
insert...select :速度快，但是可能造成死鎖，相比cursor能夠成倍提升，在服務停止的情況下遷移,速度快
數據遷移案例：
首先數據的遷移絕對不是一朝一夕能夠快速遷移完成的 ，如果可以很快完成的 dump便可以搞定，沒必要大費周折了。
既然不是一朝一夕能完成的，那么有關鍵的日志記錄表能夠良好的反應數據遷移的過程
遷移日志表腳本：
DROP TABLE IF EXISTS `cx_delete_log`; CREATE TABLE `cx_delete_log` ( `id` int(30) NOT NULL, `table_name` varchar(30) DEFAULT NULL, `start_tm` datetime DEFAULT NULL, `end_tm` datetime DEFAULT NULL, `status` int(10) DEFAULT NULL, `pro_create_time` datetime DEFAULT NULL, `pro_end_time` datetime DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -- ---------------------------- -- Records of cx_delete_log -- ---------------------------- INSERT INTO `cx_delete_log` VALUES ('1', 'cx_waybill', '2017-01-01 00:00:00', '2017-01-01 00:10:00', '1', '2017-03-21 14:06:21', null); ALTER TABLE `cx_delete_log` ADD INDEX `index_end_date` (`end_tm`) USING BTREE ;
首先遷移的增量字段確定。這里存在一個最優解，即你每分鍾的數據量，與你mysql的性能，我們可以在此之間尋找到一個平衡點（cursor與insert...select並不相同），設置每次遷移多次時間段的數據，這樣能，最大效率的利用我們數據庫的性能。
我們插入了一條日志數據，有這個來控制每次數據遷移的間隔時間段，與開始時間
如：2017-01-01 00:00:00 start_tm數據遷移增量字段的開始時間，2017-01-01 00:10:00 end_tm數據增量字段遷移的結束時間，時間間隔10分鍾。
狀態為1代表成功，2017-03-21 14:06:21 代表執行這段時間數據遷移的執行時間，pro_end_time代表執行這段時間數據遷移的執行完畢時間時間。
存儲過程：
存儲過程分為兩個部分，1.日志的寫入2.遷移操作的進行
日志寫入：
delimiter | drop procedure if exists batch_move_cx_waybill_partition_by_minute; create procedure batch_move_cx_waybill_partition_by_minute() begin DECLARE proNo INTEGER; DECLARE startDate TIMESTAMP; DECLARE endDate TIMESTAMP; DECLARE nextDate TIMESTAMP; DECLARE pro_endDate TIMESTAMP; SET pro_endDate='2017-03-20 00:00:00'; REPEAT SELECT ID, end_tm,DATE_ADD(end_tm,INTERVAL '00:10:00' hour_second) INTO proNo,startDate ,endDate from cx_delete_log ORDER BY end_tm desc limit 1; IF curtime()>'06:00:00' THEN SET startDate =pro_endDate; END IF; IF startDate <pro_endDate THEN INSERT into cx_delete_log(id,table_name,start_tm,end_tm,status,pro_create_time)VALUES(proNo+1,'cx_waybill',startDate,endDate,'0',SYSDATE()); CALL cursor_move_cx_waybill_partition(startDate); update cx_delete_log set pro_end_time =SYSDATE() , `status`=1 where id=proNo+1; COMMIT; END IF; UNTIL startDate >=pro_endDate END REPEAT; end | delimiter ;
遷移操作的進行：
實例為cursor，insert..select較為簡單，直接帶入就行，就不做示例了
delimiter || drop procedure if exists cursor_move_cx_waybill_partition; create procedure cursor_move_cx_waybill_partition(IN startDate_tmp TIMESTAMP) begin DECLARE isEXist INTEGER; DECLARE WAYBILLNO_TMP VARCHAR(32); DECLARE RESOURCECODE_TMP VARCHAR(10); DECLARE APPOINTMENTDDELIVERYTIME_TMP TIMESTAMP; DECLARE APPOINTMENTDDELIVERYENDTIME_TMP TIMESTAMP; DECLARE EXPECTEDDELIVERYTIME_TMP TIMESTAMP; DECLARE ORIGINATEID_TMP VARCHAR(32); DECLARE DESTINATIONID_TMP VARCHAR(32); DECLARE WAYBILLSTATUS_TMP VARCHAR(4); DECLARE VERSIONNO_TMP INTEGER; DECLARE STATUS_TMP CHAR(1); DECLARE FAILSTARTUS_TMP VARCHAR(10); DECLARE FAILCAUSEDESC_TMP VARCHAR(500); DECLARE MEMBERID_TMP VARCHAR(32); DECLARE HIDEFLAG_TMP VARCHAR(4); DECLARE CREATEUSERID_TMP VARCHAR(32); DECLARE CREATETIME_TMP TIMESTAMP; DECLARE UPDATEUSERID_TMP VARCHAR(32); DECLARE UPDATETIME_TMP TIMESTAMP; DECLARE APPOINTMENTNO_TMP VARCHAR(32); DECLARE SOURCECITY_TMP VARCHAR(32); DECLARE DESTCITY_TMP VARCHAR(32); DECLARE SOURCECITYCODE_TMP VARCHAR(20); DECLARE DESTCITYCODE_TMP VARCHAR(20); DECLARE EMPCODE_TMP VARCHAR(10); DECLARE RECEMPCODE_TMP VARCHAR(10); DECLARE RECMEMBERID_TMP VARCHAR(32); DECLARE WAYBILLFEE_TMP DOUBLE(10,0); DECLARE MOBILE_TMP VARCHAR(20); DECLARE RECMOBILE_TMP VARCHAR(20); DECLARE RECTIME_TMP TIMESTAMP; DECLARE ENDTIME_TMP TIMESTAMP; DECLARE EXPECTMEMBERID_TMP VARCHAR(32); DECLARE ORIGINATE_TMP VARCHAR(500); DECLARE DESTINATION_TMP VARCHAR(500); DECLARE PAYTYPE_TMP VARCHAR(4); DECLARE SECRECYTYPE_TMP VARCHAR(32); DECLARE SEND_EVALUATE_TMP VARCHAR(10); DECLARE PRODUCT_TYPE_TMP VARCHAR(32); DECLARE SERVICES_PROD_CODE_TMP VARCHAR(200); DECLARE CHANGERECORD_TMP VARCHAR(32); DECLARE IS_ISSUED_PREFRENCE_TMP VARCHAR(10); DECLARE REC_EVALUATE_TMP VARCHAR(10); DECLARE done INTEGER; DECLARE bill_cursor CURSOR for SELECT WAYBILLNO, RESOURCECODE , APPOINTMENTDDELIVERYTIME , APPOINTMENTDDELIVERYENDTIME , EXPECTEDDELIVERYTIME ,ORIGINATEID , DESTINATIONID , WAYBILLSTATUS , VERSIONNO , STATUS, FAILSTARTUS, FAILCAUSEDESC, MEMBERID, HIDEFLAG, CREATEUSERID, CREATETIME, UPDATEUSERID, UPDATETIME, APPOINTMENTNO, SOURCECITY, DESTCITY, SOURCECITYCODE, DESTCITYCODE, EMPCODE, RECEMPCODE, RECMEMBERID,WAYBILLFEE, MOBILE, RECMOBILE, RECTIME, ENDTIME, EXPECTMEMBERID, ORIGINATE, DESTINATION, PAYTYPE, SECRECYTYPE, SEND_EVALUATE, PRODUCT_TYPE, SERVICES_PROD_CODE, CHANGERECORD, IS_ISSUED_PREFRENCE, REC_EVALUATE FROM cx_waybill WHERE CREATETIME > startDate_tmp AND CREATETIME <= DATE_ADD(startDate_tmp,INTERVAL '00:10:00' hour_second); DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1; open bill_cursor; loop_bill:LOOP IF done = 1 THEN LEAVE loop_bill; END IF; FETCH bill_cursor INTO WAYBILLNO_TMP, RESOURCECODE_TMP, APPOINTMENTDDELIVERYTIME_TMP, APPOINTMENTDDELIVERYENDTIME_TMP,EXPECTEDDELIVERYTIME_TMP,ORIGINATEID_TMP, DESTINATIONID_TMP, WAYBILLSTATUS_TMP, VERSIONNO_TMP, STATUS_TMP, FAILSTARTUS_TMP, FAILCAUSEDESC_TMP, MEMBERID_TMP, HIDEFLAG_TMP, CREATEUSERID_TMP, CREATETIME_TMP, UPDATEUSERID_TMP, UPDATETIME_TMP, APPOINTMENTNO_TMP, SOURCECITY_TMP, DESTCITY_TMP, SOURCECITYCODE_TMP, DESTCITYCODE_TMP, EMPCODE_TMP, RECEMPCODE_TMP, RECMEMBERID_TMP,WAYBILLFEE_TMP, MOBILE_TMP, RECMOBILE_TMP, RECTIME_TMP, ENDTIME_TMP, EXPECTMEMBERID_TMP, ORIGINATE_TMP, DESTINATION_TMP, PAYTYPE_TMP, SECRECYTYPE_TMP, SEND_EVALUATE_TMP, PRODUCT_TYPE_TMP, SERVICES_PROD_CODE_TMP, CHANGERECORD_TMP, IS_ISSUED_PREFRENCE_TMP, REC_EVALUATE_TMP; IF WAYBILLNO_TMP is NULL THEN LEAVE loop_bill; END IF; SELECT count(1) INTO isEXist from cx_waybill_partition w where w.WAYBILLNO=WAYBILLNO_TMP; IF isEXist >0 THEN LEAVE loop_bill; ELSE INSERT INTO cx_waybill_PARTITION (WAYBILLNO, RESOURCECODE, APPOINTMENTDDELIVERYTIME, APPOINTMENTDDELIVERYENDTIME,EXPECTEDDELIVERYTIME,ORIGINATEID, DESTINATIONID, WAYBILLSTATUS, VERSIONNO, STATUS, FAILSTARTUS, FAILCAUSEDESC, MEMBERID, HIDEFLAG, CREATEUSERID, CREATETIME, UPDATEUSERID, UPDATETIME, APPOINTMENTNO, SOURCECITY, DESTCITY, SOURCECITYCODE, DESTCITYCODE, EMPCODE, RECEMPCODE, RECMEMBERID,WAYBILLFEE, MOBILE, RECMOBILE, RECTIME, ENDTIME, EXPECTMEMBERID, ORIGINATE, DESTINATION, PAYTYPE, SECRECYTYPE, SEND_EVALUATE, PRODUCT_TYPE, SERVICES_PROD_CODE, CHANGERECORD, IS_ISSUED_PREFRENCE, REC_EVALUATE)VALUES(WAYBILLNO_TMP, RESOURCECODE_TMP, APPOINTMENTDDELIVERYTIME_TMP, APPOINTMENTDDELIVERYENDTIME_TMP,EXPECTEDDELIVERYTIME_TMP,ORIGINATEID_TMP, DESTINATIONID_TMP, WAYBILLSTATUS_TMP, VERSIONNO_TMP, STATUS_TMP, FAILSTARTUS_TMP, FAILCAUSEDESC_TMP, MEMBERID_TMP, HIDEFLAG_TMP, CREATEUSERID_TMP, CREATETIME_TMP, UPDATEUSERID_TMP, UPDATETIME_TMP, APPOINTMENTNO_TMP, SOURCECITY_TMP, DESTCITY_TMP, SOURCECITYCODE_TMP, DESTCITYCODE_TMP, EMPCODE_TMP, RECEMPCODE_TMP, RECMEMBERID_TMP,WAYBILLFEE_TMP, MOBILE_TMP, RECMOBILE_TMP, RECTIME_TMP, ENDTIME_TMP, EXPECTMEMBERID_TMP, ORIGINATE_TMP, DESTINATION_TMP, PAYTYPE_TMP, SECRECYTYPE_TMP, SEND_EVALUATE_TMP, PRODUCT_TYPE_TMP, SERVICES_PROD_CODE_TMP, CHANGERECORD_TMP, IS_ISSUED_PREFRENCE_TMP, REC_EVALUATE_TMP); END IF; end Loop; close bill_cursor; COMMIT; end || delimiter ;
定時器的創建：
每天凌晨一點執行：
---輪循刪除定時器 drop event if exists waybill_move_event; create event waybill_move_event on schedule every 1 DAY STARTS '2017-01-17 01:00:00' on completion preserve ENABLE do call batch_move_cx_waybill_partition_by_minute(); alter event waybill_move_event on completion preserve enable;
其實，在中間踩過一些坑，但是很多都是百度都能解決的 。事實證明，這樣的數據遷移，其速度並不慢，而且性能相對穩定，在后續的觀察中，並未對數據庫造成太多的壓力，可看作種螞蟻搬家。