一、文檔說明
1.0 需求：需要將生產環境PICC用戶導出，在測試環境中docker 測試數據庫導入，只需要表結構；
2.0 思路：根據開發人員提供的需求，數據庫源端aix 11.2.0.4, 目標端11.2.0.3
工具：使用expdp進行導出，impdp進行導入，scp進行傳輸
expdp導出，開並行，只導出元數據，排除JOB等存儲過程對象
scp服務器相關權限端口，賬戶密碼索取
impdp導入前，存儲空間，用戶權限提前准備
二、操作記錄
--第一次導入操作failed
--失敗原因，導入表創建語法DDL失敗，原因
1.導入環境為XE學習環境，數據庫容量最大11g，如果空間足夠，不會報錯
2.導入建表語法，dump文件小於200M，導入建表語句還未插入數據，已占用11G存儲，由於表的STORAGE(INITIAL 65536 NEXT 1048576)參數導致，對每個段都分配了空間，導致創建空表占用11g達到XE閥值報錯
3 .測試11g新特效，延遲段創建對數據泵導入的對象無效，此參照只能影響手工新建的對象
1）資料獲取 數據泵INCLUDE and EXCLUDE對象類型視圖 Export/Import DataPump Parameters INCLUDE and EXCLUDE - How to Load and Unload Specific Objects (Doc ID 341733.1) ~database_export_objects /schema_export_objects /table_export_objects 2)進行導出 SQL> create directory dump as '/home/oracle/tools'; expdp \'/ as sysdba\' directory=dump dumpfile=system%u.dmp logfile=system.log SCHEMAS=system EXCLUDE=SEQUENCE,TRIGGER,REF_CONSTRAINT,CONSTRAINT,PROCDEPOBJ CONTENT=METADATA_ONLY cluster=n parallel --導出元數據，加了並行，但實質只導出一個dump文件,說明導出元數據只能串行導出 --附上導出日志，如果願意，后綴的導出類型，均可過濾，保留表、索引即可，最小化原則處理需求 Starting "SYS"."SYS_EXPORT_SCHEMA_02": "/******** AS SYSDBA" directory=dump dumpfile=system%u.dmp logfile=system.log SCHEMAS=system EXCLUDE=SEQUENCE,TRIGGER,REF_CONSTRAINT,CONSTRAINT,PROCDEPOBJ CONTENT=METADATA_ONLY cluster=n parallel=2 Processing object type SCHEMA_EXPORT/USER Processing object type SCHEMA_EXPORT/SYSTEM_GRANT Processing object type SCHEMA_EXPORT/ROLE_GRANT Processing object type SCHEMA_EXPORT/DEFAULT_ROLE Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA Processing object type SCHEMA_EXPORT/SYNONYM/SYNONYM Processing object type SCHEMA_EXPORT/TYPE/TYPE_SPEC Processing object type SCHEMA_EXPORT/TABLE/TABLE Processing object type SCHEMA_EXPORT/TABLE/PRE_TABLE_ACTION Processing object type SCHEMA_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT Processing object type SCHEMA_EXPORT/TABLE/COMMENT Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC Processing object type SCHEMA_EXPORT/FUNCTION/FUNCTION Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE Processing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC Processing object type SCHEMA_EXPORT/FUNCTION/ALTER_FUNCTION Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS Processing object type SCHEMA_EXPORT/VIEW/VIEW Processing object type SCHEMA_EXPORT/VIEW/GRANT/OWNER_GRANT/OBJECT_GRANT Processing object type SCHEMA_EXPORT/VIEW/COMMENT Processing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS Processing object type SCHEMA_EXPORT/TABLE/POST_TABLE_ACTION Processing object type SCHEMA_EXPORT/POST_SCHEMA/PROCACT_SCHEMA Master table "SYS"."SYS_EXPORT_SCHEMA_02" successfully loaded/unloaded ****************************************************************************** Dump file set for SYS.SYS_EXPORT_SCHEMA_02 is: /home/oracle/tools/system01.dmp Job "SYS"."SYS_EXPORT_SCHEMA_02" successfully completed at Thu Nov 29 14:48:14 2018 elapsed 0 00:00 3）scp 省略 4） 導入操作 --docker 命令進行SQL*Plus docker run -d --name "oradb" -v /home/dmuser/my_oracle_data:/u01/app/oracle IP:10082/sath89/oracle-xe-11g --正常環境，登陸服務器端，即可使用impdp工具，本次場景為docker 有些特殊 --創建表空間 --源端用戶所在表空間 SQL> select tablespace_name from dba_segments where owner='SYSTEM' group by tablespace_name; TABLESPACE_NAME ------------------------------ SYSAUX SYSTEM--目標端創建表空間 crate tablesapce SYSTEM datafile '/u01/app/oracle/oradata/XE/system.dbf' size 10m uniform size 128k autoextend on next 30m maxsize 10g; sqlplus as sysdba create directories abc as 'xxx'; --導入 impdp \'/ as sysdba\' dumpfile=PICCPROD%U.dmp directory=dump logfile=picc.log REMAP_TABLESPACE=LIFELOB_DATA:LIFEDATA_T_L,LIFEINDEX_T_L:LIFEDATA_T_L --導入報錯 ORA-01658: unable to create INITIAL extent for segment in tablespace SYSTEM --對system表空間進行擴容 --再次導入報錯 CREATE TABLE "PICCPROD"."T_POLICY_CHECK_ITEM" ("ITEM_ID" NUMBER(10,0) NOT NULL ENABLE, "CUSTOMER_ID" NUMBER(10,0) NOT NULL ENABLE, "CHECK_NUM" NUMBER(4,0) NOT NULL ENABLE, "NORMAL" CHAR(1 BYTE) DEFAULT 'Y' NOT NULL ENABLE, "NOTES" VARCHAR2(1000 BYTE), "CHARGE_FEE" NUMBER(10,2) DEFAULT 0 NOT NULL ENABLE) PCTFREE 10 PCTUSED 40 IN ORA-39171: Job is experiencing a resumable wait. ORA-12953: The request exceeds the maximum allowed database size of 11 GB --搜索MOS 提示，學習XE類 Express Edition (簡化版本，數據庫允許最大11g) ORA-12953: The request exceeds the maximum allowed database size of 11 GB (Doc ID 2414879.1) [Release 11.2 to 12.2] This is an Oracle Database XE environment, and Oracle Database XE has a limitation of 11 GB of user data 疑問？測試庫怎么有11g這么大？ SQL> select round( sum(bytes)/1024/1024) m from dba_segments where owner not in('PICCPROD'); M ---------- 1708 --導入用戶后 >11g出發XE峰值，報錯 SQL> select sum(bytes)/1024/1024 from dba_segments; SUM(BYTES)/1024/1024 -------------------- 11706.8125 --什么類型的對象占用空間 SQL> select sum(bytes/1024/1024) ,segment_type from dba_segments where owner='PICCPROD' group by owner,segment_type; SUM(BYTES/1024/1024) SEGMENT_TYPE -------------------- ------------------ 9999 TABLE ？dump文件大小182，導入后，表大小9G oracle@c46bf408bfad:~/dmpfile$ ls PICCPROD01.dmp picc.log oracle@c46bf408bfad:~/dmpfile$ du -sm PICCPROD01.dmp 182 PICCPROD01.dmpfile 查詢表數據： select segment_name,segment_type,bytes/1024/1024 from dba_segments where owner='PICCPROD' and segment_type='TABLE' and rownum=1; SEGMENT_NAME SEGMENT_TYPE BYTES/1024/1024 ------------------------------ ------------------ --------------- T_DIAGNOSIS_TYPE TABLE .5 select count(*) from PICCPROD.T_DIAGNOSIS_TYPE; COUNT(*) ---------- 0 查詢表的區的分布 SQL> select tablespace_name,segment_name,EXTENT_ID,BLOCK_ID,BYTES/1024,BLOCKS from dba_extents where owner='PICCPROD' and segment_name='T_DIAGNOSIS_TYPE'; TABLESPACE_NAME SEGMENT_NAME EXTENT_ID BLOCK_ID BYTES/1024 BLOCKS --------------- -------------------- ---------- ---------- ---------- ---------- LIFEDATA_T_L T_DIAGNOSIS_TYPE 0 128 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 1 144 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 2 160 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 3 176 128 16 嘗試對表move（本次不再次對數據文件resize 進行分析，如果需要resize減少數據文件，需要從數據文件max_block_id 進行move，隨后對數據文件進行resize） alter table PICCPROD.T_DIAGNOSIS_TYPE move tablespace LIFEDATA_T_L; * ERROR at line 1: ORA-12953: The request exceeds the maximum allowed database size of 11 GB *Move時，需要存在一倍對象空閑空間，此時空閑空間不足無法Move* *釋放空間，找出max最大對象 select segment_name,sum(blocks) from dba_extents where owner='PICCPROD' group by segment_name order by 2; SEGMENT_NAME SUM(BLOCKS) ------------------------------------------- T_POLICY_AUTO 221312 T_POLICY_PRODUCT 309120 drop table PICCPROD.T_POLICY_AUTO purge; drop table PICCPROD.T_POLICY_PRODUCT purge; alter table PICCPROD.T_DIAGNOSIS_TYPE move tablespace LIFEDATA_T_L; select tablespace_name,segment_name,EXTENT_ID,BLOCK_ID,BYTES/1024,BLOCKS from dba_extents where owner='PICCPROD' and segment_name='T_DIAGNOSIS_TYPE'; TABLESPACE_NAME SEGMENT_NAME EXTENT_ID BLOCK_ID BYTES/1024 BLOCKS -------------------- -------------------- ---------- ---------- ---------- ---------- LIFEDATA_T_L T_DIAGNOSIS_TYPE 0 34720 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 1 34736 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 2 34752 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 3 34768 128 16 *嘗試對數據文件進行resize* select file_id,bytes/1024/1024 from dba_data_files where tablespace_name='LIFEDATA_T_L'; FILE_ID BYTES/1024/1024 ---------- --------------- 5 10000 alter database datafile 5 resize 5000M; * ERROR at line 1: ORA-03297: file contains used data beyond requested RESIZE value --雖然數據為空，但是表對應的段，分配了存儲，占用了存儲 *truncate table SQL> truncate table PICCPROD.T_DIAGNOSIS_TYPE; Table truncated. SQL> select tablespace_name,segment_name,EXTENT_ID,BLOCK_ID,BYTES/1024,BLOCKS from dba_extents where owner='PICCPROD' and segment_name='T_DIAGNOSIS_TYPE'; TABLESPACE_NAME SEGMENT_NAME EXTENT_ID BLOCK_ID BYTES/1024 BLOCKS -------------------- -------------------- ---------- ---------- ---------- ---------- LIFEDATA_T_L T_DIAGNOSIS_TYPE 0 34720 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 1 34736 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 2 34752 128 16 LIFEDATA_T_L T_DIAGNOSIS_TYPE 3 34768 128 16 *數據庫參數，延遲段創建 SQL> show parameter defer NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ deferred_segment_creation boolean TRUE --參數並未關閉--false，但是之前的導入還是創建表，因此延遲段創建參數，對於數據泵導入的對象是不起作用的 *查詢建表語句 --導入 impdp \'/ as sysdba\' dumpfile=PICCPROD%U.dmp directory=dump logfile=picc%U.log sqlfile=sql01.sql ORA-39002: invalid operation ORA-31694: master table "SYS"."SYS_SQL_FILE_FULL_01" failed to load/unload ORA-02354: error in exporting/importing data ORA-39776: fatal Direct Path API error loading table "SYS"."SYS_SQL_FILE_FULL_01" ORA-12953: The request exceeds the maximum allowed database size of 11 GB -- CREATE TABLE "PICCPROD"."T_POLICY_CHECK_ITEM" ("ITEM_ID" NUMBER(10,0) NOT NULL ENABLE, "CUSTOMER_ID" NUMBER(10,0) NOT NULL ENABLE, "CHECK_NUM" NUMBER(4,0) NOT NULL ENABLE, "NORMAL" CHAR(1 BYTE) DEFAULT 'Y' NOT NULL ENABLE, "NOTES" VARCHAR2(1000 BYTE), "CHARGE_FEE" NUMBER(10,2) DEFAULT 0 NOT NULL ENABLE) PCTFREE 10 PCTUSED 40 IN ORA-39171: Job is experiencing a resumable wait. ORA-12953: The request exceeds the maximum allowed database size of 11 GB 為何impdp，提取ddl語句也報錯，因為數據泵導入會在數據庫內，創建相關對象，占用臨時存儲，數據庫都滿了，因此即使不導入數據，臨時創建個對象都是報錯的
--第二次操作，successful
總結：第一次導入失敗，主要原因是，數據泵導入后，分配段初始化空間，避免導入分配第一個extent，即避免現象
impdp \'/ as sysdba\' dumpfile=PICCPROD%U.dmp directory=dump logfile=picc.log REMAP_TABLESPACE=LIFELOB_DATA:LIFEDATA_T_L,LIFEINDEX_T_L:LIFEDATA_T_L
TRANSFORM = SEGMENT_ATTRIBUTES:N
三、問題反思
TRANSFORM = SEGMENT_ATTRIBUTES:N 參數作用何在，有什么特性 實驗測試： 1.創建一個測試表a, 創建主鍵，創建一個索引 2.對對象進行元數據導出 3.實驗參數，進行導入 4.查詢及驗證表及索引是否存在 5.查詢表及索引的段是否分配 6.impdp sqlfile 查詢與表 and 索引 get ddl 差異
1.創建一個測試表a, 創建主鍵，創建一個索引 SQL> create table yang.ceshi as select * from scott.emp; SQL> alter table yang.ceshi add constraint pk_id primary key(empno); SQL> create index yang.id2 on yang.ceshi(deptno); 2.對對象進行元數據導出 expdp \'/ as sysdba\' directory=dump dumpfile=yang%u.dmp logfile=yang.log SCHEMAS=yang include=TABLE:"IN('CESHI')" CONTENT=METADATA_ONLY 3.實驗參數，進行導入 impdp \'/ as sysdba\' directory=dump dumpfile=yang%u.dmp logfile=yang1.log TRANSFORM = SEGMENT_ATTRIBUTES:N REMAP_SCHEMA=yang:scott 4.查詢及驗證表及索引是否存在 select object_name,object_type,status from user_objects where created>sysdate-1 OBJECT_NAME OBJECT_TYPE STATUS -------------------- ------------------- ------- ID2 INDEX VALID PK_ID INDEX VALID CESHI TABLE VALID 5.查詢表及索引的段是否分配 SQL> select SEGMENT_NAME,SEGMENT_TYPE,BLOCKS,EXTENTS,INITIAL_EXTENT,NEXT_EXTENT from user_segments where segment_name in(select object_name from user_objects where created>sysdate-1); no rows selected 6.impdp sqlfile 查詢與表 and 索引 get ddl 差異 [oracle@t2 tools]$ impdp \'/ as sysdba\' directory=dump dumpfile=yang%u.dmp logfile=yang1.log TRANSFORM = SEGMENT_ATTRIBUTES:N REMAP_SCHEMA=yang:scott sqlfile=test_table.sql --查詢創建語法，進行說明： 第一部分 alter session level trace文件 第二部分，創建表語法，沒有了storage語法 第三部分，創建索引，對索引取消並行，在真實導入數據過程中，parallel並行能幫助快速創建索引，創建索引后數據泵自動noparallel 第四部分，導入完畢后，對數據泵臨時創建對象進行清理 [oracle@t2 tools]$ more test_table.sql -- CONNECT SYS ALTER SESSION SET EVENTS '22830 TRACE NAME CONTEXT FOREVER, LEVEL 192 '; -- new object type path: SCHEMA_EXPORT/TABLE/TABLE CREATE TABLE "SCOTT"."CESHI" ( "EMPNO" NUMBER(4,0), "ENAME" VARCHAR2(10 BYTE), "JOB" VARCHAR2(9 BYTE), "MGR" NUMBER(4,0), "HIREDATE" DATE, "SAL" NUMBER(7,2), "COMM" NUMBER(7,2), "DEPTNO" NUMBER(2,0) ) ; -- new object type path: SCHEMA_EXPORT/TABLE/INDEX/INDEX -- CONNECT SCOTT CREATE UNIQUE INDEX "SCOTT"."PK_ID" ON "SCOTT"."CESHI" ("EMPNO") ; ALTER INDEX "SCOTT"."PK_ID" NOPARALLEL; CREATE INDEX "SCOTT"."ID2" ON "SCOTT"."CESHI" ("DEPTNO") ; ALTER INDEX "SCOTT"."ID2" NOPARALLEL; -- new object type path: SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT -- CONNECT SYS ALTER TABLE "SCOTT"."CESHI" ADD CONSTRAINT "PK_ID" PRIMARY KEY ("EMPNO") USING INDEX "SCOTT"."PK_ID" ENABLE; -- new object type path: SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS DECLARE I_N VARCHAR2(60); I_O VARCHAR2(60); NV VARCHAR2(1); c DBMS_METADATA.T_VAR_COLL; df varchar2(21) := 'YYYY-MM-DD:HH24:MI:SS'; stmt varchar2(300) := ' INSERT INTO "SYS"."IMPDP_STATS" (type,version,flags,c1,c2,c3,c5,n1,n2,n3,n4,n5,n6,n7,n8,n9,n10,n11,n12,d1,cl1) VALUES (''I'',6,:1,: 2,:3,:4,:5,:6,:7,:8,:9,:10,:11,:12,:13,NULL,:14,:15,NULL,:16,:17)'; BEGIN DELETE FROM "SYS"."IMPDP_STATS"; i_n := 'PK_ID'; i_o := 'SCOTT'; EXECUTE IMMEDIATE stmt USING 2,I_N,NV,NV,I_O,14,1,14,1,1,1,0,14,NV,NV,TO_DATE('2018-11-29 15:17:22',df),NV; DBMS_STATS.IMPORT_INDEX_STATS('"' || i_o || '"','"' || i_n || '"',NULL,'"IMPDP_STATS"',NULL,'"SYS"'); DELETE FROM "SYS"."IMPDP_STATS"; END; / DECLARE I_N VARCHAR2(60); I_O VARCHAR2(60); NV VARCHAR2(1); c DBMS_METADATA.T_VAR_COLL; df varchar2(21) := 'YYYY-MM-DD:HH24:MI:SS'; stmt varchar2(300) := ' INSERT INTO "SYS"."IMPDP_STATS" (type,version,flags,c1,c2,c3,c5,n1,n2,n3,n4,n5,n6,n7,n8,n9,n10,n11,n12,d1,cl1) VALUES (''I'',6,:1,: 2,:3,:4,:5,:6,:7,:8,:9,:10,:11,:12,:13,NULL,:14,:15,NULL,:16,:17)'; BEGIN DELETE FROM "SYS"."IMPDP_STATS"; i_n := 'ID2'; i_o := 'SCOTT'; EXECUTE IMMEDIATE stmt USING 2,I_N,NV,NV,I_O,14,1,3,1,1,1,0,14,NV,NV,TO_DATE('2018-11-29 15:18:08',df),NV; DBMS_STATS.IMPORT_INDEX_STATS('"' || i_o || '"','"' || i_n || '"',NULL,'"IMPDP_STATS"',NULL,'"SYS"'); DELETE FROM "SYS"."IMPDP_STATS"; END; / ---查詢表的ddl語法 select dbms_metadata.get_ddl('TABLE','CESHI','SCOTT') ddl_text from dual DDL_TEXT -------------------------------------------------------------------------------- CREATE TABLE "SCOTT"."CESHI" ( "EMPNO" NUMBER(4,0), "ENAME" VARCHAR2(10), "JOB" VARCHAR2(9), "MGR" NUMBER(4,0), "HIREDATE" DATE, "SAL" NUMBER(7,2), "COMM" NUMBER(7,2), "DEPTNO" NUMBER(2,0), CONSTRAINT "PK_ID" PRIMARY KEY ("EMPNO") USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS TABLESPACE "USERS" ENABLE ) SEGMENT CREATION DEFERRED PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING TABLESPACE "USERS" --表不存在storage屬性后 insert 數據進行測試 SQL> insert into scott.ceshi select * from scott.emp; --查詢表及索引的extent分配，發現表其實並沒有extent storage屬性，是怎么分配的呢？ SQL> select SEGMENT_NAME,SEGMENT_TYPE,BLOCKS,EXTENTS,INITIAL_EXTENT/1024 INITIAL_EXTENT_K,NEXT_EXTENT/1024 NEXT_EXTENT_K from user_segments where segment_name in(select object_name from user_objects where created>sysdate-1); SEGMENT_NAME SEGMENT_TYPE BLOCKS EXTENTS INITIAL_EXTENT_K NEXT_EXTENT_K --------------- ------------------ ---------- ---------- ---------------- ------------- CESHI TABLE 8 1 64 1024 PK_ID INDEX 8 1 64 1024 ID2 INDEX 8 1 64 1024 --查詢表空間屬性 SQL> select tablespace_name from user_segments where segment_name in(select object_name from user_objects where created>sysdate-1) group by tablespace_name; TABLESPACE_NAME ------------------------------ USERS SQL> select TABLESPACE_NAME,BLOCK_SIZE,INITIAL_EXTENT/1024 INITIAL_EXTENT_K,NEXT_EXTENT/1024 NEXT_EXTENT_K,SEGMENT_SPACE_MANAGEMENT,ALLOCATION_TYPE,EXTENT_MANAGEMENT from dba_tablespaces where tablespace_name='USERS'; TABLESPACE BLOCK_SIZE INITIAL_EXTENT_K NEXT_EXTENT_K SEGMEN ALLOCATIO EXTENT_MAN ---------- ---------- ---------------- ------------- ------ --------- ---------- USERS 8192 64 AUTO SYSTEM LOCAL --表空間本地管理表空間，系統自動擴展分配， 8K,1M,8M,64M --可見，對於表來說，如果未配置storage屬性，則表使用表空間的默認storage屬性分配extents 大小
四、問題總結
1.根據問題現象，尋找解決方法，路不止一條，在能接受的范圍內解決即可
2.根據問題現象，尋找合適的技術點，使用自己能運用且無風險的方案解決問題，測試后方可使用