Linux大文件拆分、合並、校驗

Linux大文件拆分、合並、校驗

大文件在實際應用傳輸過程中往往經歷拆分、合並的過程。文件在多次傳輸后有可能會損壞，在使用前可以進行完整性校驗。

1.文件生成MD5碼

md5sum rhel-server-6.3-x86_64-dvd.iso
d717af33dd258945e6304f9955487017  rhel-server-6.3-x86_64-dvd.iso

2.文件拆分

split -b 600m -d rhel-server-6.3-x86_64-dvd.iso redhat6.3_x

參數說明: -b 600m 表示文件以600mb為單位分割
         -d  分割后子文件名以數字作為后綴，默認兩位
         redhat6.3_x 子文件前綴

3.拆分后生成子文件MD5碼

for var in $(ls redhat*); do md5sum $var; done
b9440b925d9e9bc640fd23ff00e15450  redhat6.3_x00
af4985a9aa1bc557b5a05de3cdad026b  redhat6.3_x01
2433ab29a85789f4aca21421307ed788  redhat6.3_x02
8cbccc256b1da1014f07e9142cf2fc4c  redhat6.3_x03
962e79bceacef5c1fa9336e4a21ce995  redhat6.3_x04
22a68fb82aaa7c8d516897740ebf052f  redhat6.3_x05
#可以直接將MD5值重定向到文件
for var in $(ls redhat*); do md5sum $var >> s_sub.md5; done

4.子文件遠程傳輸

scp redhat6.3_x00 redhat6.3_x01 redhat6.3_x02 redhat6.3_x03 redhat6.3_x04 redhat6.3_x05 user@192.168.122.36:~

5.在接收端校驗子文件完整性

#5.1 在接收端生成子文件
for var in $(ls redhat6.3_x*);do md5sum $var ;done
b9440b925d9e9bc640fd23ff00e15450  redhat6.3_x00
af4985a9aa1bc557b5a05de3cdad026b  redhat6.3_x01
2433ab29a85789f4aca21421307ed788  redhat6.3_x02
8cbccc256b1da1014f07e9142cf2fc4c  redhat6.3_x03
962e79bceacef5c1fa9336e4a21ce995  redhat6.3_x04
22a68fb82aaa7c8d516897740ebf052f  redhat6.3_x05
for var in $(ls redhat*); do md5sum $var >> d_sub.md5; done
#5.2 比較源子文件MD5碼，相同說明完整
diff s_sub.md5 d_sub.md5 > /dev/null && echo true || echo false
true

6.大文件合並

cat redhat6.3_x00 redhat6.3_x01 redhat6.3_x02 redhat6.3_x03 redhat6.3_x04 redhat6.3_x05 > rhel-server-6.3-x86_64-dvd.iso

7.合並后的大文件生成MD5碼與源MD5值比較，值相同表示完整無損

md5sum rhel-server-6.3-x86_64-dvd.iso
d717af33dd258945e6304f9955487017  rhel-server-6.3-x86_64-dvd.iso
# 比較略(字符串比較