python學習——大文件分割與合並

在平常的生活中，我們會遇到下面這樣的情況：

你下載了一個比較大型的游戲（假設有10G），現在想跟你的同學一起玩，你需要把這個游戲拷貝給他。

然后現在有一個問題是文件太大（我們不考慮你有移動硬盤什么的情況），假設現在只有一個2G或4G的優盤，該怎么辦呢？

有很多方法，例如winrar壓縮的時候分成很多小卷，這里不累述。

在學習python之后，我們自己就可以解決這個問題啦。

我們可以自己寫一個腳本去分割合並文件，將文件分割成適合優盤大小的小文件，在拷貝，然后再合並。

 

下面是文件分割腳本：

import sys,os

kilobytes = 1024
megabytes = kilobytes*1000
chunksize = int(200*megabytes)#default chunksize

def split(fromfile,todir,chunksize=chunksize):
    if not os.path.exists(todir):#check whether todir exists or not
        os.mkdir(todir)          
    else:
        for fname in os.listdir(todir):
            os.remove(os.path.join(todir,fname))
    partnum = 0
    inputfile = open(fromfile,'rb')#open the fromfile
    while True:
        chunk = inputfile.read(chunksize)
        if not chunk:             #check the chunk is empty
            break
        partnum += 1
        filename = os.path.join(todir,('part%04d'%partnum))
        fileobj = open(filename,'wb')#make partfile
        fileobj.write(chunk)         #write data into partfile
        fileobj.close()
    return partnum
if __name__=='__main__':
        fromfile  = input('File to be split?')
        todir     = input('Directory to store part files?')
        chunksize = int(input('Chunksize to be split?'))
        absfrom,absto = map(os.path.abspath,[fromfile,todir])
        print('Splitting',absfrom,'to',absto,'by',chunksize)
        try:
            parts = split(fromfile,todir,chunksize)
        except:
            print('Error during split:')
            print(sys.exc_info()[0],sys.exc_info()[1])
        else:
            print('split finished:',parts,'parts are in',absto)

下面是腳本運行的例子：

我們在F有一個X—MEN1.rar文件，1.26G大小，我們現在把它分割成400000000bit(大約380M)的文件。

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 
File to be split?F:\X-MEN1.rar
Directory to store part files?F:\split
Chunksize to be split?400000000
Splitting F:\X-MEN1.rar to F:\split by 400000000
split finished: 4 parts are in F:\split
>>> 

這是分割后的文件： 

 

下面是文件合並腳本：

 

import sys,os

def joinfile(fromdir,filename,todir):
    if not os.path.exists(todir):
        os.mkdir(todir)
    if not os.path.exists(fromdir):
        print('Wrong directory')
    outfile = open(os.path.join(todir,filename),'wb')
    files = os.listdir(fromdir) #list all the part files in the directory
    files.sort()                #sort part files to read in order
    for file in files:
        filepath = os.path.join(fromdir,file)
        infile = open(filepath,'rb')
        data = infile.read()
        outfile.write(data)
        infile.close()
    outfile.close()
if __name__=='__main__':
        fromdir = input('Directory containing part files?')
        filename = input('Name of file to be recreated?')
        todir   = input('Directory to store recreated file?')
        
        try:
            joinfile(fromdir,filename,todir)
        except:
            print('Error joining files:')
            print(sys.exc_info()[0],sys.exc_info()[1])

運行合並腳本，將上面分割腳本分割的文件重組：

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>> 
Directory containing part files?F:\split
Name of file to be recreated?xman1.rar
Directory to store recreated file?F:\
>>> 

運行之后可以看到F盤下生成了重組的xman.rar