python學習-06（模塊化編程）

 

1：模塊定義

         模塊：用來從邏輯上組織Python代碼（變量，函數、類，邏輯：實現一個功能），本質就是.py結尾的Python文件（文件名:test.py,對應的模塊名，test）。

         包：本質就是一個文件夾或者是目錄（必須帶有一個_init_.py文件），包是用來從邏輯上組織模塊的。

2.導入方法：

1):import module_test, module_test1 （調用方法的時候，需要module_test.   ）

2):form moudule_test import * （* 表示所有的方法）(不建議使用，這個就相當於導入了程序。)  容易造成覆蓋，出現問題。

3):form module_test import module_ajun  as module_ajun_test(作為一個別名來使用)

3：import本質 （路徑 搜索和搜索路徑）

1)      import module_test  相當於把module_test 的所有代碼解釋一遍，全部傳遞給module_test. 調用module_test的變量和方法。

2)      form moudule_test import name 相當於值調用了name變量，在當前模塊，就不需要加前綴了。

總結：導入模塊的本質就是把Python文件解釋一遍。(import test  test=’test.py  all code’)

           導入包的本質就是執行包下面的_init_.py文件

            Import module_name--》mudule_name.py--》module_name.py的路徑--》sys.path

 

 

os.path.abspath(_file_)=獲取 當前目錄

os.path.dirname(os.path.abspath(_file_))=獲取當前目錄的父目錄

os.path.dirname(os.path.dirname (os.path.abspath(_file_)))=獲取當前目錄的父目錄 的父目錄

form .（點表示當前目錄，就是_init_的目錄）  import 包名

4.導入優化

        

怎么優化？每次都去找，很浪費時間？

使用from module_test  import test 這個方法，節約時間。

5.模塊的分類

A:標准庫

B：開源模塊

C：自定義模塊

 

A標准庫

5.1time與datetime 兩個庫。

1）表示時間，格式化的時間格式：2017-07-28

2）時間戳：從1970.1.1日到現在總共經過了多少秒？

為什么從1970年1月1日開始呢？

3）元祖，共有九個元素表示。

UTC（Coordinated Universal Time，世界協調時）亦即格林威治天文時間，世界標准時間。在中國為UTC+8。DST（Daylight Saving Time）即夏令時。 

時間戳（timestamp）的方式：通常來說，時間戳表示的是從1970年1月1日00:00:00開始按秒計算的偏移量。我們運行“type(time.time())”，返回的是float類型。返回時間戳方式的函數主要有time()，clock()等。 

元組（struct_time）方式：struct_time元組共有9個元素，返回struct_time的函數主要有gmtime()，localtime()，strptime()。下面列出這種方式元組中的幾個元素：　

#Author:ajun
import time

#print(time.process_time())#測量處理器運算時間,不包括sleep時間,不穩定
#print(time.altzone)  #返回與utc時間的時間差,以秒計算\
#print(time.asctime()) #返回時間格式Wed Jun 28 21:38:29 2017 格式,
#print(time.localtime()) #返回本地時間 的struct time對象格式
#print(time.gmtime(time.time()-800000)) #返回utc時間的struc時間對象格式
#print(time.asctime(time.localtime())) #返回時間格式"Fri Aug 19 11:14:16 2016",
#print(time.ctime()) #返回Wed Jun 28 21:38:29 2017 格式, 同上
# 日期字符串 轉成  時間戳
string_2_struct = time.strptime("2017/06/28","%Y/%m/%d") #將 日期字符串 轉成 struct時間對象格式
print(string_2_struct)
# #
struct_2_stamp = time.mktime(string_2_struct) #將struct時間對象轉成時間戳
print(struct_2_stamp)

#將時間戳轉為字符串格式
print(time.gmtime(time.time()-86640)) #將utc時間戳轉換成struct_time格式
print(time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime()) ) #將utc struct_time格式轉成指定的字符串格式

#時間加減
import datetime

print(datetime.datetime.now()) #返回 2017-06-28 21:11:01.433280
print(datetime.date.fromtimestamp(time.time()) )  # 時間戳直接轉成日期格式 2017-06-28
# print(datetime.datetime.now() )
# print(datetime.datetime.now() + datetime.timedelta(3)) #當前時間+3天
# print(datetime.datetime.now() + datetime.timedelta(-3)) #當前時間-3天
# print(datetime.datetime.now() + datetime.timedelta(hours=3)) #當前時間+3小時
# print(datetime.datetime.now() + datetime.timedelta(minutes=30)) #當前時間+30分

#修改時間
c_time  = datetime.datetime.now()
print(c_time.replace(minute=3,hour=2)) #時間替換)) #時間替換

時間轉換圖（一個來自alex，一個來自海峰老師）

 

%a    本地（locale）簡化星期名稱    
%A    本地完整星期名稱    
%b    本地簡化月份名稱    
%B    本地完整月份名稱    
%c    本地相應的日期和時間表示    
%d    一個月中的第幾天（01 - 31）    
%H    一天中的第幾個小時（24小時制，00 - 23）    
%I    第幾個小時（12小時制，01 - 12）    
%j    一年中的第幾天（001 - 366）    
%m    月份（01 - 12）    
%M    分鍾數（00 - 59）    
%p    本地am或者pm的相應符    一    
%S    秒（01 - 61）    二    
%U    一年中的星期數。（00 - 53星期天是一個星期的開始。）第一個星期天之前的所有天數都放在第0周。    三    
%w    一個星期中的第幾天（0 - 6，0是星期天）    三    
%W    和%U基本相同，不同的是%W以星期一為一個星期的開始。    
%x    本地相應日期    
%X    本地相應時間    
%y    去掉世紀的年份（00 - 99）    
%Y    完整的年份    
%Z    時區的名字（如果不存在為空字符）    
%%    ‘%’字符

 5.2random模塊

隨機數

import random
print (random.random())  #0.6445010863311293  
#random.random()用於生成一個0到1的隨機符點數: 0 <= n < 1.0
print (random.randint(1,7)) #4
#random.randint()的函數原型為：random.randint(a, b)，用於生成一個指定范圍內的整數。
# 其中參數a是下限，參數b是上限，生成的隨機數n: a <= n <= b
print (random.randrange(1,10)) #5
#random.randrange的函數原型為：random.randrange([start], stop[, step])，
# 從指定范圍內，按指定基數遞增的集合中 獲取一個隨機數。如：random.randrange(10, 100, 2)，
# 結果相當於從[10, 12, 14, 16, ... 96, 98]序列中獲取一個隨機數。
# random.randrange(10, 100, 2)在結果上與 random.choice(range(10, 100, 2) 等效。
print(random.choice('liukuni')) #i
#random.choice從序列中獲取一個隨機元素。
# 其函數原型為：random.choice(sequence)。參數sequence表示一個有序類型。
# 這里要說明一下：sequence在python不是一種特定的類型，而是泛指一系列的類型。
# list, tuple, 字符串都屬於sequence。有關sequence可以查看python手冊數據模型這一章。
# 下面是使用choice的一些例子：
print(random.choice("學習Python"))#學
print(random.choice(["JGood","is","a","handsome","boy"]))  #List
print(random.choice(("Tuple","List","Dict")))   #List
print(random.sample([1,2,3,4,5],3))    #[1, 2, 5]
#random.sample的函數原型為：random.sample(sequence, k)，從指定序列中隨機獲取指定長度的片斷。sample函數不會修改原有序列。

#!/usr/bin/env python
# encoding: utf-8
import random
import string
#隨機整數：
print( random.randint(0,99))  #70
 
#隨機選取0到100間的偶數：
print(random.randrange(0, 101, 2)) #4
 
#隨機浮點數：
print( random.random()) #0.2746445568079129
print(random.uniform(1, 10)) #9.887001463194844
 
#隨機字符：
print(random.choice('abcdefg&#%^*f')) #f
 
#多個字符中選取特定數量的字符：
print(random.sample('abcdefghij',3)) #['f', 'h', 'd']
 
#隨機選取字符串：
print( random.choice ( ['apple', 'pear', 'peach', 'orange', 'lemon'] )) #apple
#洗牌#
items = [1,2,3,4,5,6,7]
print(items) #[1, 2, 3, 4, 5, 6, 7]
random.shuffle(items)
print(items) #[1, 4, 7, 2, 5, 3, 6]

import random
checkcode=''
for i in range(5):
    current_a=random.randint(1,6)
    if current_a%2:
        temp = chr(random.randint(65,90))
    else:
        temp = random.randint(0,9)
    checkcode+=str(temp)
print(checkcode)

 5.3os模塊

參考：https://docs.python.org/2/library/os.html?highlight=os#module-os （幫助）

http://www.cnblogs.com/alex3714/articles/5161349.html（alex）

http://egon09.blog.51cto.com/9161406/1840425（海峰老師）

#Author:ajun
os.getcwd() #獲取當前工作目錄，即當前python腳本工作的目錄路徑
os.chdir("dirname")  #改變當前腳本工作目錄；相當於shell下cd
os.curdir # 返回當前目錄: ('.')
os.pardir # 獲取當前目錄的父目錄字符串名：('..')
os.makedirs('dirname1/dirname2')    #可生成多層遞歸目錄
os.removedirs('dirname1')  #  若目錄為空，則刪除，並遞歸到上一級目錄，如若也為空，則刪除，依此類推
os.mkdir('dirname')   # 生成單級目錄；相當於shell中mkdir dirname
os.rmdir('dirname')   # 刪除單級空目錄，若目錄不為空則無法刪除，報錯；相當於shell中rmdir dirname
os.listdir('dirname')    #列出指定目錄下的所有文件和子目錄，包括隱藏文件，並以列表方式打印
os.remove()  #刪除一個文件
os.rename("oldname","newname")  #重命名文件/目錄
os.stat('path/filename')  獲取文件/目錄信息
os.sep    #輸出操作系統特定的路徑分隔符，win下為"\\",Linux下為"/"
os.linesep    #輸出當前平台使用的行終止符，win下為"\t\n",Linux下為"\n"
os.pathsep   # 輸出用於分割文件路徑的字符串
os.name   # 輸出字符串指示當前使用平台。win->'nt'; Linux->'posix'
os.system("bash command")  #運行shell命令，直接顯示
os.environ # 獲取系統環境變量
os.path.abspath(path)  #返回path規范化的絕對路徑
os.path.split(path)  #將path分割成目錄和文件名二元組返回
os.path.dirname(path)  #返回path的目錄。其實就是os.path.split(path)的第一個元素
os.path.basename(path)  #返回path最后的文件名。如何path以／或\結尾，那么就會返回空值。即os.path.split(path)的第二個元素
os.path.exists(path)  #如果path存在，返回True；如果path不存在，返回False
os.path.isabs(path)  #如果path是絕對路徑，返回True
os.path.isfile(path)  #如果path是一個存在的文件，返回True。否則返回False
os.path.isdir(path)  #如果path是一個存在的目錄，則返回True。否則返回False
os.path.join(path1[,path2[, ...]]) # 將多個路徑組合后返回，第一個絕對路徑之前的參數將被忽略
os.path.getatime(path)  #返回path所指向的文件或者目錄的最后存取時間
os.path.getmtime(path)  #返回path所指向的文件或者目錄的最后修改時間

5.4sys模塊

sys.argv           #命令行參數List，第一個元素是程序本身路徑
sys.exit(n)       # 退出程序，正常退出時exit(0)
sys.version        #獲取Python解釋程序的版本信息
sys.maxint         #最大的Int值
sys.path           返回模塊的搜索路徑，初始化時使用PYTHONPATH環境變量的值#
sys.platform     #  返回操作系統平台名稱
sys.stdout.write('please:')  #寫進度條就需要這個函數
val = sys.stdin.readline()[:-1]

5.5shutil模塊

主要就是用於文件的拷貝和壓縮包。

參考http://www.cnblogs.com/wupeiqi/articles/4963027.html 

高級的 文件、文件夾、壓縮包 處理模塊

shutil.copyfileobj(fsrc, fdst[, length])
將文件內容拷貝到另一個文件中，可以部分內容

def copyfileobj(fsrc, fdst, length=16*1024):
    """copy data from file-like object fsrc to file-like object fdst"""
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

shutil.copyfile(src, dst)
拷貝文件

def copyfile(src, dst):
    """Copy data from src to dst"""
    if _samefile(src, dst):
        raise Error("`%s` and `%s` are the same file" % (src, dst))

    for fn in [src, dst]:
        try:
            st = os.stat(fn)
        except OSError:
            # File most likely does not exist
            pass
        else:
            # XXX What about other special files? (sockets, devices...)
            if stat.S_ISFIFO(st.st_mode):
                raise SpecialFileError("`%s` is a named pipe" % fn)

    with open(src, 'rb') as fsrc:
        with open(dst, 'wb') as fdst:
            copyfileobj(fsrc, fdst)

shutil.copymode(src, dst)
僅拷貝權限。內容、組、用戶均不變

def copymode(src, dst):
    """Copy mode bits from src to dst"""
    if hasattr(os, 'chmod'):
        st = os.stat(src)
        mode = stat.S_IMODE(st.st_mode)
        os.chmod(dst, mode)

shutil.copystat(src, dst)
拷貝狀態的信息，包括：mode bits, atime, mtime, flags

def copystat(src, dst):
    """Copy all stat info (mode bits, atime, mtime, flags) from src to dst"""
    st = os.stat(src)
    mode = stat.S_IMODE(st.st_mode)
    if hasattr(os, 'utime'):
        os.utime(dst, (st.st_atime, st.st_mtime))
    if hasattr(os, 'chmod'):
        os.chmod(dst, mode)
    if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):
        try:
            os.chflags(dst, st.st_flags)
        except OSError, why:
            for err in 'EOPNOTSUPP', 'ENOTSUP':
                if hasattr(errno, err) and why.errno == getattr(errno, err):
                    break
            else:
                raise

shutil.copy(src, dst)
拷貝文件和權限

def copy(src, dst):
    """Copy data and mode bits ("cp src dst").

    The destination may be a directory.

    """
    if os.path.isdir(dst):
        dst = os.path.join(dst, os.path.basename(src))
    copyfile(src, dst)
    copymode(src, dst)

shutil.copy2(src, dst)
拷貝文件和狀態信息

def copy2(src, dst):
    """Copy data and all stat info ("cp -p src dst").

    The destination may be a directory.

    """
    if os.path.isdir(dst):
        dst = os.path.join(dst, os.path.basename(src))
    copyfile(src, dst)
    copystat(src, dst)

shutil.ignore_patterns(*patterns)
shutil.copytree(src, dst, symlinks=False, ignore=None)
遞歸的去拷貝文件

例如：copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))

def ignore_patterns(*patterns):
    """Function that can be used as copytree() ignore parameter.

    Patterns is a sequence of glob-style patterns
    that are used to exclude files"""
    def _ignore_patterns(path, names):
        ignored_names = []
        for pattern in patterns:
            ignored_names.extend(fnmatch.filter(names, pattern))
        return set(ignored_names)
    return _ignore_patterns

def copytree(src, dst, symlinks=False, ignore=None):
    """Recursively copy a directory tree using copy2().

    The destination directory must not already exist.
    If exception(s) occur, an Error is raised with a list of reasons.

    If the optional symlinks flag is true, symbolic links in the
    source tree result in symbolic links in the destination tree; if
    it is false, the contents of the files pointed to by symbolic
    links are copied.

    The optional ignore argument is a callable. If given, it
    is called with the `src` parameter, which is the directory
    being visited by copytree(), and `names` which is the list of
    `src` contents, as returned by os.listdir():

        callable(src, names) -> ignored_names

    Since copytree() is called recursively, the callable will be
    called once for each directory that is copied. It returns a
    list of names relative to the `src` directory that should
    not be copied.

    XXX Consider this example code rather than the ultimate tool.

    """
    names = os.listdir(src)
    if ignore is not None:
        ignored_names = ignore(src, names)
    else:
        ignored_names = set()

    os.makedirs(dst)
    errors = []
    for name in names:
        if name in ignored_names:
            continue
        srcname = os.path.join(src, name)
        dstname = os.path.join(dst, name)
        try:
            if symlinks and os.path.islink(srcname):
                linkto = os.readlink(srcname)
                os.symlink(linkto, dstname)
            elif os.path.isdir(srcname):
                copytree(srcname, dstname, symlinks, ignore)
            else:
                # Will raise a SpecialFileError for unsupported file types
                copy2(srcname, dstname)
        # catch the Error from the recursive copytree so that we can
        # continue with other files
        except Error, err:
            errors.extend(err.args[0])
        except EnvironmentError, why:
            errors.append((srcname, dstname, str(why)))
    try:
        copystat(src, dst)
    except OSError, why:
        if WindowsError is not None and isinstance(why, WindowsError):
            # Copying file access times may fail on Windows
            pass
        else:
            errors.append((src, dst, str(why)))
    if errors:
        raise Error, errors

shutil.rmtree(path[, ignore_errors[, onerror]])
遞歸的去刪除文件

def rmtree(path, ignore_errors=False, onerror=None):
    """Recursively delete a directory tree.

    If ignore_errors is set, errors are ignored; otherwise, if onerror
    is set, it is called to handle the error with arguments (func,
    path, exc_info) where func is os.listdir, os.remove, or os.rmdir;
    path is the argument to that function that caused it to fail; and
    exc_info is a tuple returned by sys.exc_info().  If ignore_errors
    is false and onerror is None, an exception is raised.

    """
    if ignore_errors:
        def onerror(*args):
            pass
    elif onerror is None:
        def onerror(*args):
            raise
    try:
        if os.path.islink(path):
            # symlinks to directories are forbidden, see bug #1669
            raise OSError("Cannot call rmtree on a symbolic link")
    except OSError:
        onerror(os.path.islink, path, sys.exc_info())
        # can't continue even if onerror hook returns
        return
    names = []
    try:
        names = os.listdir(path)
    except os.error, err:
        onerror(os.listdir, path, sys.exc_info())
    for name in names:
        fullname = os.path.join(path, name)
        try:
            mode = os.lstat(fullname).st_mode
        except os.error:
            mode = 0
        if stat.S_ISDIR(mode):
            rmtree(fullname, ignore_errors, onerror)
        else:
            try:
                os.remove(fullname)
            except os.error, err:
                onerror(os.remove, fullname, sys.exc_info())
    try:
        os.rmdir(path)
    except os.error:
        onerror(os.rmdir, path, sys.exc_info())

shutil.move(src, dst)
遞歸的去移動文件

def move(src, dst):
    """Recursively move a file or directory to another location. This is
    similar to the Unix "mv" command.

    If the destination is a directory or a symlink to a directory, the source
    is moved inside the directory. The destination path must not already
    exist.

    If the destination already exists but is not a directory, it may be
    overwritten depending on os.rename() semantics.

    If the destination is on our current filesystem, then rename() is used.
    Otherwise, src is copied to the destination and then removed.
    A lot more could be done here...  A look at a mv.c shows a lot of
    the issues this implementation glosses over.

    """
    real_dst = dst
    if os.path.isdir(dst):
        if _samefile(src, dst):
            # We might be on a case insensitive filesystem,
            # perform the rename anyway.
            os.rename(src, dst)
            return

        real_dst = os.path.join(dst, _basename(src))
        if os.path.exists(real_dst):
            raise Error, "Destination path '%s' already exists" % real_dst
    try:
        os.rename(src, real_dst)
    except OSError:
        if os.path.isdir(src):
            if _destinsrc(src, dst):
                raise Error, "Cannot move a directory '%s' into itself '%s'." % (src, dst)
            copytree(src, real_dst, symlinks=True)
            rmtree(src)
        else:
            copy2(src, real_dst)
            os.unlink(src)

shutil.make_archive(base_name, format,...)

創建壓縮包並返回文件路徑，例如：zip、tar

• base_name： 壓縮包的文件名，也可以是壓縮包的路徑。只是文件名時，則保存至當前目錄，否則保存至指定路徑，
  如：www                        =>保存至當前路徑
  如：/Users/wupeiqi/www =>保存至/Users/wupeiqi/
• format： 壓縮包種類，“zip”, “tar”, “bztar”，“gztar”
• root_dir： 要壓縮的文件夾路徑（默認當前目錄）
• owner： 用戶，默認當前用戶
• group： 組，默認當前組
• logger： 用於記錄日志，通常是logging.Logger對象

• #將 /Users/wupeiqi/Downloads/test 下的文件打包放置當前程序目錄
   
  import shutil
  ret = shutil.make_archive("wwwwwwwwww", 'gztar', root_dir='/Users/wupeiqi/Downloads/test')
   
   
  #將 /Users/wupeiqi/Downloads/test 下的文件打包放置 /Users/wupeiqi/目錄
  import shutil
  ret = shutil.make_archive("/Users/wupeiqi/wwwwwwwwww", 'gztar', root_dir='/Users/wupeiqi/Downloads/test')

  def make_archive(base_name, format, root_dir=None, base_dir=None, verbose=0,
                   dry_run=0, owner=None, group=None, logger=None):
      """Create an archive file (eg. zip or tar).
  
      'base_name' is the name of the file to create, minus any format-specific
      extension; 'format' is the archive format: one of "zip", "tar", "bztar"
      or "gztar".
  
      'root_dir' is a directory that will be the root directory of the
      archive; ie. we typically chdir into 'root_dir' before creating the
      archive.  'base_dir' is the directory where we start archiving from;
      ie. 'base_dir' will be the common prefix of all files and
      directories in the archive.  'root_dir' and 'base_dir' both default
      to the current directory.  Returns the name of the archive file.
  
      'owner' and 'group' are used when creating a tar archive. By default,
      uses the current owner and group.
      """
      save_cwd = os.getcwd()
      if root_dir is not None:
          if logger is not None:
              logger.debug("changing into '%s'", root_dir)
          base_name = os.path.abspath(base_name)
          if not dry_run:
              os.chdir(root_dir)
  
      if base_dir is None:
          base_dir = os.curdir
  
      kwargs = {'dry_run': dry_run, 'logger': logger}
  
      try:
          format_info = _ARCHIVE_FORMATS[format]
      except KeyError:
          raise ValueError, "unknown archive format '%s'" % format
  
      func = format_info[0]
      for arg, val in format_info[1]:
          kwargs[arg] = val
  
      if format != 'zip':
          kwargs['owner'] = owner
          kwargs['group'] = group
  
      try:
          filename = func(base_name, base_dir, **kwargs)
      finally:
          if root_dir is not None:
              if logger is not None:
                  logger.debug("changing back to '%s'", save_cwd)
              os.chdir(save_cwd)
  
      return filename

   

• shutil 對壓縮包的處理是調用 ZipFile 和 TarFile 兩個模塊來進行的，詳細：

• import zipfile
  
  # 壓縮
  z = zipfile.ZipFile('laxi.zip', 'w')
  z.write('a.log')
  z.write('data.data')
  z.close()
  
  # 解壓
  z = zipfile.ZipFile('laxi.zip', 'r')
  z.extractall()
  z.close()

  import tarfile
  
  # 壓縮
  tar = tarfile.open('your.tar','w')
  tar.add('/Users/wupeiqi/PycharmProjects/bbs2.zip', arcname='bbs2.zip')
  tar.add('/Users/wupeiqi/PycharmProjects/cmdb.zip', arcname='cmdb.zip')
  tar.close()
  
  # 解壓
  tar = tarfile.open('your.tar','r')
  tar.extractall()  # 可設置解壓地址
  tar.close()

  class ZipFile(object):
      """ Class with methods to open, read, write, close, list zip files.
  
      z = ZipFile(file, mode="r", compression=ZIP_STORED, allowZip64=False)
  
      file: Either the path to the file, or a file-like object.
            If it is a path, the file will be opened and closed by ZipFile.
      mode: The mode can be either read "r", write "w" or append "a".
      compression: ZIP_STORED (no compression) or ZIP_DEFLATED (requires zlib).
      allowZip64: if True ZipFile will create files with ZIP64 extensions when
                  needed, otherwise it will raise an exception when this would
                  be necessary.
  
      """
  
      fp = None                   # Set here since __del__ checks it
  
      def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False):
          """Open the ZIP file with mode read "r", write "w" or append "a"."""
          if mode not in ("r", "w", "a"):
              raise RuntimeError('ZipFile() requires mode "r", "w", or "a"')
  
          if compression == ZIP_STORED:
              pass
          elif compression == ZIP_DEFLATED:
              if not zlib:
                  raise RuntimeError,\
                        "Compression requires the (missing) zlib module"
          else:
              raise RuntimeError, "That compression method is not supported"
  
          self._allowZip64 = allowZip64
          self._didModify = False
          self.debug = 0  # Level of printing: 0 through 3
          self.NameToInfo = {}    # Find file info given name
          self.filelist = []      # List of ZipInfo instances for archive
          self.compression = compression  # Method of compression
          self.mode = key = mode.replace('b', '')[0]
          self.pwd = None
          self._comment = ''
  
          # Check if we were passed a file-like object
          if isinstance(file, basestring):
              self._filePassed = 0
              self.filename = file
              modeDict = {'r' : 'rb', 'w': 'wb', 'a' : 'r+b'}
              try:
                  self.fp = open(file, modeDict[mode])
              except IOError:
                  if mode == 'a':
                      mode = key = 'w'
                      self.fp = open(file, modeDict[mode])
                  else:
                      raise
          else:
              self._filePassed = 1
              self.fp = file
              self.filename = getattr(file, 'name', None)
  
          try:
              if key == 'r':
                  self._RealGetContents()
              elif key == 'w':
                  # set the modified flag so central directory gets written
                  # even if no files are added to the archive
                  self._didModify = True
              elif key == 'a':
                  try:
                      # See if file is a zip file
                      self._RealGetContents()
                      # seek to start of directory and overwrite
                      self.fp.seek(self.start_dir, 0)
                  except BadZipfile:
                      # file is not a zip file, just append
                      self.fp.seek(0, 2)
  
                      # set the modified flag so central directory gets written
                      # even if no files are added to the archive
                      self._didModify = True
              else:
                  raise RuntimeError('Mode must be "r", "w" or "a"')
          except:
              fp = self.fp
              self.fp = None
              if not self._filePassed:
                  fp.close()
              raise
  
      def __enter__(self):
          return self
  
      def __exit__(self, type, value, traceback):
          self.close()
  
      def _RealGetContents(self):
          """Read in the table of contents for the ZIP file."""
          fp = self.fp
          try:
              endrec = _EndRecData(fp)
          except IOError:
              raise BadZipfile("File is not a zip file")
          if not endrec:
              raise BadZipfile, "File is not a zip file"
          if self.debug > 1:
              print endrec
          size_cd = endrec[_ECD_SIZE]             # bytes in central directory
          offset_cd = endrec[_ECD_OFFSET]         # offset of central directory
          self._comment = endrec[_ECD_COMMENT]    # archive comment
  
          # "concat" is zero, unless zip was concatenated to another file
          concat = endrec[_ECD_LOCATION] - size_cd - offset_cd
          if endrec[_ECD_SIGNATURE] == stringEndArchive64:
              # If Zip64 extension structures are present, account for them
              concat -= (sizeEndCentDir64 + sizeEndCentDir64Locator)
  
          if self.debug > 2:
              inferred = concat + offset_cd
              print "given, inferred, offset", offset_cd, inferred, concat
          # self.start_dir:  Position of start of central directory
          self.start_dir = offset_cd + concat
          fp.seek(self.start_dir, 0)
          data = fp.read(size_cd)
          fp = cStringIO.StringIO(data)
          total = 0
          while total < size_cd:
              centdir = fp.read(sizeCentralDir)
              if len(centdir) != sizeCentralDir:
                  raise BadZipfile("Truncated central directory")
              centdir = struct.unpack(structCentralDir, centdir)
              if centdir[_CD_SIGNATURE] != stringCentralDir:
                  raise BadZipfile("Bad magic number for central directory")
              if self.debug > 2:
                  print centdir
              filename = fp.read(centdir[_CD_FILENAME_LENGTH])
              # Create ZipInfo instance to store file information
              x = ZipInfo(filename)
              x.extra = fp.read(centdir[_CD_EXTRA_FIELD_LENGTH])
              x.comment = fp.read(centdir[_CD_COMMENT_LENGTH])
              x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET]
              (x.create_version, x.create_system, x.extract_version, x.reserved,
                  x.flag_bits, x.compress_type, t, d,
                  x.CRC, x.compress_size, x.file_size) = centdir[1:12]
              x.volume, x.internal_attr, x.external_attr = centdir[15:18]
              # Convert date/time code to (year, month, day, hour, min, sec)
              x._raw_time = t
              x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F,
                                       t>>11, (t>>5)&0x3F, (t&0x1F) * 2 )
  
              x._decodeExtra()
              x.header_offset = x.header_offset + concat
              x.filename = x._decodeFilename()
              self.filelist.append(x)
              self.NameToInfo[x.filename] = x
  
              # update total bytes read from central directory
              total = (total + sizeCentralDir + centdir[_CD_FILENAME_LENGTH]
                       + centdir[_CD_EXTRA_FIELD_LENGTH]
                       + centdir[_CD_COMMENT_LENGTH])
  
              if self.debug > 2:
                  print "total", total
  
  
      def namelist(self):
          """Return a list of file names in the archive."""
          l = []
          for data in self.filelist:
              l.append(data.filename)
          return l
  
      def infolist(self):
          """Return a list of class ZipInfo instances for files in the
          archive."""
          return self.filelist
  
      def printdir(self):
          """Print a table of contents for the zip file."""
          print "%-46s %19s %12s" % ("File Name", "Modified    ", "Size")
          for zinfo in self.filelist:
              date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]
              print "%-46s %s %12d" % (zinfo.filename, date, zinfo.file_size)
  
      def testzip(self):
          """Read all the files and check the CRC."""
          chunk_size = 2 ** 20
          for zinfo in self.filelist:
              try:
                  # Read by chunks, to avoid an OverflowError or a
                  # MemoryError with very large embedded files.
                  with self.open(zinfo.filename, "r") as f:
                      while f.read(chunk_size):     # Check CRC-32
                          pass
              except BadZipfile:
                  return zinfo.filename
  
      def getinfo(self, name):
          """Return the instance of ZipInfo given 'name'."""
          info = self.NameToInfo.get(name)
          if info is None:
              raise KeyError(
                  'There is no item named %r in the archive' % name)
  
          return info
  
      def setpassword(self, pwd):
          """Set default password for encrypted files."""
          self.pwd = pwd
  
      @property
      def comment(self):
          """The comment text associated with the ZIP file."""
          return self._comment
  
      @comment.setter
      def comment(self, comment):
          # check for valid comment length
          if len(comment) > ZIP_MAX_COMMENT:
              import warnings
              warnings.warn('Archive comment is too long; truncating to %d bytes'
                            % ZIP_MAX_COMMENT, stacklevel=2)
              comment = comment[:ZIP_MAX_COMMENT]
          self._comment = comment
          self._didModify = True
  
      def read(self, name, pwd=None):
          """Return file bytes (as a string) for name."""
          return self.open(name, "r", pwd).read()
  
      def open(self, name, mode="r", pwd=None):
          """Return file-like object for 'name'."""
          if mode not in ("r", "U", "rU"):
              raise RuntimeError, 'open() requires mode "r", "U", or "rU"'
          if not self.fp:
              raise RuntimeError, \
                    "Attempt to read ZIP archive that was already closed"
  
          # Only open a new file for instances where we were not
          # given a file object in the constructor
          if self._filePassed:
              zef_file = self.fp
              should_close = False
          else:
              zef_file = open(self.filename, 'rb')
              should_close = True
  
          try:
              # Make sure we have an info object
              if isinstance(name, ZipInfo):
                  # 'name' is already an info object
                  zinfo = name
              else:
                  # Get info object for name
                  zinfo = self.getinfo(name)
  
              zef_file.seek(zinfo.header_offset, 0)
  
              # Skip the file header:
              fheader = zef_file.read(sizeFileHeader)
              if len(fheader) != sizeFileHeader:
                  raise BadZipfile("Truncated file header")
              fheader = struct.unpack(structFileHeader, fheader)
              if fheader[_FH_SIGNATURE] != stringFileHeader:
                  raise BadZipfile("Bad magic number for file header")
  
              fname = zef_file.read(fheader[_FH_FILENAME_LENGTH])
              if fheader[_FH_EXTRA_FIELD_LENGTH]:
                  zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH])
  
              if fname != zinfo.orig_filename:
                  raise BadZipfile, \
                          'File name in directory "%s" and header "%s" differ.' % (
                              zinfo.orig_filename, fname)
  
              # check for encrypted flag & handle password
              is_encrypted = zinfo.flag_bits & 0x1
              zd = None
              if is_encrypted:
                  if not pwd:
                      pwd = self.pwd
                  if not pwd:
                      raise RuntimeError, "File %s is encrypted, " \
                          "password required for extraction" % name
  
                  zd = _ZipDecrypter(pwd)
                  # The first 12 bytes in the cypher stream is an encryption header
                  #  used to strengthen the algorithm. The first 11 bytes are
                  #  completely random, while the 12th contains the MSB of the CRC,
                  #  or the MSB of the file time depending on the header type
                  #  and is used to check the correctness of the password.
                  bytes = zef_file.read(12)
                  h = map(zd, bytes[0:12])
                  if zinfo.flag_bits & 0x8:
                      # compare against the file type from extended local headers
                      check_byte = (zinfo._raw_time >> 8) & 0xff
                  else:
                      # compare against the CRC otherwise
                      check_byte = (zinfo.CRC >> 24) & 0xff
                  if ord(h[11]) != check_byte:
                      raise RuntimeError("Bad password for file", name)
  
              return ZipExtFile(zef_file, mode, zinfo, zd,
                      close_fileobj=should_close)
          except:
              if should_close:
                  zef_file.close()
              raise
  
      def extract(self, member, path=None, pwd=None):
          """Extract a member from the archive to the current working directory,
             using its full name. Its file information is extracted as accurately
             as possible. `member' may be a filename or a ZipInfo object. You can
             specify a different directory using `path'.
          """
          if not isinstance(member, ZipInfo):
              member = self.getinfo(member)
  
          if path is None:
              path = os.getcwd()
  
          return self._extract_member(member, path, pwd)
  
      def extractall(self, path=None, members=None, pwd=None):
          """Extract all members from the archive to the current working
             directory. `path' specifies a different directory to extract to.
             `members' is optional and must be a subset of the list returned
             by namelist().
          """
          if members is None:
              members = self.namelist()
  
          for zipinfo in members:
              self.extract(zipinfo, path, pwd)
  
      def _extract_member(self, member, targetpath, pwd):
          """Extract the ZipInfo object 'member' to a physical
             file on the path targetpath.
          """
          # build the destination pathname, replacing
          # forward slashes to platform specific separators.
          arcname = member.filename.replace('/', os.path.sep)
  
          if os.path.altsep:
              arcname = arcname.replace(os.path.altsep, os.path.sep)
          # interpret absolute pathname as relative, remove drive letter or
          # UNC path, redundant separators, "." and ".." components.
          arcname = os.path.splitdrive(arcname)[1]
          arcname = os.path.sep.join(x for x in arcname.split(os.path.sep)
                      if x not in ('', os.path.curdir, os.path.pardir))
          if os.path.sep == '\\':
              # filter illegal characters on Windows
              illegal = ':<>|"?*'
              if isinstance(arcname, unicode):
                  table = {ord(c): ord('_') for c in illegal}
              else:
                  table = string.maketrans(illegal, '_' * len(illegal))
              arcname = arcname.translate(table)
              # remove trailing dots
              arcname = (x.rstrip('.') for x in arcname.split(os.path.sep))
              arcname = os.path.sep.join(x for x in arcname if x)
  
          targetpath = os.path.join(targetpath, arcname)
          targetpath = os.path.normpath(targetpath)
  
          # Create all upper directories if necessary.
          upperdirs = os.path.dirname(targetpath)
          if upperdirs and not os.path.exists(upperdirs):
              os.makedirs(upperdirs)
  
          if member.filename[-1] == '/':
              if not os.path.isdir(targetpath):
                  os.mkdir(targetpath)
              return targetpath
  
          with self.open(member, pwd=pwd) as source, \
               file(targetpath, "wb") as target:
              shutil.copyfileobj(source, target)
  
          return targetpath
  
      def _writecheck(self, zinfo):
          """Check for errors before writing a file to the archive."""
          if zinfo.filename in self.NameToInfo:
              import warnings
              warnings.warn('Duplicate name: %r' % zinfo.filename, stacklevel=3)
          if self.mode not in ("w", "a"):
              raise RuntimeError, 'write() requires mode "w" or "a"'
          if not self.fp:
              raise RuntimeError, \
                    "Attempt to write ZIP archive that was already closed"
          if zinfo.compress_type == ZIP_DEFLATED and not zlib:
              raise RuntimeError, \
                    "Compression requires the (missing) zlib module"
          if zinfo.compress_type not in (ZIP_STORED, ZIP_DEFLATED):
              raise RuntimeError, \
                    "That compression method is not supported"
          if not self._allowZip64:
              requires_zip64 = None
              if len(self.filelist) >= ZIP_FILECOUNT_LIMIT:
                  requires_zip64 = "Files count"
              elif zinfo.file_size > ZIP64_LIMIT:
                  requires_zip64 = "Filesize"
              elif zinfo.header_offset > ZIP64_LIMIT:
                  requires_zip64 = "Zipfile size"
              if requires_zip64:
                  raise LargeZipFile(requires_zip64 +
                                     " would require ZIP64 extensions")
  
      def write(self, filename, arcname=None, compress_type=None):
          """Put the bytes from filename into the archive under the name
          arcname."""
          if not self.fp:
              raise RuntimeError(
                    "Attempt to write to ZIP archive that was already closed")
  
          st = os.stat(filename)
          isdir = stat.S_ISDIR(st.st_mode)
          mtime = time.localtime(st.st_mtime)
          date_time = mtime[0:6]
          # Create ZipInfo instance to store file information
          if arcname is None:
              arcname = filename
          arcname = os.path.normpath(os.path.splitdrive(arcname)[1])
          while arcname[0] in (os.sep, os.altsep):
              arcname = arcname[1:]
          if isdir:
              arcname += '/'
          zinfo = ZipInfo(arcname, date_time)
          zinfo.external_attr = (st[0] & 0xFFFF) << 16L      # Unix attributes
          if compress_type is None:
              zinfo.compress_type = self.compression
          else:
              zinfo.compress_type = compress_type
  
          zinfo.file_size = st.st_size
          zinfo.flag_bits = 0x00
          zinfo.header_offset = self.fp.tell()    # Start of header bytes
  
          self._writecheck(zinfo)
          self._didModify = True
  
          if isdir:
              zinfo.file_size = 0
              zinfo.compress_size = 0
              zinfo.CRC = 0
              zinfo.external_attr |= 0x10  # MS-DOS directory flag
              self.filelist.append(zinfo)
              self.NameToInfo[zinfo.filename] = zinfo
              self.fp.write(zinfo.FileHeader(False))
              return
  
          with open(filename, "rb") as fp:
              # Must overwrite CRC and sizes with correct data later
              zinfo.CRC = CRC = 0
              zinfo.compress_size = compress_size = 0
              # Compressed size can be larger than uncompressed size
              zip64 = self._allowZip64 and \
                      zinfo.file_size * 1.05 > ZIP64_LIMIT
              self.fp.write(zinfo.FileHeader(zip64))
              if zinfo.compress_type == ZIP_DEFLATED:
                  cmpr = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
                       zlib.DEFLATED, -15)
              else:
                  cmpr = None
              file_size = 0
              while 1:
                  buf = fp.read(1024 * 8)
                  if not buf:
                      break
                  file_size = file_size + len(buf)
                  CRC = crc32(buf, CRC) & 0xffffffff
                  if cmpr:
                      buf = cmpr.compress(buf)
                      compress_size = compress_size + len(buf)
                  self.fp.write(buf)
          if cmpr:
              buf = cmpr.flush()
              compress_size = compress_size + len(buf)
              self.fp.write(buf)
              zinfo.compress_size = compress_size
          else:
              zinfo.compress_size = file_size
          zinfo.CRC = CRC
          zinfo.file_size = file_size
          if not zip64 and self._allowZip64:
              if file_size > ZIP64_LIMIT:
                  raise RuntimeError('File size has increased during compressing')
              if compress_size > ZIP64_LIMIT:
                  raise RuntimeError('Compressed size larger than uncompressed size')
          # Seek backwards and write file header (which will now include
          # correct CRC and file sizes)
          position = self.fp.tell()       # Preserve current position in file
          self.fp.seek(zinfo.header_offset, 0)
          self.fp.write(zinfo.FileHeader(zip64))
          self.fp.seek(position, 0)
          self.filelist.append(zinfo)
          self.NameToInfo[zinfo.filename] = zinfo
  
      def writestr(self, zinfo_or_arcname, bytes, compress_type=None):
          """Write a file into the archive.  The contents is the string
          'bytes'.  'zinfo_or_arcname' is either a ZipInfo instance or
          the name of the file in the archive."""
          if not isinstance(zinfo_or_arcname, ZipInfo):
              zinfo = ZipInfo(filename=zinfo_or_arcname,
                              date_time=time.localtime(time.time())[:6])
  
              zinfo.compress_type = self.compression
              if zinfo.filename[-1] == '/':
                  zinfo.external_attr = 0o40775 << 16   # drwxrwxr-x
                  zinfo.external_attr |= 0x10           # MS-DOS directory flag
              else:
                  zinfo.external_attr = 0o600 << 16     # ?rw-------
          else:
              zinfo = zinfo_or_arcname
  
          if not self.fp:
              raise RuntimeError(
                    "Attempt to write to ZIP archive that was already closed")
  
          if compress_type is not None:
              zinfo.compress_type = compress_type
  
          zinfo.file_size = len(bytes)            # Uncompressed size
          zinfo.header_offset = self.fp.tell()    # Start of header bytes
          self._writecheck(zinfo)
          self._didModify = True
          zinfo.CRC = crc32(bytes) & 0xffffffff       # CRC-32 checksum
          if zinfo.compress_type == ZIP_DEFLATED:
              co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
                   zlib.DEFLATED, -15)
              bytes = co.compress(bytes) + co.flush()
              zinfo.compress_size = len(bytes)    # Compressed size
          else:
              zinfo.compress_size = zinfo.file_size
          zip64 = zinfo.file_size > ZIP64_LIMIT or \
                  zinfo.compress_size > ZIP64_LIMIT
          if zip64 and not self._allowZip64:
              raise LargeZipFile("Filesize would require ZIP64 extensions")
          self.fp.write(zinfo.FileHeader(zip64))
          self.fp.write(bytes)
          if zinfo.flag_bits & 0x08:
              # Write CRC and file sizes after the file data
              fmt = '<LQQ' if zip64 else '<LLL'
              self.fp.write(struct.pack(fmt, zinfo.CRC, zinfo.compress_size,
                    zinfo.file_size))
          self.fp.flush()
          self.filelist.append(zinfo)
          self.NameToInfo[zinfo.filename] = zinfo
  
      def __del__(self):
          """Call the "close()" method in case the user forgot."""
          self.close()
  
      def close(self):
          """Close the file, and for mode "w" and "a" write the ending
          records."""
          if self.fp is None:
              return
  
          try:
              if self.mode in ("w", "a") and self._didModify: # write ending records
                  pos1 = self.fp.tell()
                  for zinfo in self.filelist:         # write central directory
                      dt = zinfo.date_time
                      dosdate = (dt[0] - 1980) << 9 | dt[1] << 5 | dt[2]
                      dostime = dt[3] << 11 | dt[4] << 5 | (dt[5] // 2)
                      extra = []
                      if zinfo.file_size > ZIP64_LIMIT \
                              or zinfo.compress_size > ZIP64_LIMIT:
                          extra.append(zinfo.file_size)
                          extra.append(zinfo.compress_size)
                          file_size = 0xffffffff
                          compress_size = 0xffffffff
                      else:
                          file_size = zinfo.file_size
                          compress_size = zinfo.compress_size
  
                      if zinfo.header_offset > ZIP64_LIMIT:
                          extra.append(zinfo.header_offset)
                          header_offset = 0xffffffffL
                      else:
                          header_offset = zinfo.header_offset
  
                      extra_data = zinfo.extra
                      if extra:
                          # Append a ZIP64 field to the extra's
                          extra_data = struct.pack(
                                  '<HH' + 'Q'*len(extra),
                                  1, 8*len(extra), *extra) + extra_data
  
                          extract_version = max(45, zinfo.extract_version)
                          create_version = max(45, zinfo.create_version)
                      else:
                          extract_version = zinfo.extract_version
                          create_version = zinfo.create_version
  
                      try:
                          filename, flag_bits = zinfo._encodeFilenameFlags()
                          centdir = struct.pack(structCentralDir,
                          stringCentralDir, create_version,
                          zinfo.create_system, extract_version, zinfo.reserved,
                          flag_bits, zinfo.compress_type, dostime, dosdate,
                          zinfo.CRC, compress_size, file_size,
                          len(filename), len(extra_data), len(zinfo.comment),
                          0, zinfo.internal_attr, zinfo.external_attr,
                          header_offset)
                      except DeprecationWarning:
                          print >>sys.stderr, (structCentralDir,
                          stringCentralDir, create_version,
                          zinfo.create_system, extract_version, zinfo.reserved,
                          zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
                          zinfo.CRC, compress_size, file_size,
                          len(zinfo.filename), len(extra_data), len(zinfo.comment),
                          0, zinfo.internal_attr, zinfo.external_attr,
                          header_offset)
                          raise
                      self.fp.write(centdir)
                      self.fp.write(filename)
                      self.fp.write(extra_data)
                      self.fp.write(zinfo.comment)
  
                  pos2 = self.fp.tell()
                  # Write end-of-zip-archive record
                  centDirCount = len(self.filelist)
                  centDirSize = pos2 - pos1
                  centDirOffset = pos1
                  requires_zip64 = None
                  if centDirCount > ZIP_FILECOUNT_LIMIT:
                      requires_zip64 = "Files count"
                  elif centDirOffset > ZIP64_LIMIT:
                      requires_zip64 = "Central directory offset"
                  elif centDirSize > ZIP64_LIMIT:
                      requires_zip64 = "Central directory size"
                  if requires_zip64:
                      # Need to write the ZIP64 end-of-archive records
                      if not self._allowZip64:
                          raise LargeZipFile(requires_zip64 +
                                             " would require ZIP64 extensions")
                      zip64endrec = struct.pack(
                              structEndArchive64, stringEndArchive64,
                              44, 45, 45, 0, 0, centDirCount, centDirCount,
                              centDirSize, centDirOffset)
                      self.fp.write(zip64endrec)
  
                      zip64locrec = struct.pack(
                              structEndArchive64Locator,
                              stringEndArchive64Locator, 0, pos2, 1)
                      self.fp.write(zip64locrec)
                      centDirCount = min(centDirCount, 0xFFFF)
                      centDirSize = min(centDirSize, 0xFFFFFFFF)
                      centDirOffset = min(centDirOffset, 0xFFFFFFFF)
  
                  endrec = struct.pack(structEndArchive, stringEndArchive,
                                      0, 0, centDirCount, centDirCount,
                                      centDirSize, centDirOffset, len(self._comment))
                  self.fp.write(endrec)
                  self.fp.write(self._comment)
                  self.fp.flush()
          finally:
              fp = self.fp
              self.fp = None
              if not self._filePassed:
                  fp.close()

  class TarFile(object):
      """The TarFile Class provides an interface to tar archives.
      """
  
      debug = 0                   # May be set from 0 (no msgs) to 3 (all msgs)
  
      dereference = False         # If true, add content of linked file to the
                                  # tar file, else the link.
  
      ignore_zeros = False        # If true, skips empty or invalid blocks and
                                  # continues processing.
  
      errorlevel = 1              # If 0, fatal errors only appear in debug
                                  # messages (if debug >= 0). If > 0, errors
                                  # are passed to the caller as exceptions.
  
      format = DEFAULT_FORMAT     # The format to use when creating an archive.
  
      encoding = ENCODING         # Encoding for 8-bit character strings.
  
      errors = None               # Error handler for unicode conversion.
  
      tarinfo = TarInfo           # The default TarInfo class to use.
  
      fileobject = ExFileObject   # The default ExFileObject class to use.
  
      def __init__(self, name=None, mode="r", fileobj=None, format=None,
              tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,
              errors=None, pax_headers=None, debug=None, errorlevel=None):
          """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to
             read from an existing archive, 'a' to append data to an existing
             file or 'w' to create a new file overwriting an existing one. `mode'
             defaults to 'r'.
             If `fileobj' is given, it is used for reading or writing data. If it
             can be determined, `mode' is overridden by `fileobj's mode.
             `fileobj' is not closed, when TarFile is closed.
          """
          modes = {"r": "rb", "a": "r+b", "w": "wb"}
          if mode not in modes:
              raise ValueError("mode must be 'r', 'a' or 'w'")
          self.mode = mode
          self._mode = modes[mode]
  
          if not fileobj:
              if self.mode == "a" and not os.path.exists(name):
                  # Create nonexistent files in append mode.
                  self.mode = "w"
                  self._mode = "wb"
              fileobj = bltn_open(name, self._mode)
              self._extfileobj = False
          else:
              if name is None and hasattr(fileobj, "name"):
                  name = fileobj.name
              if hasattr(fileobj, "mode"):
                  self._mode = fileobj.mode
              self._extfileobj = True
          self.name = os.path.abspath(name) if name else None
          self.fileobj = fileobj
  
          # Init attributes.
          if format is not None:
              self.format = format
          if tarinfo is not None:
              self.tarinfo = tarinfo
          if dereference is not None:
              self.dereference = dereference
          if ignore_zeros is not None:
              self.ignore_zeros = ignore_zeros
          if encoding is not None:
              self.encoding = encoding
  
          if errors is not None:
              self.errors = errors
          elif mode == "r":
              self.errors = "utf-8"
          else:
              self.errors = "strict"
  
          if pax_headers is not None and self.format == PAX_FORMAT:
              self.pax_headers = pax_headers
          else:
              self.pax_headers = {}
  
          if debug is not None:
              self.debug = debug
          if errorlevel is not None:
              self.errorlevel = errorlevel
  
          # Init datastructures.
          self.closed = False
          self.members = []       # list of members as TarInfo objects
          self._loaded = False    # flag if all members have been read
          self.offset = self.fileobj.tell()
                                  # current position in the archive file
          self.inodes = {}        # dictionary caching the inodes of
                                  # archive members already added
  
          try:
              if self.mode == "r":
                  self.firstmember = None
                  self.firstmember = self.next()
  
              if self.mode == "a":
                  # Move to the end of the archive,
                  # before the first empty block.
                  while True:
                      self.fileobj.seek(self.offset)
                      try:
                          tarinfo = self.tarinfo.fromtarfile(self)
                          self.members.append(tarinfo)
                      except EOFHeaderError:
                          self.fileobj.seek(self.offset)
                          break
                      except HeaderError, e:
                          raise ReadError(str(e))
  
              if self.mode in "aw":
                  self._loaded = True
  
                  if self.pax_headers:
                      buf = self.tarinfo.create_pax_global_header(self.pax_headers.copy())
                      self.fileobj.write(buf)
                      self.offset += len(buf)
          except:
              if not self._extfileobj:
                  self.fileobj.close()
              self.closed = True
              raise
  
      def _getposix(self):
          return self.format == USTAR_FORMAT
      def _setposix(self, value):
          import warnings
          warnings.warn("use the format attribute instead", DeprecationWarning,
                        2)
          if value:
              self.format = USTAR_FORMAT
          else:
              self.format = GNU_FORMAT
      posix = property(_getposix, _setposix)
  
      #--------------------------------------------------------------------------
      # Below are the classmethods which act as alternate constructors to the
      # TarFile class. The open() method is the only one that is needed for
      # public use; it is the "super"-constructor and is able to select an
      # adequate "sub"-constructor for a particular compression using the mapping
      # from OPEN_METH.
      #
      # This concept allows one to subclass TarFile without losing the comfort of
      # the super-constructor. A sub-constructor is registered and made available
      # by adding it to the mapping in OPEN_METH.
  
      @classmethod
      def open(cls, name=None, mode="r", fileobj=None, bufsize=RECORDSIZE, **kwargs):
          """Open a tar archive for reading, writing or appending. Return
             an appropriate TarFile class.
  
             mode:
             'r' or 'r:*' open for reading with transparent compression
             'r:'         open for reading exclusively uncompressed
             'r:gz'       open for reading with gzip compression
             'r:bz2'      open for reading with bzip2 compression
             'a' or 'a:'  open for appending, creating the file if necessary
             'w' or 'w:'  open for writing without compression
             'w:gz'       open for writing with gzip compression
             'w:bz2'      open for writing with bzip2 compression
  
             'r|*'        open a stream of tar blocks with transparent compression
             'r|'         open an uncompressed stream of tar blocks for reading
             'r|gz'       open a gzip compressed stream of tar blocks
             'r|bz2'      open a bzip2 compressed stream of tar blocks
             'w|'         open an uncompressed stream for writing
             'w|gz'       open a gzip compressed stream for writing
             'w|bz2'      open a bzip2 compressed stream for writing
          """
  
          if not name and not fileobj:
              raise ValueError("nothing to open")
  
          if mode in ("r", "r:*"):
              # Find out which *open() is appropriate for opening the file.
              for comptype in cls.OPEN_METH:
                  func = getattr(cls, cls.OPEN_METH[comptype])
                  if fileobj is not None:
                      saved_pos = fileobj.tell()
                  try:
                      return func(name, "r", fileobj, **kwargs)
                  except (ReadError, CompressionError), e:
                      if fileobj is not None:
                          fileobj.seek(saved_pos)
                      continue
              raise ReadError("file could not be opened successfully")
  
          elif ":" in mode:
              filemode, comptype = mode.split(":", 1)
              filemode = filemode or "r"
              comptype = comptype or "tar"
  
              # Select the *open() function according to
              # given compression.
              if comptype in cls.OPEN_METH:
                  func = getattr(cls, cls.OPEN_METH[comptype])
              else:
                  raise CompressionError("unknown compression type %r" % comptype)
              return func(name, filemode, fileobj, **kwargs)
  
          elif "|" in mode:
              filemode, comptype = mode.split("|", 1)
              filemode = filemode or "r"
              comptype = comptype or "tar"
  
              if filemode not in ("r", "w"):
                  raise ValueError("mode must be 'r' or 'w'")
  
              stream = _Stream(name, filemode, comptype, fileobj, bufsize)
              try:
                  t = cls(name, filemode, stream, **kwargs)
              except:
                  stream.close()
                  raise
              t._extfileobj = False
              return t
  
          elif mode in ("a", "w"):
              return cls.taropen(name, mode, fileobj, **kwargs)
  
          raise ValueError("undiscernible mode")
  
      @classmethod
      def taropen(cls, name, mode="r", fileobj=None, **kwargs):
          """Open uncompressed tar archive name for reading or writing.
          """
          if mode not in ("r", "a", "w"):
              raise ValueError("mode must be 'r', 'a' or 'w'")
          return cls(name, mode, fileobj, **kwargs)
  
      @classmethod
      def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
          """Open gzip compressed tar archive name for reading or writing.
             Appending is not allowed.
          """
          if mode not in ("r", "w"):
              raise ValueError("mode must be 'r' or 'w'")
  
          try:
              import gzip
              gzip.GzipFile
          except (ImportError, AttributeError):
              raise CompressionError("gzip module is not available")
  
          try:
              fileobj = gzip.GzipFile(name, mode, compresslevel, fileobj)
          except OSError:
              if fileobj is not None and mode == 'r':
                  raise ReadError("not a gzip file")
              raise
  
          try:
              t = cls.taropen(name, mode, fileobj, **kwargs)
          except IOError:
              fileobj.close()
              if mode == 'r':
                  raise ReadError("not a gzip file")
              raise
          except:
              fileobj.close()
              raise
          t._extfileobj = False
          return t
  
      @classmethod
      def bz2open(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
          """Open bzip2 compressed tar archive name for reading or writing.
             Appending is not allowed.
          """
          if mode not in ("r", "w"):
              raise ValueError("mode must be 'r' or 'w'.")
  
          try:
              import bz2
          except ImportError:
              raise CompressionError("bz2 module is not available")
  
          if fileobj is not None:
              fileobj = _BZ2Proxy(fileobj, mode)
          else:
              fileobj = bz2.BZ2File(name, mode, compresslevel=compresslevel)
  
          try:
              t = cls.taropen(name, mode, fileobj, **kwargs)
          except (IOError, EOFError):
              fileobj.close()
              if mode == 'r':
                  raise ReadError("not a bzip2 file")
              raise
          except:
              fileobj.close()
              raise
          t._extfileobj = False
          return t
  
      # All *open() methods are registered here.
      OPEN_METH = {
          "tar": "taropen",   # uncompressed tar
          "gz":  "gzopen",    # gzip compressed tar
          "bz2": "bz2open"    # bzip2 compressed tar
      }
  
      #--------------------------------------------------------------------------
      # The public methods which TarFile provides:
  
      def close(self):
          """Close the TarFile. In write-mode, two finishing zero blocks are
             appended to the archive.
          """
          if self.closed:
              return
  
          if self.mode in "aw":
              self.fileobj.write(NUL * (BLOCKSIZE * 2))
              self.offset += (BLOCKSIZE * 2)
              # fill up the end with zero-blocks
              # (like option -b20 for tar does)
              blocks, remainder = divmod(self.offset, RECORDSIZE)
              if remainder > 0:
                  self.fileobj.write(NUL * (RECORDSIZE - remainder))
  
          if not self._extfileobj:
              self.fileobj.close()
          self.closed = True
  
      def getmember(self, name):
          """Return a TarInfo object for member `name'. If `name' can not be
             found in the archive, KeyError is raised. If a member occurs more
             than once in the archive, its last occurrence is assumed to be the
             most up-to-date version.
          """
          tarinfo = self._getmember(name)
          if tarinfo is None:
              raise KeyError("filename %r not found" % name)
          return tarinfo
  
      def getmembers(self):
          """Return the members of the archive as a list of TarInfo objects. The
             list has the same order as the members in the archive.
          """
          self._check()
          if not self._loaded:    # if we want to obtain a list of
              self._load()        # all members, we first have to
                                  # scan the whole archive.
          return self.members
  
      def getnames(self):
          """Return the members of the archive as a list of their names. It has
             the same order as the list returned by getmembers().
          """
          return [tarinfo.name for tarinfo in self.getmembers()]
  
      def gettarinfo(self, name=None, arcname=None, fileobj=None):
          """Create a TarInfo object for either the file `name' or the file
             object `fileobj' (using os.fstat on its file descriptor). You can
             modify some of the TarInfo's attributes before you add it using
             addfile(). If given, `arcname' specifies an alternative name for the
             file in the archive.
          """
          self._check("aw")
  
          # When fileobj is given, replace name by
          # fileobj's real name.
          if fileobj is not None:
              name = fileobj.name
  
          # Building the name of the member in the archive.
          # Backward slashes are converted to forward slashes,
          # Absolute paths are turned to relative paths.
          if arcname is None:
              arcname = name
          drv, arcname = os.path.splitdrive(arcname)
          arcname = arcname.replace(os.sep, "/")
          arcname = arcname.lstrip("/")
  
          # Now, fill the TarInfo object with
          # information specific for the file.
          tarinfo = self.tarinfo()
          tarinfo.tarfile = self
  
          # Use os.stat or os.lstat, depending on platform
          # and if symlinks shall be resolved.
          if fileobj is None:
              if hasattr(os, "lstat") and not self.dereference:
                  statres = os.lstat(name)
              else:
                  statres = os.stat(name)
          else:
              statres = os.fstat(fileobj.fileno())
          linkname = ""
  
          stmd = statres.st_mode
          if stat.S_ISREG(stmd):
              inode = (statres.st_ino, statres.st_dev)
              if not self.dereference and statres.st_nlink > 1 and \
                      inode in self.inodes and arcname != self.inodes[inode]:
                  # Is it a hardlink to an already
                  # archived file?
                  type = LNKTYPE
                  linkname = self.inodes[inode]
              else:
                  # The inode is added only if its valid.
                  # For win32 it is always 0.
                  type = REGTYPE
                  if inode[0]:
                      self.inodes[inode] = arcname
          elif stat.S_ISDIR(stmd):
              type = DIRTYPE
          elif stat.S_ISFIFO(stmd):
              type = FIFOTYPE
          elif stat.S_ISLNK(stmd):
              type = SYMTYPE
              linkname = os.readlink(name)
          elif stat.S_ISCHR(stmd):
              type = CHRTYPE
          elif stat.S_ISBLK(stmd):
              type = BLKTYPE
          else:
              return None
  
          # Fill the TarInfo object with all
          # information we can get.
          tarinfo.name = arcname
          tarinfo.mode = stmd
          tarinfo.uid = statres.st_uid
          tarinfo.gid = statres.st_gid
          if type == REGTYPE:
              tarinfo.size = statres.st_size
          else:
              tarinfo.size = 0L
          tarinfo.mtime = statres.st_mtime
          tarinfo.type = type
          tarinfo.linkname = linkname
          if pwd:
              try:
                  tarinfo.uname = pwd.getpwuid(tarinfo.uid)[0]
              except KeyError:
                  pass
          if grp:
              try:
                  tarinfo.gname = grp.getgrgid(tarinfo.gid)[0]
              except KeyError:
                  pass
  
          if type in (CHRTYPE, BLKTYPE):
              if hasattr(os, "major") and hasattr(os, "minor"):
                  tarinfo.devmajor = os.major(statres.st_rdev)
                  tarinfo.devminor = os.minor(statres.st_rdev)
          return tarinfo
  
      def list(self, verbose=True):
          """Print a table of contents to sys.stdout. If `verbose' is False, only
             the names of the members are printed. If it is True, an `ls -l'-like
             output is produced.
          """
          self._check()
  
          for tarinfo in self:
              if verbose:
                  print filemode(tarinfo.mode),
                  print "%s/%s" % (tarinfo.uname or tarinfo.uid,
                                   tarinfo.gname or tarinfo.gid),
                  if tarinfo.ischr() or tarinfo.isblk():
                      print "%10s" % ("%d,%d" \
                                      % (tarinfo.devmajor, tarinfo.devminor)),
                  else:
                      print "%10d" % tarinfo.size,
                  print "%d-%02d-%02d %02d:%02d:%02d" \
                        % time.localtime(tarinfo.mtime)[:6],
  
              print tarinfo.name + ("/" if tarinfo.isdir() else ""),
  
              if verbose:
                  if tarinfo.issym():
                      print "->", tarinfo.linkname,
                  if tarinfo.islnk():
                      print "link to", tarinfo.linkname,
              print
  
      def add(self, name, arcname=None, recursive=True, exclude=None, filter=None):
          """Add the file `name' to the archive. `name' may be any type of file
             (directory, fifo, symbolic link, etc.). If given, `arcname'
             specifies an alternative name for the file in the archive.
             Directories are added recursively by default. This can be avoided by
             setting `recursive' to False. `exclude' is a function that should
             return True for each filename to be excluded. `filter' is a function
             that expects a TarInfo object argument and returns the changed
             TarInfo object, if it returns None the TarInfo object will be
             excluded from the archive.
          """
          self._check("aw")
  
          if arcname is None:
              arcname = name
  
          # Exclude pathnames.
          if exclude is not None:
              import warnings
              warnings.warn("use the filter argument instead",
                      DeprecationWarning, 2)
              if exclude(name):
                  self._dbg(2, "tarfile: Excluded %r" % name)
                  return
  
          # Skip if somebody tries to archive the archive...
          if self.name is not None and os.path.abspath(name) == self.name:
              self._dbg(2, "tarfile: Skipped %r" % name)
              return
  
          self._dbg(1, name)
  
          # Create a TarInfo object from the file.
          tarinfo = self.gettarinfo(name, arcname)
  
          if tarinfo is None:
              self._dbg(1, "tarfile: Unsupported type %r" % name)
              return
  
          # Change or exclude the TarInfo object.
          if filter is not None:
              tarinfo = filter(tarinfo)
              if tarinfo is None:
                  self._dbg(2, "tarfile: Excluded %r" % name)
                  return
  
          # Append the tar header and data to the archive.
          if tarinfo.isreg():
              with bltn_open(name, "rb") as f:
                  self.addfile(tarinfo, f)
  
          elif tarinfo.isdir():
              self.addfile(tarinfo)
              if recursive:
                  for f in os.listdir(name):
                      self.add(os.path.join(name, f), os.path.join(arcname, f),
                              recursive, exclude, filter)
  
          else:
              self.addfile(tarinfo)
  
      def addfile(self, tarinfo, fileobj=None):
          """Add the TarInfo object `tarinfo' to the archive. If `fileobj' is
             given, tarinfo.size bytes are read from it and added to the archive.
             You can create TarInfo objects using gettarinfo().
             On Windows platforms, `fileobj' should always be opened with mode
             'rb' to avoid irritation about the file size.
          """
          self._check("aw")
  
          tarinfo = copy.copy(tarinfo)
  
          buf = tarinfo.tobuf(self.format, self.encoding, self.errors)
          self.fileobj.write(buf)
          self.offset += len(buf)
  
          # If there's data to follow, append it.
          if fileobj is not None:
              copyfileobj(fileobj, self.fileobj, tarinfo.size)
              blocks, remainder = divmod(tarinfo.size, BLOCKSIZE)
              if remainder > 0:
                  self.fileobj.write(NUL * (BLOCKSIZE - remainder))
                  blocks += 1
              self.offset += blocks * BLOCKSIZE
  
          self.members.append(tarinfo)
  
      def extractall(self, path=".", members=None):
          """Extract all members from the archive to the current working
             directory and set owner, modification time and permissions on
             directories afterwards. `path' specifies a different directory
             to extract to. `members' is optional and must be a subset of the
             list returned by getmembers().
          """
          directories = []
  
          if members is None:
              members = self
  
          for tarinfo in members:
              if tarinfo.isdir():
                  # Extract directories with a safe mode.
                  directories.append(tarinfo)
                  tarinfo = copy.copy(tarinfo)
                  tarinfo.mode = 0700
              self.extract(tarinfo, path)
  
          # Reverse sort directories.
          directories.sort(key=operator.attrgetter('name'))
          directories.reverse()
  
          # Set correct owner, mtime and filemode on directories.
          for tarinfo in directories:
              dirpath = os.path.join(path, tarinfo.name)
              try:
                  self.chown(tarinfo, dirpath)
                  self.utime(tarinfo, dirpath)
                  self.chmod(tarinfo, dirpath)
              except ExtractError, e:
                  if self.errorlevel > 1:
                      raise
                  else:
                      self._dbg(1, "tarfile: %s" % e)
  
      def extract(self, member, path=""):
          """Extract a member from the archive to the current working directory,
             using its full name. Its file information is extracted as accurately
             as possible. `member' may be a filename or a TarInfo object. You can
             specify a different directory using `path'.
          """
          self._check("r")
  
          if isinstance(member, basestring):
              tarinfo = self.getmember(member)
          else:
              tarinfo = member
  
          # Prepare the link target for makelink().
          if tarinfo.islnk():
              tarinfo._link_target = os.path.join(path, tarinfo.linkname)
  
          try:
              self._extract_member(tarinfo, os.path.join(path, tarinfo.name))
          except EnvironmentError, e:
              if self.errorlevel > 0:
                  raise
              else:
                  if e.filename is None:
                      self._dbg(1, "tarfile: %s" % e.strerror)
                  else:
                      self._dbg(1, "tarfile: %s %r" % (e.strerror, e.filename))
          except ExtractError, e:
              if self.errorlevel > 1:
                  raise
              else:
                  self._dbg(1, "tarfile: %s" % e)
  
      def extractfile(self, member):
          """Extract a member from the archive as a file object. `member' may be
             a filename or a TarInfo object. If `member' is a regular file, a
             file-like object is returned. If `member' is a link, a file-like
             object is constructed from the link's target. If `member' is none of
             the above, None is returned.
             The file-like object is read-only and provides the following
             methods: read(), readline(), readlines(), seek() and tell()
          """
          self._check("r")
  
          if isinstance(member, basestring):
              tarinfo = self.getmember(member)
          else:
              tarinfo = member
  
          if tarinfo.isreg():
              return self.fileobject(self, tarinfo)
  
          elif tarinfo.type not in SUPPORTED_TYPES:
              # If a member's type is unknown, it is treated as a
              # regular file.
              return self.fileobject(self, tarinfo)
  
          elif tarinfo.islnk() or tarinfo.issym():
              if isinstance(self.fileobj, _Stream):
                  # A small but ugly workaround for the case that someone tries
                  # to extract a (sym)link as a file-object from a non-seekable
                  # stream of tar blocks.
                  raise StreamError("cannot extract (sym)link as file object")
              else:
                  # A (sym)link's file object is its target's file object.
                  return self.extractfile(self._find_link_target(tarinfo))
          else:
              # If there's no data associated with the member (directory, chrdev,
              # blkdev, etc.), return None instead of a file object.
              return None
  
      def _extract_member(self, tarinfo, targetpath):
          """Extract the TarInfo object tarinfo to a physical
             file called targetpath.
          """
          # Fetch the TarInfo object for the given name
          # and build the destination pathname, replacing
          # forward slashes to platform specific separators.
          targetpath = targetpath.rstrip("/")
          targetpath = targetpath.replace("/", os.sep)
  
          # Create all upper directories.
          upperdirs = os.path.dirname(targetpath)
          if upperdirs and not os.path.exists(upperdirs):
              # Create directories that are not part of the archive with
              # default permissions.
              os.makedirs(upperdirs)
  
          if tarinfo.islnk() or tarinfo.issym():
              self._dbg(1, "%s -> %s" % (tarinfo.name, tarinfo.linkname))
          else:
              self._dbg(1, tarinfo.name)
  
          if tarinfo.isreg():
              self.makefile(tarinfo, targetpath)
          elif tarinfo.isdir():
              self.makedir(tarinfo, targetpath)
          elif tarinfo.isfifo():
              self.makefifo(tarinfo, targetpath)
          elif tarinfo.ischr() or tarinfo.isblk():
              self.makedev(tarinfo, targetpath)
          elif tarinfo.islnk() or tarinfo.issym():
              self.makelink(tarinfo, targetpath)
          elif tarinfo.type not in SUPPORTED_TYPES:
              self.makeunknown(tarinfo, targetpath)
          else:
              self.makefile(tarinfo, targetpath)
  
          self.chown(tarinfo, targetpath)
          if not tarinfo.issym():
              self.chmod(tarinfo, targetpath)
              self.utime(tarinfo, targetpath)
  
      #--------------------------------------------------------------------------
      # Below are the different file methods. They are called via
      # _extract_member() when extract() is called. They can be replaced in a
      # subclass to implement other functionality.
  
      def makedir(self, tarinfo, targetpath):
          """Make a directory called targetpath.
          """
          try:
              # Use a safe mode for the directory, the real mode is set
              # later in _extract_member().
              os.mkdir(targetpath, 0700)
          except EnvironmentError, e:
              if e.errno != errno.EEXIST:
                  raise
  
      def makefile(self, tarinfo, targetpath):
          """Make a file called targetpath.
          """
          source = self.extractfile(tarinfo)
          try:
              with bltn_open(targetpath, "wb") as target:
                  copyfileobj(source, target)
          finally:
              source.close()
  
      def makeunknown(self, tarinfo, targetpath):
          """Make a file from a TarInfo object with an unknown type
             at targetpath.
          """
          self.makefile(tarinfo, targetpath)
          self._dbg(1, "tarfile: Unknown file type %r, " \
                       "extracted as regular file." % tarinfo.type)
  
      def makefifo(self, tarinfo, targetpath):
          """Make a fifo called targetpath.
          """
          if hasattr(os, "mkfifo"):
              os.mkfifo(targetpath)
          else:
              raise ExtractError("fifo not supported by system")
  
      def makedev(self, tarinfo, targetpath):
          """Make a character or block device called targetpath.
          """
          if not hasattr(os, "mknod") or not hasattr(os, "makedev"):
              raise ExtractError("special devices not supported by system")
  
          mode = tarinfo.mode
          if tarinfo.isblk():
              mode |= stat.S_IFBLK
          else:
              mode |= stat.S_IFCHR
  
          os.mknod(targetpath, mode,
                   os.makedev(tarinfo.devmajor, tarinfo.devminor))
  
      def makelink(self, tarinfo, targetpath):
          """Make a (symbolic) link called targetpath. If it cannot be created
            (platform limitation), we try to make a copy of the referenced file
            instead of a link.
          """
          if hasattr(os, "symlink") and hasattr(os, "link"):
              # For systems that support symbolic and hard links.
              if tarinfo.issym():
                  if os.path.lexists(targetpath):
                      os.unlink(targetpath)
                  os.symlink(tarinfo.linkname, targetpath)
              else:
                  # See extract().
                  if os.path.exists(tarinfo._link_target):
                      if os.path.lexists(targetpath):
                          os.unlink(targetpath)
                      os.link(tarinfo._link_target, targetpath)
                  else:
                      self._extract_member(self._find_link_target(tarinfo), targetpath)
          else:
              try:
                  self._extract_member(self._find_link_target(tarinfo), targetpath)
              except KeyError:
                  raise ExtractError("unable to resolve link inside archive")
  
      def chown(self, tarinfo, targetpath):
          """Set owner of targetpath according to tarinfo.
          """
          if pwd and hasattr(os, "geteuid") and os.geteuid() == 0:
              # We have to be root to do so.
              try:
                  g = grp.getgrnam(tarinfo.gname)[2]
              except KeyError:
                  g = tarinfo.gid
              try:
                  u = pwd.getpwnam(tarinfo.uname)[2]
              except KeyError:
                  u = tarinfo.uid
              try:
                  if tarinfo.issym() and hasattr(os, "lchown"):
                      os.lchown(targetpath, u, g)
                  else:
                      if sys.platform != "os2emx":
                          os.chown(targetpath, u, g)
              except EnvironmentError, e:
                  raise ExtractError("could not change owner")
  
      def chmod(self, tarinfo, targetpath):
          """Set file permissions of targetpath according to tarinfo.
          """
          if hasattr(os, 'chmod'):
              try:
                  os.chmod(targetpath, tarinfo.mode)
              except EnvironmentError, e:
                  raise ExtractError("could not change mode")
  
      def utime(self, tarinfo, targetpath):
          """Set modification time of targetpath according to tarinfo.
          """
          if not hasattr(os, 'utime'):
              return
          try:
              os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
          except EnvironmentError, e:
              raise ExtractError("could not change modification time")
  
      #--------------------------------------------------------------------------
      def next(self):
          """Return the next member of the archive as a TarInfo object, when
             TarFile is opened for reading. Return None if there is no more
             available.
          """
          self._check("ra")
          if self.firstmember is not None:
              m = self.firstmember
              self.firstmember = None
              return m
  
          # Read the next block.
          self.fileobj.seek(self.offset)
          tarinfo = None
          while True:
              try:
                  tarinfo = self.tarinfo.fromtarfile(self)
              except EOFHeaderError, e:
                  if self.ignore_zeros:
                      self._dbg(2, "0x%X: %s" % (self.offset, e))
                      self.offset += BLOCKSIZE
                      continue
              except InvalidHeaderError, e:
                  if self.ignore_zeros:
                      self._dbg(2, "0x%X: %s" % (self.offset, e))
                      self.offset += BLOCKSIZE
                      continue
                  elif self.offset == 0:
                      raise ReadError(str(e))
              except EmptyHeaderError:
                  if self.offset == 0:
                      raise ReadError("empty file")
              except TruncatedHeaderError, e:
                  if self.offset == 0:
                      raise ReadError(str(e))
              except SubsequentHeaderError, e:
                  raise ReadError(str(e))
              break
  
          if tarinfo is not None:
              self.members.append(tarinfo)
          else:
              self._loaded = True
  
          return tarinfo
  
      #--------------------------------------------------------------------------
      # Little helper methods:
  
      def _getmember(self, name, tarinfo=None, normalize=False):
          """Find an archive member by name from bottom to top.
             If tarinfo is given, it is used as the starting point.
          """
          # Ensure that all members have been loaded.
          members = self.getmembers()
  
          # Limit the member search list up to tarinfo.
          if tarinfo is not None:
              members = members[:members.index(tarinfo)]
  
          if normalize:
              name = os.path.normpath(name)
  
          for member in reversed(members):
              if normalize:
                  member_name = os.path.normpath(member.name)
              else:
                  member_name = member.name
  
              if name == member_name:
                  return member
  
      def _load(self):
          """Read through the entire archive file and look for readable
             members.
          """
          while True:
              tarinfo = self.next()
              if tarinfo is None:
                  break
          self._loaded = True
  
      def _check(self, mode=None):
          """Check if TarFile is still open, and if the operation's mode
             corresponds to TarFile's mode.
          """
          if self.closed:
              raise IOError("%s is closed" % self.__class__.__name__)
          if mode is not None and self.mode not in mode:
              raise IOError("bad operation for mode %r" % self.mode)
  
      def _find_link_target(self, tarinfo):
          """Find the target member of a symlink or hardlink member in the
             archive.
          """
          if tarinfo.issym():
              # Always search the entire archive.
              linkname = "/".join(filter(None, (os.path.dirname(tarinfo.name), tarinfo.linkname)))
              limit = None
          else:
              # Search the archive before the link, because a hard link is
              # just a reference to an already archived file.
              linkname = tarinfo.linkname
              limit = tarinfo
  
          member = self._getmember(linkname, tarinfo=limit, normalize=True)
          if member is None:
              raise KeyError("linkname %r not found" % linkname)
          return member
  
      def __iter__(self):
          """Provide an iterator object.
          """
          if self._loaded:
              return iter(self.members)
          else:
              return TarIter(self)
  
      def _dbg(self, level, msg):
          """Write debugging output to sys.stderr.
          """
          if level <= self.debug:
              print >> sys.stderr, msg
  
      def __enter__(self):
          self._check()
          return self
  
      def __exit__(self, type, value, traceback):
          if type is None:
              self.close()
          else:
              # An exception occurred. We must not call close() because
              # it would try to write end-of-archive blocks and padding.
              if not self._extfileobj:
                  self.fileobj.close()
              self.closed = True
  # class TarFile

   

5.6json和pickle模塊

json主要應用於Python中的數據交換，pickle應用於不用的編程語言之間的數據交換。

用於序列化的兩個模塊

• json，用於字符串 和 python數據類型間進行轉換
• pickle，用於python特有的類型 和 python的數據類型間進行轉換

Json模塊提供了四個功能：dumps、dump、loads、load

pickle模塊提供了四個功能：dumps、dump、loads、load

下面例子為Python2.7中的例子。

5.7shelve模塊

shelve模塊是一個簡單的k,v將內存數據通過文件持久化的模塊，可以持久化任何pickle可支持的python數據格式。是對pickle的更上一次的封裝。

import shelve
 
d = shelve.open('shelve_test') #打開一個文件
 
class Test(object):
    def __init__(self,n):
        self.n = n
 
 
t = Test(123) 
t2 = Test(123334)
 
name = ["alex","rain","test"]
d["test"] = name #持久化列表
d["t1"] = t      #持久化類
d["t2"] = t2
 
d.close()

5.8xml模塊

xml是實現不同語言或程序之間進行數據交換的協議，跟json差不多，但json使用起來更簡單，不過，古時候，在json還沒誕生的黑暗年代，大家只能選擇用xml呀，至今很多傳統公司如金融行業的很多系統的接口還主要是xml。

xml的格式如下，就是通過<>節點來區別數據結構的:

<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank updated="yes">2</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank updated="yes">5</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank updated="yes">69</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>

xml協議在各個語言里的都 是支持的，在python中可以用以下模塊操作xml

import xml.etree.ElementTree as ET
 
tree = ET.parse("xmltest.xml")
root = tree.getroot()
print(root.tag)
 
#遍歷xml文檔
for child in root:
    print(child.tag, child.attrib)
    for i in child:
        print(i.tag,i.text)
 
#只遍歷year 節點
for node in root.iter('year'):
    print(node.tag,node.text)

修改和刪除xml文檔內容

import xml.etree.ElementTree as ET
 
tree = ET.parse("xmltest.xml")
root = tree.getroot()
 
#修改
for node in root.iter('year'):
    new_year = int(node.text) + 1
    node.text = str(new_year)
    node.set("updated","yes")
 
tree.write("xmltest.xml")
 
 
#刪除node
for country in root.findall('country'):
   rank = int(country.find('rank').text)
   if rank > 50:
     root.remove(country)
 
tree.write('output.xml')

自己 創建Xml

import xml.etree.ElementTree as ET
 
 
new_xml = ET.Element("namelist")
name = ET.SubElement(new_xml,"name",attrib={"enrolled":"yes"})
age = ET.SubElement(name,"age",attrib={"checked":"no"})
sex = ET.SubElement(name,"sex")
sex.text = '33'
name2 = ET.SubElement(new_xml,"name",attrib={"enrolled":"no"})
age = ET.SubElement(name2,"age")
age.text = '19'
 
et = ET.ElementTree(new_xml) #生成文檔對象
et.write("test.xml", encoding="utf-8",xml_declaration=True)
 
ET.dump(new_xml) #打印生成的格式

5.9PyYAML模塊

pyyaml主要實現配置文檔的書寫。

load一下，編程字典的格式。

Python也可以很容易的處理ymal文檔格式，只不過需要安裝一個模塊，參考文檔：http://pyyaml.org/wiki/PyYAMLDocumentation

5.10ConfigParser模塊

用於生成和修改常見配置文檔，當前模塊的名稱在 python 3.x 版本中變更為 configparser。

一般都是讀配置文件，解析配置文件。修改用的不是很多。

來看一個好多軟件的常見文檔格式如下：

[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
 
[bitbucket.org]
User = hg
 
[topsecret.server.com]
Port = 50022
ForwardX11 = no

如果想用python生成一個這樣的文檔怎么做呢？

import configparser
 
config = configparser.ConfigParser()
config["DEFAULT"] = {'ServerAliveInterval': '45',
                      'Compression': 'yes',
                     'CompressionLevel': '9'}
 
config['bitbucket.org'] = {}
config['bitbucket.org']['User'] = 'hg'
config['topsecret.server.com'] = {}
topsecret = config['topsecret.server.com']
topsecret['Host Port'] = '50022'     # mutates the parser
topsecret['ForwardX11'] = 'no'  # same here
config['DEFAULT']['ForwardX11'] = 'yes'
with open('example.ini', 'w') as configfile:
   config.write(configfile)

寫完了還可以再讀出來哈。

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.sections()
[]
>>> config.read('example.ini')
['example.ini']
>>> config.sections()
['bitbucket.org', 'topsecret.server.com']
>>> 'bitbucket.org' in config
True
>>> 'bytebong.com' in config
False
>>> config['bitbucket.org']['User']
'hg'
>>> config['DEFAULT']['Compression']
'yes'
>>> topsecret = config['topsecret.server.com']
>>> topsecret['ForwardX11']
'no'
>>> topsecret['Port']
'50022'
>>> for key in config['bitbucket.org']: print(key)
...
user
compressionlevel
serveraliveinterval
compression
forwardx11
>>> config['bitbucket.org']['ForwardX11']
'yes'

configparser增刪改查語法

[section1]
k1 = v1
k2:v2
  
[section2]
k1 = v1
 
import ConfigParser
  
config = ConfigParser.ConfigParser()
config.read('i.cfg')
  
# ########## 讀 ##########
#secs = config.sections()
#print secs
#options = config.options('group2')
#print options
  
#item_list = config.items('group2')
#print item_list
  
#val = config.get('group1','key')
#val = config.getint('group1','key')
  
# ########## 改寫 ##########
#sec = config.remove_section('group1')
#config.write(open('i.cfg', "w"))
  
#sec = config.has_section('wupeiqi')
#sec = config.add_section('wupeiqi')
#config.write(open('i.cfg', "w"))
  
  
#config.set('group2','k1',11111)
#config.write(open('i.cfg', "w"))
  
#config.remove_option('group2','age')
#config.write(open('i.cfg', "w"))

5.11hashlib模塊　　

用於加密相關的操作，3.x里代替了md5模塊和sha模塊，主要提供 SHA1, SHA224, SHA256, SHA384, SHA512 ，MD5 算法

MD5不能反解的

import hashlib
 
m = hashlib.md5()
m.update(b"Hello")
m.update(b"It's me")
print(m.digest())
m.update(b"It's been a long time since last time we ...")
 
print(m.digest()) #2進制格式hash
print(len(m.hexdigest())) #16進制格式hash
'''
def digest(self, *args, **kwargs): # real signature unknown
    """ Return the digest value as a string of binary data. """
    pass
 
def hexdigest(self, *args, **kwargs): # real signature unknown
    """ Return the digest value as a string of hexadecimal digits. """
    pass
 
'''
import hashlib
 
# ######## md5 ########
 
hash = hashlib.md5()
hash.update('admin')
print(hash.hexdigest())
 
# ######## sha1 ########
 
hash = hashlib.sha1()
hash.update('admin')
print(hash.hexdigest())
 
# ######## sha256 ########
 
hash = hashlib.sha256()
hash.update('admin')
print(hash.hexdigest())
 
 
# ######## sha384 ########
 
hash = hashlib.sha384()
hash.update('admin')
print(hash.hexdigest())
 
# ######## sha512 ########
 
hash = hashlib.sha512()
hash.update('admin')
print(hash.hexdigest())

python 還有一個 hmac 模塊，它內部對我們創建 key 和 內容 再進行處理然后再加密

散列消息鑒別碼，簡稱HMAC，是一種基於消息鑒別碼MAC（Message Authentication Code）的鑒別機制。使用HMAC時,消息通訊的雙方，通過驗證消息中加入的鑒別密鑰K來鑒別消息的真偽；

一般用於網絡通信中消息加密，前提是雙方先要約定好key,就像接頭暗號一樣，然后消息發送把用key把消息加密，接收方用key ＋ 消息明文再加密，拿加密后的值 跟 發送者的相對比是否相等，這樣就能驗證消息的真實性，及發送者的合法性了。

 

import hmac
h = hmac.new(b'天王蓋地虎', b'寶塔鎮河妖')
print h.hexdigest()

更多關於md5,sha1,sha256等介紹的文章看這里https://www.tbs-certificates.co.uk/FAQ/en/sha256.html 

5.12Subprocess模塊 

常用subprocess方法示例

#執行命令，返回命令執行狀態 ， 0 or 非0
>>> retcode = subprocess.call(["ls", "-l"])

#執行命令，如果命令結果為0，就正常返回，否則拋異常
>>> subprocess.check_call(["ls", "-l"])
0

#接收字符串格式命令，返回元組形式，第1個元素是執行狀態，第2個是命令結果 
>>> subprocess.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')

#接收字符串格式命令，並返回結果
>>> subprocess.getoutput('ls /bin/ls')
'/bin/ls'

#執行命令，並返回結果，注意是返回結果，不是打印，下例結果返回給res
>>> res=subprocess.check_output(['ls','-l'])
>>> res
b'total 0\ndrwxr-xr-x 12 alex staff 408 Nov 2 11:05 OldBoyCRM\n'

#上面那些方法，底層都是封裝的subprocess.Popen
poll()
Check if child process has terminated. Returns returncode

wait()
Wait for child process to terminate. Returns returncode attribute.

terminate() 殺掉所啟動進程
communicate() 等待任務結束

stdin 標准輸入

stdout 標准輸出

stderr 標准錯誤

pid
The process ID of the child process.

#例子
>>> p = subprocess.Popen("df -h|grep disk",stdin=subprocess.PIPE,stdout=subprocess.PIPE,shell=True)
>>> p.stdout.read()
b'/dev/disk1 465Gi 64Gi 400Gi 14% 16901472 104938142 14% /\n'

調用subprocess.run(...)是推薦的常用方法，在大多數情況下能滿足需求，但如果你可能需要進行一些復雜的與系統的交互的話，你還可以用subprocess.Popen(),語法如下：

p = subprocess.Popen("find / -size +1000000 -exec ls -shl {} \;",shell=True,stdout=subprocess.PIPE)
print(p.stdout.read())

可用參數：

• args：shell命令，可以是字符串或者序列類型（如：list，元組）
• bufsize：指定緩沖。0 無緩沖,1 行緩沖,其他 緩沖區大小,負值 系統緩沖
• stdin, stdout, stderr：分別表示程序的標准輸入、輸出、錯誤句柄
• preexec_fn：只在Unix平台下有效，用於指定一個可執行對象（callable object），它將在子進程運行之前被調用
• close_sfs：在windows平台下，如果close_fds被設置為True，則新創建的子進程將不會繼承父進程的輸入、輸出、錯誤管道。
  所以不能將close_fds設置為True同時重定向子進程的標准輸入、輸出與錯誤(stdin, stdout, stderr)。
• shell：同上
• cwd：用於設置子進程的當前目錄
• env：用於指定子進程的環境變量。如果env = None，子進程的環境變量將從父進程中繼承。
• universal_newlines：不同系統的換行符不同，True -> 同意使用 \n
• startupinfo與createionflags只在windows下有效
  將被傳遞給底層的CreateProcess()函數，用於設置子進程的一些屬性，如：主窗口的外觀，進程的優先級等等

終端輸入的命令分為兩種：

• 輸入即可得到輸出，如：ifconfig
• 輸入進行某環境，依賴再輸入，如：python

需要交互的命令示例

import subprocess
 
obj = subprocess.Popen(["python"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
obj.stdin.write('print 1 \n ')
obj.stdin.write('print 2 \n ')
obj.stdin.write('print 3 \n ')
obj.stdin.write('print 4 \n ')
 
out_error_list = obj.communicate(timeout=10)
print out_error_list

subprocess實現sudo 自動輸入密碼

import subprocess
 
def mypass():
    mypass = '123' #or get the password from anywhere
    return mypass
 
echo = subprocess.Popen(['echo',mypass()],
                        stdout=subprocess.PIPE,
                        )
 
sudo = subprocess.Popen(['sudo','-S','iptables','-L'],
                        stdin=echo.stdout,
                        stdout=subprocess.PIPE,
                        )
 
end_of_pipe = sudo.stdout
 
print "Password ok \n Iptables Chains %s" % end_of_pipe.read()

5.13logging模塊

很多程序都有記錄日志的需求，並且日志中包含的信息即有正常的程序訪問日志，還可能有錯誤、警告等信息輸出，python的logging模塊提供了標准的日志接口，你可以通過它存儲各種格式的日志，logging的日志可以分為 debug(), info(), warning(), error() and critical() 5個級別，下面我們看一下怎么用。

import logging
 
logging.warning("user [alex] attempted wrong password more than 3 times")
logging.critical("server is down")
 
#輸出
WARNING:root:user [alex] attempted wrong password more than 3 times
CRITICAL:root:server is down

如果想把日志寫到文件里，也很簡單

import logging
 
logging.basicConfig(filename='example.log',level=logging.INFO)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')

其中下面這句中的level=loggin.INFO意思是，把日志紀錄級別設置為INFO，也就是說，只有比日志是INFO或比INFO級別更高的日志才會被紀錄到文件里，在這個例子， 第一條日志是不會被紀錄的，如果希望紀錄debug的日志，那把日志級別改成DEBUG就行了。

logging.basicConfig(filename='example.log',level=logging.INFO)

感覺上面的日志格式忘記加上時間啦，日志不知道時間怎么行呢，下面就來加上!

import logging
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('is when this event was logged.')
 
#輸出
12/12/2010 11:46:36 AM is when this event was logged.

如果想同時把log打印在屏幕和文件日志里，就需要了解一點復雜的知識 了

Python 使用logging模塊記錄日志涉及四個主要類，使用官方文檔中的概括最為合適：

logger提供了應用程序可以直接使用的接口；

handler將(logger創建的)日志記錄發送到合適的目的輸出；

filter提供了細度設備來決定輸出哪條日志記錄；

formatter決定日志記錄的最終輸出格式。

 

日志的格式

%(name)s	Logger的名字
%(levelno)s	數字形式的日志級別
%(levelname)s	文本形式的日志級別
%(pathname)s	調用日志輸出函數的模塊的完整路徑名，可能沒有
%(filename)s	調用日志輸出函數的模塊的文件名
%(module)s	調用日志輸出函數的模塊名
%(funcName)s	調用日志輸出函數的函數名
%(lineno)d	調用日志輸出函數的語句所在的代碼行
%(created)f	當前時間，用UNIX標准的表示時間的浮 點數表示
%(relativeCreated)d	輸出日志信息時的，自Logger創建以 來的毫秒數
%(asctime)s	字符串形式的當前時間。默認格式是 “2003-07-08 16:49:45,896”。逗號后面的是毫秒
%(thread)d	線程ID。可能沒有
%(threadName)s	線程名。可能沒有
%(process)d	進程ID。可能沒有
%(message)s	用戶輸出的消息

logger
每個程序在輸出信息之前都要獲得一個Logger。Logger通常對應了程序的模塊名，比如聊天工具的圖形界面模塊可以這樣獲得它的Logger：
LOG=logging.getLogger(”chat.gui”)
而核心模塊可以這樣：
LOG=logging.getLogger(”chat.kernel”)

Logger.setLevel(lel):指定最低的日志級別，低於lel的級別將被忽略。debug是最低的內置級別，critical為最高
Logger.addFilter(filt)、Logger.removeFilter(filt):添加或刪除指定的filter
Logger.addHandler(hdlr)、Logger.removeHandler(hdlr)：增加或刪除指定的handler
Logger.debug()、Logger.info()、Logger.warning()、Logger.error()、Logger.critical()：可以設置的日志級別

 

handler

handler對象負責發送相關的信息到指定目的地。Python的日志系統有多種Handler可以使用。有些Handler可以把信息輸出到控制台，有些Logger可以把信息輸出到文件，還有些 Handler可以把信息發送到網絡上。如果覺得不夠用，還可以編寫自己的Handler。可以通過addHandler()方法添加多個多handler
Handler.setLevel(lel):指定被處理的信息級別，低於lel級別的信息將被忽略
Handler.setFormatter()：給這個handler選擇一個格式
Handler.addFilter(filt)、Handler.removeFilter(filt)：新增或刪除一個filter對象

每個Logger可以附加多個Handler。接下來我們就來介紹一些常用的Handler：
1) logging.StreamHandler
使用這個Handler可以向類似與sys.stdout或者sys.stderr的任何文件對象(file object)輸出信息。它的構造函數是：
StreamHandler([strm])
其中strm參數是一個文件對象。默認是sys.stderr

2) logging.FileHandler
和StreamHandler類似，用於向一個文件輸出日志信息。不過FileHandler會幫你打開這個文件。它的構造函數是：
FileHandler(filename[,mode])
filename是文件名，必須指定一個文件名。
mode是文件的打開方式。參見Python內置函數open()的用法。默認是’a'，即添加到文件末尾。

3) logging.handlers.RotatingFileHandler
這個Handler類似於上面的FileHandler，但是它可以管理文件大小。當文件達到一定大小之后，它會自動將當前日志文件改名，然后創建 一個新的同名日志文件繼續輸出。比如日志文件是chat.log。當chat.log達到指定的大小之后，RotatingFileHandler自動把 文件改名為chat.log.1。不過，如果chat.log.1已經存在，會先把chat.log.1重命名為chat.log.2。。。最后重新創建 chat.log，繼續輸出日志信息。它的構造函數是：
RotatingFileHandler( filename[, mode[, maxBytes[, backupCount]]])
其中filename和mode兩個參數和FileHandler一樣。
maxBytes用於指定日志文件的最大文件大小。如果maxBytes為0，意味着日志文件可以無限大，這時上面描述的重命名過程就不會發生。
backupCount用於指定保留的備份文件的個數。比如，如果指定為2，當上面描述的重命名過程發生時，原有的chat.log.2並不會被更名，而是被刪除。

4) logging.handlers.TimedRotatingFileHandler
這個Handler和RotatingFileHandler類似，不過，它沒有通過判斷文件大小來決定何時重新創建日志文件，而是間隔一定時間就 自動創建新的日志文件。重命名的過程與RotatingFileHandler類似，不過新的文件不是附加數字，而是當前時間。它的構造函數是：
TimedRotatingFileHandler( filename [,when [,interval [,backupCount]]])
其中filename參數和backupCount參數和RotatingFileHandler具有相同的意義。
interval是時間間隔。
when參數是一個字符串。表示時間間隔的單位，不區分大小寫。它有以下取值：
S 秒
M 分
H 小時
D 天
W 每星期（interval==0時代表星期一）
midnight 每天凌晨

import logging
 
#create logger
logger = logging.getLogger('TEST-LOG')
logger.setLevel(logging.DEBUG)
 
 
# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
 
# create file handler and set level to warning
fh = logging.FileHandler("access.log")
fh.setLevel(logging.WARNING)
# create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
 
# add formatter to ch and fh
ch.setFormatter(formatter)
fh.setFormatter(formatter)
 
# add ch and fh to logger
logger.addHandler(ch)
logger.addHandler(fh)
 
# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')

文件自動截斷例子

import logging

from logging import handlers

logger = logging.getLogger(__name__)

log_file = "timelog.log"
#fh = handlers.RotatingFileHandler(filename=log_file,maxBytes=10,backupCount=3)
fh = handlers.TimedRotatingFileHandler(filename=log_file,when="S",interval=5,backupCount=3)


formatter = logging.Formatter('%(asctime)s %(module)s:%(lineno)d %(message)s')

fh.setFormatter(formatter)

logger.addHandler(fh)


logger.warning("test1")
logger.warning("test12")
logger.warning("test13")
logger.warning("test14")

5.14 RE模塊

常用正則表達式符號

'.'     默認匹配除\n之外的任意一個字符，若指定flag DOTALL,則匹配任意字符，包括換行
'^'     匹配字符開頭，若指定flags MULTILINE,這種也可以匹配上(r"^a","\nabc\neee",flags=re.MULTILINE)
'$'     匹配字符結尾，或e.search("foo$","bfoo\nsdfsf",flags=re.MULTILINE).group()也可以
'*'     匹配*號前的字符0次或多次，re.findall("ab*","cabb3abcbbac")  結果為['abb', 'ab', 'a']
'+'     匹配前一個字符1次或多次，re.findall("ab+","ab+cd+abb+bba") 結果['ab', 'abb']
'?'     匹配前一個字符1次或0次
'{m}'   匹配前一個字符m次
'{n,m}' 匹配前一個字符n到m次，re.findall("ab{1,3}","abb abc abbcbbb") 結果'abb', 'ab', 'abb']
'|'     匹配|左或|右的字符，re.search("abc|ABC","ABCBabcCD").group() 結果'ABC'
'(...)' 分組匹配，re.search("(abc){2}a(123|456)c", "abcabca456c").group() 結果 abcabca456c
 
 
'\A'    只從字符開頭匹配，re.search("\Aabc","alexabc") 是匹配不到的
'\Z'    匹配字符結尾，同$
'\d'    匹配數字0-9
'\D'    匹配非數字
'\w'    匹配[A-Za-z0-9]
'\W'    匹配非[A-Za-z0-9]
's'     匹配空白字符、\t、\n、\r , re.search("\s+","ab\tc1\n3").group() 結果 '\t'
 
'(?P<name>...)' 分組匹配 re.search("(?P<province>[0-9]{4})(?P<city>[0-9]{2})(?P<birthday>[0-9]{4})","371481199306143242").groupdict("city") 結果{'province': '3714', 'city': '81', 'birthday': '1993'}

最常用的匹配語法

re.match 從頭開始匹配
re.search 匹配包含
re.findall 把所有匹配到的字符放到以列表中的元素返回
re.split 以匹配到的字符當做列表分隔符
re.sub      匹配字符並替換

反斜杠的困擾

轉義\ 和分組（2）
與大多數編程語言相同，正則表達式里使用"\"作為轉義字符，這就可能造成反斜杠困擾。假如你需要匹配文本中的字符"\"，那么使用編程語言表示的正則表達式里將需要4個反斜杠"\\\\"：前兩個和后兩個分別用於在編程語言里轉義成反斜杠，轉換成兩個反斜杠后再在正則表達式里轉義成一個反斜杠。Python里的原生字符串很好地解決了這個問題，這個例子中的正則表達式可以使用r"\\"表示。同樣，匹配一個數字的"\\d"可以寫成r"\d"。有了原生字符串，你再也不用擔心是不是漏寫了反斜杠，寫出來的表達式也更直觀。

 

僅需輕輕知道的幾個匹配模式

re.I(re.IGNORECASE): 忽略大小寫（括號內是完整寫法，下同）
M(MULTILINE): 多行模式，改變'^'和'$'的行為
S(DOTALL): 點任意匹配模式，改變'.'的行為

 六、作業

6.1  開發一個簡單的python計算器

1. 實現加減乘除及拓號優先級解析
2. 用戶輸入 1 - 2 * ( (60-30 +(-40/5) * (9-2*5/3 + 7 /3*99/4*2998 +10 * 568/14 )) - (-4*3)/ (16-3*2) )等類似公式后，必須自己解析里面的(),+,-,*,/符號和公式(不能調用eval等類似功能偷懶實現)，運算后得出結果，結果必須與真實的計算器所得出的結果一致

 6.2模塊化練習：

#實現效果：隨機生成一個6位驗證碼（包含數字和字母）

import random
def make_code(n):
    res=''
    for i in range(n):
        s1=chr(random.randint(65,90))
        s2=str(random.randint(0,9))
        res+=random.choice([s1,s2])
    return res
print(make_code(6))

6.3 模擬一個搶紅包的例子， 實現效果： 10塊錢，隨機分層5份，要求精確到分。

num=5
total=1000#先擴大100倍，在縮小100倍 剩余的錢
list_1=[]
for i in range(4):
    a=random.randint(1,total-num+1)
    list_1.append(a/100)
    total=total-a
    num=num-1
list_1.append(total/100)
print(list_1)