python 操作excel工具及xlsxwriter處理數據量大的方法

前言

- 在我們的web項目中一些圖表的預覽頁面常常伴隨着數據導出至excel的功能。在python中我們常常使用xlrd、xlwt、xlutils、openpyxl、xlsxwriter來達成我們的需求。

方法簡介

一.xlsxwriter 只寫 (推薦)

注：xlsxwriter 只能寫，不能讀，但功能強大，可進行添加圖表等復雜操作，詳見https://xlsxwriter.readthedocs.io/

1.本地生成(數據量大時建議保存在本地)

簡單的例子

import xlsxwriter as xw
#新建excel，這樣在本地會生成這個文件，下文介紹與寫入內存中
workbook  = xw.Workbook('myexcel.xlsx')
#新建工作薄
worksheet = workbook.add_worksheet()
#寫入數據
worksheet.wirte('A1',1)
#關閉保存
workbook.close()

插入圖表的例子

import xlsxwriter
 
def get_chart(series):
    chart = workbook.add_chart({'type': 'line'})
    for ses in series:
        name = ses["name"]
        values = ses["values"]
        chart.add_series({
            'name': name,
            'categories': 'A2:A10',
            'values':values
        })
    chart.set_size({'width': 600, 'height': 250})
    return chart
 
if __name__ == '__main__':
    workbook = xlsxwriter.Workbook('xxxx.xlsx')
    worksheet = workbook.add_worksheet("工作簿1")
    headings = ['日期', '平均值']
    worksheet.write_row('A1', headings)
    index=0
    for row in range(1,10):
        for com in [0,1]:
            worksheet.write(row,com,index)
            index+=1
    series = [{"name":"平均值","values":"B2:B10"}]
    chart = get_chart(series)
    chart.set_title ({'name': '每日頁面分享數據'}) 
    worksheet.insert_chart('H7', chart)
    workbook.close()

2.內存生成(web中常用直接返回響應數據)

python3 中操作需 借助 BytesIO， Python2 需借助 StringIO

場景 python3 + django 實現excel導出

from io import BytesIO

import xlsxwriter as xw
from django.http import FileResponse, HttpResponse


def export(request, *arg, **kwargs):
    
    out_put = BytesIO()
    #新建excel
    workbook  = xw.Workbook(out_put)
    #新建工作薄
    worksheet = workbook.add_worksheet()
    #寫入數據
    worksheet.wirte('A1',1)
    #關閉保存
    workbook.close()
    # make 響應對象
    response = HttpResponse()
    response['Content-Type'] = "application/octet-stream"
    disposition = 'attachment;filename={}.xlsx'.format(escape_uri_path(file_name))
    response['Content-Disposition'] = disposition
    response.write(out_put.getvalue())
    
    return response

3.xlsxwriter處理大數據

注：默認情況下，XlsxWriter將所有單元數據保存在內存中。當格式化與數據分開應用時，這將允許將來使用功能。這樣做的結果是XlsxWriter會消耗大量內存，並且在創建大文件時可能會耗盡內存。

解決辦法：

- 我們可以通過Workbook() 的 constant_memory 屬性來幾乎完全減少內存使用量

workbook = xlsxwriter.Workbook(filename, {'constant_memory': True})

源碼：

當設置為True時源碼解釋        
    # Write out the worksheet data as a single row with cells.
    # This method is used when constant_memory is on. A single
    # row is written and the data table is reset. That way only
    # one row of data is kept in memory at any one time. We don't
    # write span data in the optimized case since it is optional.

官網解釋：
# Ok. With 'constant_memory' you must write data in row by column order.
for row in range(0, row_max):
    for col in range(0, col_max):
        worksheet.write(row, col, some_data)

# Not ok. With 'constant_memory' this will only write the first column of data.
for col in range(0, col_max):
    for row in range(0, row_max):
        worksheet.write(row, col, some_data)

缺點：

無法在寫入單元數據后利用任何新功能來操縱單元數據。當前，該add_table()方法不適用於此模式，merge_range()並且set_row()僅適用於當前行。

二.openpyxl可以對excel文件進行讀寫操作

示例1：

from openpyxl import Workbook
from openpyxl import load_workbook
from openpyxl.writer.excel import ExcelWriter 
 
workbook_ = load_workbook(u"test1477881109469.xlsx")
sheetnames =workbook_.get_sheet_names() #獲得表單名字
print sheetnames
sheet = workbook_.get_sheet_by_name(sheetnames[0])
print sheet.cell(row=3,column=3).value
sheet['A1'] = '47'
workbook_.save(u"test1477881109469_new.xlsx") 
wb = Workbook()
ws = wb.active
ws['A1'] = 4
wb.save("test.xlsx")

示例2

import openpyxl
# 新建文件
workbook = openpyxl.Workbook()
# 寫入文件
sheet = workbook.activesheet['A1']='A1'
# 保存文件
workbook.save('test.xlsx')

三.xlrd 主要是用來讀取excel文件

import xlrd
data = xlrd.open_workbook('abcd.xls') # 打開xls文件
table = data.sheets()[0] # 打開第一張表
nrows = table.nrows # 獲取表的行數
for i in range(nrows): # 循環逐行打印
    if i == 0:# 跳過第一行
        continue
    print (table.row_values(i)[:13]) # 取前十三列

四.xlutils結合xlrd可以達到修改excel文件目的

import xlrd
from xlutils.copy import copy
workbook = xlrd.open_workbook('xxxxx.xls')
workbooknew = copy(workbook)
ws = workbooknew.get_sheet(0)
ws.write(3, 0, 'changed!')
workbooknew.save('xxxx.xls')

五.xlwt主要是用來寫excel文件

import xlwt
wbk = xlwt.Workbook()
sheet = wbk.add_sheet('sheet 1')
sheet.write(0,1,'test text')#第0行第一列寫入內容
wbk.save('xxxx.xls')