Python-爬蟲-（Json和Csv）文件存儲

用解析器解析出的數據后，可以通過TXT、JSON、CSV等文件形式進行存儲；

1、TXT形式此處略；

2、JSON文件存儲

json即js 對象標記，是一種數據格式；

json格式：

json對象：{"username":"ADMIN","pwd":"xxxx","address":"北京"}

json數組：[{"name":"smith","age":"23"},{"name":"jones","age":"23"},{"name":"zye","age":"23"}]

 

例如：

import json
str='''
[{"name":"小明","age":"23"},{"name":"jones","age":"23"},{"name":"zye","age":"23"}]
'''
print(type(str))
obj=json.loads(str)#將json格式字符串轉為json
print(type(obj))
#取值
print(obj[0]['name'])
print(obj[1]['age'])
print(obj[1].get('name'))
print(obj[1].get('address','北京'))#即如果該字段為None或者該key不存在時，會采用默認值；即第二個參數為默認值

#加載一個json文件
with open("read.json",'r') as file:
    s=file.read()
    j=json.loads(s)
    print(j[0]['name'])

#輸出json
jsonArray=[{'name':'sd'},{'name':'ss'}]
with open('test.json','w') as  file:
    file.write(json.dumps(jsonArray))

#或者：
jsonObj=json.loads(str)
with open('test.json','w') as file:
    file.write(json.dumps(jsonObj))

#或者：如果有中文，且想格式化
jsonObj=json.loads(str)
with open('test.json','w',encoding='utf-8') as file:
    file.write(json.dumps(jsonObj,indent=2,ensure_ascii=False))#格式化了json,

寫入test.json后格式：

[
  {
    "name": "小明",
 "age": "23"
 },
 {
 "name": "jones",
 "age": "23"
 },
 {
 "name": "zye",
 "age": "23"
 }
]

 2、CSV文件操作

CSV通過分隔符進行風格以純文本形式存儲表格數據；該文件是一個字符序列，可以任意數目組成的記錄，記錄之間通過換行符分離；

將數據寫入csv文件

import csv
with open ('data .csv',  'w',newline='') as csvfile:# newline=''設置換行，否則會出現空行
     writer = csv.writer(csvfile,delimiter=':')#delimiter=':' 設置：號隔開
     writer.writerow (['id', 'ame','age'])
     writer. writerow( ['10001', 'Mike',20])
     writer .writerow (['10002','Bob',22])

     writer.writerows([['1003','JEL',23],['1004','JESR',34],['1005','ESR',34]])
     csvfile.close()

#字典形式寫入
with open('abc.csv','w',newline='',encoding='utf-8 ') as myfile:#設置w為a則此時會像文檔中追加內容
     fieldnames=['id','name','addr']
     writer=csv.DictWriter(myfile,delimiter='!',fieldnames=fieldnames)
     writer.writeheader()
     writer.writerow({'id':'1001','name':'JONES','addr':'北京'})
     writer.writerow({'id': '1002', 'name': 'smith', 'addr': '北京'})
     writer.writerow({'id': '1003', 'name': 'jenes', 'addr': '北京'})

讀取csv文件：

#讀取數據
with open('abc.csv','r',encoding='utf-8') as myfile:
     csvReader=csv.reader(myfile)
     for r in csvReader:
          print(r)