python 讀取json大文件


奇葩json文件:{}{}{}{}{}

一行,919m

#!/usr/bin/env python3.8
# _*_ coding: utf-8 _*_
# Description:
# Author: gaozengzeng <gaozengzeng@sglcapital.com.cn>
# Date: 2021/11/18

def each_chunk(stream, separator):
    buffer = ''
    while True:  # until EOF
        chunk = stream.read(4096)  # I propose 4096 or so
        if not chunk:  # EOF?
            yield buffer
            break
        buffer += chunk
        while True:  # until no separator is found
            try:
                part, buffer = buffer.split(separator, 1)

            except ValueError:
                break
            else:
                yield part+'}'

def main():
    with open('./SA_SRN_RECORD.json') as myFile:
        for chunk in each_chunk(myFile, separator='}'):
            print(chunk)  # not holding in memory, but printing chunk by chunk

if __name__ == '__main__':
    main()

感謝gaozengzeng老鐵的解決方案。


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM