mongodb python image 圖像存儲讀取

最近做一些數據庫調研的工作，目標是實現影像更快的入庫、出庫、查詢，並實現並行訪問等操作。

將結果總結成一個mongoImg類，也算是小結吧。

 

'''
Created on 2013-8-6
class mongoInsert
@author: tree
'''
__metaclass__ = type

import os
from pymongo.database import Database
import time
import gridfs

class mongoImg(object):
    """mongoInsert is a class for inserting document
    
    
    """
    def __init__(self, database, dir):
        """Create a new instance of :class:mongoInsert
        :Parameters:
          - `database`: database to use
          - `dir` : directory of document 
          """
        if not isinstance(database, Database):
            raise TypeError("database must be an instance of Database")
        if len(dir) < 1:
            raise TypeError("dir must be an string of directory")
        
#         self.__con = Connection()
        self.__imgdb = database
        self.__imgfs = gridfs.GridFS (self.__imgdb)
        self.__dir = dir
        self.__filelist=[]

    #save filepath in list.txt
    def __dirwalk(self,topdown=True):
        """traverse the documents of self.__dir and save in self.__filelist
        """
        sum=0
        self.__filelist.clear()
        
        for root,dirs,files in os.walk(self.__dir,topdown):
            for name in files:
                sum+=1
                temp=os.path.join(root,name)
                self.__filelist.append(temp)
        print(sum)

    #insert image 
    def insert(self):
        """insert images in mongodb
        """
        self.__dirwalk()

        tStart = time.time()        
        for fi in self.__filelist:       
            with open (fi,'rb') as myimage:
                data=myimage.read()                  
                self.__imgfs.put(data, content_type = "jpg", filename =fi)
    
        tEnd =time.time ()
        print ("It cost %f sec" % (tEnd - tStart))
        
    #get image by filename
    def getbyname(self,filename,savepath):
        """get img from mongdb by filename
        """
        if len(savepath) < 1:
            raise TypeError("dir must be an string of directory")
        dataout=self.__imgfs.get_version(filename)
        try:
            imgout=open(savepath,'wb')
            data=dataout.read()
            imgout.write(data)
        finally:
            imgout.close()
        

 使用示例：也可以將數據庫連接寫在類內部

from pymongo import Connection
import mongoImg

filedir=r'D:\image'
con = Connection()
db = con.imgdb
imgmongo=mongoImg.mongoImg(db,filedir)
imgmongo.insert()

 

感覺mongodb存儲影像切片還是蠻快的，1w多個圖片，大約100-200秒左右。

tip：

gridfs.GridFS.put 函數

put(data, **kwargs)
Put data in GridFS as a new file.

Equivalent to doing:

try:
    f = new_file(**kwargs)
    f.write(data)
    finally
f.close()

在存儲讀取圖像時，犯了低級錯誤，將open得到的file實例當做數據存儲，讀取的時候怎么也讀不出數據。。。囧

另外以字節流形式讀取圖像數據比較適合。

pipe = open('/dev/input/js0','rb')

如果以str形式存儲的話，可能會出現UnicodeDecodeError錯誤，貌似是因為圖像數據有些超出了python默認編碼的存儲區間。

 

ps：初學python 數據庫操作也忘得差不多 歡迎大家批評和指正~