背景：實現相應的目標檢測網絡需要能夠測試mAP
目的：實現mAP的測試。
參考代碼：https://github.com/Cartucho/mAP#create-the-ground-truth-files
目錄
一、mAP概覽
1.1 mAP概覽
mAP為目標檢測領域的基礎指標。
首先標簽相同交並比IoU>0.5表示網絡檢測正確。
然后畫出相應的查全率與查准率的曲線，積分得到的藍色區域即為mAP。
各類的平均AP即mAP
1.2 測試需要的步驟
•Create a separate ground-truth text file for each image. In these files, each line should be in the following format: <class_name> <left> <top> <right> <bottom> [<difficult>]
生成相應的標注文檔，無論是GroundTruth還是網絡生成的預測，都需要按照上面的格式標注。
•Use matching names (e.g. image: "image_1.jpg", ground-truth: "image_1.txt"; "image_2.jpg", "image_2.txt"...).
然后圖片放入相應的文件夾之中。
1.Create the ground-truth files 創建相應的GroundTruth文件
2.Move the ground-truth files into the folder ground-truth/ 放入相應文件夾
3.Create the predicted objects files 創建相應的預測的文件
4.Move the predictions files into the folder predicted/ 放入相應文件夾
5.Run the mAP code 運行mAP的代碼
二、GroundTruth文檔的生成
按照要求將文件拷入文件夾與生成相應的標注。
<class_name> <left> <top> <right> <bottom> [<difficult>] 標注需要與文件同名且每一行按照這種格式生成。
""" author:xing xiangrui time :2018.9.26 10:34 copy image and change name to the new directory write image name and labels to the list.txt """ import os import shutil oldDir = "data" newDir = "head_data" width = 352 height = 288 fw = open('list.txt','w') for root,dirs,files in os.walk(oldDir): if 'images' in root: continue for file in files: if 'jpg' in file: # copy jpg oldPic = root + os.path.sep + file newPic = newDir + os.path.sep + root.split(os.path.sep)[-2] + "_" + root.split(os.path.sep)[-1] + "_" + file shutil.copyfile(oldPic,newPic) fw.write(newPic) # write txt oldTxt = root + os.path.sep + file.replace('jpg','txt') fo = open(oldTxt,'r') lines = fo.readlines() for line in lines: line = line.split() cx, cy, w, h = float(line[1])*width, float(line[2])*height, float(line[3])*width, float(line[4])*height #[x, y, w, h] = [float(line[i]) for i in range(1,5)] x1 = round(cx - w/2, 2) y1 = round(cy - h/2, 2) x2 = round(cx + w/2, 2) y2 = round(cy + h/2, 2) print(x1,y1,x2,y2) fw.write(" " + str(x1) + " " + str(y1) + " " + str(x2) + " " + str(y2)) fw.write("\n") fo.close() #print("fine") print("done") fw.close()此部分代碼實現了
- 舊用於預測的圖像重命名放入新目錄下
- 舊標簽按照mAP的要求格式生成新標簽
三、網絡預測結果生成
for tmp_file in jpg_list: img=cv2.imread(tmp_file) # add ROI region ROI=img[ROI_idx[0]:ROI_idx[1],ROI_idx[2]:ROI_idx[3]] ROI_temp=ROI.copy() img[:,:,:]=0 img[ROI_idx[0]:ROI_idx[1],ROI_idx[2]:ROI_idx[3]]=ROI_temp #create txt file tmp_file=tmp_file.replace("jpg","txt") txt_filename=tmp_file.replace("images","predicted") print("LOACTION!!!predict:"+tmp_file) # start_time = time.time() #print("LOCATION!!!detect_face function start"+"\n") rectangles, points = detect_face(img, args.minsize, pnet_fun, rnet_fun, onet_fun, args.threshold, args.factor) #print("LOCATION!!!idetect_face function done"+"\n") # duration = time.time() - start_time # print("duration:"+str(duration)) #print(type(rectangles)) points = np.transpose(points) #print("LOCATION!!!loop rectangles"+"\n") with open(txt_filename,'w') as result_file: for rectangle in rectangles: result_file.write("head" + " " + str(rectangle[4]) + " " + str(rectangle[0]) + " " + str(rectangle[1]) + " " + str(rectangle[2]) + " " + str(rectangle[3])+"\n") #print("LOCATION!!!Write done!"+"\n") print(ROI_idx) os.chdir("mAP/") os.system("python main.py -na")根據網絡預測將所有圖片運行一遍，結果寫入相應txt文件之中，然后調用mAP測試函數對結果進行預測。
四、預測mAP代碼
4.1 運算IoU
# load prediction bounding-box bb = [ float(x) for x in prediction["bbox"].split() ] for obj in ground_truth_data: # look for a class_name match if obj["class_name"] == class_name: bbgt = [ float(x) for x in obj["bbox"].split() ] bi = [max(bb[0],bbgt[0]), max(bb[1],bbgt[1]), min(bb[2],bbgt[2]), min(bb[3],bbgt[3])] iw = bi[2] - bi[0] + 1 ih = bi[3] - bi[1] + 1 if iw > 0 and ih > 0: # compute overlap (IoU) = area of intersection / area of union ua = (bb[2] - bb[0] + 1) * (bb[3] - bb[1] + 1) + (bbgt[2] - bbgt[0] + 1) * (bbgt[3] - bbgt[1] + 1) - iw * ih ov = iw * ih / ua if ov > ovmax: ovmax = ov gt_match = obj4.2 運算AP
先設置兩個點，即precision為1的時候，recall為0；precision為0的時候，recall為1
def voc_ap(rec, prec): """ --- Official matlab code VOC2012--- mrec=[0 ; rec ; 1]; mpre=[0 ; prec ; 0]; for i=numel(mpre)-1:-1:1 mpre(i)=max(mpre(i),mpre(i+1)); end i=find(mrec(2:end)~=mrec(1:end-1))+1; ap=sum((mrec(i)-mrec(i-1)).*mpre(i)); """ rec.insert(0, 0.0) # insert 0.0 at begining of list rec.append(1.0) # insert 1.0 at end of list mrec = rec[:] prec.insert(0, 0.0) # insert 0.0 at begining of list prec.append(0.0) # insert 0.0 at end of list mpre = prec[:]precision隨着recall的增減，逐漸減少
""" This part makes the precision monotonically decreasing (goes from the end to the beginning) matlab: for i=numel(mpre)-1:-1:1 mpre(i)=max(mpre(i),mpre(i+1)); """ # matlab indexes start in 1 but python in 0, so I have to do: # range(start=(len(mpre) - 2), end=0, step=-1) # also the python function range excludes the end, resulting in: # range(start=(len(mpre) - 2), end=-1, step=-1) for i in range(len(mpre)-2, -1, -1): mpre[i] = max(mpre[i], mpre[i+1])創建recall的變化
""" This part creates a list of indexes where the recall changes matlab: i=find(mrec(2:end)~=mrec(1:end-1))+1; """ i_list = [] for i in range(1, len(mrec)): if mrec[i] != mrec[i-1]: i_list.append(i) # if it was matlab would be i + 1AP即為曲線下積分的面積
""" The Average Precision (AP) is the area under the curve (numerical integration) matlab: ap=sum((mrec(i)-mrec(i-1)).*mpre(i)); """ ap = 0.0 for i in i_list: ap += ((mrec[i]-mrec[i-1])*mpre[i]) return ap, mrec, mpre4.3 mAP
根據每類的AP算出mAP
""" Calculate the AP for each class """ sum_AP = 0.0 ap_dictionary = {} # open file to store the results with open(results_files_path + "/results.txt", 'w') as results_file: results_file.write("# AP and precision/recall per class\n") count_true_positives = {}。。。代碼中有大量的代碼用於畫出曲線和顯示出ground truth和圖像上的prediction等，我們省去此部分。
results_file.write("\n# mAP of all classes\n") mAP = sum_AP / n_classes text = "mAP = {0:.2f}%".format(mAP*100) results_file.write(text + "\n") print(text)