一. 准備工作
1）實驗環境：
darknet 是由 C 和 CUDA 開發的，不需要配置其他深度學習的框架（如，tensorflow、caffe 等），支持 CPU 和 GPU 運算，而且安裝過程非常簡單。本文使用的 CUDA 的版本如下所示：
CUDA：9.0
CUDNN：7.0
2）下載 github 源碼：
git clone https://github.com/pjreddie/darknet.git
3）配置darknet編譯環境：
| Ⅰ. 編譯一直在darknet文件夾下； Ⅱ. 每次修改 Makefile 文件后需重新 make 一下才能生效； Ⅲ. 默認的 Makefile 是使用 CPU； |
修改Makefile編譯環境配置文件：
GPU=1 # 是否打開GPU CUDNN=1 # 是否打開cudnn OPENCV=0 # 是否打開opencv OPENMP=0 DEBUG=1 # 是否進行debug
ARCH= -gencode arch=compute_61,code=compute_61 # 根據GPU計算能力選擇對應數值（GTX 1080Ti:61、Tesla K80:37）官網查看：https://developer.nvidia.com/cuda-gpus
...
NVCC=/usr/local/cuda-9.0/bin/nvcc #修改nvcc路徑
...
ifeq ($(GPU), 1) #修改cuda路徑--黃色部分，若不需要更改刪除黃色部分即可
COMMON+= -DGPU -I/usr/local/cuda-9.0/include
CFLAGS+= -DGPU
LDFLAGS+= -L/usr/local/cuda-9.0/lib64 -lcuda -lcudart -lcublas -lcurand
endif
ifeq ($(CUDNN), 1) #修改cudnn路徑--黃色部分，若不需要更改刪除黃色部分即可
COMMON+= -DCUDNN -I/usr/local/cuda-9.0/include
CFLAGS+= -DCUDNN
LDFLAGS+= -L/usr/local/cuda-9.0/lib64 -lcudnn
endif
重新編譯
make clean make
執行./darknet就可以執行編譯
4) 實驗環境測試
下載預訓練模型權重yolov3.weights
下載地址：https://pjreddie.com/media/files/yolov3.weights
基於yolov3.weights模型權重的測試
測試單張圖片(下面兩個指令相同)
./darknet detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg
./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg
測試多張圖片
./darknet detect cfg/yolov3.cfg yolov3.weights
~根據提示輸入圖片路徑
輸出：保存./darknet目錄下的predict.png
5) ./darknet編譯格式
./darknet detector test <data_cfg> <models_cfg> <weights> <test_file> [-thresh] [-out] ./darknet detector train <data_cfg> <models_cfg> <weights> [-thresh] [-gpu] [-gpus] [-clear] ./darknet detector valid <data_cfg> <models_cfg> <weights> [-out] [-thresh] ./darknet detector recall <data_cfg> <models_cfg> <weights> [-thresh]
'<>'必選項，’[ ]‘可選項
data_cfg：數據配置文件，eg：cfg/voc.data
models_cfg：模型配置文件，eg：cfg/yolov3-voc.cfg
weights：權重配置文件，eg：weights/yolov3.weights
test_file：測試文件，eg：*/*/*/test.txt
-thresh：顯示被檢測物體中confidence大於等於 [-thresh] 的bounding-box，默認0.005
-out：輸出文件名稱，默認路徑為results文件夾下，eg：-out "" //輸出class_num個文件，文件名為class_name.txt；若不選擇此選項，則默認輸出文件名為comp4_det_test_"class_name".txt
-i/-gpu：指定單個gpu，默認為0，eg：-gpu 2
-gpus：指定多個gpu，默認為0，eg：-gpus 0,1,2
二. 實驗步驟
| 基於darknet在VOC格式的數據集上訓練yolov3 |
1）下載 Pascal VOC 2007 和 2012 的數據集
wget https://pjreddie.com/media/files/VOCtrainval_11-May-2012.tar wget https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar wget https://pjreddie.com/media/files/VOCtest_06-Nov-2007.tar tar xf VOCtrainval_11-May-2012.tar tar xf VOCtrainval_06-Nov-2007.tar tar xf VOCtest_06-Nov-2007.tar
注：若非voc格式的數據集需先生成voc格式的數據集，數據集的目錄嚴格按照voc數據集的目錄結構
2）生成 VOC 數據集的標簽
darknet 需要一個“.txt”格式的標簽文件，每行表示一張圖像的信息，包括（x, y, w, h）格式為：<object-class> <x> <y> <width> <height>
darknet 官網提供了一個針對 VOC 數據集，處理標簽的腳本，darknet/scripts文件夾下的voc_label.py文件，若無執行如下命令下載文件：
wget https://pjreddie.com/media/files/voc_label.py # 獲取腳本
修改 voc_label.py（4處）
①sets=[('2007', 'train'), ('2007', 'val'), ('2007', 'test')] #替換為自己的數據集 ②classes = ["head", "eye", "nose"] #修改為自己的類別 ③in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml'%(year, image_id)) #將數據集放於當前目錄下 ④os.system("cat 2007_train.txt 2007_val.txt > train.txt") #修改為自己的數據集用作訓練
修改完成后執行：
python voc_label.py # 執行腳本，獲得所需的“.txt”文件
生成的xml文件在 VOCdevkit/VOC%s/labels 目錄下
標簽處理完成后，在voc_label.py所在目錄下生成如下幾個文件：
2007_text.txt、2007_train.txt、2007_val.txt、2012_text.txt、2012_train.txt、train.txt、train_all.txt
3）修改配置文件
Ⅰ. 數據配置文件——cfg/voc.data
classes= 20 # 類別總數 train = /home/xieqi/project/train.txt # 訓練數據所在的位置 valid = /home/xieqi/project/2007_test.txt # 測試數據所在的位置 names = data/voc.names # 修改見voc.names backup = backup # 輸出的權重信息保存的文件夾
eval =
results =
Ⅱ. 模型配置文件——cfg/yolov3-voc.cfg
batch=64 # 一批訓練樣本的樣本數量，每batch個樣本更新一次參數 subdivisions=32 # 它會讓你的每一個batch不是一下子都丟到網絡里。而是分成subdivision對應數字的份數，一份一份的跑完后，在一起打包算作完成一次iteration width=416 # 只可以設置成32的倍數 height=416 # 只可以設置成32的倍數 channels=3 # 若為灰度圖，則chennels=1，另外還需修改/scr/data.c文件中的load_data_detection函數；若為RGB則 channels=3 ，無需修改/scr/data.c文件 momentum=0.9 # 最優化方法的動量參數，這個值影響着梯度下降到最優值得速度 decay=0.0005 # 權重衰減正則項，防止過擬合 angle=0 # 通過旋轉角度來生成更多訓練樣本 saturation = 1.5 # 通過調整飽和度來生成更多訓練樣本 exposure = 1.5 # 通過調整曝光量來生成更多訓練樣本 hue=.1 # 通過調整色調來生成更多訓練樣本 learning_rate=0.001 # 學習率, 剛開始訓練時, 以 0.01 ~ 0.001 為宜, 一定輪數過后,逐漸減緩。 burn_in=1000 # 在迭代次數小於burn_in時，其學習率的更新有一種方式，大於burn_in時，才采用policy的更新方式 max_batches = 50200 # 訓練步數 policy=steps # 學習率調整的策略 steps=40000,45000 # 開始衰減的步數 scales=.1,.1 # 在第40000和第45000次迭代時，學習率衰減10倍 ... [convolutional]——YOLO層前一層卷積層 ... filters=24 # 每一個[yolo]層前的最后一個卷積層中的 filters=num(yolo層個數)*(classes+5) ... [yolo] mask = 6,7,8 anchors = 10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326 #如果想修改默認anchors數值，使用k-means即可； classes=3 # 修改為自己的類別數 num=9 # 每個grid cell預測幾個box,和anchors的數量一致。調大num后訓練時Obj趨近0的話可以嘗試調大object_scale jitter=.3 # 利用數據抖動產生更多數據, jitter是crop的參數, jitter=.3，就是在0~0.3中進行crop ignore_thresh = .5 # 決定是否需要計算IOU誤差的參數，大於thresh，IOU誤差不會夾在cost function中 truth_thresh = 1 random=1 # 如果為1，每次迭代圖片大小隨機從320到608，步長為32，如果為0，每次訓練大小與輸入大小一致 ...
Ⅲ. 標簽配置文件——data/voc.names
head #自己需要探測的類別，一行一個 eye nose
Ⅳ. 修改數據格式——scr/data.c
data load_data_detection(int n, char **paths, int m, int w, int h, int boxes, int classes, float jitter, float hue, float saturation, float exposure) { char **random_paths = get_random_paths(paths, n, m); int i; data d = {0}; d.shallow = 0; d.X.rows = n; d.X.vals = calloc(d.X.rows, sizeof(float*)); d.X.cols = h*w; //灰階圖 //d.X.cols = h*w*3; //RGB圖 ...
4）下載預訓練的參數（卷積權重）
“darknet53.conv.74”是使用 Imagenet 數據集進行預訓練：
wget https://pjreddie.com/media/files/darknet53.conv.74 # 下載預訓練的網絡模型參數
獲取其他模型類似 darknet53.conv.74 的預訓練權重
Ⅰ. 下載官方預訓練模型的權重
https://pjreddie.com/darknet/yolo/ //COCO數據集訓練的
https://pjreddie.com/darknet/imagenet/ //Imagenet數據集訓練的
eg： wget https://pjreddie.com/media/files/yolov3-tiny.weights
Ⅱ. 轉換為類似darknet.conv.74的預訓練權重
eg： ./darknet partial cfg/yolov3-tiny yolov3-tiny.weights yolov-tiny.conv.15 15
5）訓練模型：
Ⅰ. 單GPU訓練: ./darknet detector train <data_cfg> <train_cfg> <weights> -gpu <gpu_id>
./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -i 1
Ⅱ. 多GPU訓練: ./darknet detector train <data_cfg> <model_cfg> <weights> -gpus <gpu_list>
./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg darknet53.conv.74 -gpus 0,1,2,3
Ⅲ. 從checkpoint繼續訓練
./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc.backup -gpus 0,1,2,3
Ⅳ. CPU訓練:
./darknet detector train <data_cfg> <model_cfg> <weights> -nogpu
Ⅴ. 生成loss-iter曲線
在執行訓練命令的時候加一下管道，tee一下log：
./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg | tee result/log/training.log
將下面的python代碼保存為drawcurve.py。並執行
python drawcurve.py training.log 0
~drawcurve.py
import argparse import sys import matplotlib.pyplot as plt def main(argv): parser = argparse.ArgumentParser() parser.add_argument("log_file", help = "path to log file" ) parser.add_argument( "option", help = "0 -> loss vs iter" ) args = parser.parse_args() f = open(args.log_file) lines = [line.rstrip("\n") for line in f.readlines()] # skip the first 3 lines lines = lines[3:] numbers = {'1','2','3','4','5','6','7','8','9','0'} iters = [] loss = [] for line in lines: if line[0] in numbers: args = line.split(" ") if len(args) >3: iters.append(int(args[0][:-1])) loss.append(float(args[2])) plt.plot(iters,loss) plt.xlabel('iters') plt.ylabel('loss') plt.grid() plt.show() if __name__ == "__main__": main(sys.argv)
6）訓練過程：
訓練的log格式如下：
Loaded: 4.533954 seconds Region Avg IOU: 0.262313, Class: 1.000000, Obj: 0.542580, No Obj: 0.514735, Avg Recall: 0.162162, count: 37 Region Avg IOU: 0.175988, Class: 1.000000, Obj: 0.499655, No Obj: 0.517558, Avg Recall: 0.070423, count: 71 Region Avg IOU: 0.200012, Class: 1.000000, Obj: 0.483404, No Obj: 0.514622, Avg Recall: 0.075758, count: 66 Region Avg IOU: 0.279284, Class: 1.000000, Obj: 0.447059, No Obj: 0.515849, Avg Recall: 0.134615, count: 52 1: 629.763611, 629.763611 avg, 0.001000 rate, 6.098687 seconds, 64 images Loaded: 2.957771 seconds Region Avg IOU: 0.145857, Class: 1.000000, Obj: 0.051285, No Obj: 0.031538, Avg Recall: 0.069767, count: 43 Region Avg IOU: 0.257284, Class: 1.000000, Obj: 0.048616, No Obj: 0.027511, Avg Recall: 0.078947, count: 38 Region Avg IOU: 0.174994, Class: 1.000000, Obj: 0.030197, No Obj: 0.029943, Avg Recall: 0.088889, count: 45 Region Avg IOU: 0.196278, Class: 1.000000, Obj: 0.076030, No Obj: 0.030472, Avg Recall: 0.087719, count: 57 2: 84.804230, 575.267700 avg, 0.001000 rate, 5.959159 seconds, 128 images
iter 總損失 平均損失 學習率 花費時間 參與訓練的圖片總數
| Region | cfg文件中yolo-layer的索引 |
| Avg IOU | 當前迭代中，預測的box與標注的box的平均交並比，越大越好，期望數值為1 |
| Class | 標注物體的分類准確率，越大越好，期望數值為1 |
| obj | 越大越好，期望數值為1 |
| No obj | 越小越好，但不為零 |
| .5R | 以IOU=0.5為閾值時候的recall; recall = 檢出的正樣本/實際的正樣本 |
| .75R | 以IOU=0.75為閾值時候的recall |
| count | 正樣本數目 |
Region 82 Avg IOU: 0.798032, Class: 0.559781, Obj: 0.515851, No Obj: 0.006533, .5R: 1.000000, .75R: 1.000000, count: 2 Region 94 Avg IOU: 0.725307, Class: 0.830518, Obj: 0.506567, No Obj: 0.000680, .5R: 1.000000, .75R: 0.750000, count: 4 Region 106 Avg IOU: 0.579333, Class: 0.322556, Obj: 0.020537, No Obj: 0.000070, .5R: 1.000000, .75R: 0.000000, count: 2
以上輸出顯示了所有訓練圖片的一個批次（batch），批次大小的划分根據我們在 .cfg 文件中設置的subdivisions參數。
在我使用的 .cfg 文件中 batch = 64 ，subdivision = 16，所以在訓練輸出中，訓練迭代包含了16組，每組又包含了4張圖片，跟設定的batch和subdivision的值一致。
但是此處有16*3條信息，每組包含三條信息，分別是：Region 82、Region 94、Region 106。
三個尺度上預測不同大小的框：
82卷積層 為最大的預測尺度，使用較大的mask，但是可以預測出較小的物體；
94卷積層 為中間的預測尺度，使用中等的mask；
106卷積層為最小的預測尺度，使用較小的mask，可以預測出較大的物體
每個batch都會有這樣一個輸出：
2706: 1.350835, 1.386559 avg, 0.001000 rate, 3.323842 seconds, 173184 images
batch 總損失 平均損失 學習率 花費時間 參與訓練的圖片總數 = 2706 * 64
三. 模型評價
1）更改配置文件：
Ⅰ. 模型參數(cfg/yoloc3.cfg)：
batch=1
subdivisions=1
| 注：測評模型時batch、subdivisions必須為1 |
Ⅱ. 更改為批處理(example/detector.c)：
①. 在detector.c中增加頭文件：
#include <unistd.h> /* Many POSIX functions (but not all, by a large margin) */ #include <fcntl.h> /* open(), creat() - and fcntl() */
②. 在前面添加GetFilename(char *fullname)函數
#include <unistd.h> #include <fcntl.h> #include "darknet.h" #include <sys/stat.h> #include <stdio.h> #include <time.h> #include <sys/types.h> static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,
36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,
67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90};
//產生的文件名與原文件名相同（去除路徑和格式） char *GetFilename(char *fullname) { int from,to,i; char *newstr,*temp; if(fullname!=NULL){ //if not find dot if((temp=strchr(fullname,'.'))==NULL){ newstr = fullname; } else { from = strlen(fullname) - strlen(temp); to = (temp-fullname); //the first dot's index for (i=from; i<=to; i--){ if (fullname[i]=='.') break;//find the last dot } newstr = (char*)malloc(i+1); strncpy(newstr,fullname,i); *(newstr+i)=0; } } static char name[50] = {""}; char *q = strrchr(newstr,'/') + 1; strncpy(name,q,40); return name; }
③. 用下面代碼替換detector.c文件的void test_detector函數（注意有3處要改成自己的路徑）
void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen)
{
list *options = read_data_cfg(datacfg);
char *name_list = option_find_str(options, "names", "data/names.list");
char **names = get_labels(name_list);
image **alphabet = load_alphabet();
network *net = load_network(cfgfile, weightfile, 0);
set_batch_network(net, 1);
srand(2222222);
double time;
char buff[256];
char *input = buff;
float nms=.45;
int i=0;
while(1){
if(filename){
strncpy(input, filename, 256);
image im = load_image_color(input,0,0);
image sized = letterbox_image(im, net->w, net->h);
//image sized = resize_image(im, net->w, net->h);
//image sized2 = resize_max(im, net->w);
//image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
//resize_network(net, sized.w, sized.h);
layer l = net->layers[net->n-1];
float *X = sized.data;
time=what_time_is_it_now();
network_predict(net, X);
printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time);
int nboxes = 0;
detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
//printf("%d\n", nboxes);
//if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
free_detections(dets, nboxes);
if(outfile)
{
save_image(im, outfile);
}
else{
save_image(im, "predictions");
#ifdef OPENCV
cvNamedWindow("predictions", CV_WINDOW_NORMAL);
if(fullscreen){
cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
}
show_image(im, "predictions");
cvWaitKey(0);
cvDestroyAllWindows();
#endif
}
free_image(im);
free_image(sized);
if (filename) break;
}
else {
printf("Enter Image Path: ");
fflush(stdout);
input = fgets(input, 256, stdin);
if(!input) return;
strtok(input, "\n");
list *plist = get_paths(input);
char **paths = (char **)list_to_array(plist);
printf("Start Testing!\n");
int m = plist->size;
if(access("/home/xieqi/darknet/data/out_img",0)==-1) //修改成自己的路徑
{
if (mkdir("/home/xieqi/darknet/data/out_img",0777)) //修改成自己的路徑
{
printf("creat file bag failed!!!");
}
}
for(i = 0; i < m; ++i){
char *path = paths[i];
image im = load_image_color(path,0,0);
image sized = letterbox_image(im, net->w, net->h);
//image sized = resize_image(im, net->w, net->h);
//image sized2 = resize_max(im, net->w);
//image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h);
//resize_network(net, sized.w, sized.h);
layer l = net->layers[net->n-1];
float *X = sized.data;
time=what_time_is_it_now();
network_predict(net, X);
printf("Try Very Hard:");
printf("%s: Predicted in %f seconds.\n", path, what_time_is_it_now()-time);
int nboxes = 0;
detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes);
//printf("%d\n", nboxes);
//if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms);
if (nms) do_nms_sort(dets, nboxes, l.classes, nms);
draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes);
free_detections(dets, nboxes);
if(outfile){
save_image(im, outfile);
}
else{
char b[2048];
sprintf(b,"/home/xieqi/darknet/data/out_img/%s",GetFilename(path)); //修改成自己的路徑
save_image(im, b);
printf("save %s successfully!\n",GetFilename(path));
#ifdef OPENCV
cvNamedWindow("predictions", CV_WINDOW_NORMAL);
if(fullscreen){
cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN);
}
show_image(im, "predictions");
cvWaitKey(0);
cvDestroyAllWindows();
#endif
}
free_image(im);
free_image(sized);
if (filename) break;
}
}
}
}
④. validate_detector_recall函數定義和調用改為：
void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile) validate_detector_recall(datacfg, cfg, weights);
⑤.validate_detector_recall內的plist和paths的如下初始化代碼：
list *plist = get_paths("data/voc.2007.test");
char **paths = (char **)list_to_array(plist);
修改為：
list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "data/train.list"); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist);
Ⅲ. 在darknet下重新make
make clean make
2). 測試單張圖片
Ⅰ. ./darknet detector test <data_cfg> <models_cfg> <weights> <image_path> # 本次測試無opencv支持
Ⅱ. <models_cfg>文件中batch和subdivisions兩項必須為1；
Ⅲ. 測試時還可以用-thresh和-hier選項指定對應參數；
Ⅳ. 結果都保存在./data/out_img 文件夾下
./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights Eminem.jpg # 測試單張圖片
3). 生成預測結果
Ⅰ. 批量測試——輸出文本檢測結果
./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights -i 1 -out ""
① ./darknet detector valid <data_cfg> <models_cfg> <weights>；
② 結果生成在<data_cfg>的指定的目錄下以<out_file>開頭的若干文件中，若<data_cfg>沒有指定results，那么默認為<darknet_root>/results；
③ <models_cfg>文件中batch和subdivisions兩項必須為1；
④ 若-out 未指定字符串，則在results文件夾下生成comp4_det_test_[類名].txt文件並保存測試結果；
⑤ 本次實驗在results文件夾下生成 [類名].txt 文件；
Ⅱ. 批量測試——輸出圖片檢測結果
./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights -i 2 #enter
Enter Image Path: data/voc/2007_test.txt
① ./darknet detector test <data_cfg> <models_cfg> <weights>；
② <models_cfg>文件中batch和subdivisions兩項必須為1；
③ 本次實驗結果，在<darknet_root>/data/out_img文件夾下（detector.c/test_detector函數更改的路徑），文件名以原圖片名前6位命名（detector.c/GetFilename函數定義的位數）；
4). 計算recall（執行這個命令需要修改detector.c文件，修改信息請參考“detector.c修改”）
Ⅰ. ./darknet detector recall <data_cfg> <test_cfg> <weights>；
Ⅱ. <test_cfg>文件中batch和subdivisions兩項必須為1；
Ⅲ. 輸出在stderr里，重定向時請注意；
Ⅳ. RPs/Img、IOU、Recall都是到當前測試圖片的均值；
Ⅴ. detector.c中對目錄處理有錯誤，可以參照validate_detector對validate_detector_recall最開始幾行的處理進行修改；
./darknet detector valid cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_20000.weights
最后得到的log如下：
306 746 783 RPs/Img: 21.45 IOU: 75.59% Recall:95.27% 307 748 785 RPs/Img: 21.43 IOU: 75.62% Recall:95.29% 308 750 787 RPs/Img: 21.42 IOU: 75.59% Recall:95.30% 309 752 789 RPs/Img: 21.43 IOU: 75.62% Recall:95.31% 310 754 791 RPs/Img: 21.44 IOU: 75.63% Recall:95.32%
No. Correct total
輸出的具體格式為：
| Number | 處理的第幾張圖 |
| Correct | 正確的識別出了多少bbox。這個值算出來的步驟是這樣的，丟進網絡一張圖片，網絡會預測出很多bbox，每個bbox都有其置信概率，概率大於threshold的bbox與實際的bbox，也就是labels中txt的內容計算IOU， 找出IOU最大的bbox，如果這個最大值大於預設的IOU的threshold，那么correct加一 |
| Total | 實際有多少個bbox |
| Rps/img | 平均每個圖片會預測出來多少個bbox |
| IOU | 預測出的bbox和實際標注的bbox的交集 除以 他們的並集。顯然，這個數值越大，說明預測的結果越好 |
| Recall | 召回率， 檢測出物體的個數 除以 標注的所有物體個數。通過代碼我們也能看出來就是Correct除以Total的值 |
error：
本次實驗出現 IOU：inf% —— 交並比 數值爆炸
打印bbox詳細信息(修改detector.c/validate_detector_recall)
for (j = 0; j < num_labels; ++j) { ++total; box t = {truth[j].x, truth[j].y, truth[j].w, truth[j].h}; printf("truth x=%f, y=%f, w=%f,h=%f\n",truth[j].x,truth[j].y,truth[j].w,truth[j].h);//添加代碼 float best_iou = 0; for(k = 0; k < l.w*l.h*l.n; ++k){ float iou = box_iou(dets[k].bbox, t); if(dets[k].objectness > thresh && iou > best_iou){ printf("predict=%f x=%f, y=%f, w=%f,h=%f\n",dets[k].objectness,dets[k].bbox.x,dets[k].bbox.y,dets[k].bbox.w,dets[k].bbox.h);//添加代碼 best_iou = iou; } }
由顯示結果可以看出，預測框的中心坐標x和y出現-nan
解決方法：修改函數
void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile) { ... //layer l = net->layers[net->n-1]; //注釋掉 ... for(k = 0; k < l.w*l.h*l.n; ++k){ //改為for(k = 0; k < nboxes; ++k){ ...
5). 計算AP、mAP
先使用 3)-Ⅰ計算出驗證集結果，再使用py-faster-rcnn下的voc_eval.py計算AP、mAP
Ⅰ. 下載voc_eval.py到 darknet 的根目錄
voc_eval.py下載地址：https://github.com/rbgirshick/py-faster-rcnn/blob/master/lib/datasets/voc_eval.py
根據標簽修改voc_eval.py的內容
.xml文件結構中 ①若無pose 22 #obj_struct['pose'] = obj.find('pose').text # 注釋掉 ②若無truncated 23 #obj_struct['truncated'] = int(obj.find('truncated').text) # 注釋掉 ③若無difficult 24 #obj_struct['difficult'] = int(obj.find('difficult').text) # 注釋掉 137 #if use_diff: # 注釋掉 138 #difficult = np.array([False for x in R]).astype(np.bool) # 注釋掉 139 #else: # 注釋掉 140 #difficult = np.array([x['difficult'] for x in R]).astype(np.bool) # 注釋掉 142 npos = npos +len(R) # 更改npos = npos + sum(~difficult) 145 #'difficult': difficult, # 注釋掉 197 #if not R['difficult'][jmax]: # 注釋掉
Ⅱ. 計算單類AP
①. 在darknet根目錄下新建computer_mAP.py
#!/home/xieqi/anaconda2/envs/py2.7/bin python2.7 # -*- coding:utf-8 -*- from voc_eval import voc_eval print(voc_eval('/home/xieqi/darknet/results/{}.txt', '/home/xieqi/project/traffic_object_detection/voc_data/Annotations/{}.xml',
'/home/xieqi/project/traffic_object_detection/voc_data/ImageSets/Main/test.txt', 'vehicle', '/home/xieqi/project/traffic_object_detection/result/')) # "第一個參數為detector valid 按類別分類后的txt路徑" # "第二個參數為驗證集對應的xml標簽路徑" # "第三個為驗證集txt文本路徑，內容必須是無路徑無后綴的圖片名" # "第四個為待驗證的類別名" # "第五個為pkl文件保存的路徑"
②. 用python2執行computer_mAP.py
① 重復執行，檢測其他類別需要刪除生成的annots.pkl文件或改變computer_mAP.py中pkl文件保存的路徑
② 輸出兩個array()，分別為rec和prec，最后一個數字為單類AP。
Ⅲ. 計算總mAP
在darknet根目錄下新建computer_all_mAP.py
#!/home/xieqi/anaconda2/envs/py2.7/bin python2.7 # -*- coding:utf-8 -*- from voc_eval import voc_eval import os current_path = os.getcwd() results_path = current_path+"/results" sub_files = os.listdir(results_path) mAP = [] for i in range(len(sub_files)): class_name = sub_files[i].split(".txt")[0] rec, prec, ap = voc_eval('/home/xieqi/darknet/results/{}.txt',
'/home/xieqi/project/traffic_object_detection/voc_data/Annotations/{}.xml',
'/home/xieqi/project/traffic_object_detection/voc_data/ImageSets/Main/test.txt',
class_name, '/home/xieqi/project/traffic_object_detection/result/mAP') print("{} :\t {} ".format(class_name, ap)) mAP.append(ap) mAP = tuple(mAP) print("***************************") print("mAP :\t {}".format( float( sum(mAP)/len(mAP)) )) # results文件夾只能有'類名.txt'文件 # test.txt文件包含所有results中txt文件的圖片名（保證是共同的驗證集） # 保證最后的輸出路徑下無pkl文件。 # 上述代碼中的ap就是針對輸入單一類別計算出的AP。
四、文件解析
1). ~detector.c
#include <unistd.h> #include <fcntl.h> #include "darknet.h" #include <sys/stat.h> #include <stdio.h> #include <time.h> #include <sys/types.h> static int coco_ids[] = {1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,34,35,36,37,38,39,40,
41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90}; char *GetFilename(char *fullname) { int from,to,i; char *newstr,*temp; if(fullname!=NULL){ //if not find dot if((temp=strchr(fullname,'.'))==NULL){ newstr = fullname; } else { from = strlen(fullname) - strlen(temp); to = (temp-fullname); //the first dot's index for (i=from; i<=to; i--){ if (fullname[i]=='.') break;//find the last dot } newstr = (char*)malloc(i+1); strncpy(newstr,fullname,i); *(newstr+i)=0; } } static char name[50] = {""}; char *q = strrchr(newstr,'/') + 1; strncpy(name,q,40); return name; } void train_detector(char *datacfg, char *cfgfile, char *weightfile, int *gpus, int ngpus, int clear) { list *options = read_data_cfg(datacfg); //解析data文件，用自定義鏈表options存儲訓練集基本信息，函數位於option_list.c char *train_images = option_find_str(options, "train", "data/train.list"); //從options中找訓練集 char *backup_directory = option_find_str(options, "backup", "/backup/"); //從options中找backup路徑 srand(time(0)); //初始化隨機種子數 char *base = basecfg(cfgfile); //此函數位於utils.c,返回cfg文件不帶后綴的名字 printf("%s\n", base); float avg_loss = -1; network **nets = calloc(ngpus, sizeof(network)); srand(time(0)); int seed = rand(); int i; for(i = 0; i < ngpus; ++i){ srand(seed); #ifdef GPU cuda_set_device(gpus[i]); #endif nets[i] = load_network(cfgfile, weightfile, clear); nets[i]->learning_rate *= ngpus; } srand(time(0)); network *net = nets[0]; int imgs = net->batch * net->subdivisions * ngpus; printf("Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); data train, buffer; layer l = net->layers[net->n - 1]; int classes = l.classes; float jitter = l.jitter; list *plist = get_paths(train_images); //int N = plist->size; char **paths = (char **)list_to_array(plist); load_args args = get_base_args(net); args.coords = l.coords; args.paths = paths; args.n = imgs; args.m = plist->size; args.classes = classes; args.jitter = jitter; args.num_boxes = l.max_boxes; args.d = &buffer; args.type = DETECTION_DATA; //args.type = INSTANCE_DATA; args.threads = 64; pthread_t load_thread = load_data(args); double time; int count = 0; //while(i*imgs < N*120){ while(get_current_batch(net) < net->max_batches){ if(l.random && count++%10 == 0){ printf("Resizing\n"); int dim = (rand() % 10 + 10) * 32; if (get_current_batch(net)+200 > net->max_batches) dim = 608; //int dim = (rand() % 4 + 16) * 32; printf("%d\n", dim); args.w = dim; args.h = dim; pthread_join(load_thread, 0); train = buffer; free_data(train); load_thread = load_data(args); #pragma omp parallel for for(i = 0; i < ngpus; ++i){ resize_network(nets[i], dim, dim); } net = nets[0]; } time=what_time_is_it_now(); pthread_join(load_thread, 0); train = buffer; load_thread = load_data(args); /* int k; for(k = 0; k < l.max_boxes; ++k){ box b = float_to_box(train.y.vals[10] + 1 + k*5); if(!b.x) break; printf("loaded: %f %f %f %f\n", b.x, b.y, b.w, b.h); } */ /* int zz; for(zz = 0; zz < train.X.cols; ++zz){ image im = float_to_image(net->w, net->h, 3, train.X.vals[zz]); int k; for(k = 0; k < l.max_boxes; ++k){ box b = float_to_box(train.y.vals[zz] + k*5, 1); printf("%f %f %f %f\n", b.x, b.y, b.w, b.h); draw_bbox(im, b, 1, 1,0,0); } show_image(im, "truth11"); cvWaitKey(0); save_image(im, "truth11"); } */ printf("Loaded: %lf seconds\n", what_time_is_it_now()-time); time=what_time_is_it_now(); float loss = 0; #ifdef GPU if(ngpus == 1){ loss = train_network(net, train); } else { loss = train_networks(nets, ngpus, train, 4); } #else loss = train_network(net, train); #endif if (avg_loss < 0) avg_loss = loss; avg_loss = avg_loss*.9 + loss*.1; i = get_current_batch(net); printf("%ld: %f, %f avg, %f rate, %lf seconds, %d images\n", get_current_batch(net), loss, avg_loss, get_current_rate(net), what_time_is_it_now()-time, i*imgs); if(i%100==0){ #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s.backup", backup_directory, base); save_weights(net, buff); } if(i%10000==0 || (i < 1000 && i%100 == 0)){ #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s_%d.weights", backup_directory, base, i); save_weights(net, buff); } free_data(train); } #ifdef GPU if(ngpus != 1) sync_nets(nets, ngpus, 0); #endif char buff[256]; sprintf(buff, "%s/%s_final.weights", backup_directory, base); save_weights(net, buff); } static int get_coco_image_id(char *filename) { char *p = strrchr(filename, '/'); char *c = strrchr(filename, '_'); if(c) p = c; return atoi(p+1); } static void print_cocos(FILE *fp, char *image_path, detection *dets, int num_boxes, int classes, int w, int h) { int i, j; int image_id = get_coco_image_id(image_path); for(i = 0; i < num_boxes; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2.; float xmax = dets[i].bbox.x + dets[i].bbox.w/2.; float ymin = dets[i].bbox.y - dets[i].bbox.h/2.; float ymax = dets[i].bbox.y + dets[i].bbox.h/2.; if (xmin < 0) xmin = 0; if (ymin < 0) ymin = 0; if (xmax > w) xmax = w; if (ymax > h) ymax = h; float bx = xmin; float by = ymin; float bw = xmax - xmin; float bh = ymax - ymin; for(j = 0; j < classes; ++j){ if (dets[i].prob[j]) fprintf(fp, "{\"image_id\":%d, \"category_id\":%d, \"bbox\":[%f, %f, %f, %f], \"score\":%f},\n", image_id, coco_ids[j], bx, by, bw, bh, dets[i].prob[j]); } } } void print_detector_detections(FILE **fps, char *id, detection *dets, int total, int classes, int w, int h) { int i, j; for(i = 0; i < total; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2. + 1; float xmax = dets[i].bbox.x + dets[i].bbox.w/2. + 1; float ymin = dets[i].bbox.y - dets[i].bbox.h/2. + 1; float ymax = dets[i].bbox.y + dets[i].bbox.h/2. + 1; if (xmin < 1) xmin = 1; if (ymin < 1) ymin = 1; if (xmax > w) xmax = w; if (ymax > h) ymax = h; for(j = 0; j < classes; ++j){ if (dets[i].prob[j]) fprintf(fps[j], "%s %f %f %f %f %f\n", id, dets[i].prob[j], xmin, ymin, xmax, ymax); } } } void print_imagenet_detections(FILE *fp, int id, detection *dets, int total, int classes, int w, int h) { int i, j; for(i = 0; i < total; ++i){ float xmin = dets[i].bbox.x - dets[i].bbox.w/2.; float xmax = dets[i].bbox.x + dets[i].bbox.w/2.; float ymin = dets[i].bbox.y - dets[i].bbox.h/2.; float ymax = dets[i].bbox.y + dets[i].bbox.h/2.; if (xmin < 0) xmin = 0; if (ymin < 0) ymin = 0; if (xmax > w) xmax = w; if (ymax > h) ymax = h; for(j = 0; j < classes; ++j){ int class = j; if (dets[i].prob[class]) fprintf(fp, "%d %d %f %f %f %f %f\n", id, j+1, dets[i].prob[class], xmin, ymin, xmax, ymax); } } } void validate_detector_flip(char *datacfg, char *cfgfile, char *weightfile, char *outfile) { int j; list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "data/train.list"); char *name_list = option_find_str(options, "names", "data/names.list"); char *prefix = option_find_str(options, "results", "results"); char **names = get_labels(name_list); char *mapf = option_find_str(options, "map", 0); int *map = 0; if (mapf) map = read_map(mapf); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 2); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); layer l = net->layers[net->n-1]; int classes = l.classes; char buff[1024]; char *type = option_find_str(options, "eval", "voc"); FILE *fp = 0; FILE **fps = 0; int coco = 0; int imagenet = 0; if(0==strcmp(type, "coco")){ if(!outfile) outfile = "coco_results"; snprintf(buff, 1024, "%s/%s.json", prefix, outfile); fp = fopen(buff, "w"); fprintf(fp, "[\n"); coco = 1; } else if(0==strcmp(type, "imagenet")){ if(!outfile) outfile = "imagenet-detection"; snprintf(buff, 1024, "%s/%s.txt", prefix, outfile); fp = fopen(buff, "w"); imagenet = 1; classes = 200; } else { if(!outfile) outfile = "comp4_det_test_"; fps = calloc(classes, sizeof(FILE *)); for(j = 0; j < classes; ++j){ snprintf(buff, 1024, "%s/%s%s.txt", prefix, outfile, names[j]); fps[j] = fopen(buff, "w"); } } int m = plist->size; int i=0; int t; float thresh = .005; float nms = .45; int nthreads = 4; image *val = calloc(nthreads, sizeof(image)); image *val_resized = calloc(nthreads, sizeof(image)); image *buf = calloc(nthreads, sizeof(image)); image *buf_resized = calloc(nthreads, sizeof(image)); pthread_t *thr = calloc(nthreads, sizeof(pthread_t)); image input = make_image(net->w, net->h, net->c*2); load_args args = {0}; args.w = net->w; args.h = net->h; //args.type = IMAGE_DATA; args.type = LETTERBOX_DATA; for(t = 0; t < nthreads; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } double start = what_time_is_it_now(); for(i = nthreads; i < m+nthreads; i += nthreads){ fprintf(stderr, "%d\n", i); for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ pthread_join(thr[t], 0); val[t] = buf[t]; val_resized[t] = buf_resized[t]; } for(t = 0; t < nthreads && i+t < m; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ char *path = paths[i+t-nthreads]; char *id = basecfg(path); copy_cpu(net->w*net->h*net->c, val_resized[t].data, 1, input.data, 1); flip_image(val_resized[t]); copy_cpu(net->w*net->h*net->c, val_resized[t].data, 1, input.data + net->w*net->h*net->c, 1); network_predict(net, input.data); int w = val[t].w; int h = val[t].h; int num = 0; detection *dets = get_network_boxes(net, w, h, thresh, .5, map, 0, &num); if (nms) do_nms_sort(dets, num, classes, nms); if (coco){ print_cocos(fp, path, dets, num, classes, w, h); } else if (imagenet){ print_imagenet_detections(fp, i+t-nthreads+1, dets, num, classes, w, h); } else { print_detector_detections(fps, id, dets, num, classes, w, h); } free_detections(dets, num); free(id); free_image(val[t]); free_image(val_resized[t]); } } for(j = 0; j < classes; ++j){ if(fps) fclose(fps[j]); } if(coco){ fseek(fp, -2, SEEK_CUR); fprintf(fp, "\n]\n"); fclose(fp); } fprintf(stderr, "Total Detection Time: %f Seconds\n", what_time_is_it_now() - start); } void validate_detector(char *datacfg, char *cfgfile, char *weightfile, char *outfile) { int j; list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "/home/xieqi/project/traffic_object_detection/voc_data/test.txt"); char *name_list = option_find_str(options, "names", "data/53w.names"); char *prefix = option_find_str(options, "results", "results"); char **names = get_labels(name_list); char *mapf = option_find_str(options, "map", 0); int *map = 0; if (mapf) map = read_map(mapf); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); layer l = net->layers[net->n-1]; int classes = l.classes; char buff[1024]; char *type = option_find_str(options, "eval", "voc"); FILE *fp = 0; FILE **fps = 0; int coco = 0; int imagenet = 0; if(0==strcmp(type, "coco")){ if(!outfile) outfile = "coco_results"; snprintf(buff, 1024, "%s/%s.json", prefix, outfile); fp = fopen(buff, "w"); fprintf(fp, "[\n"); coco = 1; } else if(0==strcmp(type, "imagenet")){ if(!outfile) outfile = "imagenet-detection"; snprintf(buff, 1024, "%s/%s.txt", prefix, outfile); fp = fopen(buff, "w"); imagenet = 1; classes = 200; } else { if(!outfile) outfile = "comp4_det_test_"; fps = calloc(classes, sizeof(FILE *)); for(j = 0; j < classes; ++j){ snprintf(buff, 1024, "%s/%s%s.txt", prefix, outfile, names[j]); fps[j] = fopen(buff, "w"); } } int m = plist->size; int i=0; int t; float thresh = .005; float nms = .45; int nthreads = 4; image *val = calloc(nthreads, sizeof(image)); image *val_resized = calloc(nthreads, sizeof(image)); image *buf = calloc(nthreads, sizeof(image)); image *buf_resized = calloc(nthreads, sizeof(image)); pthread_t *thr = calloc(nthreads, sizeof(pthread_t)); load_args args = {0}; args.w = net->w; args.h = net->h; //args.type = IMAGE_DATA; args.type = LETTERBOX_DATA; for(t = 0; t < nthreads; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } double start = what_time_is_it_now(); for(i = nthreads; i < m+nthreads; i += nthreads){ fprintf(stderr, "%d\n", i); for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ pthread_join(thr[t], 0); val[t] = buf[t]; val_resized[t] = buf_resized[t]; } for(t = 0; t < nthreads && i+t < m; ++t){ args.path = paths[i+t]; args.im = &buf[t]; args.resized = &buf_resized[t]; thr[t] = load_data_in_thread(args); } for(t = 0; t < nthreads && i+t-nthreads < m; ++t){ char *path = paths[i+t-nthreads]; char *id = basecfg(path); float *X = val_resized[t].data; network_predict(net, X); int w = val[t].w; int h = val[t].h; int nboxes = 0; detection *dets = get_network_boxes(net, w, h, thresh, .5, map, 0, &nboxes); if (nms) do_nms_sort(dets, nboxes, classes, nms); if (coco){ print_cocos(fp, path, dets, nboxes, classes, w, h); } else if (imagenet){ print_imagenet_detections(fp, i+t-nthreads+1, dets, nboxes, classes, w, h); } else { print_detector_detections(fps, id, dets, nboxes, classes, w, h); } free_detections(dets, nboxes); free(id); free_image(val[t]); free_image(val_resized[t]); } } for(j = 0; j < classes; ++j){ if(fps) fclose(fps[j]); } if(coco){ fseek(fp, -2, SEEK_CUR); fprintf(fp, "\n]\n"); fclose(fp); } fprintf(stderr, "Total Detection Time: %f Seconds\n", what_time_is_it_now() - start); } void validate_detector_recall(char *datacfg, char *cfgfile, char *weightfile) { network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); fprintf(stderr, "Learning Rate: %g, Momentum: %g, Decay: %g\n", net->learning_rate, net->momentum, net->decay); srand(time(0)); list *options = read_data_cfg(datacfg); char *valid_images = option_find_str(options, "valid", "/home/xieqi/project/traffic_object_detection/voc_data/test.txt"); list *plist = get_paths(valid_images); char **paths = (char **)list_to_array(plist); //layer l = net->layers[net->n-1]; int j, k; int m = plist->size; //測試的圖片總數 int i=0; float thresh = .001; float iou_thresh = .5; float nms = .4; int total = 0; //實際有多少個bbox int correct = 0; //正確識別出了多少個bbox int proposals = 0; //測試集預測的bbox總數 float avg_iou = 0; //printf("l.w*l.h*l.n = %d\n",l.w*l.h*l.n); for(i = 0; i < m; ++i){ char *path = paths[i]; image orig = load_image_color(path, 0, 0); image sized = resize_image(orig, net->w, net->h); char *id = basecfg(path); network_predict(net, sized.data); int nboxes = 0; detection *dets = get_network_boxes(net, sized.w, sized.h, thresh, .5, 0, 1, &nboxes); if (nms) do_nms_obj(dets, nboxes, 1, nms); char labelpath[4096]; find_replace(path, "images", "labels", labelpath); find_replace(labelpath, "JPEGImages", "labels", labelpath); find_replace(labelpath, ".jpg", ".txt", labelpath); find_replace(labelpath, ".JPEG", ".txt", labelpath); int num_labels = 0; //測試集實際的標注框數量 box_label *truth = read_boxes(labelpath, &num_labels); for(k = 0; k < nboxes; ++k){ if(dets[k].objectness > thresh){ ++proposals; } } for (j = 0; j < num_labels; ++j) { ++total; box t = {truth[j].x, truth[j].y, truth[j].w, truth[j].h}; //printf("truth x=%f, y=%f, w=%f,h=%f\n",truth[j].x,truth[j].y,truth[j].w,truth[j].h); float best_iou = 0; //對每一個標注的框 for(k = 0; k < nboxes; ++k){ //for(k = 0; k < l.w*l.h*l.n; ++k){ float iou = box_iou(dets[k].bbox, t); //printf("predict=%f iou=%f x=%f, y=%f, w=%f,h=%f\n",dets[k].objectness,iou,dets[k].bbox.x,dets[k].bbox.y,dets[k].bbox.w,dets[k].bbox.h); if(dets[k].objectness > thresh && iou > best_iou){ best_iou = iou; } } //printf("best_iou=%f\n",best_iou); avg_iou += best_iou; if(best_iou > iou_thresh){ ++correct; } } fprintf(stderr, "%5d %5d %5d\tRPs/Img: %.2f\tIOU: %.2f%%\tRecall:%.2f%%\n", i, correct, total, (float)proposals/(i+1), avg_iou*100/total, 100.*correct/total); free(id); free_image(orig); free_image(sized); } } void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen) { list *options = read_data_cfg(datacfg); //options存儲分類的標簽等基本訓練信息 char *name_list = option_find_str(options, "names", "data/names.list"); //抽取標簽名稱 char **names = get_labels(name_list); image **alphabet = load_alphabet(); //加載位於data/labels下的字符圖片，用於顯示矩形框名稱 network *net = load_network(cfgfile, weightfile, 0); //用netweork.h中自定義的network結構體存儲模型文件,函數位於parser.c set_batch_network(net, 1); srand(2222222); double start_time; double end_time; double img_time; double sum_time=0.0; char buff[256]; char *input = buff; float nms=.45; int i=0; while(1){ //讀取結構對應的權重文件 if(filename){ strncpy(input, filename, 256); image im = load_image_color(input,0,0); image sized = letterbox_image(im, net->w, net->h); //輸入圖片大小經過resize至輸入大小 //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; //X指向圖片的data元素，即圖片像素 start_time=what_time_is_it_now(); network_predict(net, X); //network_predict函數負責預測當前圖片的數據X end_time=what_time_is_it_now(); img_time= end_time - start_time; printf("%s: Predicted in %f seconds.\n", input, img_time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile) { save_image(im, outfile); } else{ //save_image(im, "predictions"); char image[2048]; sprintf(image,"./data/predict/%s",GetFilename(filename)); save_image(im,image); printf("predict %s successfully!\n",GetFilename(filename)); #ifdef OPENCV cvNamedWindow("predictions", CV_WINDOW_NORMAL); if(fullscreen){ cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN); } show_image(im, "predictions"); cvWaitKey(0); cvDestroyAllWindows(); #endif } free_image(im); free_image(sized); if (filename) break; } else { printf("Enter Image Path: "); fflush(stdout); input = fgets(input, 256, stdin); if(!input) return; strtok(input, "\n"); list *plist = get_paths(input); char **paths = (char **)list_to_array(plist); printf("Start Testing!\n"); int m = plist->size; if(access("/home/xieqi/darknet/data/out_img",0)==-1)//修改成自己的路徑 { if (mkdir("/home/xieqi/darknet/data/out_img",0777))//修改成自己的路徑 { printf("creat file bag failed!!!"); } } for(i = 0; i < m; ++i){ char *path = paths[i]; image im = load_image_color(path,0,0); image sized = letterbox_image(im, net->w, net->h); //輸入圖片大小經過resize至輸入大小 //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; //X指向圖片的data元素，即圖片像素 start_time = what_time_is_it_now(); network_predict(net, X); //network_predict函數負責預測當前圖片的數據X end_time = what_time_is_it_now(); img_time = end_time - start_time; sum_time = sum_time+img_time; printf("Try Very Hard:"); printf("%s: Predicted in %f seconds.\n", path, img_time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile){ save_image(im, outfile); } else{ char b[2048]; sprintf(b,"/home/xieqi/darknet/data/out_img/%s",GetFilename(path));//修改成自己的路徑 save_image(im, b); printf("save %s successfully!\n",GetFilename(path)); #ifdef OPENCV cvNamedWindow("predictions", CV_WINDOW_NORMAL); if(fullscreen){ cvSetWindowProperty("predictions", CV_WND_PROP_FULLSCREEN, CV_WINDOW_FULLSCREEN); } show_image(im, "predictions"); cvWaitKey(0); cvDestroyAllWindows(); #endif } free_image(im); free_image(sized); if (filename) break; } printf("fps: %.2f totall image %d\n",(float)m/sum_time,m); } } } /* void test_detector(char *datacfg, char *cfgfile, char *weightfile, char *filename, float thresh, float hier_thresh, char *outfile, int fullscreen) { list *options = read_data_cfg(datacfg); char *name_list = option_find_str(options, "names", "data/names.list"); char **names = get_labels(name_list); image **alphabet = load_alphabet(); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); double time; char buff[256]; char *input = buff; float nms=.45; while(1){ if(filename){ strncpy(input, filename, 256); } else { printf("Enter Image Path: "); fflush(stdout); input = fgets(input, 256, stdin); if(!input) return; strtok(input, "\n"); } image im = load_image_color(input,0,0); image sized = letterbox_image(im, net->w, net->h); //image sized = resize_image(im, net->w, net->h); //image sized2 = resize_max(im, net->w); //image sized = crop_image(sized2, -((net->w - sized2.w)/2), -((net->h - sized2.h)/2), net->w, net->h); //resize_network(net, sized.w, sized.h); layer l = net->layers[net->n-1]; float *X = sized.data; time=what_time_is_it_now(); network_predict(net, X); printf("%s: Predicted in %f seconds.\n", input, what_time_is_it_now()-time); int nboxes = 0; detection *dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 1, &nboxes); //printf("%d\n", nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); draw_detections(im, dets, nboxes, thresh, names, alphabet, l.classes); free_detections(dets, nboxes); if(outfile){ save_image(im, outfile); } else{ save_image(im, "predictions"); #ifdef OPENCV make_window("predictions", 512, 512, 0); show_image(im, "predictions", 0); #endif } free_image(im); free_image(sized); if (filename) break; } } void censor_detector(char *datacfg, char *cfgfile, char *weightfile, int cam_index, const char *filename, int class, float thresh, int skip) { #ifdef OPENCV char *base = basecfg(cfgfile); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); CvCapture * cap; int w = 1280; int h = 720; if(filename){ cap = cvCaptureFromFile(filename); }else{ cap = cvCaptureFromCAM(cam_index); } if(w){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_WIDTH, w); } if(h){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_HEIGHT, h); } if(!cap) error("Couldn't connect to webcam.\n"); cvNamedWindow(base, CV_WINDOW_NORMAL); cvResizeWindow(base, 512, 512); float fps = 0; int i; float nms = .45; while(1){ image in = get_image_from_stream(cap); //image in_s = resize_image(in, net->w, net->h); image in_s = letterbox_image(in, net->w, net->h); layer l = net->layers[net->n-1]; float *X = in_s.data; network_predict(net, X); int nboxes = 0; detection *dets = get_network_boxes(net, in.w, in.h, thresh, 0, 0, 0, &nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); for(i = 0; i < nboxes; ++i){ if(dets[i].prob[class] > thresh){ box b = dets[i].bbox; int left = b.x-b.w/2.; int top = b.y-b.h/2.; censor_image(in, left, top, b.w, b.h); } } show_image(in, base); cvWaitKey(10); free_detections(dets, nboxes); free_image(in_s); free_image(in); float curr = 0; fps = .9*fps + .1*curr; for(i = 0; i < skip; ++i){ image in = get_image_from_stream(cap); free_image(in); } } #endif } void extract_detector(char *datacfg, char *cfgfile, char *weightfile, int cam_index, const char *filename, int class, float thresh, int skip) { #ifdef OPENCV char *base = basecfg(cfgfile); network *net = load_network(cfgfile, weightfile, 0); set_batch_network(net, 1); srand(2222222); CvCapture * cap; int w = 1280; int h = 720; if(filename){ cap = cvCaptureFromFile(filename); }else{ cap = cvCaptureFromCAM(cam_index); } if(w){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_WIDTH, w); } if(h){ cvSetCaptureProperty(cap, CV_CAP_PROP_FRAME_HEIGHT, h); } if(!cap) error("Couldn't connect to webcam.\n"); cvNamedWindow(base, CV_WINDOW_NORMAL); cvResizeWindow(base, 512, 512); float fps = 0; int i; int count = 0; float nms = .45; while(1){ image in = get_image_from_stream(cap); //image in_s = resize_image(in, net->w, net->h); image in_s = letterbox_image(in, net->w, net->h); layer l = net->layers[net->n-1]; show_image(in, base); int nboxes = 0; float *X = in_s.data; network_predict(net, X); detection *dets = get_network_boxes(net, in.w, in.h, thresh, 0, 0, 1, &nboxes); //if (nms) do_nms_obj(boxes, probs, l.w*l.h*l.n, l.classes, nms); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); for(i = 0; i < nboxes; ++i){ if(dets[i].prob[class] > thresh){ box b = dets[i].bbox; int size = b.w*in.w > b.h*in.h ? b.w*in.w : b.h*in.h; int dx = b.x*in.w-size/2.; int dy = b.y*in.h-size/2.; image bim = crop_image(in, dx, dy, size, size); char buff[2048]; sprintf(buff, "results/extract/%07d", count); ++count; save_image(bim, buff); free_image(bim); } } free_detections(dets, nboxes); free_image(in_s); free_image(in); float curr = 0; fps = .9*fps + .1*curr; for(i = 0; i < skip; ++i){ image in = get_image_from_stream(cap); free_image(in); } } #endif } */ /* void network_detect(network *net, image im, float thresh, float hier_thresh, float nms, detection *dets) { network_predict_image(net, im); layer l = net->layers[net->n-1]; int nboxes = num_boxes(net); fill_network_boxes(net, im.w, im.h, thresh, hier_thresh, 0, 0, dets); if (nms) do_nms_sort(dets, nboxes, l.classes, nms); } */ // ./darknet [xxx]中如果命令如果第二個xxx參數是detector，則調用這個 void run_detector(int argc, char **argv) { char *prefix = find_char_arg(argc, argv, "-prefix", 0); //尋找是否有參數prefix, 默認參數0，argv為二維數組，存儲了參數字符串 float thresh = find_float_arg(argc, argv, "-thresh", .5); //尋找是否有參數thresh，thresh為輸出的閾值,默認參數0.24 float hier_thresh = find_float_arg(argc, argv, "-hier", .5); //尋找是否有參數hier_thresh,默認0.5 int cam_index = find_int_arg(argc, argv, "-c", 0); //尋找是否有參數c，默認0 int frame_skip = find_int_arg(argc, argv, "-s", 0); //尋找是否有參數s，默認0 int avg = find_int_arg(argc, argv, "-avg", 3); //如果輸入參數小於4個，輸出正確語法如何使用 //printf 等價於 fprintf(stdout, ...)，這里stderr和stdout默認輸出設備都是屏幕，但是stderr一般指標准出錯輸入設備 if(argc < 4){ fprintf(stderr, "usage: %s %s [train/test/valid] [cfg] [weights (optional)]\n", argv[0], argv[1]); return; } char *gpu_list = find_char_arg(argc, argv, "-gpus", 0); //尋找是否有參數gpus，默認0 char *outfile = find_char_arg(argc, argv, "-out", 0); //檢查是否指定GPU運算,默認0 int *gpus = 0; int gpu = 0; int ngpus = 0; if(gpu_list){ printf("%s\n", gpu_list); int len = strlen(gpu_list); ngpus = 1; int i; for(i = 0; i < len; ++i){ if (gpu_list[i] == ',') ++ngpus; } gpus = calloc(ngpus, sizeof(int)); for(i = 0; i < ngpus; ++i){ gpus[i] = atoi(gpu_list); gpu_list = strchr(gpu_list, ',')+1; } } else { gpu = gpu_index; gpus = &gpu; ngpus = 1; } int clear = find_arg(argc, argv, "-clear"); //檢查clear參數 int fullscreen = find_arg(argc, argv, "-fullscreen"); int width = find_int_arg(argc, argv, "-w", 0); int height = find_int_arg(argc, argv, "-h", 0); int fps = find_int_arg(argc, argv, "-fps", 0); //int class = find_int_arg(argc, argv, "-class", 0); char *datacfg = argv[3]; //存data文件路徑 char *cfg = argv[4]; //存cfg文件路徑 char *weights = (argc > 5) ? argv[5] : 0; //存weight文件路徑 char *filename = (argc > 6) ? argv[6]: 0; //存待檢測文件路徑 //根據第三個參數的內容，調用不同的函數，並傳入之前解析的參數 if(0==strcmp(argv[2], "test")) test_detector(datacfg, cfg, weights, filename, thresh, hier_thresh, outfile, fullscreen); else if(0==strcmp(argv[2], "train")) train_detector(datacfg, cfg, weights, gpus, ngpus, clear); else if(0==strcmp(argv[2], "valid")) validate_detector(datacfg, cfg, weights, outfile); else if(0==strcmp(argv[2], "valid2")) validate_detector_flip(datacfg, cfg, weights, outfile); else if(0==strcmp(argv[2], "recall")) validate_detector_recall(datacfg, cfg, weights); else if(0==strcmp(argv[2], "demo")) { list *options = read_data_cfg(datacfg); int classes = option_find_int(options, "classes", 20); char *name_list = option_find_str(options, "names", "data/names.list"); char **names = get_labels(name_list); demo(cfg, weights, thresh, cam_index, filename, names, classes, frame_skip, prefix, avg, hier_thresh, width, height, fps, fullscreen); } //else if(0==strcmp(argv[2], "extract")) extract_detector(datacfg, cfg, weights, cam_index, filename, class, thresh, frame_skip); //else if(0==strcmp(argv[2], "censor")) censor_detector(datacfg, cfg, weights, cam_index, filename, class, thresh, frame_skip); }
"""
Note:If during training you see nan values for avg (loss) field - then training goes wrong,
but if nan is in some other lines - then training goes well.
When should I stop training:
When you see that average loss 0.xxxxxx avg no longer decreases at many iterations then you should stop training.
Once training is stopped, you should take some of last .weights-files from darknet\build\darknet\x64\backup and choose the best of them.
Overfitting - is case when you can detect objects on images from training-dataset,
but can't detect objects on any others images. You should get weights from Early Stopping Point.
IoU (intersect of union) - average instersect of union of objects and detections for a certain threshold = 0.24
How to improve object detection:
Before training:
set flag random=1 in your .cfg-file - it will increase precision by training Yolo for different resolutions.
increase network resolution in your .cfg-file (height=608, width=608 or any value multiple of 32) - it will increase precision.
recalculate anchors for your dataset for width and height from cfg-file:
darknet.exe detector calc_anchors data/obj.data -num_of_clusters 9 -width 416 -height 416 then set the same 9 anchors in each of 3 [yolo]-layers in your cfg-file
設置錨點
desirable that your training dataset include images with objects at diffrent:
scales, rotations, lightings, from different sides, on different backgrounds
樣本特點盡量多樣化，亮度，旋轉，背景，目標位置，尺寸
desirable that your training dataset include images with non-labeled objects that you do not want to detect - negative samples without bounded box (empty .txt files)
可以添加沒有標注框的圖片和其空的txt文件，作為negative數據
for training with a large number of objects in each image, add the parameter max=200 or higher value in the last layer [region] in your cfg-file
to speedup training (with decreasing detection accuracy) do Fine-Tuning instead of Transfer-Learning,
set param stopbackward=1 in one of the penultimate convolutional layers before the 1-st [yolo]-layer,
for example here: https://github.com/AlexeyAB/darknet/blob/0039fd26786ab5f71d5af725fc18b3f521e7acfd/cfg/yolov3.cfg#L598
可以在第一個[yolo]層之前的倒數第二個[convolutional]層末尾添加 stopbackward=1,以此提升訓練速度
After training - for detection:
Increase network-resolution by set in your .cfg-file (height=608 and width=608) or (height=832 and width=832)
or (any value multiple of 32) - this increases the precision and makes it possible to detect small objects,
you do not need to train the network again, just use .weights-file already trained for 416x416 resolution.
即使在用416*416訓練完之后，也可以在cfg文件中設置較大的width和height，增加網絡對圖像的分辨率，從而更可能檢測出圖像中的小目標，而不需要重新訓練
if error Out of memory occurs then in .cfg-file you should increase subdivisions=16, 32 or 64
"""