caffe版faster-RCNN環境搭建

本文轉載自查看原文 2017-11-19 16:26 7679 深度學習DeepLearning/ 檢測網絡/ 環境搭建

faster-rcnn提出論文：《Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks》
faster-rcnn 的算法詳解可看這篇博文（清晰易懂，良心博文！）： http://blog.csdn.net/shenxiaolu1984/article/details/51152614
faster-rcnn Python版本源碼地址：https://github.com/rbgirshick/py-faster-rcnn

- 基本按照官網上的readme

1.電腦上已經有可運行caffe所需的環境

2.下載faster-rcnn python版本源碼

git clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git

3. 進入/py-faster-rcnn/lib 進行編譯，build the Cython modules

    cd py-faster-rcnn/lib  
    make

如果重新升級或者降級numpy依賴包，需要重新去掉之前生成的文件，重新make; 否則會報這里面的錯

問題：

python setup.py build_ext --inplace
running build_ext
cythoning utils/bbox.pyx to utils/bbox.c

Error compiling Cython file:
------------------------------------------------------------
...
# Copyright (c) 2015 Microsoft
# Licensed under The MIT License [see LICENSE for details]
# Written by Sergey Karayev
# --------------------------------------------------------

cimport cython
       ^
------------------------------------------------------------

utils/bbox.pyx:8:8: Compiler crash in AnalyseDeclarationsTransform

File 'ModuleNode.py', line 122, in analyse_declarations: ModuleNode(bbox.pyx:1:0,
    full_module_name = 'utils.cython_bbox')
File 'Nodes.py', line 408, in analyse_declarations: StatListNode(bbox.pyx:8:0)
File 'Nodes.py', line 408, in analyse_declarations: StatListNode(bbox.pyx:8:8)
File 'Nodes.py', line 7396, in analyse_declarations: CImportStatNode(bbox.pyx:8:8,
    module_name = u'cython')


  File "/home/dsp/anaconda2/lib/python2.7/site-packages/Cython/Utils.py", line 148, in search_include_directories
    path = os.path.join(dir, dotted_filename)
  File "/home/dsp/anaconda2/lib/python2.7/posixpath.py", line 73, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 10: ordinal not in range(128)
building 'utils.cython_bbox' extension
gcc -pthread -B /home/dsp/anaconda2/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/dsp/anaconda2/lib/python2.7/site-packages/numpy/core/include -I/home/dsp/anaconda2/include/python2.7 -c utils/bbox.c -o build/temp.linux-x86_64-2.7/utils/bbox.o -Wno-cpp -Wno-unused-function
utils/bbox.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
 #error Do not use this file, it is the result of a failed Cython compilation.
  ^
error: command 'gcc' failed with exit status 1
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 1

先各種百度，安裝cython等；結果發現時路徑的坑
神坑：路徑；py-faster-rcnn存放位置路徑中不能有中文，否則報錯UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position
最后放到 /home/uername/ 下直接makec成功

4.編譯/py-faster-rcnn/caffe-fast-rcnn

cd py-faster-rcnn/caffe-fast-rcnn
make -j32 && make pycaffe

Makefile.config文件直接用的本機配置caffe的文件
問題：

dsp@dsp-PowerEdge-R730:/home/user/ran/Desktop/protobuf-2.6.1$ protoc
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.4.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1.  Please update your library.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "google/protobuf/any.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  This program requires version 3.4.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1.  Please update your library.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "google/protobuf/any.pb.cc".)
Aborted (core dumped)

這個問題是服務器上的錯誤：http://blog.csdn.net/m0_37477175/article/details/78233983

https://www.cnblogs.com/javaee6/p/4849051.html

https://github.com/BVLC/caffe/issues/5711

前面兩種方法都是了沒有解決，后面github上的issue也沒有給出解決方法，看以后能不能解決！！！

In file included from ./include/caffe/util/device_alternate.hpp:40:0,
                 from ./include/caffe/common.hpp:19,
                 from ./include/caffe/blob.hpp:8,
                 from src/caffe/blob.cpp:4:
./include/caffe/util/cudnn.hpp:8:34: fatal error: caffe/proto/caffe.pb.h: 沒有那個文件或目錄

- 按照：解決方法：用protoc從caffe/src/caffe/proto/caffe.proto生成caffe.pb.h和caffe.pb.cc ，http://blog.csdn.net/xmzwlw/article/details/48270225 沒有作用；后面還有一串錯誤

In file included from ./include/caffe/util/device_alternate.hpp:40:0,
                 from ./include/caffe/common.hpp:19,
                 from ./include/caffe/blob.hpp:8,
                 from ./include/caffe/layers/loss_layer.hpp:6,
                 from src/caffe/layers/loss_layer.cpp:3:
./include/caffe/util/cudnn.hpp: In function ‘const char* cudnnGetErrorString(cudnnStatus_t)’:
./include/caffe/util/cudnn.hpp:21:10: warning: enumeration value ‘CUDNN_STATUS_RUNTIME_PREREQUISITE_MISSING’ not handled in switch [-Wswitch]
   switch (status) {
          ^
./include/caffe/util/cudnn.hpp: In function ‘void caffe::cudnn::setConvolutionDesc(cudnnConvolutionStruct**, cudnnTensorDescriptor_t, cudnnFilterDescriptor_t, int, int, int, int)’:
./include/caffe/util/cudnn.hpp:108:70: error: too few arguments to function ‘cudnnStatus_t cudnnSetConvolution2dDescriptor(cudnnConvolutionDescriptor_t, int, int, int, int, int, int, cudnnConvolutionMode_t, cudnnDataType_t)’
       pad_h, pad_w, stride_h, stride_w, 1, 1, CUDNN_CROSS_CORRELATION));
                                                                      ^
./include/caffe/util/cudnn.hpp:15:28: note: in definition of macro ‘CUDNN_CHECK’
     cudnnStatus_t status = condition; \
                            ^
In file included from ./include/caffe/util/cudnn.hpp:5:0,
                 from ./include/caffe/util/device_alternate.hpp:40,
                 from ./include/caffe/common.hpp:19,
                 from ./include/caffe/blob.hpp:8,
                 from ./include/caffe/layers/loss_layer.hpp:6,
                 from src/caffe/layers/loss_layer.cpp:3:
/usr/local/cuda/include/cudnn.h:500:27: note: declared here
 cudnnStatus_t CUDNNWINAPI cudnnSetConvolution2dDescriptor( cudnnConvolutionDescriptor_t convDesc,
                           ^
In file included from ./include/caffe/util/device_alternate.hpp:40:0,
                 from ./include/caffe/common.hpp:19,
                 from ./include/caffe/blob.hpp:8,
                 from ./include/caffe/layers/loss_layer.hpp:6,
                 from src/caffe/layers/loss_layer.cpp:3:
./include/caffe/util/cudnn.hpp: In function ‘void caffe::cudnn::createPoolingDesc(cudnnPoolingStruct**, caffe::PoolingParameter_PoolMethod, cudnnPoolingMode_t*, int, int, int, int, int, int)’:
./include/caffe/util/cudnn.hpp:127:41: error: too few arguments to function ‘cudnnStatus_t cudnnSetPooling2dDescriptor(cudnnPoolingDescriptor_t, cudnnPoolingMode_t, cudnnNanPropagation_t, int, int, int, int, int, int)’
         pad_h, pad_w, stride_h, stride_w));
                                         ^
./include/caffe/util/cudnn.hpp:15:28: note: in definition of macro ‘CUDNN_CHECK’
     cudnnStatus_t status = condition; \
                            ^
In file included from ./include/caffe/util/cudnn.hpp:5:0,
                 from ./include/caffe/util/device_alternate.hpp:40,
                 from ./include/caffe/common.hpp:19,
                 from ./include/caffe/blob.hpp:8,
                 from ./include/caffe/layers/loss_layer.hpp:6,
                 from src/caffe/layers/loss_layer.cpp:3:

View Code

后面發現時cndnn版本不兼容：Faster R-CNN 和最新的版本cuDNN V5.0不兼容問題

解決方法：

cd py-faster-rcnn/caffe-fast-rcnn  
Git remote add caffe https://github.com/BVLC/caffe.git  
git fetch caffe  
git merge caffe/master 

在合並之后注釋掉include/caffe/layers/python_layer.hppa文件里的self_.attr(“phase”) = static_cast(this->phase_)

重新從caffe中fetch and merge
遇到merge沖突，直接編輯

  // Caffe utility functions
  bp::def("init_log", &InitLog);
  bp::def("init_log", &InitLogLevel);
  bp::def("init_log", &InitLogLevelPipe);
  bp::def("log", &Log);
  bp::def("has_nccl", &HasNCCL);
  bp::def("set_mode_cpu", &set_mode_cpu);
  bp::def("set_mode_gpu", &set_mode_gpu);
  bp::def("set_random_seed", &set_random_seed);
  bp::def("set_device", &Caffe::SetDevice);
<<<<<<< HEAD
  bp::def("set_random_seed", &Caffe::set_random_seed);
=======
  bp::def("solver_count", &Caffe::solver_count);
  bp::def("set_solver_count", &Caffe::set_solver_count);
  bp::def("solver_rank", &Caffe::solver_rank);
  bp::def("set_solver_rank", &Caffe::set_solver_rank);
  bp::def("set_multiprocess", &Caffe::set_multiprocess);
>>>>>>> caffe/master

- 最后編譯成功；faster-rcnn 安裝里面也提到了處理版本不兼容，進行文件替換的方法

AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/train_net.bin
CXX/LD -o .build_release/tools/finetune_net.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/test_net.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX/LD -o .build_release/tools/net_speed_benchmark.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
CXX/LD -o .build_release/examples/cpp_classification/classification.bin
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
touch python/caffe/proto/__init__.py
PROTOC (python) src/caffe/proto/caffe.proto

View Code

5、下載Faster R-CNN的預訓練模型

cd py-faster-rcnn ./data/scripts/fetch_faster_rcnn_models.sh

model:https://dl.dropboxusercontent.com/s/o6ii098bu51d139/faster_rcnn_models.tgz?dl=0

6、運行demo.py

cd py-faster-rcnn ./tools/demo.py

7.Usage:訓練

To train and test a Faster R-CNN detector using the alternating optimization algorithm from our NIPS 2015 paper, use experiments/scripts/faster_rcnn_alt_opt.sh. Output is written underneath $FRCN_ROOT/output.

cd $FRCN_ROOT ./experiments/scripts/faster_rcnn_alt_opt.sh [GPU_ID] [NET] [--set ...] # GPU_ID is the GPU you want to train on # NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use # --set ... allows you to specify fast_rcnn.config options, e.g. # --set EXP_DIR seed_rng1701 RNG_SEED 1701

AttributeError: 'module' object has no attribute 'text_format'

在文件./lib/fast_rcnn/train.py增加一行import google.protobuf.text_format 即可解決問題

8.后續問題

I1119 21:29:42.451519 21365 net.cpp:744] Ignoring source layer pool5_spm6
I1119 21:29:42.451537 21365 net.cpp:744] Ignoring source layer pool5_spm6_flatten
I1119 21:29:42.481129 21365 net.cpp:744] Ignoring source layer fc8
I1119 21:29:42.481151 21365 net.cpp:744] Ignoring source layer prob
Solving...
Process Process-3:
Traceback (most recent call last):
  File "/home/dsp/anaconda2/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/home/dsp/anaconda2/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "./tools/train_faster_rcnn_alt_opt.py", line 195, in train_fast_rcnn
    max_iters=max_iters)
  File "/home/dsp/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 162, in train_net
    model_paths = sw.train_model(max_iters)
  File "/home/dsp/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 103, in train_model
    self.solver.step(1)
  File "/home/dsp/py-faster-rcnn/tools/../lib/roi_data_layer/layer.py", line 144, in forward
    blobs = self._get_next_minibatch()
  File "/home/dsp/py-faster-rcnn/tools/../lib/roi_data_layer/layer.py", line 63, in _get_next_minibatch
    return get_minibatch(minibatch_db, self._num_classes)
  File "/home/dsp/py-faster-rcnn/tools/../lib/roi_data_layer/minibatch.py", line 55, in get_minibatch
    num_classes)
  File "/home/dsp/py-faster-rcnn/tools/../lib/roi_data_layer/minibatch.py", line 100, in _sample_rois
    fg_inds, size=fg_rois_per_this_image, replace=False)
  File "mtrand.pyx", line 1187, in mtrand.RandomState.choice
TypeError: 'numpy.float64' object cannot be interpreted as an index


TypeError: 'numpy.float64' object cannot be interpreted as an index
還是numpy版本的問題，直接換一個版本好了
sudo pip install -U numpy==1.11.0


--這個錯 “ImportError: numpy.core.multiarray failed to import”好像就是lib庫里面沒有重新make,去掉之前

+ ./tools/train_faster_rcnn_alt_opt.py --gpu 0 --net_name ZF --weights data/imagenet_models/ZF.v2.caffemodel --imdb voc_2007_trainval --cfg experiments/cfgs/faster_rcnn_alt_opt.yml Traceback (most recent call last): File "./tools/train_faster_rcnn_alt_opt.py", line 19, in <module> from datasets.factory import get_imdb File "/home/dsp/py-faster-rcnn/tools/../lib/datasets/factory.py", line 13, in <module> from datasets.coco import coco File "/home/dsp/py-faster-rcnn/tools/../lib/datasets/coco.py", line 20, in <module> from pycocotools.coco import COCO File "/home/dsp/py-faster-rcnn/tools/../lib/pycocotools/coco.py", line 58, in <module> import mask File "/home/dsp/py-faster-rcnn/tools/../lib/pycocotools/mask.py", line 3, in <module> import pycocotools._mask as _mask File "pycocotools/_mask.pyx", line 20, in init pycocotools._mask File "__init__.pxd", line 989, in numpy.import_array ImportError: numpy.core.multiarray failed to import

Reading annotation for 4901/4952
Saving cached annotations to /home/dsp/py-faster-rcnn/data/VOCdevkit2007/annotations_cache/annots.pkl
AP for aeroplane = 0.0339
AP for bicycle = 0.1139
AP for bird = 0.0069
AP for boat = 0.0646
Traceback (most recent call last):
  File "./tools/test_net.py", line 90, in <module>
    test_net(net, imdb, max_per_image=args.max_per_image, vis=args.vis)
  File "/home/dsp/py-faster-rcnn/tools/../lib/fast_rcnn/test.py", line 295, in test_net
    imdb.evaluate_detections(all_boxes, output_dir)
  File "/home/dsp/py-faster-rcnn/tools/../lib/datasets/pascal_voc.py", line 322, in evaluate_detections
    self._do_python_eval(output_dir)
  File "/home/dsp/py-faster-rcnn/tools/../lib/datasets/pascal_voc.py", line 285, in _do_python_eval
    use_07_metric=use_07_metric)
  File "/home/dsp/py-faster-rcnn/tools/../lib/datasets/voc_eval.py", line 148, in voc_eval
    BB = BB[sorted_ind, :]
IndexError: too many indices for array

http://blog.csdn.net/weinaonao5418/article/details/54234684

File "/home/dl-box/wei/py-faster-rcnn/tools/../lib/datasets/voc_eval.py", line 149, in voc_eval

BB = BB[sorted_ind, :]

IndexError: too many indices for array

這個錯是說沒有學習到東西，我用了[4000,2000,4000,2000]試了一下也不行。不過能看到這個錯誤，如果你的數據集做的沒有問題的話，用 [80000,40000,80000,40000]（親測大概16小時，1341張圖）運行你自己的數據集就沒問題了。為了省時間我用的是[40000,20000,40000,2000]（大概8小時，50張圖）。這個時間好像與圖片大小和圖片數量關系不大。感覺只與迭代次數有關。

Reference

CNN目標檢測（一）：Faster RCNN詳解

faster-rcnn 安裝

Faster R-CNN 的Caffe實現

http://blog.csdn.net/mydear_11000/article/details/70241139

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 faster-rcnn Pytorch復現Faster-RCNN Faster-RCNN 訓練自己的數據 faster-rcnn原理講解 faster-rcnn anchor 介紹關於Faster-RCNN訓練細節 faster-rcnn系列筆記(一) mmdetection faster-rcnn yolo faster-rcnn 結構雜談 Caffe使用step by step：faster-rcnn目標檢測matlab代碼