centos7下安裝caffe以及pycaffe流程及問題解決

一、環境准備

安裝依賴包

python、tesseract依賴：

yum install openssl openssl-devel gcc gcc-c++ autoconf automake libtool libjpeg libpng libtiff zlib libjpeg-devel libpng-devel libtiff-devel zlib-devel python-devel ImageMagick ImageMagick-devel ghostscript python-imaging vim (libyaml)

caffe依賴：

yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel  openblas-devel

源碼位置

caffe: https://github.com/BVLC/caffe

下載部署包

• python366：https://www.python.org/downloads/release/python-366/

二、安裝Python3

• （1）創建文件夾：

mkdir /usr/local/python3

• （2）解壓下載的文件：
  tar xvJf Python-3.6.6.tar.xz
• （3）進入解壓后目錄，執行編譯命令

./configure --prefix=/usr/local/python3
make && make install

• （4）創建軟連接命令：

ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

三、安裝caffe

• （1）下載額外的rpm包（可能yum源沒有），然后安裝。

gflags-2.1.1-6.el7.x86_64.rpm        glog-devel-0.3.3-8.el7.x86_64.rpm    leveldb-1.12.0-11.el7.x86_64.rpm        libaec-devel-1.0.4-1.el7.x86_64.rpm  lmdb-libs-0.9.22-2.el7.x86_64.rpm
gflags-devel-2.1.1-6.el7.x86_64.rpm  hdf5-1.8.12-10.el7.x86_64.rpm        leveldb-devel-1.12.0-11.el7.x86_64.rpm  lmdb-0.9.22-2.el7.x86_64.rpm
glog-0.3.3-8.el7.x86_64.rpm          hdf5-devel-1.8.12-10.el7.x86_64.rpm  libaec-1.0.4-1.el7.x86_64.rpm           lmdb-devel-0.9.22-2.el7.x86_64.rpm

• （2）yum安裝依賴包。

yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel

-（3）解壓caffe源碼包，然后執行：

cp Makefile.config.example Makefile.config

-（4）修改文件“Makefile.conf”，內容如下：

CPU_ONLY := 1
USE_OPENCV := 0


WITH_PYTHON_LAYER := 1
 
 
……
 PYTHON_INCLUDE := /usr/local/python3/include/python3.6m \
                 /usr/local/python3/lib/python3.6/site-packages/numpy/core/include \
                 /usr/local/python3/

• （5）開始執行編譯：

make all -j8
make test -j16
make runtest

注意：這里“-j8”等參數，表示使用多少個線程進行編譯。

• （6） 編譯pycaffe。

make pycaffe -j8

然后將編譯完成之后的目錄“/opt/soft/caffe/caffe-master/python/caffe”拷貝到python的site-packages目錄中。

• 異常1: 遇到如下錯誤：

[root@aidm-1 caffe-master]# vim Makefile.config
[root@aidm-1 caffe-master]# make all -j8
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: 找不到 -lcblas
/usr/bin/ld: 找不到 -latlas
collect2: 錯誤：ld 返回 1
make: *** [.build_release/lib/libcaffe.so.1.0.0] 錯誤 1

則進入 “usr/lib63/atlas”，創建兩個軟連接：

sudo ln -sv libsatlas.so.3.10 libcblas.so
sudo ln -sv libsatlas.so.3.10 libatlas.so

然后修改“Makefile.config”文件，在“LIBRARY_DIRS”后面追加 “/usr/lib64/atlas”。

• 異常2: 安裝完成后，通過python調用出現如下錯誤：

[root@aidm-1 ~]# python3
Python 3.6.6 (default, Jun 26 2019, 17:33:11)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: /usr/local/python3/lib/python3.6/site-packages/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

要在“Makefile.config”文件中去掉這行注釋：

WITH_PYTHON_LAYER := 1

然后，在/usr/lib64中添加如下兩個文件：

libpython3.4m.so.1.0
libboost_python34.so.1.69.0

然后，重新編譯。

• 異常3: python調用出現如下錯誤：

Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/abc/workplace/caffe/python/caffe/__init__.py", line 4, in <module>
    from .proto.caffe_pb2 import TRAIN, TEST
  File "/home/abc/workplace/caffe/python/caffe/proto/caffe_pb2.py", line 17, in <module>
    serialized_pb='\n\x0b\x63\x61\x66\x66\x65.proto\x12\x05\x63\x61\x66\x66\x65\"\x1c\n\tBlobShape\x12\x0f\n\x03\x64i...'
  File "/home/abc/anaconda3/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 824, in __new__
    return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found

原因是之前編譯的python包“caffe”中的proto/caffe_pb2.py文件有問題，找一個正確的替換下即可。

• 有用的linux命令

ls -l|grep "7月 2" | awk -F ' ' '{print $9;}' | xargs -i cp {} /home/aidm-algorithm/libs/ext-lib-all

問題

安裝Python完成之后，執行pip3 list 命令之后報如下錯誤：

[root@aidm-2 ~]# pip3 list
Package    Version
---------- -------
pip        10.0.1
setuptools 39.0.1
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

• 原因：新版本pip默認使用https，而Python里面沒有找到ssl模塊（由於編譯python的時候沒有指定）。
• 解決方式1: 配置一個http的pip源，然后將這個源的域名設置為受信任的，參考操作如下：

mkdir ~/.pip
cd ~/.pip 
touch pip.conf

然后添加如下內容：

[global] 
index-url = http://pypi.douban.com/simple 
[install]
trusted-host = pypi.douban.com

• 解決方式2: 首先檢查系統中是否包含openssl和opeenssl-devel兩個包，如果沒有，則首先進行安裝。然后重新編譯python：

./configure --prefix=/usr/local/python3  --with-ssl
make && make install