win10 + VS2017 + MNN 編譯使用

1. VS2017 安裝

1. 從這里下載在線安裝器，隨后在 cmd 窗口中離線下載安裝包：

vs_Professional.exe --layout D:\vs2017-offline --lang zh-CN 

下載完后，雙擊 vs_setup.exe 安裝，選擇

 2. 安裝插件（可選）

2. 配置高版本 cmake 到環境變量

 去這里下載好了 ，然后將可執行程序所在文件夾添加到環境變量

D:\cmake-3.20.2-windows-x86_64\bin

3. MNN 推理框架編譯

去官網下載 MNN 源碼到本地

3.1 編譯 flatbuffers

右鍵管理員運行 “VS 2017 的開發人員命令提示符”

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN\3rd_party\flatbuffers
mkdir tmp && cd tmp
cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release ..
ninja

3.2 編譯 推理框架

右鍵管理員運行 “VS 2017 的開發人員命令提示符”, 這里需要借助 powershell 

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
powershell set-executionpolicy -executionpolicy unrestricted
powershell ./schema/generate.ps1
powershell .\package_scripts\win\build_lib.ps1 -path MNN-CPU/lib/x64

完成后在 D:\MNN\MNN-CPU 這地方會有各種版本的推理庫

4. MNN Demo& Convert 編譯

4.1 編譯 protobuf

參考官方教程

這里我選擇使用 VS 編譯 Debug 和 Release 版本

右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd cmake
mkdir build & cd build
mkdir solution & cd solution
cmake -G "Visual Studio 15 2017 Win64" -Dprotobuf_MSVC_STATIC_RUNTIME=ON -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=../../../../install ../..

然后用 VS2017 分別編譯出 Debug 和 Release 版本，雙擊 extract_includes.bat 復制頭文件

最后將 include 和 Release(Debug）生成文件夾加入環境變量

4.2 單獨編譯 demo

右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
mkdir build_demo
cd build_demo
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DMNN_BUILD_DEMO=ON ..
nmake

 

4.3 單獨編譯 convert

 右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
powershell set-executionpolicy -executionpolicy unrestricted
powershell .\package_scripts\win\build_tools.ps1 -path MNN-CPU/lib/x64

4.4 換個姿勢

這里其實可以修改 build_tools.ps1

mkdir build_convert  # mkdir build  
pushd build_convert  # pushd build

並添加

DMNN_BUILD_DEMO=ON

借助 powershell 

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
powershell set-executionpolicy -executionpolicy unrestricted
powershell .\package_scripts\win\build_tools.ps1 -path MNN-CPU/lib/x64

5. MNN Debug 編譯

5.1 Debug

 右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
mkdir build_debug& cd build_debug
cmake -G "Visual Studio 15 2017 Win64" -DMNN_BUILD_SHARED_LIBS=ON -DMNN_BUILD_CONVERTER=ON -DMNN_BUILD_DEMO=ON -DCMAKE_BUILD_TYPE=Debug ..

隨后打開創建的 VS 工程編譯即可

注意：這里的運行庫方式要和 protobuf Debug 生成時保持一致

 

5.2 OpenCL

同上面一樣，換個姿勢再來一次

右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
mkdir build_opencl & cd build_opencl
cmake -G "Visual Studio 15 2017 Win64" -DMNN_OPENCL=ON DMNN_BUILD_DEMO=ON -DCMAKE_BUILD_TYPE=Debug ..

隨后打開創建的 VS 工程編譯即可

5.2 Vulkan

。。。, 換個姿勢再來一次

右鍵管理員運行 “VS 2017 的開發人員命令提示符”:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvars64.bat"
cd D:\MNN
mkdir build_vulkan & cd build_vulkan cmake -G "Visual Studio 15 2017 Win64" -DMNN_VULKAN=ON -DMNN_BUILD_DEMO=ON -DCMAKE_BUILD_TYPE=Debug ..

隨后打開創建的 VS 工程編譯即可，注意 VulkanRuntime.cpp 需要改下編碼格式為 ANSI

5.4 測試代碼

選擇 caffe mobilenet 的分類網絡

5.2.1 MNN 轉換模型 MNNConvert

-f CAFFE --modelFile D:/MNN/demo/model/MobileNet/Caffe_MobileNet_V2/mobilenet_v2.caffemodel --prototxt D:/MNN/demo/model/MobileNet/Caffe_MobileNet_V2/mobilenet_v2_deploy.prototxt --MNNModel D:/MNN/demo/model/MobileNet/mobilenet_v2.mnn --bizCode biz

5.2.2 MNN 推理 pictureRecognition.out

D:/MNN/demo/model/MobileNet/mobilenet_v2.mnn D:/MNN/demo/model/MobileNet/testcat.jpg D:/MNN/demo/model/MobileNet/synset_words.txt

Can't Find type=4 backend, use 0 instead
Can't Find type=4 backend, use 0 instead
input: w:224 , h:224, bpp: 3
origin size: 480, 360 output size:1000 Egyptian cat: 0.196567 tiger cat: 0.125614 tabby, tabby cat: 0.097382 red fox, Vulpes vulpes: 0.092684 kit fox, Vulpes macrotis: 0.077640 hare: 0.058969 wood rabbit, cottontail, cottontail rabbit: 0.033735 grey fox, gray fox, Urocyon cinereoargenteus: 0.026056 Norwich terrier: 0.022280 Pomeranian: 0.016270

5.2.3 原始 python Caffe

import argparse
import numpy as np
import caffe


def eval(opt):
    nh, nw = opt.img_size, opt.img_size
    img_mean = np.array(opt.img_mean, dtype=np.float32)

    # caffe.set_mode_cpu()
    caffe.set_device(0)
    caffe.set_mode_gpu()
    net = caffe.Net(opt.proto, opt.model, caffe.TEST)

    im = caffe.io.load_image(opt.image)
    h, w, _ = im.shape
    if h < w:
        off = int((w - h) / 2)
        im = im[:, off:off + h]
    else:
        off = (h - w) / 2
        im = im[off:off + h, :]
    im = caffe.io.resize_image(im, [nh, nw])

    transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
    transformer.set_transpose('data', (2, 0, 1))  # row to col
    transformer.set_channel_swap('data', (2, 1, 0))  # RGB to BGR
    transformer.set_raw_scale('data', 255)  # [0,1] to [0,255]
    transformer.set_mean('data', img_mean)
    transformer.set_input_scale('data', opt.img_scale)

    net.blobs['data'].reshape(1, 3, nh, nw)
    net.blobs['data'].data[...] = transformer.preprocess('data', im)
    # np.save('data', net.blobs['data'].data[0])
    out = net.forward()
    prob = out['prob']
    prob = np.squeeze(prob)
    idx = np.argsort(-prob)

    label_names = np.loadtxt('synset.txt', str, delimiter='\t')
    for i in range(5):
        label = idx[i]
        print('%.2f - %s' % (prob[label], label_names[label]))
    return


if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='evaluate pretrained mobilenet models')
    parser.add_argument('--proto', dest='proto', default='MobileNet V2/mobilenet_v2_deploy.prototxt',
                        help="path to deploy prototxt.", type=str)
    parser.add_argument('--model', dest='model', default='MobileNet V2/mobilenet_v2.caffemodel',
                        help='path to pretrained weights', type=str)
    parser.add_argument('--image', dest='image', default='testcat.jpg',
                        help='path to color image', type=str)
    parser.add_argument('--img-size', type=int, default=224, help='inference size (pixels)')
    parser.add_argument('--img-mean', type=list, default=[103.94,116.78,123.68], help='image mean [103.94,116.78,123.68]')
    parser.add_argument('--img-scale', type=float, default=0.017, help='scale image 0.017/1')
    opt = parser.parse_args()
    eval(opt)

0.25 - 'tiger cat'
0.17 - ‘Egyptian cat'
0.13 - 'tabby, tabby cat'
0.06 - 'red fox, Vulpes vulpes'
0.03 - 'hare'

5.2.4 調用 python MNN

pip install MNN
pip install MNNCV

import numpy as np
import MNN
import cv2


def inference():
    """ inference mobilenet_v1 using a specific picture """
    interpreter = MNN.Interpreter("D:/MNN/demo/model/MobileNet/mobilenet_v2.mnn")
    interpreter.setCacheFile('.tempcache')
    config = {}
    config['precision'] = 'low'
    session = interpreter.createSession()
    input_tensor = interpreter.getSessionInput(session)
    image = cv2.imread('D:/MNN/demo/model/MobileNet/testcat.jpg')
    # cv2 read as bgr format
    # image = image[..., ::-1] # convert to RGB
    # change to rgb format
    image = cv2.resize(image, (224, 224))
    # resize to mobile_net tensor size
    image = image - (103.94, 116.78, 123.68)
    image = image * (0.017, 0.017, 0.017)
    # preprocess it
    image = image.transpose((2, 0, 1))
    # change numpy data type as np.float32 to match tensor's format
    image = image.astype(np.float32)
    # image = np.load('data.npy')
    # cv2 read shape is NHWC, Tensor's need is NCHW,transpose it
    tmp_input = MNN.Tensor((1, 3, 224, 224), MNN.Halide_Type_Float,
                           image, MNN.Tensor_DimensionType_Caffe)
    input_tensor.copyFrom(tmp_input)
    interpreter.runSession(session)
    output_tensor = interpreter.getSessionOutput(session)
    # constuct a tmp tensor and copy/convert in case output_tensor is nc4hw4
    tmp_output = MNN.Tensor((1, 1000), MNN.Halide_Type_Float, np.ones([1, 1000]).astype(np.float32),
                            MNN.Tensor_DimensionType_Caffe)
    output_tensor.copyToHostTensor(tmp_output)
    print("expect 983")
    print("output belong to class: {}".format(np.argmax(tmp_output.getData())))
    prob = np.array(tmp_output.getData())
    idx = np.argsort(-prob)
    label_names = np.loadtxt('synset.txt', str, delimiter='\t')
    for i in range(5):
        label = idx[i]
        print('%.2f - %s' % (prob[label], label_names[label]))


if __name__ == "__main__":
    inference()

0.21 - 'Egyptian cat'
0.12 - 'tiger cat'
0.10 - 'tabby, tabby cat'
0.08 - 'red fox, Vulpes vulpes'
0.07 - 'kit fox, Vulpes macrotis'

三者的差別在於輸入圖片讀取方式（python mnn 的輸入替換成 python caffe 的輸入層，二者推理結果一致）

PS: 注意，在源代碼上添加中文注釋的話，記得要手動將源代碼的編碼改成 ANSI，否則會出現出人意料的 bug！

6. Others

6.1 模型可視化

官方推薦 netron 來可視化網絡結構

6.2 MNN 工作台

 

6.3 MNN 代碼

1. 這里控制推理方式

ScheduleConfig config;
config.type = MNN_FORWARD_AUTO;

2. MNN 中數據按照 HxWx(C+1）存儲

auto values = input->host<float>();