LibTorch實戰六：C++版本YOLOV5.4的部署<一>

目錄

• 一、環境配置
• 二、.torchscript.pt版本模型導出
• 三、C++版本yolov5.4實現
• 四、問題記錄

 

一、環境配置

• win10
• vs2017
• libtorch-win-shared-with-deps-debug-1.8.1+cpu
• opencv349

　　由於yolov5代碼，作者還在更新(寫這篇博客的時候，最新是5.4)，模型結構可能會有改變，所以咱們使用的libtorch必須滿足其要求，最好是一致。我這里提供本博客采用的yolov5版本python源碼。

百度雲網盤分享：

鏈接：https://pan.baidu.com/s/1VVns4hzJdDN0hFNtSnUZ2w 
提取碼：6c1p 
復制這段內容后打開百度網盤手機App，操作更方便哦

　　在源碼中的requirments.txt中要求依賴庫版本如下；在c++環境中，咱們這里用的libtorch1.8.1（今天我也測試了環境：libtorch-win-shared-with-deps-1.7.1+cu110，也能夠正常檢測，和本博客最終結果一致）；同時用opencv&c++作圖像處理，不需要c++版本torchvision：

# pip install -r requirements.txt

# base ----------------------------------------
matplotlib>=3.2.2
numpy>=1.18.5
opencv-python>=4.1.2
Pillow
PyYAML>=5.3.1
scipy>=1.4.1
10 torch>=1.7.0
11 torchvision>=0.8.1
# 以下內容神略

　　為了便於調試，我這里下載的是debug版本libtorch，而且是cpu版本，代碼調好后，轉GPU也很簡單吧。opencv版本其實隨意，opencv3++就行。

二、.torchscript.pt版本模型導出

　　打開yolov5.4源碼目錄下models文件夾，編輯export.py腳本，如下，將58行注釋，新增59行（GPU版本還需要修改一些內容，GPU版本后續更新，這篇博客只管CPU版本）

"""Exports a YOLOv5 *.pt model to ONNX and TorchScript formats

Usage:
    $ export PYTHONPATH="$PWD" && python models/export.py --weights ./weights/yolov5s.pt --img 640 --batch 1
"""

import argparse
import sys
import time

sys.path.append('./')  # to run '$ python *.py' files in subdirectories

import torch
import torch.nn as nn

import models
from models.experimental import attempt_load
from utils.activations import Hardswish, SiLU
from utils.general import set_logging, check_img_size
from utils.torch_utils import select_device

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--weights', type=str, default='./yolov5s.pt', help='weights path')  # from yolov5/models/
    parser.add_argument('--img-size', nargs='+', type=int, default=[640, 640], help='image size')  # height, width
    parser.add_argument('--batch-size', type=int, default=1, help='batch size')
    parser.add_argument('--dynamic', action='store_true', help='dynamic ONNX axes')
    parser.add_argument('--grid', action='store_true', help='export Detect() layer grid')
    parser.add_argument('--device', default='cpu', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')
    opt = parser.parse_args()
    opt.img_size *= 2 if len(opt.img_size) == 1 else 1  # expand
    print(opt)
    set_logging()
    t = time.time()

    # Load PyTorch model
    device = select_device(opt.device)
    model = attempt_load(opt.weights, map_location=device)  # load FP32 model
    labels = model.names

    # Checks
    gs = int(max(model.stride))  # grid size (max stride)
    opt.img_size = [check_img_size(x, gs) for x in opt.img_size]  # verify img_size are gs-multiples

    # Input
    img = torch.zeros(opt.batch_size, 3, *opt.img_size).to(device)  # image size(1,3,320,192) iDetection

    # Update model
    for k, m in model.named_modules():
        m._non_persistent_buffers_set = set()  # pytorch 1.6.0 compatibility
        if isinstance(m, models.common.Conv):  # assign export-friendly activations
            if isinstance(m.act, nn.Hardswish):
                m.act = Hardswish()
            elif isinstance(m.act, nn.SiLU):
                m.act = SiLU()
        # elif isinstance(m, models.yolo.Detect):
        #     m.forward = m.forward_export  # assign forward (optional)
    #model.model[-1].export = not opt.grid  # set Detect() layer grid export
    model.model[-1].export = False
    y = model(img)  # dry run

    # TorchScript export
    try:
        print('\nStarting TorchScript export with torch %s...' % torch.__version__)
        f = opt.weights.replace('.pt', '.torchscript.pt')  # filename
        ts = torch.jit.trace(model, img)
        ts.save(f)
        print('TorchScript export success, saved as %s' % f)
    except Exception as e:
        print('TorchScript export failure: %s' % e)
# 以下代碼省略，無需求改
......

　　接着在conda環境激活yolov5.4的虛擬環境，執行下面腳本：

(提示：如何配置yolov5.4環境？參考我這篇Win10環境下YOLO5 快速配置與測試：https://www.cnblogs.com/winslam/p/13474330.html)

python models/export.py --weights ./weights/yolov5s.pt --img 640 --batch 1

　　錯誤解決：1、bash窗口可能提示 not module utils；這是因為沒有將源碼根目錄添加進環境變量，linux下執行以下命令就行

export PYTHONPATH="$PWD"

　　win下，我建議直接用pycharm打開yolov5.4工程，在ide中去執行export.py就行，如果你沒有下載好yolovs.pt，他會自動下載，下載鏈接會打印在控制台，如下，如果下不動，可以嘗試復制鏈接到迅雷

Downloading https://github.com/ultralytics/yolov5/releases/download/v4.0/yolov5s.pt to yolov5s.pt...

執行export.py后出現如下警告：

D:\yolov5-0327\models\yolo.py:50: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if self.grid[i].shape[2:4] != x[i].shape[2:4]:
D:\Program Files\Anaconda\envs\yolov5\lib\site-packages\torch\jit\_trace.py:934: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
  module._c._create_method_from_trace(
TorchScript export success, saved as ./yolov5s.torchscript.pt
ONNX export failure: No module named 'onnx'
CoreML export failure: No module named 'coremltools'

Export complete (10.94s). Visualize with https://github.com/lutzroeder/netron.

警告內容以后分析，不影響部署

三、C++版本yolov5.4實現

libtorch在vs環境中配置(在項目屬性中設置下面加粗項目)：

include：

D:\libtorch-win-shared-with-deps-debug-1.8.1+cpu\libtorch\include

D:\libtorch-win-shared-with-deps-debug-1.8.1+cpu\libtorch\include\torch\csrc\api\include

lib：

D:\libtorch-win-shared-with-deps-debug-1.8.1+cpu\libtorch\lib

依賴庫（你可能用的更新的libtorch，所以具體lib目錄下所有.lib文件都要自己貼到連接器-附加依賴中）：

 

asmjit.lib
c10.lib
c10d.lib
c10_cuda.lib
caffe2_detectron_ops_gpu.lib
caffe2_module_test_dynamic.lib
caffe2_nvrtc.lib
clog.lib
cpuinfo.lib
dnnl.lib
fbgemm.lib
fbjni.lib
gloo.lib
gloo_cuda.lib
libprotobuf-lite.lib
libprotobuf.lib
libprotoc.lib
mkldnn.lib
pthreadpool.lib
pytorch_jni.lib
torch.lib
torch_cpu.lib
torch_cuda.lib
XNNPACK.lib

 

 

 

環境變量（需要重啟）：

D:\libtorch-win-shared-with-deps-debug-1.8.1+cpu\libtorch\lib

配置好之后，vs2017 設置為debug X64模式，下面是yolov5.4版本c++代碼

輸入是：

• 上述轉好的.torchscript.pt格式的模型文件
• coco.names
• 一張圖

#include <torch/script.h>
#include <torch/torch.h>
#include<opencv2/opencv.hpp>
#include <iostream>


std::vector<std::string> LoadNames(const std::string& path) 
{
    // load class names
    std::vector<std::string> class_names;
    std::ifstream infile(path);
    if (infile.is_open()) {
        std::string line;
        while (std::getline(infile, line)) {
            class_names.emplace_back(line);
        }
        infile.close();
    }
    else {
        std::cerr << "Error loading the class names!\n";
    }

    return class_names;
}

std::vector<float> LetterboxImage(const cv::Mat& src, cv::Mat& dst, const cv::Size& out_size)
{
    auto in_h = static_cast<float>(src.rows);
    auto in_w = static_cast<float>(src.cols);
    float out_h = out_size.height;
    float out_w = out_size.width;

    float scale = std::min(out_w / in_w, out_h / in_h);

    int mid_h = static_cast<int>(in_h * scale);
    int mid_w = static_cast<int>(in_w * scale);

    cv::resize(src, dst, cv::Size(mid_w, mid_h));

    int top = (static_cast<int>(out_h) - mid_h) / 2;
    int down = (static_cast<int>(out_h) - mid_h + 1) / 2;
    int left = (static_cast<int>(out_w) - mid_w) / 2;
    int right = (static_cast<int>(out_w) - mid_w + 1) / 2;

    cv::copyMakeBorder(dst, dst, top, down, left, right, cv::BORDER_CONSTANT, cv::Scalar(114, 114, 114));

    std::vector<float> pad_info{ static_cast<float>(left), static_cast<float>(top), scale };
    return pad_info;
}

enum Det 
{
    tl_x = 0,
    tl_y = 1,
    br_x = 2,
    br_y = 3,
    score = 4,
    class_idx = 5
};

struct Detection 
{
    cv::Rect bbox;
    float score;
    int class_idx;
};

void Tensor2Detection(const at::TensorAccessor<float, 2>& offset_boxes,
    const at::TensorAccessor<float, 2>& det,
    std::vector<cv::Rect>& offset_box_vec,
    std::vector<float>& score_vec)
{

    for (int i = 0; i < offset_boxes.size(0); i++) {
        offset_box_vec.emplace_back(
            cv::Rect(cv::Point(offset_boxes[i][Det::tl_x], offset_boxes[i][Det::tl_y]),
                cv::Point(offset_boxes[i][Det::br_x], offset_boxes[i][Det::br_y]))
        );
        score_vec.emplace_back(det[i][Det::score]);
    }
}

void ScaleCoordinates(std::vector<Detection>& data, float pad_w, float pad_h,
    float scale, const cv::Size& img_shape)
{
    auto clip = [](float n, float lower, float upper)
    {
        return std::max(lower, std::min(n, upper));
    };

    std::vector<Detection> detections;
    for (auto & i : data) {
        float x1 = (i.bbox.tl().x - pad_w) / scale;  // x padding
        float y1 = (i.bbox.tl().y - pad_h) / scale;  // y padding
        float x2 = (i.bbox.br().x - pad_w) / scale;  // x padding
        float y2 = (i.bbox.br().y - pad_h) / scale;  // y padding

        x1 = clip(x1, 0, img_shape.width);
        y1 = clip(y1, 0, img_shape.height);
        x2 = clip(x2, 0, img_shape.width);
        y2 = clip(y2, 0, img_shape.height);

        i.bbox = cv::Rect(cv::Point(x1, y1), cv::Point(x2, y2));
    }
}


torch::Tensor xywh2xyxy(const torch::Tensor& x)
{
    auto y = torch::zeros_like(x);
    // convert bounding box format from (center x, center y, width, height) to (x1, y1, x2, y2)
    y.select(1, Det::tl_x) = x.select(1, 0) - x.select(1, 2).div(2);
    y.select(1, Det::tl_y) = x.select(1, 1) - x.select(1, 3).div(2);
    y.select(1, Det::br_x) = x.select(1, 0) + x.select(1, 2).div(2);
    y.select(1, Det::br_y) = x.select(1, 1) + x.select(1, 3).div(2);
    return y;
}

std::vector<std::vector<Detection>> PostProcessing(const torch::Tensor& detections,
    float pad_w, float pad_h, float scale, const cv::Size& img_shape,
    float conf_thres, float iou_thres)
{
    /***
     * 結果緯度為batch index(0), top-left x/y (1,2), bottom-right x/y (3,4), score(5), class id(6)
     * 13*13*3*(1+4)*80
     */
    constexpr int item_attr_size = 5;
    int batch_size = detections.size(0);
    // number of classes, e.g. 80 for coco dataset
    auto num_classes = detections.size(2) - item_attr_size;

    // get candidates which object confidence > threshold
    auto conf_mask = detections.select(2, 4).ge(conf_thres).unsqueeze(2);

    std::vector<std::vector<Detection>> output;
    output.reserve(batch_size);

    // iterating all images in the batch
    for (int batch_i = 0; batch_i < batch_size; batch_i++) {
        // apply constrains to get filtered detections for current image
        auto det = torch::masked_select(detections[batch_i], conf_mask[batch_i]).view({ -1, num_classes + item_attr_size });

        // if none detections remain then skip and start to process next image
        if (0 == det.size(0)) {
            continue;
        }

        // compute overall score = obj_conf * cls_conf, similar to x[:, 5:] *= x[:, 4:5]
        det.slice(1, item_attr_size, item_attr_size + num_classes) *= det.select(1, 4).unsqueeze(1);

        // box (center x, center y, width, height) to (x1, y1, x2, y2)
        torch::Tensor box = xywh2xyxy(det.slice(1, 0, 4));

        // [best class only] get the max classes score at each result (e.g. elements 5-84)
        std::tuple<torch::Tensor, torch::Tensor> max_classes = torch::max(det.slice(1, item_attr_size, item_attr_size + num_classes), 1);

        // class score
        auto max_conf_score = std::get<0>(max_classes);
        // index
        auto max_conf_index = std::get<1>(max_classes);

        max_conf_score = max_conf_score.to(torch::kFloat).unsqueeze(1);
        max_conf_index = max_conf_index.to(torch::kFloat).unsqueeze(1);

        // shape: n * 6, top-left x/y (0,1), bottom-right x/y (2,3), score(4), class index(5)
        det = torch::cat({ box.slice(1, 0, 4), max_conf_score, max_conf_index }, 1);

        // for batched NMS
        constexpr int max_wh = 4096;
        auto c = det.slice(1, item_attr_size, item_attr_size + 1) * max_wh;
        auto offset_box = det.slice(1, 0, 4) + c;

        std::vector<cv::Rect> offset_box_vec;
        std::vector<float> score_vec;

        // copy data back to cpu
        auto offset_boxes_cpu = offset_box.cpu();
        auto det_cpu = det.cpu();
        const auto& det_cpu_array = det_cpu.accessor<float, 2>();

        // use accessor to access tensor elements efficiently
        Tensor2Detection(offset_boxes_cpu.accessor<float, 2>(), det_cpu_array, offset_box_vec, score_vec);

        // run NMS
        std::vector<int> nms_indices;
        cv::dnn::NMSBoxes(offset_box_vec, score_vec, conf_thres, iou_thres, nms_indices);

        std::vector<Detection> det_vec;
        for (int index : nms_indices) {
            Detection t;
            const auto& b = det_cpu_array[index];
            t.bbox =
                cv::Rect(cv::Point(b[Det::tl_x], b[Det::tl_y]),
                    cv::Point(b[Det::br_x], b[Det::br_y]));
            t.score = det_cpu_array[index][Det::score];
            t.class_idx = det_cpu_array[index][Det::class_idx];
            det_vec.emplace_back(t);
        }

        ScaleCoordinates(det_vec, pad_w, pad_h, scale, img_shape);

        // save final detection for the current image
        output.emplace_back(det_vec);
    } // end of batch iterating

    return output;
}

void Demo(cv::Mat& img,
    const std::vector<std::vector<Detection>>& detections,
    const std::vector<std::string>& class_names,
    bool label = true)
{
    if (!detections.empty()) {
        for (const auto& detection : detections[0]) {
            const auto& box = detection.bbox;
            float score = detection.score;
            int class_idx = detection.class_idx;

            cv::rectangle(img, box, cv::Scalar(0, 0, 255), 2);

            if (label) {
                std::stringstream ss;
                ss << std::fixed << std::setprecision(2) << score;
                std::string s = class_names[class_idx] + " " + ss.str();

                auto font_face = cv::FONT_HERSHEY_DUPLEX;
                auto font_scale = 1.0;
                int thickness = 1;
                int baseline = 0;
                auto s_size = cv::getTextSize(s, font_face, font_scale, thickness, &baseline);
                cv::rectangle(img,
                    cv::Point(box.tl().x, box.tl().y - s_size.height - 5),
                    cv::Point(box.tl().x + s_size.width, box.tl().y),
                    cv::Scalar(0, 0, 255), -1);
                cv::putText(img, s, cv::Point(box.tl().x, box.tl().y - 5),
                    font_face, font_scale, cv::Scalar(255, 255, 255), thickness);
            }
        }
    }

    cv::namedWindow("Result", cv::WINDOW_NORMAL);
    cv::imshow("Result", img);

}

int main()
{
    // yolov5Ns.torchscript.pt 報錯，所以僅能讀取yolov5.4模型
    torch::jit::script::Module module = torch::jit::load("yolov5sxxx.torchscript.pt");
    torch::DeviceType device_type = torch::kCPU;
    module.to(device_type);
    /*module.to(torch::kHalf);*/
    module.eval();

    // img 必須讀取3-channels圖片
    cv::Mat img = cv::imread("zidane.jpg", -1);
    // 讀取類別
    std::vector<std::string> class_names = LoadNames("coco.names");
    if (class_names.empty()) {
        return -1;
    }

    // set up threshold
    float conf_thres = 0.4;
    float iou_thres = 0.5;

    //inference
    torch::NoGradGuard no_grad;
    cv::Mat img_input = img.clone();
    std::vector<float> pad_info = LetterboxImage(img_input, img_input, cv::Size(640, 640));
    const float pad_w = pad_info[0];
    const float pad_h = pad_info[1];
    const float scale = pad_info[2];
    cv::cvtColor(img_input, img_input, cv::COLOR_BGR2RGB);  // BGR -> RGB
    //歸一化需要是浮點類型
    img_input.convertTo(img_input, CV_32FC3, 1.0f / 255.0f);  // normalization 1/255
    // 加載圖像到設備
    auto tensor_img = torch::from_blob(img_input.data, { 1, img_input.rows, img_input.cols, img_input.channels() }).to(device_type);
    // BHWC -> BCHW
    tensor_img = tensor_img.permute({ 0, 3, 1, 2 }).contiguous();  // BHWC -> BCHW (Batch, Channel, Height, Width)
    
    std::vector<torch::jit::IValue> inputs;
    // 在容器尾部添加一個元素，這個元素原地構造，不需要觸發拷貝構造和轉移構造
    inputs.emplace_back(tensor_img);
    
    torch::jit::IValue output = module.forward(inputs);
    
    // 解析結果
    auto detections = output.toTuple()->elements()[0].toTensor();
    auto result = PostProcessing(detections, pad_w, pad_h, scale, img.size(), conf_thres, iou_thres);
    // visualize detections
    if (true) {
        Demo(img, result, class_names);
        cv::waitKey(0);
    }
    return 1;
}

四、問題記錄

我參考的是鏈接[1][2]代碼，非常坑，[1][2]代碼是一樣的，也不知道誰抄誰的，代碼中沒有說明yolov5具體版本，而且有很多問題，不過還是感謝給了參考。

原版代碼：

鏈接：https://pan.baidu.com/s/1KFJZV3KxAoXUcN2UKiT2gg 
提取碼：r5c9 
復制這段內容后打開百度網盤手機App，操作更方便哦

整理后的代碼：

鏈接：https://pan.baidu.com/s/1SvN6cEniUwKJ8_MH-EwAPw 
提取碼：br7i 
復制這段內容后打開百度網盤手機App，操作更方便哦

在原版代碼整理之后，再將其改為第三節中的cpp，，第三節中的cpp相對原版libtorch實現，我做了如下修改(改了一些錯誤)，參考了資料[3]：

1、注釋 detector.h中，注釋如下頭文件
//#include <c10/cuda/CUDAStream.h>
#//include <ATen/cuda/CUDAEvent.h>

2、錯誤： “std”: 不明確的符號

解決辦法1：項目->屬性->c/c++->語言->符合模式->選擇否

(看清楚vs項目屬性窗口對應的到底是Debug還是Release，血的教訓！）

解決辦法2：還有有個老哥給出的方法是，在std報錯的地方改為："::std"，不推薦！

3、建議常被debug版本libtorch

libtorch中，執行到加載模型那一行代碼，跳進libtorch庫中的Assert，提示錯誤：AT_ASSERT(isTuple(), "Expected Tuple but got ", tagKind());（咱們是libtorch debug版本，還能跳到這一行，要是release，你都不知道錯在哪里，所以常備debug版本，很有必要）

可能是你轉模型的yolov5版本不是5.4，而是5.3、5.3.1、5.3、5.1；還有可能是你export.py腳本中沒有按照上面設置。

參考：https://blog.csdn.net/weixin_42398658/article/details/111954760

4、問題：編譯成功后，運行代碼，發現torch::cuda::is_available()返回false

解決：a、配置環境的時候，請將庫lib文件夾下所有“.lib”文件名粘貼到項目屬性(Release)-鏈接器 - 輸入 - 附加依賴項

b、在項目屬性(Release)-鏈接器 - 命令行 - 其他選項貼入下面命令

/INCLUDE:?warp_size@cuda@at@@YAHXZ

完美解決！

5、導出模型，命令行有警告

上面導出模型控制台打印的警告信息還沒解決，但是部署后，檢測效果和python版本有差別(其實幾乎差不多)，如下：

如下：左邊是官方結果，右邊是libtorch模型部署結構，置信度不相上下，開心！

 

可以看到右邊那個人領帶沒有檢測出來，這是因為咱們用的是5s模型，在yolov5最新版本中，作者對模型的修改更加注重5x模型的精度，5s性能確實略微下降。

  

reference：

[1]　libtorch代碼參考；https://zhuanlan.zhihu.com/p/338167520

[2]　libtorch代碼參考；https://gitee.com/goodtn/libtorch-yolov5-gpu/tree/master

[3]　libtorch相關報錯總結(非常nice！)：https://blog.csdn.net/qq_18305555/article/details/114013236