• 机器视觉系列5：C++部署pytorch模型onnxruntime

     系列文章目录

    第一章：Visual Studio 2019 动态链接库DLL建立

    第二章：VS动态链接库DLL调试

    第三章：VS2019 OpenCV环境配置 

    第四章：C++部署pytorch模型Libtorch

    第五章：C++部署pytorch模型onnxruntime

    目录

     系列文章目录

    前言

    一、C++部署pytorch？

    二、onnxruntime配置

    1.下载onnxruntime

    2.VS2019配置onnxruntime

    2.1配置VC++目录

     2.2配置链接器

     2.3 onnxruntime环境变量配置

    三、pytorch模型转换为onnx

    四、C++中onnxruntime的使用

    五、python中onnxruntime的使用

    参考文献

    前言

    环境：visual studio 2019；OpenCV4.5.5；pytorch1.8；onnxruntime1.8.1；

    一、C++部署pytorch？

    pytorch模型在C++部署，上一章是使用pytorch对应版本的Libtorch部署。其实转onnx部署可能更方便，之前语义分割精度相差太大是因为数据预处理的问题，一般图像在输入网络之前需要标准化和转RGB等相关操作。onnx部署的好处是微软中间格式，兼容各种平台，比较方便。如果部署平台有GPU可能用tensorRT会更好。

    二、onnxruntime配置

    注意事项：注意pt模型转onnx时的版本，onnxruntime版本对应

    1.下载onnxruntime

    官网：NuGet Gallery | Home

    onnxruntime-win-x64-1.8.1链接如下：

    https://objects.githubusercontent.com/github-production-release-asset-2e65be/156939672/7bc42680-deed-11eb-89d9-81963670045b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220809%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220809T093837Z&X-Amz-Expires=300&X-Amz-Signature=4c9cc7ec4ba93d83ccb3307533b47e5c40303c43c003b2a791d4d321570dc5b8&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=156939672&response-content-disposition=attachment%3B%20filename%3Donnxruntime-win-x64-1.8.1.zip&response-content-type=application%2Foctet-streamhttps://objects.githubusercontent.com/github-production-release-asset-2e65be/156939672/7bc42680-deed-11eb-89d9-81963670045b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220809%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220809T093837Z&X-Amz-Expires=300&X-Amz-Signature=4c9cc7ec4ba93d83ccb3307533b47e5c40303c43c003b2a791d4d321570dc5b8&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=156939672&response-content-disposition=attachment%3B%20filename%3Donnxruntime-win-x64-1.8.1.zip&response-content-type=application%2Foctet-stream

    2.VS2019配置onnxruntime

    2.1配置VC++目录

    首先配置包含目录和库目录，对应opencv一样的方法。

     2.2配置链接器

    依赖项添加所有lib，cmd中进入lib目录，使用dir /b *.lib>1.txt命令可生成目录，复制使用。

     2.3 onnxruntime环境变量配置

    把所有DLL复制进Release或者Debug目录。

    三、pytorch模型转换为onnx

    注意事项：模型输入BCHW，推理模式model.eval()，输出版本opset_version=11

    1. import torch
    2.  
    3. x = torch.randn(1, 3, 512, 512, device="cpu")
    4. model = torch.load('best_model.pth', map_location=torch.device('cpu'))
    5. model.eval()
    6. input_names = ["input"]
    7. output_names = ["output"]
    8.  
    9. torch.onnx.export(model, x, "GlandUnet.onnx", verbose=True, input_names=input_names, output_names=output_names, opset_version=11)

    四、C++中onnxruntime的使用

    注意事项：1，一定要注意图像输入模型之前的预处理，是否标准化，是否转RGB

                      2，C++中onnxruntime推理输出为数组的首地址，可以用指针取出生成Mat

    ------------------------------------------------20230802更新argmax-----------------------------------------------

    1. /****************************************
    2. @brief     :  分割onnxruntime
    3. @input	   :  图像
    4. @output    :  掩膜
    5. *****************************************/
    6. void SegmentAIONNX(Mat& imgSrc, int width, int height)
    7. {
    8. 	模型信息/
    9. 	Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "OnnxModel");
    10. 	Ort::SessionOptions session_options;
    11. 	session_options.SetIntraOpNumThreads(1);
    12. #ifdef _WIN32
    13. 	const wchar_t* model_path = L"GlandUnet.onnx";
    14. #else
    15. 	const char* model_path = "RedUnet.onnx";
    16. #endif
    17. 	Ort::Session session(env, model_path, session_options);
    18. 	Ort::AllocatorWithDefaultOptions allocator;
    19. 	size_t num_input_nodes = session.GetInputCount();	//batchsize
    20. 	size_t num_output_nodes = session.GetOutputCount();
    21. 	const char* input_name = session.GetInputName(0, allocator);
    22. 	const char* output_name = session.GetOutputName(0, allocator);
    23. 	auto input_dims = session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();	//输入输出维度
    24. 	auto output_dims = session.GetOutputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();
    25. 	std::vector<const char*> input_names{ input_name };
    26. 	std::vector<const char*> output_names = { output_name };
    27. 	输入处理//
    28. 	Mat imgBGR = imgSrc;	//输入图片预处理
    29. 	Mat imgBGRresize;
    30. 	resize(imgBGR, imgBGRresize, Size(input_dims[3], input_dims[2]), InterpolationFlags::INTER_CUBIC);
    31. 	Mat imgRGBresize = imgBGRresize;
    32. 	//cvtColor(imgBGRresize, imgRGBresize, COLOR_BGR2RGB);	//smp未转RGB
    33. 	Mat resize_img;
    34. 	imgRGBresize.convertTo(resize_img, CV_32F, 1.0 / 255);  //divided by 255转float
    35. 	cv::Mat channels[3]; //分离通道进行HWC->CHW
    36. 	cv::split(resize_img, channels);
    37. 	std::vector<float> inputTensorValues;
    38. 	float mean[] = { 0.485f, 0.456f, 0.406f };	//
    39. 	float std_val[] = { 0.229f, 0.224f, 0.225f };
    40. 	for (int i = 0; i < resize_img.channels(); i++)	//标准化ImageNet
    41. 	{
    42. 		channels[i] -= mean[i];  // mean均值
    43. 		channels[i] /= std_val[i];   // std方差
    44. 	}
    45. 	for (int i = 0; i < resize_img.channels(); i++)  //HWC->CHW
    46. 	{
    47. 		std::vector<float> data = std::vector<float>(channels[i].reshape(1, resize_img.cols * resize_img.rows));
    48. 		inputTensorValues.insert(inputTensorValues.end(), data.begin(), data.end());
    49. 	}
    50. 	Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu(OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
    51. 	vector inputTensors;
    52. 	inputTensors.push_back(Ort::Value::CreateTensor<float>(memoryInfo, inputTensorValues.data(), inputTensorValues.size(), input_dims.data(), input_dims.size()));
    53. 	//clock_t startTime, endTime;	//计算推理时间
    54. 	//startTime = clock();
    55. 	auto outputTensor = session.Run(Ort::RunOptions{ nullptr }, input_names.data(), inputTensors.data(), 1, output_names.data(), 1);   // 开始推理
    56. 	//endTime = clock();
    57. 	打印模型信息/
    58. 	//printf("Using Onnxruntime C++ API\n");
    59. 	//printf("Number of inputs = %zu\n", num_input_nodes);
    60. 	//printf("Number of output = %zu\n", num_output_nodes);
    61. 	//std::cout << "input_name:" << input_name << std::endl;
    62. 	//std::cout << "output_name: " << output_name << std::endl;
    63. 	//std::cout << "input_dims:" << input_dims[0] << input_dims[1] << input_dims[2] << input_dims[3] << std::endl;
    64. 	//std::cout << "output_dims:" << output_dims[0] << output_dims[1] << output_dims[2] << output_dims[3] << std::endl;
    65. 	//std::cout << "The run time is:" << (double)(endTime - startTime) / CLOCKS_PER_SEC << "s" << std::endl;
    66. 	
    67. 	//输出处理//
    68. 	float* mask_ptr = outputTensor[0].GetTensorMutableData<float>(); //outtensor首地址
    69. 	vector< unsigned char >results(512 * 512);
    70. 	for (int i = 0; i < 512 * 512; i++)
    71. 	{
    72. 		if (mask_ptr[i] >= 0.5)
    73. 		{
    74. 			results[i] = 0;
    75. 		}
    76. 		else
    77. 		{
    78. 			results[i] = 255;
    79. 		}
    80. 	}
    81. 	unsigned char* ptr = &results[0];
    82. 	Mat mask = Mat(output_dims[2], output_dims[3], CV_8U, ptr);
    83. 	resize(mask, imgSrc, Size(imgBGR.cols, imgBGR.rows));
    84. 	//原图展示分割结果//
    85. 	//cvtColor(imgSrc, imgSrc, COLOR_GRAY2BGR);
    86. 	//Mat imgAdd;
    87. 	//addWeighted(imgBGR, 1, imgSrc, 0.3, 0, imgAdd);
    88. }

                    3，由于C++中没有argmax函数，可以在模型结构中集成argmax，推理后处理改一下

    1. #pragma region AI分割
    2. Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "OnnxModel");
    3. Ort::SessionOptions session_options;
    4. session_options.SetIntraOpNumThreads(1);
    5. const wchar_t* model_path = L"Segformer.onnx";    //载入模型文件
    6. Ort::Session session(env, model_path, session_options);
    7. Ort::AllocatorWithDefaultOptions allocator;
    8. size_t num_input_nodes = session.GetInputCount();	//batchsize
    9. size_t num_output_nodes = session.GetOutputCount();
    10. const char* input_name = session.GetInputName(0, allocator);
    11. const char* output_name = session.GetOutputName(0, allocator);
    12. auto input_dims = session.GetInputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();	//输入输出维度
    13. auto output_dims = session.GetOutputTypeInfo(0).GetTensorTypeAndShapeInfo().GetShape();
    14. std::vector<const char*> input_names{ input_name };
    15. std::vector<const char*> output_names = { output_name };
    16. ///输入处理///
    17. Mat imgBGR = vailSrcRef;	//输入图片预处理
    18. Mat imgBGRresize;
    19. resize(imgBGR, imgBGRresize, Size(input_dims[3], input_dims[2]), InterpolationFlags::INTER_CUBIC);
    20. Mat imgRGBresize;
    21. cvtColor(imgBGRresize, imgRGBresize, COLOR_BGR2RGB);
    22. Mat resize_img;
    23. imgRGBresize.convertTo(resize_img, CV_32F, 1.0 / 255);  //divided by 255;
    24. cv::Mat channels[3]; //借用来进行HWC->CHW
    25. cv::split(resize_img, channels);
    26. std::vector<float> inputTensorValues;
    27. float mean[] = { 0.485f, 0.456f, 0.406f };
    28. float std_val[] = { 0.229f, 0.224f, 0.225f };
    29. for (int i = 0; i < resize_img.channels(); i++)	//标准化
    30. {
    31. 	channels[i] -= mean[i];  // mean
    32. 	channels[i] /= std_val[i];   // std
    33. }
    34. for (int i = 0; i < resize_img.channels(); i++)  //HWC->CHW
    35. {
    36. 	std::vector<float> data = std::vector<float>(channels[i].reshape(1, resize_img.cols * resize_img.rows));
    37. 	inputTensorValues.insert(inputTensorValues.end(), data.begin(), data.end());
    38. }
    39. Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu(OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
    40. vector inputTensors;
    41. inputTensors.push_back(Ort::Value::CreateTensor<float>(memoryInfo, inputTensorValues.data(), inputTensorValues.size(), input_dims.data(), input_dims.size()));
    42. auto outputTensor = session.Run(Ort::RunOptions{ nullptr }, input_names.data(), inputTensors.data(), 1, output_names.data(), 1);   // 开始推理
    43. 输出处理/
    44. long long* mask_ptr = outputTensor[0].GetTensorMutableData<long long>(); //outtensor首地址
    45. vector< unsigned char >results(262144);
    46. for (int i = 0; i < 512 * 512; i++)
    47. {
    48. 	results[i] = unsigned char(mask_ptr[i]) * 255;
    49. }
    50. unsigned char* ptr = &results[0];
    51. Mat mask = Mat(output_dims[2], output_dims[3], CV_8U, ptr);
    52. #pragma endregion

    五、python中onnxruntime的使用

    1. # -*- coding:utf-8 -*-
    2. import cv2
    3. import numpy as np
    4. import onnxruntime as ort
    5. import imgviz
    6. import time
    7.  
    8. class_names = ['_background_', 'conjunctiva_area']
    9. ### 定义一些数据前后处理的工具
    10. def preprocess(input_data):
    11.     # convert the input data into the float32 input
    12.     img_data = input_data.astype('float32')
    13.     # normalize
    14.     mean_vec = np.array([0.485, 0.456, 0.406])
    15.     stddev_vec = np.array([0.229, 0.224, 0.225])
    16.     norm_img_data = np.zeros(img_data.shape).astype('float32')
    17.     for i in range(img_data.shape[0]):
    18.         norm_img_data[i, :, :] = (img_data[i, :, :] / 255 - mean_vec[i]) / stddev_vec[i]
    19.     # add batch channel
    20.     norm_img_data = norm_img_data.reshape(1, 3, 512, 512).astype('float32')
    21.     return norm_img_data
    22.  
    23. def softmax(x):
    24.     x = x.reshape(-1)
    25.     e_x = np.exp(x - np.max(x))
    26.     return e_x / e_x.sum(axis=0)
    27.  
    28. def postprocess(result):
    29.     return softmax(np.array(result)).tolist()
    30.  
    31.  
    32. session = ort.InferenceSession('GlandUnet.onnx')
    33. img0 = cv2.imread('test.bmp')
    34. h0, w0 = img0.shape[0:2]
    35. img = cv2.cvtColor(img0, cv2.COLOR_BGR2RGB)
    36. img = cv2.resize(img, [512, 512])
    37. image_data = np.array(img).transpose(2, 0, 1) # HWC->CHW
    38. input_data = preprocess(image_data)
    39. time_start = time.time()  # 记录开始时间
    40. raw_result = session.run([], {'input': input_data})
    41. time_end = time.time()  # 记录结束时间
    42. time_sum = time_end - time_start  # 计算的时间差为程序的执行时间，单位为秒/s
    43. print(time_sum)
    44. # label_result = np.argmax(raw_result, dim=1)  # 缺argmax
    45. # out = np.squeeze(raw_result)
    46. # result_img = np.array(out, dtype=np.uint8)
    47. # result_img = cv2.resize(result_img, (w0, h0))
    48.  
    49. out1 = raw_result[0][0]
    50.  
    51. cv2.imshow('2', out1[1])
    52. cv2.waitKey(0)

    参考文献

    https://onnxruntime.ai/

    https://github.com/leimao/ONNX-Runtime-Inference/blob/main/src/inference.cpp

    神经网络语义分割模型C++部署(VS2019+ONNXRuntime+OpenCV)_Shijunfeng00的博客-CSDN博客

    opencv 图片HWC格式转CHW格式_wuqingshan2010的博客-CSDN博客

  • 相关阅读:
    1151：素数个数
    Java中String类型的hashCode实现
    qizhidao参数分析
    TiDB 监控框架概述
    Linux/麒麟系统上部署Vue+SpringBoot前后端分离项目
    Java程序设计——集元数据（JDBC编程）
    html+css制作3D七夕表白旋转相册特效
    CEC2013（MATLAB）：狐猴优化算法（Lemurs Optimizer，LO）求解CEC2013（提供MATLAB代码及参考文献）
    C++ 提高编程
    快速创建django项目管理系统流程
  • 原文地址：https://blog.csdn.net/weixin_42748604/article/details/127078298