• 基于ONNXRuntime C#实现的高性能YOLO推理框架

    YoloSharpOnnx是一个高性能、内存复用、跨平台的 C# YOLO 推理库，基于 OpenCV 和 ONNX 运行时实现。

    背景

    刚开始做工业检测项目时，在网上找了一些现成的YOLO推理库，发现都不怎么好用，性能也一般，并且部署不是很方便，只支持少数几种硬件部署，另外项目需要使用批量检测的功能，网上的一些YOLO推理框架都没有批量推理的功能，于是干脆就自己动手封装一个YOLO推理框架，方便修改优化，可以随时根据需求调整功能或添加功能。

    完成基本的目标检测功能后，后面一个月时间，陆陆续续把YOLO所有的功能都完善了，就把代码放到GitHub上面开源了

    功能特色

    • 支持所有YOLO任务：目标检测、图片分类、实例分割、姿势估计、OBB
    • 支持多种部署方案：CPU, CUDA / TensorRT, OpenVINO, CoreML, DirectML
    • 可以批量检测图片，性能大幅优于单线程串行执行
    • 图像处理使用OpenCVSharp
    • 推理引擎：ONNX Runtime 是一个跨平台的机器学习模型加速器

    示例Demo

    Object Detection Image Classification 
    bus_detectdet_000000270579 cls_000000063409cls_000000000009

     

    Instance Segmentation
    res_seg_zidane
    res_seg_02

     

     

    Pose Estimation
    res_pose_01
    res_pose_02
    OBB Detection
    res_obb_01
    res_obb_02

     

     

    使用示例

    1 导出模型为onnx格式

    复制代码
    from ultralytics import YOLO

# Load a model
model = YOLO('path/to/best.pt')

# Export the model to ONNX format
model.export(format='onnx')
    复制代码

    2 YoloSharpOnnx初始化

    安装Nuget 包YoloSharpOnnx, OnnxRuntime, OpenCvSharp4.runtime

    CPU推理

    dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime
    using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));

    CoreML推理

    dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.osx.10.15-x64
dotnet add package Microsoft.ML.OnnxRuntime
    using YoloSharp yolo = new YoloSharp(new ExecutionProviderCoreML("yolo11n.onnx"));

    CUDA/TensorRT推理

    dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime.Gpu.Windows
    using YoloSharp yolo = new YoloSharp(new ExecutionProviderCUDA("yolo11n.onnx",0));
using YoloSharp yolo = new YoloSharp(new ExecutionProviderTensorRT("yolo11n.onnx",0));

    DirectML推理

    dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Microsoft.ML.OnnxRuntime.DirectML
    using YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML("yolo11n.onnx",0));

    OpenVINO Inference

    dotnet add package YoloSharpOnnx
dotnet add package OpenCvSharp4.runtime.win
dotnet add package Intel.ML.OnnxRuntime.OpenVino
    using YoloSharp yolo = new YoloSharp(new ExecutionProviderOpenVINO("yolo11n.onnx", IntelDeviceType.NPU));

     

    基本的API，加载模型并进行预测

    复制代码
    using Mat image = Cv2.ImRead("bus.jpg");
using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));

List res = yolo.RunDetect(image);

yolo.DrawDetections(image,res);
Cv2.ImWrite("bus_res.jpg", image);

string printString = res.Summary();
Console.WriteLine(printString);
    复制代码

    性能测试API

    复制代码
    using Mat image = Cv2.ImRead("bus.jpg");
     
using YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML("yolo11n.onnx",1));
var res = yolo.RunDetectWithTime(item.FullName);

Console.WriteLine($"{res.ToString()}, {res.SpeedResult.ToString()}");
    复制代码

    配置参数

    复制代码
    using Mat image = Cv2.ImRead("bus.jpg");
using YoloSharp yolo = new YoloSharp(new ExecutionProviderCPU("yolo11n.onnx"));
yolo.YoloConfiguration.IoU = 0.4f;
yolo.YoloConfiguration.Confidence = 0.3f;
yolo.YoloConfiguration.ResizeAlgorithm = InterpolationFlags.Linear;
yolo.YoloConfiguration.ImageExtsBatch = [".jpg", ".png"];
var res = yolo.RunDetect(image);
    复制代码

     

    批量处理API

    复制代码
    private static void TestBatchInfer()
{
    string modelPath = @"D:\code\model\best.onnx";
    string dir = @"D:\code\model\TestImages"
    DirectoryInfo directory = new DirectoryInfo(dir);
    var files = directory.GetFiles()
    System.Diagnostics.Stopwatch _stopwatch = new System.Diagnostics.Stopwatch();
    _stopwatch.Start();
    int num=files.Length;
    using (YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML(modelPath, 0)))
    {
        var list = yolo.RunBatchDetect(dir,new ProcessCallback(), ReceiveProcess, 30)
    }
    _stopwatch.Stop()
    Console.WriteLine($"detect {num} images, time:{_stopwatch.Elapsed}");

}
private static void ReceiveProcess(DetectionBatchResult e)
{
    string res = e.Results.Summary();
}
internal class ProcessCallback : IBatchProcessCallback
{
   
    public void ReceiveProcessResult(DetectionBatchResult e)
    {
       
        string res = e.Results.Summary();
      
    }
}
    复制代码

     

    批量处理Foreach API

    复制代码
    private static async Task TestBatchForeachInfer()
{
    var files = Directory.GetFiles(dir);
    System.Diagnostics.Stopwatch _stopwatch = new System.Diagnostics.Stopwatch();
    _stopwatch.Start();
    int num = files.Length;
    using (YoloSharp yolo = new YoloSharp(new ExecutionProviderDirectML(modelPath, _deviceId)))
    {
        yolo.YoloConfiguration.BatchPoolSize = 30;
        await foreach (var item in yolo.BatchDetectForeachAsync(files.ToList()))
        {
            Console.WriteLine($"{item.ImagePath} {item.Results.Summary()}");
        }
    }
    _stopwatch.Stop();
    Console.WriteLine($"detect {num} images, time:{_stopwatch.Elapsed}");
}
    复制代码

     

    性能测试

    Yolo C# inference libraryVersionImage Processing libraryImage Resize AlgorithmSequence inferenceBatch inference
    YoloSharp 6.1.0 SixLabors.ImageSharp 3.1.12 Triangle(Bilinear) support not support
    YoloDotNet 4.2.0 SkiaSharp 3.119.1 Linear(Bilinear) support support
    YoloSharpOnnx 1.3.3 OpenCvSharp4 4.13.0.20260318 Linear(Bilinear) support support

     

    测试工具

    YoloOnnxWinform

    测试电脑

    HardwareSummary
    Windows Windows 10 OS Version 19045.6466
    CPU AMD Ryzen 7 5800X 8-Core Processor 3.8GHz
    RAM DDR4 3200 MHz 32GB
    GPU AMD Radeom RX6800 16GB
    Storage SSD 2TB

    测试数据

    Images: 300 images (image size: 2480x3494)

    Yolo Model: Yolo11n.onnx InputShape float32[1,3,1280,1280]

    Inference Provider: DirectML Inference Microsoft.ML.OnnxRuntime.DirectML 1.24.3

    YoloSharp测试结果

    单线程推理时间: 18.707s Memory Usage: 1374M

     

    yolosharp

     

    YoloDotNet测试结果

    单线程推理时间: 17.665s Memory Usage: 169M

    批量处理图片时间: 10.587s Memory Usage: 639M

    yolodotnet

    yolodotnet2

     

    YoloSharpOnnx 测试结果

    单线程推理时间: 13.693s Memory Usage: 169M

    批量处理图片时间: 2.980s Memory Usage: 601M

    yolosharponnx1
    yolosharponnx2

     

     

    测试结果汇总

    Yolo C# inference libraryVersionImage Processing libraryImage Resize AlgorithmSequence inference (Time/Memory)Batch inference (Time/Memory)
    YoloSharp 6.1.0 SixLabors.ImageSharp 3.1.12 Triangle(Bilinear) 18.707s, 1374M -
    YoloDotNet 4.2.0 SkiaSharp 3.119.1 Linear(Bilinear) 17.665s, 169M 10.587s, 639M
    YoloSharpOnnx 1.3.3 OpenCvSharp4 4.13.0.20260318 Linear(Bilinear) 13.693s, 169M 2.980s, 601M

     

    后续需要完善的功能

    RT-DETR、SAM等检测模型的API开发

     

  • 相关阅读:
    51单片机K型热电偶温度采集及控制温控模块MAX6675热电偶LCD1602
    IBM bladecenter H刀箱BladeCenter北电交换机VLAN配置
    【广州华锐互动】灭火器使用VR教学系统应用于高校消防演练有什么好处？
    功率放大器在超声导波中的应用有哪些
    最小生成树
    [附源码]Python计算机毕业设计宠物短期寄养平台
    2022最新一线大厂Java八股文合集PDF版震撼开源，堪称史上最强
    prompt提示词：影响力营销文案，让AI 帮你写营销文案
    MiniGPT-4：看图聊天、教学、创作、搭网站......还开源了
    04 Spring MVC 源码总结 - 启动流程
  • 原文地址：https://www.cnblogs.com/luoht/p/20176232