• onnxruntime C++推理

    Opencv C++ onnx模型推理

    最先想用的就是opencv的dnn模块对onnx模型推理，其实之前也用过dnn模块不过以前推理的是tensorflow模型，当时是成功的。我以为这次onnx模型推理也没问题，结果：

    1. cv::dnn::Net thenet=cv::dnn::readNetFromONNX("model.onnx");
    2. 		//为什么图像blob行列错误，为什么是-1,-1呢 ？？
    3. 		Mat blob=cv::dnn::blobFromImage(inputimg,1.0/255);
    4. 		thenet.setInput(blob);
    5. 		Mat output=thenet.forward();

    加载模型是ok的，结果blobFromImage返回的blob行列是-1、-1？？？为什么blob是3维或4维？！

    onnxruntime C++ ubuntu源码编译

    准备编译出动态库，然后调用推理。先看ONNX Runtime | Home官网，纠结了一下还是先搞cpu版本。

    适合的onnxruntime版本

    根据NVIDIA - CUDA | onnxruntime 知道CUDA版本与配套的onnxruntime版本，我根据自己的CUDA版本需要下载 onnxruntime: Github onnxruntime镜像 - Gitee.com 这个版本，网好的也可以在https://github.com/microsoft/onnxruntime/releases?page=2 这里下载。下载完毕，解压：然后运行

    ./build.sh --config RelWithDebInfo --build_shared_lib --parallel --skip_submodule_sync --skip_tests

    numpy版本报错

    报错日志如下，原因就是网不好，需要的版本无法下载下来。

    1. 2023-08-09 14:49:01,895 build [DEBUG] - Defaulting to running update, build [and test for native builds].
    2. 2023-08-09 14:49:01,896 build [INFO] - Build started
    3. 2023-08-09 14:49:01,896 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0'
    4. ['/usr/bin/python3', '-m', 'pip', 'install', '--trusted-host', 'files.pythonhosted.org', 'setuptools', 'wheel', 'pytest', 'numpy>=1.16.6', 'sympy>=1.1', 'packaging', 'cerberus']
    5. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
    6. Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (41.0.1)
    7. Requirement already satisfied: wheel in /root/.local/lib/python3.5/site-packages (0.33.4)
    8. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
    9. WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
    10. WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/pytest/
    11. Collecting pytest
    12.   Using cached pytest-6.1.2-py3-none-any.whl (272 kB)
    13. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)",)': /simple/numpy/
    14. Collecting numpy>=1.16.6
    15.   WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)",)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
    16.   WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9668>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
    17.   WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)",)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
    18.   WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9ac8>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
    19.   WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f8f783b9c50>: Failed to establish a new connection: [Errno 101] 网络不可达',)': /packages/b5/36/88723426b4ff576809fec7d73594fe17a35c27f8d01f93637637a29ae25b/numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl
    20.   Downloading numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl (19.9 MB)
    21.      |▎                               | 204 kB 27 kB/s eta 0:12:01ERROR: Exception:
    22. Traceback (most recent call last):
    23.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    24.     crit = self.state.criteria[name]
    25. KeyError: 'numpy'
    26.  
    27. During handling of the above exception, another exception occurred:
    28.  
    29. Traceback (most recent call last):
    30.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher
    31.     yield
    32.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 519, in read
    33.     data = self._fp.read(amt) if not fp_closed else b""
    34.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read
    35.     data = self.__fp.read(amt)
    36.   File "/usr/lib/python3.5/http/client.py", line 462, in read
    37.     n = self.readinto(b)
    38.   File "/usr/lib/python3.5/http/client.py", line 502, in readinto
    39.     n = self.fp.readinto(b)
    40.   File "/usr/lib/python3.5/socket.py", line 575, in readinto
    41.     return self._sock.recv_into(b)
    42.   File "/usr/lib/python3.5/ssl.py", line 929, in recv_into
    43.     return self.read(nbytes, buffer)
    44.   File "/usr/lib/python3.5/ssl.py", line 791, in read
    45.     return self._sslobj.read(len, buffer)
    46.   File "/usr/lib/python3.5/ssl.py", line 575, in read
    47.     v = self._sslobj.read(len, buffer)
    48. socket.timeout: The read operation timed out
    49.  
    50. During handling of the above exception, another exception occurred:
    51.  
    52. Traceback (most recent call last):
    53.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/base_command.py", line 223, in _main
    54.     status = self.run(options, args)
    55.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    56.     return func(self, options, args)
    57.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/install.py", line 321, in run
    58.     reqs, check_supported_wheels=not options.target_dir
    59.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    60.     requirements, max_rounds=try_to_avoid_resolution_too_deep,
    61.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    62.     state = resolution.resolve(requirements, max_rounds=max_rounds)
    63.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    64.     name, crit = self._merge_into_criterion(r, parent=None)
    65.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    66.     crit = Criterion.from_requirement(self._p, requirement, parent)
    67.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement
    68.     if not cands:
    69.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/resolvelib/structs.py", line 124, in __bool__
    70.     return bool(self._sequence)
    71.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in __bool__
    72.     return any(self)
    73.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 244, in iter_index_candidates
    74.     version=ican.version,
    75.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 169, in _make_candidate_from_link
    76.     name=name, version=version,
    77.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in __init__
    78.     version=version,
    79.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 144, in __init__
    80.     self.dist = self._prepare()
    81.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    82.     dist = self._prepare_distribution()
    83.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    84.     self._ireq, parallel_builds=True,
    85.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 480, in prepare_linked_requirement
    86.     return self._prepare_linked_requirement(req, parallel_builds)
    87.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 505, in _prepare_linked_requirement
    88.     self.download_dir, hashes,
    89.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 257, in unpack_url
    90.     hashes=hashes,
    91.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/operations/prepare.py", line 130, in get_http_url
    92.     from_path, content_type = download(link, temp_dir.path)
    93.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/network/download.py", line 163, in __call__
    94.     for chunk in chunks:
    95.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/progress_bars.py", line 168, in iter
    96.     for x in it:
    97.   File "/usr/local/lib/python3.5/dist-packages/pip/_internal/network/utils.py", line 88, in response_chunks
    98.     decode_content=False,
    99.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 576, in stream
    100.     data = self.read(amt=amt, decode_content=decode_content)
    101.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 541, in read
    102.     raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
    103.   File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    104.     self.gen.throw(type, value, traceback)
    105.   File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
    106.     raise ReadTimeoutError(self._pool, None, "Read timed out.")
    107. pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
    108. Traceback (most recent call last):
    109.   File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 2088, in <module>
    110.     sys.exit(main())
    111.   File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 2003, in main
    112.     install_python_deps()
    113.   File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 585, in install_python_deps
    114.     'files.pythonhosted.org'] + dep_packages)
    115.   File "/home/jumper/workspace/onnxruntime_1.6.0/tools/ci_build/build.py", line 523, in run_subprocess
    116.     return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
    117.   File "/home/jumper/workspace/onnxruntime_1.6.0/tools/python/util/run.py", line 42, in run
    118.     env=env, shell=shell)
    119.   File "/usr/lib/python3.5/subprocess.py", line 708, in run
    120.     output=stdout, stderr=stderr)

    解决办法：另开窗口下载指定版本

    1. pip install -U numpy==1.16.6 -i https://pypi.tuna.tsinghua.edu.cn/simple
    2. pip install -U sympy==1.6.2 -i https://pypi.tuna.tsinghua.edu.cn/simple

    pyinstaller报错

    重新运行后，出现pyinstaller报错，日志如下：

    1. Getting requirements to build wheel ... error
    2.   ERROR: Command errored out with exit status 1:
    3.    command: /usr/bin/python3 /usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpkg3cye79
    4.        cwd: /tmp/pip-install-__e1hgq0/cerberus_f4b541de90114b768be7d7f6ac4d2370
    5.   Complete output (10 lines):
    6.   Traceback (most recent call last):
    7.     File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
    8.       main()
    9.     File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
    10.       json_out['return_val'] = hook(**hook_input['kwargs'])
    11.     File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 108, in get_requires_for_build_wheel
    12.       backend = _build_backend()
    13.     File "/usr/local/lib/python3.5/dist-packages/pip/_vendor/pep517/_in_process.py", line 99, in _build_backend
    14.       obj = getattr(obj, path_part)
    15.   AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__'

    解决办法如下：

    pip install pyinstaller

    第三方库缺少报错

    重新运行后，又报错说缺少googletest、flatbuffers、nsync、date、protobuf、re2等这些第三方库，日志如下：

    1. CMake Error at CMakeLists.txt:478 (add_subdirectory):
    2.   The source directory
    3.  
    4.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/googletest
    5.  
    6.   does not contain a CMakeLists.txt file.
    7.  
    8.  
    9. CMake Error at CMakeLists.txt:479 (set_target_properties):
    10.   set_target_properties Can not find target to add properties to: gmock
    11.  
    12.  
    13. CMake Error at CMakeLists.txt:480 (set_target_properties):
    14.   set_target_properties Can not find target to add properties to: gmock_main
    15.  
    16.  
    17. CMake Error at CMakeLists.txt:481 (set_target_properties):
    18.   set_target_properties Can not find target to add properties to: gtest
    19.  
    20.  
    21. CMake Error at CMakeLists.txt:482 (set_target_properties):
    22.   set_target_properties Can not find target to add properties to: gtest_main
    23.  
    24.  
    25. CMake Error at CMakeLists.txt:483 (add_library):
    26.   add_library cannot create ALIAS target "GTest::gmock" because target
    27.   "gmock" does not already exist.
    28.  
    29.  
    30. CMake Error at CMakeLists.txt:484 (add_library):
    31.   add_library cannot create ALIAS target "GTest::gmock_main" because target
    32.   "gmock_main" does not already exist.
    33.  
    34.  
    35. CMake Error at CMakeLists.txt:485 (add_library):
    36.   add_library cannot create ALIAS target "GTest::gtest" because target
    37.   "gtest" does not already exist.
    38.  
    39.  
    40. CMake Error at CMakeLists.txt:486 (add_library):
    41.   add_library cannot create ALIAS target "GTest::gtest_main" because target
    42.   "gtest_main" does not already exist.
    43.  
    44.  
    45. CMake Error at CMakeLists.txt:526 (add_subdirectory):
    46.   The source directory
    47.  
    48.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/nsync
    49.  
    50.   does not contain a CMakeLists.txt file.
    51.  
    52.  
    53. Use protobuf from submodule
    54. CMake Error at CMakeLists.txt:565 (add_subdirectory):
    55.   add_subdirectory given source
    56.   "/home/jumper/workspace/onnxruntime_1.6.0/cmake/external/protobuf/cmake"
    57.   which is not an existing directory.
    58.  
    59.  
    60. CMake Error at CMakeLists.txt:577 (add_library):
    61.   add_library cannot create ALIAS target "protobuf::libprotobuf" because
    62.   target "libprotobuf-lite" does not already exist.
    63.  
    64.  
    65. CMake Error at CMakeLists.txt:626 (add_subdirectory):
    66.   The source directory
    67.  
    68.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/date
    69.  
    70.   does not contain a CMakeLists.txt file.
    71.  
    72.  
    73. CMake Error at CMakeLists.txt:642 (add_subdirectory):
    74.   The source directory
    75.  
    76.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2
    77.  
    78.   does not contain a CMakeLists.txt file.
    79.  
    80.  
    81. CMake Error at CMakeLists.txt:643 (set_target_properties):
    82.   set_target_properties Can not find target to add properties to: re2
    83.  
    84.  
    85. CMake Error at CMakeLists.txt:644 (add_library):
    86.   add_library cannot create ALIAS target "re2::re2" because target "re2" does
    87.   not already exist.
    88.  
    89.  
    90. CMake Error at CMakeLists.txt:837 (add_subdirectory):
    91.   The source directory
    92.  
    93.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/onnx
    94.  
    95.   does not contain a CMakeLists.txt file.
    96.  
    97.  
    98. CMake Error at CMakeLists.txt:842 (target_compile_definitions):
    99.   Cannot specify compile definitions for target "onnx" which is not built by
    100.   this project.
    101.  
    102.  
    103. CMake Error at CMakeLists.txt:844 (target_compile_definitions):
    104.   Cannot specify compile definitions for target "onnx" which is not built by
    105.   this project.
    106.  
    107.  
    108. CMake Error at CMakeLists.txt:846 (set_target_properties):
    109.   set_target_properties Can not find target to add properties to: onnx
    110.  
    111.  
    112. CMake Error at CMakeLists.txt:847 (set_target_properties):
    113.   set_target_properties Can not find target to add properties to: onnx_proto
    114.  
    115.  
    116. CMake Error at CMakeLists.txt:885 (add_subdirectory):
    117.   The source directory
    118.  
    119.     /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/flatbuffers
    120.  
    121.   does not contain a CMakeLists.txt file.
    122.  
    123.  
    124. CMake Warning at flake8.cmake:19 (message):
    125.   Could not find 'flake8' to check python scripts.  Please install flake8
    126.   using pip.
    127. Call Stack (most recent call first):
    128.   CMakeLists.txt:1501 (include)

    解决办法：可以看到onnxruntime: Forked from OnnxRuntime - Gitee.com 这里每个第三方库点开都是空的，所以需要自己另行下载解压后拷贝到对应位置。以googletest为例：

    https://github.com/google/googletest

    下载完，然后运行：

    1. cd googletest         
    2. mkdir build           
    3. cd build
    4. cmake .. 
    5. make
    6. make install
    7.  
    8. 可选是否运行test : make test && make test install

    再如缺少 nsync 就是在GitHub - google/nsync: nsync is a C library that exports various synchronization primitives, such as mutexes 这里下载，然后也是如googletest一样编译。

    对于protobuf：

    1. 在这里下载解压并拷贝到onnxruntime对应位置 https://github.com/protocolbuffers/protobuf/tree/498de9f761bef56a032815ee44b6e6dbe0892cc4
    2. ./autogen.sh
    3. ./configure
    4. make
    5. make install
    6. ldconfig

    所有第三方库安装好后，重新运行。

    date_inference报错

    日志如下，

    1. 2023-08-09 20:21:03,982 build [DEBUG] - Defaulting to running update, build [and test for native builds].
    2. 2023-08-09 20:21:03,983 build [INFO] - Build started
    3. 2023-08-09 20:21:03,983 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0'
    4. ['/usr/bin/python3', '-m', 'pip', 'install', '--trusted-host', 'files.pythonhosted.org', 'setuptools', 'wheel', 'pytest', 'numpy>=1.16.6', 'sympy>=1.1', 'packaging', 'cerberus']
    5. DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
    6. Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (41.0.1)
    7. Requirement already satisfied: wheel in /root/.local/lib/python3.5/site-packages (0.33.4)
    8. Requirement already satisfied: pytest in /usr/local/lib/python3.5/dist-packages (6.1.2)
    9. Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.5/dist-packages (1.16.6)
    10. Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.5/dist-packages (1.6.2)
    11. Requirement already satisfied: packaging in /usr/local/lib/python3.5/dist-packages (20.9)
    12. Requirement already satisfied: cerberus in /usr/local/lib/python3.5/dist-packages (1.3.3)
    13. Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.5/dist-packages (from sympy>=1.1) (1.3.0)
    14. Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3/dist-packages (from packaging) (2.0.3)
    15. Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.5/dist-packages (from pytest) (22.1.0)
    16. Requirement already satisfied: importlib-metadata>=0.12 in /usr/local/lib/python3.5/dist-packages (from pytest) (2.1.3)
    17. Requirement already satisfied: pluggy<1.0,>=0.12 in /usr/local/lib/python3.5/dist-packages (from pytest) (0.13.1)
    18. Requirement already satisfied: iniconfig in /usr/local/lib/python3.5/dist-packages (from pytest) (1.1.1)
    19. Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.5/dist-packages (from pytest) (1.11.0)
    20. Requirement already satisfied: toml in /usr/local/lib/python3.5/dist-packages (from pytest) (0.10.2)
    21. Requirement already satisfied: pathlib2>=2.2.0 in /usr/local/lib/python3.5/dist-packages (from pytest) (2.3.7.post1)
    22. Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.5/dist-packages (from importlib-metadata>=0.12->pytest) (1.2.0)
    23. Requirement already satisfied: six in /root/.local/lib/python3.5/site-packages (from pathlib2>=2.2.0->pytest) (1.12.0)
    24. 2023-08-09 20:21:04,630 util.run [DEBUG] - Subprocess completed. Return code: 0
    25. 2023-08-09 20:21:04,630 build [INFO] - Generating CMake build tree
    26. 2023-08-09 20:21:04,631 util.run [INFO] - Running subprocess in '/home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo'
    27. ['/usr/local/bin/cmake', '/home/jumper/workspace/onnxruntime_1.6.0/cmake', '-Donnxruntime_RUN_ONNX_TESTS=OFF', '-Donnxruntime_BUILD_WINML_TESTS=ON', '-Donnxruntime_GENERATE_TEST_REPORTS=ON', '-Donnxruntime_DEV_MODE=ON', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-Donnxruntime_USE_CUDA=OFF', '-Donnxruntime_CUDNN_HOME=', '-Donnxruntime_USE_FEATURIZERS=OFF', '-Donnxruntime_CUDA_HOME=', '-Donnxruntime_USE_JEMALLOC=OFF', '-Donnxruntime_USE_MIMALLOC_STL_ALLOCATOR=OFF', '-Donnxruntime_USE_MIMALLOC_ARENA_ALLOCATOR=OFF', '-Donnxruntime_ENABLE_PYTHON=OFF', '-Donnxruntime_BUILD_CSHARP=OFF', '-Donnxruntime_BUILD_JAVA=OFF', '-Donnxruntime_BUILD_NODEJS=OFF', '-Donnxruntime_BUILD_SHARED_LIB=ON', '-Donnxruntime_USE_EIGEN_FOR_BLAS=ON', '-Donnxruntime_USE_OPENBLAS=OFF', '-Donnxruntime_USE_DNNL=OFF', '-Donnxruntime_DNNL_GPU_RUNTIME=', '-Donnxruntime_DNNL_OPENCL_ROOT=', '-Donnxruntime_USE_MKLML=OFF', '-Donnxruntime_USE_NNAPI_BUILTIN=OFF', '-Donnxruntime_USE_RKNPU=OFF', '-Donnxruntime_USE_OPENMP=OFF', '-Donnxruntime_USE_TVM=OFF', '-Donnxruntime_USE_LLVM=OFF', '-Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF', '-Donnxruntime_USE_VITISAI=OFF', '-Donnxruntime_USE_NUPHAR=OFF', '-Donnxruntime_USE_TENSORRT=OFF', '-Donnxruntime_TENSORRT_HOME=', '-Donnxruntime_USE_MIGRAPHX=OFF', '-Donnxruntime_MIGRAPHX_HOME=', '-Donnxruntime_CROSS_COMPILING=OFF', '-Donnxruntime_DISABLE_CONTRIB_OPS=OFF', '-Donnxruntime_DISABLE_ML_OPS=OFF', '-Donnxruntime_DISABLE_RTTI=OFF', '-Donnxruntime_DISABLE_EXCEPTIONS=OFF', '-Donnxruntime_DISABLE_ORT_FORMAT_LOAD=OFF', '-Donnxruntime_MINIMAL_BUILD=OFF', '-Donnxruntime_EXTENDED_MINIMAL_BUILD=OFF', '-Donnxruntime_REDUCED_OPS_BUILD=OFF', '-Donnxruntime_MSVC_STATIC_RUNTIME=OFF', '-Donnxruntime_ENABLE_LANGUAGE_INTEROP_OPS=OFF', '-Donnxruntime_USE_DML=OFF', '-Donnxruntime_USE_WINML=OFF', '-Donnxruntime_USE_TELEMETRY=OFF', '-Donnxruntime_ENABLE_LTO=OFF', '-Donnxruntime_USE_ACL=OFF', '-Donnxruntime_USE_ACL_1902=OFF', '-Donnxruntime_USE_ACL_1905=OFF', '-Donnxruntime_USE_ACL_1908=OFF', '-Donnxruntime_USE_ACL_2002=OFF', '-Donnxruntime_USE_ARMNN=OFF', '-Donnxruntime_ARMNN_RELU_USE_CPU=ON', '-Donnxruntime_ARMNN_BN_USE_CPU=ON', '-Donnxruntime_ENABLE_NVTX_PROFILE=OFF', '-Donnxruntime_ENABLE_TRAINING=OFF', '-Donnxruntime_USE_HOROVOD=OFF', '-Donnxruntime_USE_NCCL=ON', '-Donnxruntime_BUILD_BENCHMARKS=OFF', '-Donnxruntime_USE_ROCM=OFF', '-Donnxruntime_ROCM_HOME=', '-Donnxruntime_PYBIND_EXPORT_OPSCHEMA=OFF', '-Donnxruntime_ENABLE_MEMLEAK_CHECKER=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo']
    28. Use gtest from submodule
    29. -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3.4") 
    30. Use protobuf from submodule
    31. -- 
    32. -- 3.11.3.0
    33. -- Check if compiler accepts -pthread
    34. -- Check if compiler accepts -pthread - yes
    35. -- Found PythonInterp: /usr/bin/python3 (found version "3.5.2") 
    36. Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-ml.proto
    37. Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-operators-ml.proto
    38. Generated: /home/jumper/workspace/onnxruntime_1.6.0/build/Linux/RelWithDebInfo/external/onnx/onnx/onnx-data.proto
    39. -- 
    40. -- ******** Summary ********
    41. --   CMake version                     : 3.18.4
    42. --   CMake command                     : /usr/local/bin/cmake
    43. --   System                            : Linux
    44. --   C++ compiler                      : /usr/bin/c++
    45. --   C++ compiler version              : 6.5.0
    46. --   CXX flags                         :  -Wnon-virtual-dtor
    47. --   Build type                        : RelWithDebInfo
    48. --   Compile definitions               : ENABLE_ORT_FORMAT_LOAD;EIGEN_MPL2_ONLY;USE_EIGEN_FOR_BLAS;__STDC_FORMAT_MACROS
    49. --   CMAKE_PREFIX_PATH                 : 
    50. --   CMAKE_INSTALL_PREFIX              : /usr/local
    51. --   CMAKE_MODULE_PATH                 : /home/jumper/workspace/onnxruntime_1.6.0/cmake/external
    52. -- 
    53. --   ONNX version                      : 1.15.0
    54. --   ONNX NAMESPACE                    : onnx
    55. --   ONNX_USE_LITE_PROTO               : ON
    56. --   USE_PROTOBUF_SHARED_LIBS          : OFF
    57. --   Protobuf_USE_STATIC_LIBS          : ON
    58. --   ONNX_DISABLE_EXCEPTIONS           : OFF
    59. --   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
    60. --   ONNX_WERROR                       : OFF
    61. --   ONNX_BUILD_TESTS                  : OFF
    62. --   ONNX_BUILD_BENCHMARKS             : OFF
    63. -- 
    64. --   Protobuf compiler                 : 
    65. --   Protobuf includes                 : 
    66. --   Protobuf libraries                : 
    67. --   BUILD_ONNX_PYTHON                 : OFF
    68. -- Proceeding with version: 23.5.26.0
    69. -- CMAKE_CXX_FLAGS: 
    70. -- Configuring done
    71. CMake Error at CMakeLists.txt:673 (target_compile_definitions):
    72.   Error evaluating generator expression:
    73.  
    74.     $<TARGET_PROPERTY:date_interface,INTERFACE_COMPILE_DEFINITIONS>
    75.  
    76.   Target "date_interface" not found.
    77. Call Stack (most recent call first):
    78.   onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
    79.   onnxruntime_unittests.cmake:888 (AddTest)
    80.   CMakeLists.txt:1475 (include)
    81.  
    82.  
    83. CMake Error at CMakeLists.txt:672 (target_include_directories):
    84.   Error evaluating generator expression:
    85.  
    86.     $<TARGET_PROPERTY:date_interface,INTERFACE_INCLUDE_DIRECTORIES>
    87.  
    88.   Target "date_interface" not found.
    89. Call Stack (most recent call first):
    90.   onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
    91.   onnxruntime_unittests.cmake:888 (AddTest)
    92.   CMakeLists.txt:1475 (include)
    93.  
    94.  
    95. CMake Error at CMakeLists.txt:673 (target_compile_definitions):
    96.   Error evaluating generator expression:
    97.  
    98.     $<TARGET_PROPERTY:date_interface,INTERFACE_COMPILE_DEFINITIONS>
    99.  
    100.   Target "date_interface" not found.
    101. Call Stack (most recent call first):
    102.   onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
    103.   onnxruntime_unittests.cmake:888 (AddTest)
    104.   CMakeLists.txt:1475 (include)
    105.  
    106.  
    107. CMake Error at CMakeLists.txt:672 (target_include_directories):
    108.   Error evaluating generator expression:
    109.  
    110.     $<TARGET_PROPERTY:date_interface,INTERFACE_INCLUDE_DIRECTORIES>
    111.  
    112.   Target "date_interface" not found.
    113. Call Stack (most recent call first):
    114.   onnxruntime_unittests.cmake:64 (onnxruntime_add_include_to_target)
    115.   onnxruntime_unittests.cmake:888 (AddTest)
    116.   CMakeLists.txt:1475 (include)

    解决办法：按compile errors · Issue #4379 · microsoft/onnxruntime · GitHub 这样解决。即将onnxruntime_common.cmake and onnxruntime_unittests.cmake 中的"date_interface" 改成 "date"即可。

    GCC警告

    上面将所有的第三方库都安装好了，但重新运行时报错如下：

    1. [  6%] Built target onnxruntime_generate_def
    2. Scanning dependencies of target gtest
    3. [ 12%] Built target onnxruntime_mlas
    4. [ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/compile.cc.o
    5. [ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o
    6. [ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o
    7. [ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o
    8. [ 13%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o
    9. [ 22%] Built target libprotobuf
    10. [ 22%] Building CXX object external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o
    11. [ 22%] Building CXX object external/googletest/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
    12. In file included from /usr/local/include/absl/base/config.h:86:0,
    13.                  from /usr/local/include/absl/base/attributes.h:37,
    14.                  from /usr/local/include/absl/base/macros.h:34,
    15.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/compile.cc:15:
    16. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    17.  #error "This package requires GCC 7 or higher."
    18.   ^~~~~
    19. In file included from /usr/local/include/absl/base/config.h:86:0,
    20.                  from /usr/local/include/absl/base/internal/invoke.h:40,
    21.                  from /usr/local/include/absl/base/call_once.h:34,
    22.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/dfa.cc:36:
    23. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    24.  #error "This package requires GCC 7 or higher."
    25.   ^~~~~
    26. In file included from /usr/local/include/absl/base/config.h:86:0,
    27.                  from /usr/local/include/absl/base/attributes.h:37,
    28.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
    29.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitstate.cc:26:
    30. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    31.  #error "This package requires GCC 7 or higher."
    32.   ^~~~~
    33. In file included from /usr/local/include/absl/base/config.h:86:0,
    34.                  from /usr/local/include/absl/base/attributes.h:37,
    35.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
    36.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/mimics_pcre.cc:25:
    37. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    38.  #error "This package requires GCC 7 or higher."
    39.   ^~~~~
    40. In file included from /usr/local/include/absl/base/config.h:86:0,
    41.                  from /usr/local/include/absl/base/attributes.h:37,
    42.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/util/logging.h:16,
    43.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitmap256.h:14,
    44.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/bitmap256.cc:5:
    45. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    46.  #error "This package requires GCC 7 or higher."
    47.   ^~~~~
    48. In file included from /usr/local/include/absl/base/config.h:86:0,
    49.                  from /usr/local/include/absl/base/attributes.h:37,
    50.                  from /usr/local/include/absl/strings/string_view.h:39,
    51.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/filtered_re2.h:28,
    52.                  from /home/jumper/workspace/onnxruntime_1.6.0/cmake/external/re2/re2/filtered_re2.cc:5:
    53. /usr/local/include/absl/base/policy_checks.h:57:2: error: #error "This package requires GCC 7 or higher."
    54.  #error "This package requires GCC 7 or higher."
    55.   ^~~~~
    56. external/re2/CMakeFiles/re2.dir/build.make:81: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o' failed
    57. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o] Error 1
    58. make[2]: *** 正在等待未完成的任务....
    59. Scanning dependencies of target libprotoc
    60. external/re2/CMakeFiles/re2.dir/build.make:146: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o' failed
    61. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o] Error 1
    62. [ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/code_generator.cc.o
    63. [ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/command_line_interface.cc.o
    64. external/re2/CMakeFiles/re2.dir/build.make:94: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o' failed
    65. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o] Error 1
    66. [ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum.cc.o
    67. external/re2/CMakeFiles/re2.dir/build.make:133: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o' failed
    68. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o] Error 1
    69. [ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_enum_field.cc.o
    70. external/re2/CMakeFiles/re2.dir/build.make:107: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/compile.cc.o' failed
    71. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/compile.cc.o] Error 1
    72. [ 22%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_extension.cc.o
    73. [ 22%] Built target pep8_check
    74. [ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_field.cc.o
    75. external/re2/CMakeFiles/re2.dir/build.make:120: recipe for target 'external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o' failed
    76. make[2]: *** [external/re2/CMakeFiles/re2.dir/re2/dfa.cc.o] Error 1
    77. CMakeFiles/Makefile2:2137: recipe for target 'external/re2/CMakeFiles/re2.dir/all' failed
    78. make[1]: *** [external/re2/CMakeFiles/re2.dir/all] Error 2
    79. make[1]: *** 正在等待未完成的任务....
    80. [ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_file.cc.o
    81. [ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_generator.cc.o
    82. [ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_helpers.cc.o
    83. [ 23%] Building CXX object external/protobuf/cmake/CMakeFiles/libprotoc.dir/__/src/google/protobuf/compiler/cpp/cpp_map_field.cc.o

    但实际我的gcc版本是9.1，是满足要求的，为什么会报这个错呢？我的电脑里也没有多个版本的gcc，只有一个9.1啊！？

    使用预编译的onnxruntime-cpu

    暂时只能使用预编译的onnxruntime了，我这次下载1.12.0 Releases · microsoft/onnxruntime · GitHub版本onnxruntime-linux-x64-1.12.0.tgz，然后将include和lib拷贝到系统对应位置，即：

    cp -rfv onnxruntime-linux-x64-1.6.0/include/* /usr/local/include/ && \    cp -rfv onnxruntime-linux-x64-1.6.0/lib/* /usr/local/lib/ && \    ldconfig

    mnist实例

    按照https://github.com/microsoft/onnxruntime-inference-examples/tree/main/c_cxx/MNIST 这个

    1. #include <onnxruntime_cxx_api.h>
    2.  
    3. #include <opencv2/opencv.hpp>
    4. #include <opencv2/highgui/highgui.hpp>
    5. #include <opencv2/imgproc/imgproc.hpp>
    6. #include <opencv2/core/core.hpp>
    7. #include <opencv2/imgproc.hpp>
    8. using namespace cv;
    9. using namespace std;
    10.  
    11. //这是分类推理
    12. template <typename T>
    13. static void softmax(T& input) {
    14.   float rowmax = *std::max_element(input.begin(), input.end());
    15.   std::vector<float> y(input.size());
    16.   float sum = 0.0f;
    17.   for (size_t i = 0; i != input.size(); ++i) {
    18.     sum += y[i] = std::exp(input[i] - rowmax);
    19.   }
    20.   for (size_t i = 0; i != input.size(); ++i) {
    21.     input[i] = y[i] / sum;
    22.   }
    23. }
    24.  
    25. int main()//
    26. {
    27. 	Ort::Env env;
    28. //	Ort::Session session(env, L"mnist.onnx", Ort::SessionOptions(nullptr));
    29. 	const char *modelpath="model/mnist.onnx";
    30. //	Ort::Session session_{env, modelpath, Ort::SessionOptions{nullptr}};
    31. 	Ort::SessionOptions sessopt;
    32. 	sessopt.SetIntraOpNumThreads(1);
    33. 	sessopt.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);
    34. 	Ort::Session sess_(env,modelpath,sessopt);
    35.  
    36.     const int modelsize=28;
    37.     Ort::Value input_tensor_{nullptr};
    38.     std::array<int64_t, 4> input_shape_{1, 1, modelsize, modelsize};
    39.  
    40.     const int labelsnum=10;
    41.     Ort::Value output_tensor_{nullptr};
    42.     std::array<int64_t, 2> output_shape_{1, labelsnum};
    43.  
    44.     std::array<float, modelsize * modelsize> input_image_{};
    45.     std::array<float, labelsnum> results_{};
    46.     int64_t result_{0};
    47.  
    48.     auto memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
    49. 	input_tensor_ = Ort::Value::CreateTensor<float>(memory_info, input_image_.data(), input_image_.size(),
    50. 													input_shape_.data(), input_shape_.size());
    51. 	output_tensor_ = Ort::Value::CreateTensor<float>(memory_info, results_.data(), results_.size(),
    52. 													 output_shape_.data(), output_shape_.size());
    53.  
    54. 	// We need to convert the true-color data in the DIB into the model's floating point format
    55. 	float* inputdata = input_image_.data();
    56. 	//std::fill(input_image_.begin(), input_image_.end(), 0.f);
    58. 	const char* input_names[] = {"Input3"};
    59. 	const char* output_names[] = {"Plus214_Output_0"};
    61. 	char srcimgs[300];
    62. 	for(int imgid=0;imgid!=6;imgid++)
    63. 	{
    64. 		sprintf(srcimgs,"images/%d.jpg",imgid);
    65. 		Mat inputimg=imread(srcimgs,0);
    66. 		if(inputimg.empty())
    67. 			continue;
    69. 		cout<<"image "<
    71. 		inputimg.convertTo(inputimg,CV_32FC1);
    72. 		for (unsigned y = 0; y < modelsize; y++)
    73. 		{
    74. 			for (unsigned x = 0; x < modelsize; x++)
    75. 			{
    76. 				inputdata[y*modelsize+x] = inputimg.at(y,x);
    77. 			}
    78. 		}
    80. 		Ort::RunOptions run_options;
    81. 		sess_.Run(run_options, input_names, &input_tensor_, 1, output_names, &output_tensor_, 1);
    82. 		//get result ...
    83. 		softmax(results_);//results_和容器一样使用，得到每个labels的结果，以及最终的分类结果
    84. 		std::ptrdiff_t result_2 = std::distance(results_.begin(), std::max_element(results_.begin(), results_.end()));
    85. 		cout<<"      result_2:"<
    86. 		//or get result ...
    87. //		for (size_t i = 0; i != results_.size(); ++i) {
    88. //		    float score = results_[i] ;
    89. //		    printf("    label %d , score :%f \n",i,score);
    90. //		}
    91. 	}
    93. 	return 0;
    94. }

    可以看到结果是正确的：

     电池材料检测

    使用电池分解材料模型和图片，如下：

    1. #include <onnxruntime_cxx_api.h>
    2.  
    3. #include <opencv2/opencv.hpp>
    4. #include <opencv2/highgui/highgui.hpp>
    5. #include <opencv2/imgproc/imgproc.hpp>
    6. #include <opencv2/core/core.hpp>
    7. #include <opencv2/imgproc.hpp>
    8. using namespace cv;
    9. using namespace std;
    10.  
    11.  
    12. int main()//
    13. {
    14. 	Ort::Env env;
    15. 	const char *modelpath="model/segment.onnx";
    16. 	Ort::SessionOptions sessopt;
    17. 	sessopt.SetIntraOpNumThreads(1);
    18. 	sessopt.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_EXTENDED);
    19. 	Ort::Session sess_(env,modelpath,sessopt);
    20.  
    21.     const int modelsize=512;
    22.     Ort::Value input_tensor_{nullptr};
    23.     std::array<int64_t, 4> input_shape_{1, 1, modelsize, modelsize};
    24.  
    25.     Ort::Value output_tensor_{nullptr};
    26.     std::array<int64_t, 4> output_shape_{1, 1, modelsize, modelsize};
    27.  
    28.     std::array<float, modelsize * modelsize> input_image_{};
    29.     std::array<float, modelsize * modelsize> results_{};
    30.  
    31.     auto memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
    32. 	input_tensor_ = Ort::Value::CreateTensor<float>(memory_info, input_image_.data(), input_image_.size(),
    33. 													input_shape_.data(), input_shape_.size());
    34. 	output_tensor_ = Ort::Value::CreateTensor<float>(memory_info, results_.data(), results_.size(),
    35. 													 output_shape_.data(), output_shape_.size());
    36.  
    37. 	// We need to convert the true-color data in the DIB into the model's floating point format
    38. 	float* inputdata = input_image_.data();
    40. 	const char* input_names[] = {"input.1"};
    41. 	const char* output_names[] = {"376"};
    43. 	char srcimgs[300];
    44. 	for(int imgid=1;imgid!=2;imgid++)
    45. 	{
    46. 		sprintf(srcimgs,"images2/contrast_%d.png",imgid);
    47. 		Mat inputimg3=imread(srcimgs);
    48. 		if(inputimg3.empty())
    49. 			continue;
    51. 		Mat inputimg;
    52. 		cvtColor(inputimg3,inputimg,COLOR_BGR2GRAY);
    53. 		cout<<"image "<
    56. 		inputimg.convertTo(inputimg,CV_32FC1);
    57. 		inputimg/=255;
    58. 		cv::resize(inputimg,inputimg,Size(modelsize,modelsize));
    59. 		for (unsigned y = 0; y < modelsize; y++)
    60. 		{
    61. 			for (unsigned x = 0; x < modelsize; x++)
    62. 			{
    63. 				inputdata[y*modelsize+x] = inputimg.at(y,x);
    64. 			}
    65. 		}
    67. 		Ort::RunOptions run_options;
    68. 		sess_.Run(run_options, input_names, &input_tensor_, 1, output_names, &output_tensor_, 1);
    69. 		
    70. 		//or get result ...
    71. 		float* outputdata = results_.data();
    72. 		Mat outimg(inputimg.size(),CV_32FC1);
    73. 		for (unsigned y = 0; y < modelsize; y++)
    74. 		{
    75. 			for (unsigned x = 0; x < modelsize; x++)
    76. 			{
    77. 				outimg.at(y,x)=outputdata[y*modelsize+x];
    78. 			}
    79. 		}
    80. 		cv::resize(outimg,outimg,inputimg3.size());
    81. 		imwrite("result.jpg",outimg*255);
    83. 	}
    85. 	return 0;
    86. }

    推理结果正确：

     如上，左边是原图，右边是推理结果，CPU版本的预编译库可正常推理。耗时尚未测试。

    经过Performance | onnxruntime优化后，发现推理耗时少了很多，从而整体耗时少了很多，左边是优化前，右图是优化后，少了整整5秒，目前这还只是初步优化：

    另外看到这已经是利用上了所有核都已经到了100%。下一步我不知道怎么让多张图片并行推理？如果我那样做是不是就是将核都利用过去了，而推理时就没有核可利用了？

    ###################################################################

    采图装置是0.1mm比色皿，来自下图：

  • 相关阅读:
    javaIO流07：处理流BufferedReader和BufferedWriter，Buffer拷贝
    【算法设计与分析 李春葆】计算几何（二）——求解凸包问题
    Beyond Compare 4 试用期到期
    npm 安装到指定文件夹
    python气象科研学习路线和常用技巧
    黑马-web服务器与http协议
    uni-app--》基于小程序开发的电商平台项目实战（六）
    jenkins目录下的vue3项目——pnpm install后运行报错——奇葩问题解决
    前端下载文件重命名
    mysql索引详解
  • 原文地址：https://blog.csdn.net/wd1603926823/article/details/132270747