• deepstream python yolov5使用记录

    deepstream_python_apps

    1、下载nvidia官方发布的 deepstream_python_apps到 /opt/nvidia/deepstream/deepstream/sour ces 目录下，根据deepstrema版本下载对应版本代码，我使用得deepstream6.0，所以我克隆v1.1.0版本代码

    2、根据HOWTO.md文件安装依赖，主要是安装使用得Gst Python和pyds模块

    3、运行官方例子，检查环境是否安装成功

    deepstream_python_yolov5

    1、下载yolov5得yolov5-deepstream-python代码，

    2、编译，CUDA_VER版本根据自己版本设置

    CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo/

    3、根据tensorrt-yolov5将模型转换成engine模型

    首先根据需要修改参数，主要是input w、h，batct_size：

    yololayer.h    

    1.     static constexpr int CLASS_NUM = 10;
    2.     static constexpr int INPUT_H = 1088;  // yolov5's input height and width must be divisible by 32.
    3.     static constexpr int INPUT_W = 1088;

    yolov5.cpp

    1. #define USE_FP16  // set USE_INT8 or USE_FP16 or USE_FP32
    2. #define DEVICE 0  // GPU id
    3. #define NMS_THRESH 0.4
    4. #define CONF_THRESH 0.5
    5. #define BATCH_SIZE 20
    6. #define MAX_IMAGE_INPUT_SIZE_THRESH 3000 * 3000 // ensure it exceed the maximum size in the input images !
    1. // clone code according to above #Different versions of yolov5
    2. // download https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5s.pt
    3. cp {tensorrtx}/yolov5/gen_wts.py {ultralytics}/yolov5
    4. cd {ultralytics}/yolov5
    5. python gen_wts.py -w yolov5s.pt -o yolov5s.wts
    6. // a file 'yolov5s.wts' will be generated.
    1. cd {tensorrtx}/yolov5/
    2. // update CLASS_NUM in yololayer.h if your model is trained on custom dataset
    3. mkdir build
    4. cd build
    5. cp {ultralytics}/yolov5/yolov5s.wts {tensorrtx}/yolov5/build
    6. cmake ..
    7. make
    8. sudo ./yolov5 -s [.wts] [.engine] [n/s/m/l/x/n6/s6/m6/l6/x6 or c/c6 gd gw]  // serialize model to plan file
    9. sudo ./yolov5 -d [.engine] [image folder]  // deserialize and run inference, the images in [image folder] will be processed.
    10. // For example yolov5s
    11. sudo ./yolov5 -s yolov5s.wts yolov5s.engine s
    12. sudo ./yolov5 -d yolov5s.engine ../samples
    13. // For example Custom model with depth_multiple=0.17, width_multiple=0.25 in yolov5.yaml
    14. sudo ./yolov5 -s yolov5_custom.wts yolov5.engine c 0.17 0.25
    15. sudo ./yolov5 -d yolov5.engine ../samples

    4、检查 deepstream_yolov5_config.txt and main.py中得路径，步骤3会生成libmyplugins.so插件，因为我这边导入import ctypes报错，所以将其注销

    1. #import ctypes
    2.  
    3. import pyds
    4.  
    5. #ctypes.cdll.LoadLibrary('/home/nvidia/lefugang/tensorrtx/yolov5/build/libmyplugins.so')

    5、 修改main.py中sink插件，使得可以在终端输出结果，不需要再显示器上显示画面，主要是去掉nvegltransform插件，因为该插件时跟随nveglglessink使用的。

    1. #!/usr/bin/env python3
    2.  
    3. ################################################################################
    4. # SPDX-FileCopyrightText: Copyright (c) 2019-2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
    5. # SPDX-License-Identifier: Apache-2.0
    6. #
    7. # Licensed under the Apache License, Version 2.0 (the "License");
    8. # you may not use this file except in compliance with the License.
    9. # You may obtain a copy of the License at
    10. #
    11. # http://www.apache.org/licenses/LICENSE-2.0
    12. #
    13. # Unless required by applicable law or agreed to in writing, software
    14. # distributed under the License is distributed on an "AS IS" BASIS,
    15. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    16. # See the License for the specific language governing permissions and
    17. # limitations under the License.
    18. ################################################################################
    19.  
    20. import sys
    21. # import keyboard
    22. sys.path.append('../')
    23. import gi
    24. gi.require_version('Gst', '1.0')
    25. from gi.repository import GObject, Gst
    26. from common.is_aarch_64 import is_aarch64
    27. from common.bus_call import bus_call
    28. #import ctypes
    29.  
    30. import pyds
    31.  
    32. #ctypes.cdll.LoadLibrary('/home/nvidia/lefugang/tensorrtx/yolov5/build/libmyplugins.so')
    33.  
    34. PGIE_CLASS_ID_VEHICLE = 0
    35. PGIE_CLASS_ID_BICYCLE = 1
    36. PGIE_CLASS_ID_PERSON = 2
    37. PGIE_CLASS_ID_ROADSIGN = 3
    38.  
    39.  
    40. def osd_sink_pad_buffer_probe(pad,info,u_data):
    41.     frame_number=0
    42.     #Intiallizing object counter with 0.
    43.     num_rects=0
    44.     gst_buffer = info.get_buffer()
    45.     if not gst_buffer:
    46.         print("Unable to get GstBuffer ")
    47.         return
    48.  
    49.     # Retrieve batch metadata from the gst_buffer
    50.     # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    51.     # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    52.     batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    53.     
    54.     l_frame = batch_meta.frame_meta_list
    55.     while l_frame is not None:
    56.         try:
    57.             # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
    58.             # The casting is done by pyds.glist_get_nvds_frame_meta()
    59.             # The casting also keeps ownership of the underlying memory
    60.             # in the C code, so the Python garbage collector will leave
    61.             # it alone.
    62.             #frame_meta = pyds.glist_get_nvds_frame_meta(l_frame.data)
    63.             frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    64.         except StopIteration:
    65.             break
    66.         
    67.  
    68.         display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
    69.  
    70.         frame_number=frame_meta.frame_num
    71.         num_rects = frame_meta.num_obj_meta
    72.         l_obj=frame_meta.obj_meta_list
    73.         while l_obj is not None:
    74.             try:
    75.                 # Casting l_obj.data to pyds.NvDsObjectMeta
    76.                 #obj_meta=pyds.glist_get_nvds_object_meta(l_obj.data)
    77.                 obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
    78.             except StopIteration:
    79.                 break
    80.             # set bbox color in rgba
    81.             print(obj_meta.class_id, obj_meta.obj_label, obj_meta.confidence)
    82.             # set the border width in pixel
    83.             obj_meta.rect_params.border_width=0
    84.             obj_meta.rect_params.has_bg_color=1
    85.             obj_meta.rect_params.bg_color.set(0.0, 0.5, 0.3, 0.4)
    86.  
    87.             try: 
    88.                 l_obj=l_obj.next
    89.             except StopIteration:
    90.                 break
    91.  
    92.         # Acquiring a display meta object. The memory ownership remains in
    93.         # the C code so downstream plugins can still access it. Otherwise
    94.         # the garbage collector will claim it when this probe function exits.
    95.         
    96.         display_meta.num_labels = 1
    97.         py_nvosd_text_params = display_meta.text_params[0]
    98.         # Setting display text to be shown on screen
    99.         # Note that the pyds module allocates a buffer for the string, and the
    100.         # memory will not be claimed by the garbage collector.
    101.         # Reading the display_text field here will return the C address of the
    102.         # allocated string. Use pyds.get_string() to get the string content.
    103.         # py_nvosd_text_params.display_text = "Frame Number={} Number of Objects={} Vehicle_count={} Person_count={}".format(frame_number, num_rects, obj_counter[PGIE_CLASS_ID_VEHICLE], obj_counter[PGIE_CLASS_ID_PERSON])
    104.  
    105.         # Now set the offsets where the string should appear
    106.         py_nvosd_text_params.x_offset = 10
    107.         py_nvosd_text_params.y_offset = 12
    108.  
    109.         # Font , font-color and font-size
    110.         py_nvosd_text_params.font_params.font_name = "Serif"
    111.         py_nvosd_text_params.font_params.font_size = 10
    112.         # set(red, green, blue, alpha); set to White
    113.         py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)
    114.  
    115.         # Text background color
    116.         py_nvosd_text_params.set_bg_clr = 1
    117.         # set(red, green, blue, alpha); set to Black
    118.         py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
    119.         # Using pyds.get_string() to get display_text as string
    120.         # print(pyds.get_string(py_nvosd_text_params.display_text))
    121.         pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    122.         try:
    123.             l_frame=l_frame.next
    124.         except StopIteration:
    125.             break
    126. 			
    127.     return Gst.PadProbeReturn.OK	
    128.  
    129.  
    130.  
    131. def main(args):
    132.     # Check input arguments
    133.     if len(args) != 2:
    134.         sys.stderr.write("usage: %s \n" % args[0])
    135.         sys.exit(1)
    136.  
    137.     # Standard GStreamer initialization
    138.     GObject.threads_init()
    139.     Gst.init(None)
    140.  
    141.     # Create gstreamer elements
    142.     # Create Pipeline element that will form a connection of other elements
    143.     print("Creating Pipeline \n ")
    144.     pipeline = Gst.Pipeline()
    145.  
    146.     if not pipeline:
    147.         sys.stderr.write(" Unable to create Pipeline \n")
    148.  
    149.     # Source element for reading from the file
    150.     print("Creating Source \n ")
    151.     source = Gst.ElementFactory.make("filesrc", "file-source")
    152.     if not source:
    153.         sys.stderr.write(" Unable to create Source \n")
    154.  
    155.     # Since the data format in the input file is elementary h264 stream,
    156.     # we need a h264parser
    157.     print("Creating H264Parser \n")
    158.     h264parser = Gst.ElementFactory.make("h264parse", "h264-parser")
    159.     if not h264parser:
    160.         sys.stderr.write(" Unable to create h264 parser \n")
    161.  
    162.     # Use nvdec_h264 for hardware accelerated decode on GPU
    163.     print("Creating Decoder \n")
    164.     decoder = Gst.ElementFactory.make("nvv4l2decoder", "nvv4l2-decoder")
    165.     if not decoder:
    166.         sys.stderr.write(" Unable to create Nvv4l2 Decoder \n")
    167.  
    168.     # Create nvstreammux instance to form batches from one or more sources.
    169.     streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
    170.     if not streammux:
    171.         sys.stderr.write(" Unable to create NvStreamMux \n")
    172.  
    173.     # Use nvinfer to run inferencing on decoder's output,
    174.     # behaviour of inferencing is set through config file
    175.     pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
    176.     if not pgie:
    177.         sys.stderr.write(" Unable to create pgie \n")
    178.  
    179.     # Use convertor to convert from NV12 to RGBA as required by nvosd
    180.     nvvidconv = Gst.ElementFactory.make("nvvideoconvert", "convertor")
    181.     if not nvvidconv:
    182.         sys.stderr.write(" Unable to create nvvidconv \n")
    183.  
    184.     # Create OSD to draw on the converted RGBA buffer
    185.     nvosd = Gst.ElementFactory.make("nvdsosd", "onscreendisplay")
    186.  
    187.     if not nvosd:
    188.         sys.stderr.write(" Unable to create nvosd \n")
    189.  
    190.     # Finally render the osd output
    191.     if is_aarch64():
    192.         transform = Gst.ElementFactory.make("nvegltransform", "nvegl-transform")
    193.  
    194.     print("Creating EGLSink \n")
    195.     sink = Gst.ElementFactory.make("fakesink", "nvvideo-renderer")
    196.     if not sink:
    197.         sys.stderr.write(" Unable to create egl sink \n")
    198.  
    199.     print("Playing file %s " %args[1])
    200.     source.set_property('location', args[1])
    201.     streammux.set_property('width', 1920)
    202.     streammux.set_property('height', 1080)
    203.     streammux.set_property('batch-size', 1)
    204.     streammux.set_property('batched-push-timeout', 4000000)
    205.     pgie.set_property('config-file-path', "config/deepstream_yolov5_config.txt")
    206.  
    207.     print("Adding elements to Pipeline \n")
    208.     pipeline.add(source)
    209.     pipeline.add(h264parser)
    210.     pipeline.add(decoder)
    211.     pipeline.add(streammux)
    212.     pipeline.add(pgie)
    213.     pipeline.add(nvvidconv)
    214.     pipeline.add(nvosd)
    215.     pipeline.add(sink)
    216.  
    217.     # we link the elements together
    218.     # file-source -> h264-parser -> nvh264-decoder ->
    219.     # nvinfer -> nvvidconv -> nvosd -> video-renderer
    220.     print("Linking elements in the Pipeline \n")
    221.     source.link(h264parser)
    222.     h264parser.link(decoder)
    223.  
    224.     sinkpad = streammux.get_request_pad("sink_0")
    225.     if not sinkpad:
    226.         sys.stderr.write(" Unable to get the sink pad of streammux \n")
    227.     srcpad = decoder.get_static_pad("src")
    228.     if not srcpad:
    229.         sys.stderr.write(" Unable to get source pad of decoder \n")
    230.     srcpad.link(sinkpad)
    231.     streammux.link(pgie)
    232.     pgie.link(nvvidconv)
    233.     nvvidconv.link(nvosd)
    234.     nvosd.link(sink)
    235.  
    236.     # create an event loop and feed gstreamer bus mesages to it
    237.     #GObject.timeout_add_seconds(5, pipeline_pause(pipeline))
    238.     loop = GObject.MainLoop()
    239.     bus = pipeline.get_bus()
    240.     bus.add_signal_watch()
    241.     bus.connect ("message", bus_call, loop)
    242.  
    243.     # Lets add probe to get informed of the meta data generated, we add probe to
    244.     # the sink pad of the osd element, since by that time, the buffer would have
    245.     # had got all the metadata.
    246.     osdsinkpad = nvosd.get_static_pad("sink")
    247.     if not osdsinkpad:
    248.         sys.stderr.write(" Unable to get sink pad of nvosd \n")
    249.  
    250.     osdsinkpad.add_probe(Gst.PadProbeType.BUFFER, osd_sink_pad_buffer_probe, 0)
    251.  
    252.     print("Starting pipeline \n")
    253.     pipeline.set_state(Gst.State.PLAYING)
    254.     try:
    255.         loop.run()
    256.  
    257.     except:
    258.         pass
    259.     # cleanup
    260.     pipeline.set_state(Gst.State.NULL)
    261.  
    262. if __name__ == '__main__':
    263.     sys.exit(main(sys.argv))
    264.

    6、运行

    LD_PRELOAD=/home/nvidia/lfg/tensorrtx/yolov5/build/libmyplugins.so python main.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
    1. Creating Pipeline 
    2.  
    3. Creating streamux 
    4.  
    5. Creating source_bin  0  
    6.  
    7. Creating source bin
    8. source-bin-00
    9. Creating Pgie 
    10.  
    11. Creating tiler 
    12.  
    13. Creating nvvidconv 
    14.  
    15. Creating nvosd 
    16.  
    17. Creating transform 
    18.  
    19. Creating EGLSink 
    20.  
    21. Atleast one of the sources is live
    22. Adding elements to Pipeline 
    23.  
    24. Linking elements in the Pipeline 
    25.  
    26. Now playing...
    27. 1 :  rtsp://admin:asdf1234@10.1.7.220:554
    28. Starting pipeline 
    29.  
    30. 0:00:04.399294673  9779   0x5561021e30 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/yolov5-deepstream-python/best.engine
    31. INFO: [Implicit Engine Info]: layers num: 2
    32. 0   INPUT  kFLOAT data            3x1088x1088     
    33. 1   OUTPUT kFLOAT prob            6001x1x1        
    34.  
    35. 0:00:04.399614801  9779   0x5561021e30 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/yolov5-deepstream-python/best.engine
    36. 0:00:04.429867023  9779   0x5561021e30 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:config/deepstream_yolov5_config.txt sucessfully
    37. Decodebin child added: source 
    38.  
    39. Decodebin child added: decodebin0 
    40.  
    41. Decodebin child added: rtph264depay0 
    42.  
    43. Decodebin child added: h264parse0 
    44.  
    45. Decodebin child added: capsfilter0 
    46.  
    47. Decodebin child added: nvv4l2decoder0 
    48.  
    49. Opening in BLOCKING MODE 
    50. NvMMLiteOpen : Block : BlockType = 261 
    51. NVMEDIA: Reading vendor.tegra.display-size : status: 6 
    52. NvMMLiteBlockCreate : Block : BlockType = 261 
    53. In cb_newpad
    54.  
    55. gstname= video/x-raw
    56. features= <Gst.CapsFeatures object at 0x7fa3fd3e88 (GstCapsFeatures at 0x7f3009cda0)>
    57. Frame Number= 0 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    58. Frame Number= 1 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    59. Frame Number= 2 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    60. Frame Number= 3 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    61. Frame Number= 4 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    62. Frame Number= 5 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    63. Frame Number= 6 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    64. Frame Number= 7 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    65. Frame Number= 8 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    66. Frame Number= 9 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    67. Frame Number= 10 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    68. Frame Number= 11 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    69. Frame Number= 12 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    70. Frame Number= 13 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    71. Frame Number= 14 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    72. Frame Number= 15 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    73. Frame Number= 16 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    74. Frame Number= 17 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    75. Frame Number= 18 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
    76. Frame Number= 19 Number of Objects= 0 Vehicle_count= 0 Person_count= 0

     7、成功

  • 相关阅读:
    【老生谈算法】matlab实现图像增强处理算法源码——图像增强处理算法
    ContentProvider的执行时机
    电脑硬件及电脑配置知识大全
    vue2.x封装svg组件并使用
    MATLAB | kmeans聚类如何绘制更强的聚类边界(决策边界)
    WEB 渗透之信息收集
    字符串左旋解法和子字符串判断法
    LT-mapper,LT-removert代码运行与学习
    使用序列化技术保存数据 改进 IO流完成项目实战水果库存系统
    【面试宝藏】Redis 常见面试题解析
  • 原文地址：https://blog.csdn.net/LFGxiaogang/article/details/126481441