• 【工程部署】在RK3588上部署OCR(文字检测识别)（DBNet+CRNN）

    硬件平台：

    1、firefly安装Ubuntu系统的RK3588；

    2、安装Windows系统的电脑一台，其上安装Ubuntu18.04系统虚拟机。

    参考手册：《00-Rockchip_RKNPU_User_Guide_RKNN_API_V1.3.0_CN》

    《RKNN Toolkit Lite2 用户使用指南》

    1、文字检测

    项目地址：

    GitHub - WenmuZhou/PytorchOCR: 基于Pytorch的OCR工具库，支持常用的文字检测和识别算法

    DBNet（Dynamic-Link Bi-directional Network）是一种用于文本检测的深度学习模型。该模型于2019年由Minghui Liao等人提出，并在文本检测领域取得了显著的成果。DBNet的设计目标是在保持高精度的同时，提高文本检测的效率。传统的文本检测模型通常使用单向的横向连接或纵向连接来处理文本实例。然而，这种单向连接可能导致信息的不完整传递或信息冗余，影响了检测性能和速度。

    为了解决这些问题，DBNet引入了双向动态连接机制，允许横向和纵向两个方向上的信息流动。具体来说，DBNet由两个关键组成部分构成：

    (1) Bi-directional FFM（Feature Fusion Module）：这是DBNet的核心组件之一。它包括横向和纵向两个方向的子模块。在横向子模块中，DBNet通过可变形卷积（deformable convolution）从不同尺度的特征图中提取并融合文本实例的特征。而在纵向子模块中，DBNet使用自适应的特征选择机制，动态选择最具有代表性的特征。这些子模块的组合使得文本实例的特征能够全面而高效地进行建模。

    (2) Aggregation Decoder：这是DBNet的另一个重要组件，用于从特征图中生成文本实例的边界框和对应的文本分数。该解码器结合了横向和纵向的特征，通过逐步聚合来预测文本的位置和形状。由于使用了双向动态连接，解码器能够更准确地还原文本实例的形态。

    DBNet的训练过程包括前向传播和反向传播。在前向传播中，DBNet将图像输入网络，经过一系列卷积、特征融合和解码操作，得到文本检测的结果。然后，通过计算预测结果和真实标签之间的损失函数，使用反向传播算法来更新网络参数，从而不断优化模型的性能。

    DBNet在文本检测任务中取得了非常好的效果。其双向动态连接机制允许更好地利用横向和纵向的信息，提高了文本检测的准确性和鲁棒性。此外，相比传统的文本检测模型，DBNet在保持高精度的情况下，大幅提升了检测速度，使得它在实际应用中更具可用性和实用性。因此，DBNet在文字检测、自动化办公、图像识别等领域都具有广泛的应用前景。论文地址：https://arxiv.org/abs/1911.08947

    图1. DBNet网络结构

    2、文字识别

    项目地址：

    GitHub - WenmuZhou/PytorchOCR: 基于Pytorch的OCR工具库，支持常用的文字检测和识别算法

    CRNN（Convolutional Recurrent Neural Network）是一种深度学习模型，结合了卷积神经网络（CNN）和循环神经网络（RNN）的优势，广泛应用于图像文本识别（OCR）任务。CRNN模型于2015年由Baoguang Shi等人首次提出，并在OCR领域取得了显著的突破。

    CRNN的设计思想是将卷积神经网络用于图像的特征提取，并利用循环神经网络来对序列建模，从而使得CRNN能够直接从图像级别到序列级别进行端到端的学习。

    CRNN模型通常由以下几个部分组成：

    (1) 卷积层（Convolutional Layers）：CRNN利用多个卷积层来提取图像中的局部特征。这些卷积层可以学习不同层次的图像表示，从低级特征（如边缘和纹理）到高级特征（如形状和模式）。

    (2) RNN层（Recurrent Layers）：在卷积层后面，CRNN采用RNN层来处理序列数据。RNN能够捕捉序列的上下文信息，因此对于OCR任务而言，它可以有效地处理不同长度的文本序列。

    (3) 转录层（Transcription Layer）：在RNN层之后，CRNN使用转录层来将RNN输出映射到字符类别。这通常是一个全连接层，将RNN输出映射到预定义的字符集合，从而实现对文本的识别。

    CRNN的训练过程包括两个主要步骤：前向传播和反向传播。在前向传播中，CRNN将图像输入模型，经过卷积和循环层，最终得到文本序列的预测。然后，通过计算预测结果和真实标签之间的损失函数，使用反向传播算法来更新网络参数，从而使得模型的预测结果逐渐接近真实标签。

    CRNN在OCR领域的应用广泛，能够识别不同尺寸、字体、颜色和背景的文本。它在识别长文本序列方面表现优秀，并且由于端到端的设计，避免了传统OCR系统中复杂的流水线处理。因此，CRNN在很多实际场景中都取得了很好的效果，如车牌识别、文字检测和手写体识别等。

    总结来说，CRNN是一种将CNN和RNN结合起来的深度学习模型，用于图像文本识别任务。其端到端的设计、优秀的序列建模能力和在OCR领域的广泛应用，使得CRNN成为了一种重要的OCR模型，为自动化文本处理和识别带来了巨大的便利。论文地址：https://arxiv.org/abs/1507.05717

    图2. CRNN结构

    环境搭建

    rknn-toolkit以及rknpu_sdk环境搭建

    （手把手）rknn-toolkit以及rknpu_sdk环境搭建--以rk3588为例_warren@伟_的博客-CSDN博客

    模型的导出与验证

    文字检测

    导出onnx模型

    1. '''
    3. Author: warren
    5. Date: 2023-06-07 14:52:27
    7. LastEditors: warren
    9. LastEditTime: 2023-06-12 15:20:28
    11. FilePath: /warren/VanillaNet1/export_onnx.py
    13. Description: export onnx model
    16. Copyright (c) 2023 by ${git_name_email}, All Rights Reserved.
    18. '''
    19.  
    20. #!/usr/bin/env python3
    21.  
    22. import torch
    23.  
    24. from torchocr.networks import build_model
    25.  
    26. MODEL_PATH='./model/det_db_mbv3_new.pth'
    27.  
    28. DEVICE='cuda:0' if torch.cuda.is_available() else 'cpu'
    29.  
    30. print("-----------------------devices",DEVICE)
    31.  
    32.  
    33. class DetInfer:
    34.  
    35.     def __init__(self, model_path):
    36.  
    37.         ckpt = torch.load(model_path, map_location=DEVICE)
    38.  
    39.         cfg = ckpt['cfg']
    40.  
    41.         self.model = build_model(cfg['model'])
    42.  
    43.         state_dict = {}
    44.  
    45.         for k, v in ckpt['state_dict'].items():
    46.  
    47.             state_dict[k.replace('module.', '')] = v
    48.  
    49.         self.model.load_state_dict(state_dict)
    50.  
    51.         self.device = torch.device(DEVICE)
    52.  
    53.         self.model.to(self.device)
    54.  
    55.         self.model.eval()
    56.  
    57.  
    58.         checkpoint = torch.load(MODEL_PATH, map_location=DEVICE)
    59.  
    60.  
    61.         # Prepare input tensor
    62.  
    63.         input = torch.randn(1, 3, 640, 640, requires_grad=False).float().to(torch.device(DEVICE))
    64.  
    65.  
    66.         # Export the torch model as onnx
    67.  
    68.         print("-------------------export")
    69.  
    70.         torch.onnx.export(self.model,
    71.  
    72.             input,
    73.  
    74.             'detect_model_small.onnx', # name of the exported onnx model
    75.  
    76.             export_params=True,
    77.  
    78.             opset_version=12,
    79.  
    80.             do_constant_folding=False)
    81.  
    82.  
    83. # Load the pretrained model and export it as onnx
    84.  
    85. model = DetInfer(MODEL_PATH)

    验证

    1. import numpy as np
    2.  
    3. import cv2
    4.  
    5. import torch
    6.  
    7. from torchvision import transforms
    8.  
    9. # from label_convert import CTCLabelConverter
    10.  
    11. import cv2
    12.  
    13. import numpy as np
    14.  
    15. import pyclipper
    16.  
    17. from shapely.geometry import Polygon 
    18.  
    19. import onnxruntime
    20.  
    21.  
    22. class DBPostProcess():
    23.  
    24.     def __init__(self, thresh=0.3, box_thresh=0.7, max_candidates=1000, unclip_ratio=2):
    25.  
    26.         self.min_size = 3
    27.  
    28.         self.thresh = thresh
    29.  
    30.         self.box_thresh = box_thresh
    31.  
    32.         self.max_candidates = max_candidates
    33.  
    34.         self.unclip_ratio = unclip_ratio
    35.  
    36.  
    37.     def __call__(self, pred, h_w_list, is_output_polygon=False):
    38.  
    39.         '''
    41.         batch: (image, polygons, ignore_tags
    43.         h_w_list: 包含[h,w]的数组
    45.         pred:
    47.             binary: text region segmentation map, with shape (N, 1,H, W)
    49.         '''
    50.  
    51.         pred = pred[:, 0, :, :]
    52.  
    53.         segmentation = self.binarize(pred)
    54.  
    55.         boxes_batch = []
    56.  
    57.         scores_batch = []
    58.  
    59.         for batch_index in range(pred.shape[0]):
    60.  
    61.             height, width = h_w_list[batch_index]
    62.  
    63.             boxes, scores = self.post_p(pred[batch_index], segmentation[batch_index], width, height,
    64.  
    65.                                         is_output_polygon=is_output_polygon)
    66.  
    67.             boxes_batch.append(boxes)
    68.  
    69.             scores_batch.append(scores)
    70.  
    71.         return boxes_batch, scores_batch
    72.  
    73.  
    74.     def binarize(self, pred):
    75.  
    76.         return pred > self.thresh
    77.  
    78.  
    79.     def post_p(self, pred, bitmap, dest_width, dest_height, is_output_polygon=False):
    80.  
    81.         '''
    83.         _bitmap: single map with shape (H, W),
    85.             whose values are binarized as {0, 1}
    87.         '''
    88.  
    89.         height, width = pred.shape
    90.  
    91.         boxes = []
    92.  
    93.         new_scores = []
    94.  
    95.         # bitmap = bitmap.cpu().numpy()
    96.  
    97.         if cv2.__version__.startswith('3'):
    98.  
    99.             _, contours, _ = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    100.  
    101.         if cv2.__version__.startswith('4'):
    102.  
    103.             contours, _ = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    104.  
    105.         for contour in contours[:self.max_candidates]:
    106.  
    107.             epsilon = 0.005 * cv2.arcLength(contour, True)
    108.  
    109.             approx = cv2.approxPolyDP(contour, epsilon, True)
    110.  
    111.             points = approx.reshape((-1, 2))
    112.  
    113.             if points.shape[0] < 4:
    114.  
    115.                 continue
    116.  
    117.             score = self.box_score_fast(pred, contour.squeeze(1))
    118.  
    119.             if self.box_thresh > score:
    120.  
    121.                 continue
    122.  
    123.             if points.shape[0] > 2:
    124.  
    125.                 box = self.unclip(points, unclip_ratio=self.unclip_ratio)
    126.  
    127.                 if len(box) > 1:
    128.  
    129.                     continue
    130.  
    131.             else:
    132.  
    133.                 continue
    134.  
    135.             four_point_box, sside = self.get_mini_boxes(box.reshape((-1, 1, 2)))
    136.  
    137.             if sside < self.min_size + 2:
    138.  
    139.                 continue
    140.  
    141.             if not isinstance(dest_width, int):
    142.  
    143.                 dest_width = dest_width.item()
    144.  
    145.                 dest_height = dest_height.item()
    146.  
    147.             if not is_output_polygon:
    148.  
    149.                 box = np.array(four_point_box)
    150.  
    151.             else:
    152.  
    153.                 box = box.reshape(-1, 2)
    154.  
    155.             box[:, 0] = np.clip(np.round(box[:, 0] / width * dest_width), 0, dest_width)
    156.  
    157.             box[:, 1] = np.clip(np.round(box[:, 1] / height * dest_height), 0, dest_height)
    158.  
    159.             boxes.append(box)
    160.  
    161.             new_scores.append(score)
    162.  
    163.         return boxes, new_scores
    164.  
    165.  
    166.     def unclip(self, box, unclip_ratio=1.5):
    167.  
    168.         poly = Polygon(box)
    169.  
    170.         distance = poly.area * unclip_ratio / poly.length
    171.  
    172.         offset = pyclipper.PyclipperOffset()
    173.  
    174.         offset.AddPath(box, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)
    175.  
    176.         expanded = np.array(offset.Execute(distance))
    177.  
    178.         return expanded
    179.  
    180.  
    181.     def get_mini_boxes(self, contour):
    182.  
    183.         bounding_box = cv2.minAreaRect(contour)
    184.  
    185.         points = sorted(list(cv2.boxPoints(bounding_box)), key=lambda x: x[0])
    186.  
    187.  
    188.         index_1, index_2, index_3, index_4 = 0, 1, 2, 3
    189.  
    190.         if points[1][1] > points[0][1]:
    191.  
    192.             index_1 = 0
    193.  
    194.             index_4 = 1
    195.  
    196.         else:
    197.  
    198.             index_1 = 1
    199.  
    200.             index_4 = 0
    201.  
    202.         if points[3][1] > points[2][1]:
    203.  
    204.             index_2 = 2
    205.  
    206.             index_3 = 3
    207.  
    208.         else:
    209.  
    210.             index_2 = 3
    211.  
    212.             index_3 = 2
    213.  
    214.  
    215.         box = [points[index_1], points[index_2], points[index_3], points[index_4]]
    216.  
    217.         return box, min(bounding_box[1])
    218.  
    219.  
    220.     def box_score_fast(self, bitmap, _box):
    221.  
    222.         # bitmap = bitmap.detach().cpu().numpy()
    223.  
    224.         h, w = bitmap.shape[:2]
    225.  
    226.         box = _box.copy()
    227.  
    228.         xmin = np.clip(np.floor(box[:, 0].min()).astype(np.int), 0, w - 1)
    229.  
    230.         xmax = np.clip(np.ceil(box[:, 0].max()).astype(np.int), 0, w - 1)
    231.  
    232.         ymin = np.clip(np.floor(box[:, 1].min()).astype(np.int), 0, h - 1)
    233.  
    234.         ymax = np.clip(np.ceil(box[:, 1].max()).astype(np.int), 0, h - 1)
    235.  
    236.  
    237.         mask = np.zeros((ymax - ymin + 1, xmax - xmin + 1), dtype=np.uint8)
    238.  
    239.         box[:, 0] = box[:, 0] - xmin
    240.  
    241.         box[:, 1] = box[:, 1] - ymin
    242.  
    243.         cv2.fillPoly(mask, box.reshape(1, -1, 2).astype(np.int32), 1)
    244.  
    245.         return cv2.mean(bitmap[ymin:ymax + 1, xmin:xmax + 1], mask)[0]
    246.  
    247. def narrow_224_32(image, expected_size=(224,32)):
    248.  
    249.     ih, iw = image.shape[0:2]
    250.  
    251.     ew, eh = expected_size
    252.  
    253.     # scale = eh / ih
    254.  
    255.     scale = min((eh/ih),(ew/iw))
    256.  
    257.     # scale = eh / max(iw,ih)
    258.  
    259.     nh = int(ih * scale)
    260.  
    261.     nw = int(iw * scale)
    262.  
    263.     image = cv2.resize(image, (nw, nh), interpolation=cv2.INTER_CUBIC)
    264.  
    265.  
    266.     top = 0
    267.  
    268.     bottom = eh - nh
    269.  
    270.     left = 0
    271.  
    272.     right = ew - nw
    273.  
    274.  
    275.     new_img = cv2.copyMakeBorder(image, top, bottom, left, right, cv2.BORDER_CONSTANT, value=(114, 114, 114))
    276.  
    277.     return image,new_img
    278.  
    279. def draw_bbox(img_path, result, color=(0, 0, 255), thickness=2):
    280.  
    281.     import cv2
    282.  
    283.     if isinstance(img_path, str):
    284.  
    285.         img_path = cv2.imread(img_path)
    286.  
    287.         # img_path = cv2.cvtColor(img_path, cv2.COLOR_BGR2RGB)
    288.  
    289.     img_path = img_path.copy()
    290.  
    291.     for point in result:
    292.  
    293.         point = point.astype(int)
    294.  
    295.         cv2.polylines(img_path, [point], True, color, thickness)
    296.  
    297.     return img_path
    298.  
    299.  
    300. if __name__ == '__main__':
    301.  
    302.     onnx_model = onnxruntime.InferenceSession("detect_model_small.onnx")
    303.  
    304.     input_name = onnx_model.get_inputs()[0].name
    305.  
    306.     # Set inputs
    307.  
    308.     img = cv2.imread('./pic/6.jpg')
    309.  
    310.     img0 , image= narrow_224_32(img,expected_size=(640,640))
    311.  
    312.    
    313.  
    314.     transform_totensor = transforms.ToTensor()
    315.  
    316.     tensor=transform_totensor(image)
    317.  
    318.     tensor_nor=transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    319.  
    320.     tensor=tensor_nor(tensor)
    321.  
    322.     tensor = np.array(tensor,dtype=np.float32).reshape(1,3,640,640)
    323.  
    324.  
    325.     post_proess = DBPostProcess()
    326.  
    327.     is_output_polygon = False
    328.  
    329.     #run
    330.  
    331.     outputs = onnx_model.run(None, {input_name:tensor})
    332.  
    333.     #post process
    334.  
    335.     feat_2 = torch.from_numpy(outputs[0])
    336.  
    337.     print(feat_2.size())
    338.  
    339.     box_list, score_list = post_proess(outputs[0], [image.shape[:2]], is_output_polygon=is_output_polygon)
    340.  
    341.     box_list, score_list = box_list[0], score_list[0]
    342.  
    343.     if len(box_list) > 0:
    344.  
    345.         idx = [x.sum() > 0 for x in box_list]
    346.  
    347.         box_list = [box_list[i] for i, v in enumerate(idx) if v]
    348.  
    349.         score_list = [score_list[i] for i, v in enumerate(idx) if v]
    350.  
    351.     else:
    352.  
    353.         box_list, score_list = [], []
    354.  
    355.     print("-----------------box list",box_list)
    356.  
    357.     img = draw_bbox(image, box_list)
    358.  
    359.     img = img[0:img0.shape[0],0:img0.shape[1]]
    360.  
    361.     print("============save pic")
    362.  
    363.     img1=np.array(img,dtype=np.uint8).reshape(640,640,3)
    364.  
    365.     cv2.imwrite("img.jpg",img1)
    366.  
    367.     cv2.waitKey()

    文字识别

    onnx模型导出

    1. #!/usr/bin/env python3
    2.  
    3. import os
    4.  
    5. import sys
    6.  
    7. import pathlib
    8.  
    9. # 将 torchocr路径加到python路径里
    10.  
    11. __dir__ = pathlib.Path(os.path.abspath(__file__))
    12.  
    13. import numpy as np
    14.  
    15. sys.path.append(str(__dir__))
    16.  
    17. sys.path.append(str(__dir__.parent.parent))
    18.  
    19. import torch
    20.  
    21. from torchocr.networks import build_model
    22.  
    23.  
    24. MODEL_PATH='./model/ch_rec_moblie_crnn_mbv3.pth'
    25.  
    26. DEVICE='cuda:0' if torch.cuda.is_available() else 'cpu'
    27.  
    28. print("-----------------------devices",DEVICE)
    29.  
    30.  
    31. class RecInfer:
    32.  
    33.     def __init__(self, model_path, batch_size=1):
    34.  
    35.         ckpt = torch.load(model_path, map_location=DEVICE)
    36.  
    37.         cfg = ckpt['cfg']
    38.  
    39.         self.model = build_model(cfg['model'])
    40.  
    41.         state_dict = {}
    42.  
    43.         for k, v in ckpt['state_dict'].items():
    44.  
    45.             state_dict[k.replace('module.', '')] = v
    46.  
    47.         self.model.load_state_dict(state_dict)
    48.  
    49.         self.batch_size = batch_size
    50.  
    51.         self.device = torch.device(DEVICE)
    52.  
    53.         self.model.to(self.device)
    54.  
    55.         self.model.eval()
    56.  
    57. # Prepare input tensor
    58.  
    59.         input = torch.randn(1, 3, 32, 224, requires_grad=False).float().to(torch.device(DEVICE))
    60.  
    61. # Export the torch model as onnx
    62.  
    63.         print("-------------------export")
    64.  
    65.         torch.onnx.export(self.model,
    66.  
    67.             input,
    68.  
    69.             'rego_model_small.onnx',
    70.  
    71.             export_params=True,
    72.  
    73.             opset_version=12,
    74.  
    75.             do_constant_folding=False)
    76.  
    77.        
    78.  
    79. # Load the pretrained model and export it as onnx
    80.  
    81. model = RecInfer(MODEL_PATH)

    验证

    1. import onnxruntime
    2.  
    3. import numpy as np
    4.  
    5. import cv2
    6.  
    7. import torch
    8.  
    9. DEVICE='cuda:0' if torch.cuda.is_available() else 'cpu'
    10.  
    11. IMG_WIDTH=448
    12.  
    13. ONNX_MODEL='./onnx_model/repvgg_s.onnx'
    14.  
    15. LABEL_FILE='/root/autodl-tmp/warren/PytorchOCR_OLD/torchocr/datasets/alphabets/dict_text.txt'
    16.  
    17. #ONNX_MODEL='./onnx_model/rego_model_small.onnx'
    18.  
    19. #LABEL_FILE='/root/autodl-tmp/warren/PytorchOCR_OLD/torchocr/datasets/alphabets/ppocr_keys_v1.txt'
    20.  
    21. PIC='./pic/img.jpg'
    22.  
    23.  
    24. class CTCLabelConverter(object):
    25.  
    26.     """ Convert between text-label and text-index """
    27.  
    28.  
    29.     def __init__(self, character):
    30.  
    31.         # character (str): set of the possible characters.
    32.  
    33.         dict_character = []
    34.  
    35.         with open(character, "rb") as fin:
    36.  
    37.             lines = fin.readlines()
    38.  
    39.             for line in lines:
    40.  
    41.                 line = line.decode('utf-8').strip("\n").strip("\r\n")
    42.  
    43.                 dict_character += list(line)
    44.  
    45.         self.dict = {}
    46.  
    47.         for i, char in enumerate(dict_character):
    48.  
    49.             # NOTE: 0 is reserved for 'blank' token required by CTCLoss
    50.  
    51.             self.dict[char] = i + 1
    52.  
    53.         #TODO replace ‘ ’ with special symbol
    54.  
    55.         self.character = ['[blank]'] + dict_character+[' ']  # dummy '[blank]' token for CTCLoss (index 0)
    56.  
    57.  
    58.     def decode(self, preds, raw=False):
    59.  
    60.         """ convert text-index into text-label. """
    61.  
    62.         preds_idx = preds.argmax(axis=2)
    63.  
    64.         preds_prob = preds.max(axis=2)
    65.  
    66.         result_list = []
    67.  
    68.         for word, prob in zip(preds_idx, preds_prob):
    69.  
    70.             if raw:
    71.  
    72.                 result_list.append((''.join([self.character[int(i)] for i in word]), prob))
    73.  
    74.             else:
    75.  
    76.                 result = []
    77.  
    78.                 conf = []
    79.  
    80.                 for i, index in enumerate(word):
    81.  
    82.                     if word[i] != 0 and (not (i > 0 and word[i - 1] == word[i])):
    83.  
    84.                         result.append(self.character[int(index)])
    85.  
    86.                         conf.append(prob[i])
    87.  
    88.                 result_list.append((''.join(result), conf))
    89.  
    90.         return result_list
    91.  
    92.    
    93.  
    94. def decode(preds, raw=False):
    95.  
    96.     """ convert text-index into text-label. """
    97.  
    98.     dict_character = []
    99.  
    100.     dict = {}
    101.  
    102.     character=LABEL_FILE
    103.  
    104.     with open(character, "rb") as fin:
    105.  
    106.         lines = fin.readlines()
    107.  
    108.         for line in lines:
    109.  
    110.             line = line.decode('utf-8').strip("\n").strip("\r\n")
    111.  
    112.             dict_character += list(line)
    113.  
    114.     for i, char in enumerate(dict_character):
    115.  
    116.         # NOTE: 0 is reserved for 'blank' token required by CTCLoss
    117.  
    118.         dict[char] = i + 1
    119.  
    120.         #TODO replace ‘ ’ with special symbol
    121.  
    122.         character = ['[blank]'] + dict_character+[' ']  # dummy '[blank]' token for CTCLoss (index 0)
    123.  
    124.     preds_idx = preds.argmax(axis=2)
    125.  
    126.     preds_prob = preds.max(axis=2)
    127.  
    128.     result_list = []
    129.  
    130.     for word, prob in zip(preds_idx, preds_prob):
    131.  
    132.         if raw:
    133.  
    134.             result_list.append((''.join([character[int(i)] for i in word]), prob))
    135.  
    136.         else:
    137.  
    138.             result = []
    139.  
    140.             conf = []
    141.  
    142.             for i, index in enumerate(word):
    143.  
    144.                 if word[i] != 0 and (not (i > 0 and word[i - 1] == word[i])):
    145.  
    146.                     result.append(character[int(index)])
    147.  
    148.                     conf.append(prob[i])
    149.  
    150.             result_list.append((''.join(result), conf))
    151.  
    152.     return result_list
    153.  
    154.  
    155. def width_pad_img(_img, _target_width, _pad_value=0):
    156.  
    157.         _height, _width, _channels = _img.shape
    158.  
    159.         to_return_img = np.ones([_height, _target_width, _channels], dtype=_img.dtype) * _pad_value
    160.  
    161.         to_return_img[:_height, :_width, :] = _img
    162.  
    163.         return to_return_img
    164.  
    165. def resize_with_specific_height(_img):
    166.  
    167.         resize_ratio = 32 / _img.shape[0]
    168.  
    169.         return cv2.resize(_img, (0, 0), fx=resize_ratio, fy=resize_ratio, interpolation=cv2.INTER_LINEAR)
    170.  
    171. def normalize_img(_img):
    172.  
    173.         return (_img.astype(np.float32) / 255 - 0.5) / 0.5
    174.  
    175.  
    176. if __name__ == '__main__':
    177.  
    178.     onnx_model = onnxruntime.InferenceSession(ONNX_MODEL)
    179.  
    180.     input_name = onnx_model.get_inputs()[0].name
    181.  
    182.     # Set inputs
    183.  
    184.     imgs = cv2.imread(PIC)
    185.  
    186.     if not isinstance(imgs,list):
    187.  
    188.         imgs = [imgs]
    189.  
    190.     imgs = [normalize_img(resize_with_specific_height(img)) for img in imgs]
    191.  
    192.     widths = np.array([img.shape[1] for img in imgs])
    193.  
    194.     idxs = np.argsort(widths)
    195.  
    196.     txts = []
    197.  
    198.     label_convert=CTCLabelConverter(LABEL_FILE)
    199.  
    200.     for idx in range(len(imgs)):
    201.  
    202.         batch_idxs = idxs[idx:min(len(imgs), idx+1)]
    203.  
    204.         batch_imgs = [width_pad_img(imgs[idx],IMG_WIDTH) for idx in batch_idxs]
    205.  
    206.         batch_imgs = np.stack(batch_imgs)
    207.  
    208.         print(batch_imgs.shape)
    209.  
    210.         tensor =batch_imgs.transpose([0,3, 1, 2]).astype(np.float32)
    211.  
    212.         out = onnx_model.run(None, {input_name:tensor})
    213.  
    214.         tensor_out = torch.tensor(out)
    215.  
    216.         tensor_out = torch.squeeze(tensor_out,dim=1)
    217.  
    218.         softmax_output = tensor_out.softmax(dim=2)
    219.  
    220.         print("---------------out shape is",softmax_output.shape)
    221.  
    222.         txts.extend([label_convert.decode(np.expand_dims(txt, 0)) for txt in softmax_output])
    223.  
    224.     idxs = np.argsort(idxs)
    225.  
    226.     out_txts = [txts[idx] for idx in idxs]
    227.  
    228.     import sys
    229.  
    230.     import codecs
    231.  
    232.  
    233.     sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach())
    234.  
    235.     print(out_txts)

    至此 导出验证成功

    rk3588板端部署

    转化为rknn模型

    1. from rknn.api import RKNN
    2.  
    3. ONNX_MODEL = 'xxx.onnx'
    4. RKNN_MODEL = 'xxxx.rknn'
    5. DATASET = './dataset.txt'
    6.  
    7. if __name__ == '__main__':
    8.  
    9.     # Create RKNN object
    10.     rknn = RKNN(verbose=True)
    11.  
    12.     # pre-process config
    13.     print('--> Config model')
    14.     ret=rknn.config(mean_values=[[0, 0, 0]], std_values=[[0, 0, 0]],target_platform='rk3588')  #wzw
    15.     if ret != 0:
    16.         print('config model failed!')
    17.         exit(ret)
    18.     print('done')
    19.  
    20.     # Load ONNX model
    21.     print('--> Loading model')
    22.     ret = rknn.load_onnx(model=ONNX_MODEL, outputs=['output', '345', '346'])  
    23.     if ret != 0:
    24.         print('Load model failed!')
    25.         exit(ret)
    26.     print('done')
    27.  
    28.     # Build model
    29.     print('--> Building model')
    30.     ret = rknn.build(do_quantization=True, dataset=DATASET)
    31.     #ret = rknn.build(do_quantization=False)
    32.     if ret != 0:
    33.         print('Build model failed!')
    34.         exit(ret)
    35.     print('done')
    36.  
    37.     # Export RKNN model
    38.     print('--> Export rknn model')
    39.     ret = rknn.export_rknn(RKNN_MODEL)
    40.     if ret != 0:
    41.         print('Export rknn model failed!')
    42.         exit(ret)
    43.     print('done')
    44.     #release rknn
    45.     rknn.release()

    使用pyqt进行开发

    PyQt软件设计

    使用pyqt进行开发，ui界面如图所示

    UI

    图6. 基于PYQT的ui界面

    该界面包含了三个功能按钮，其中包裹一个选择静态图片，一个使用相机，一个检测按钮，TextEdit用于显示识别结果，label用于显示处理完成后的图片。

    软件流程图如下：

    总体目录参照

    下面依次介绍图片检测的相关代码：

    1. import platform
    2.  
    3. import sys
    4.  
    5. import cv2
    6.  
    7. import numpy as np
    8.  
    9. import torch
    10.  
    11. import pyclipper
    12.  
    13. from shapely.geometry import Polygon
    14.  
    15. from torchvision import transforms
    16.  
    17. import time
    18.  
    19. import os
    20.  
    21. import glob
    22.  
    23. import threading
    24.  
    25. from PyQt5.QtGui import *
    26.  
    27. from PyQt5.QtWidgets import *
    28.  
    29. from PyQt5.QtCore import *
    30.  
    31. import platform
    32.  
    33. from rknnlite.api import RKNNLite
    34.  
    35. import os
    36.  
    37. os.environ.pop("QT_QPA_PLATFORM_PLUGIN_PATH")
    38.  
    39. DETECT_MODEL = './model/model_small.rknn'
    40.  
    41. REGO_MODEL='./model/repvgg_s.rknn'
    42.  
    43. LABEL_FILE='./dict/dict_text.txt'
    44.  
    45. LABEL_SIZE_PRIVIOUS=0
    46.  
    47. LABEL_SIZE_LATTER=0
    48.  
    49. # 文件夹路径
    50.  
    51. folder_path = './crop_pic'
    52.  
    53. # 使用 glob 来获取所有图片文件的路径
    54.  
    55. image_files = glob.glob(os.path.join(folder_path, '*.png')) + glob.glob(os.path.join(folder_path, '*.jpg'))
    56.  
    57.  
    58. def  resize_img_self(image,reszie_size=(0,0)):
    59.  
    60.     ih,iw=image.shape[0:2]
    61.  
    62.     ew,eh=reszie_size
    63.  
    64.     scale=eh/ih
    65.  
    66.     width=int(iw*scale)
    67.  
    68.     height=int(ih*scale)
    69.  
    70.     if height!=eh:
    71.  
    72.         height=eh
    73.  
    74.     image=cv2.resize(image,(width,height),interpolation=cv2.INTER_LINEAR)
    75.  
    76.     top = 0
    77.  
    78.     bottom = 0
    79.  
    80.     left = 0
    81.  
    82.     right = ew-width
    83.  
    84.     new_img = cv2.copyMakeBorder(image, top, bottom, left, right, cv2.BORDER_CONSTANT, value=(114, 114, 114))
    85.  
    86.     #print("new image shape",new_img.shape)
    87.  
    88.     return new_img
    89.  
    90. def narrow_224_32(image, expected_size=(224,32)):
    91.  
    92.     ih, iw = image.shape[0:2]
    93.  
    94.     ew, eh = expected_size
    95.  
    96.     # scale = eh / ih
    97.  
    98.     scale = min((eh/ih),(ew/iw))
    99.  
    100.     # scale = eh / max(iw,ih)
    101.  
    102.     nh = int(ih * scale)
    103.  
    104.     nw = int(iw * scale)
    105.  
    106.     image = cv2.resize(image, (nw, nh), interpolation=cv2.INTER_CUBIC)
    107.  
    108.  
    109.     top = 0
    110.  
    111.     bottom = eh - nh
    112.  
    113.     left = 0
    114.  
    115.     right = ew - nw
    116.  
    117.  
    118.     new_img = cv2.copyMakeBorder(image, top, bottom, left, right, cv2.BORDER_CONSTANT, value=(114, 114, 114))
    119.  
    120.     return image,new_img
    121.  
    122. def draw_bbox(img_path, result, color=(0, 0, 255), thickness=2):
    123.  
    124.     import cv2
    125.  
    126.     if isinstance(img_path, str):
    127.  
    128.         img_path = cv2.imread(img_path)
    129.  
    130.         # img_path = cv2.cvtColor(img_path, cv2.COLOR_BGR2RGB)
    131.  
    132.     img_path = img_path.copy()
    133.  
    134.     for point in result:
    135.  
    136.         point = point.astype(int)
    137.  
    138.         cv2.polylines(img_path, [point], True, color, thickness)
    139.  
    140.     return img_path
    141.  
    142. def delay_milliseconds(milliseconds):
    143.  
    144.     seconds = milliseconds / 1000.0
    145.  
    146.     time.sleep(seconds)
    147.  
    148.  
    149. """ Convert between text-label and text-index """
    150.  
    151. class CTCLabelConverter(object):
    152.  
    153.     def __init__(self, character):
    154.  
    155.         # character (str): set of the possible characters.
    156.  
    157.         dict_character = []
    158.  
    159.         with open(character, "rb") as fin:
    160.  
    161.             lines = fin.readlines()
    162.  
    163.             for line in lines:
    164.  
    165.                 line = line.decode('utf-8').strip("\n").strip("\r\n")
    166.  
    167.                 dict_character += list(line)
    168.  
    169.         self.dict = {}
    170.  
    171.         for i, char in enumerate(dict_character):
    172.  
    173.             # NOTE: 0 is reserved for 'blank' token required by CTCLoss
    174.  
    175.             self.dict[char] = i + 1
    176.  
    177.         #TODO replace ‘ ’ with special symbol
    178.  
    179.         self.character = ['[blank]'] + dict_character+[' ']  # dummy '[blank]' token for CTCLoss (index 0)
    180.  
    181.  
    182.     def decode(self, preds, raw=False):
    183.  
    184.         """ convert text-index into text-label. """
    185.  
    186.         preds_idx = preds.argmax(axis=2)
    187.  
    188.         preds_prob = preds.max(axis=2)
    189.  
    190.         result_list = []
    191.  
    192.         for word, prob in zip(preds_idx, preds_prob):
    193.  
    194.             if raw:
    195.  
    196.                 result_list.append((''.join([self.character[int(i)] for i in word]), prob))
    197.  
    198.             else:
    199.  
    200.                 result = []
    201.  
    202.                 conf = []
    203.  
    204.                 for i, index in enumerate(word):
    205.  
    206.                     if word[i] != 0 and (not (i > 0 and word[i - 1] == word[i])):
    207.  
    208.                         result.append(self.character[int(index)])
    209.  
    210.                         #conf.append(prob[i])
    211.  
    212.                 #result_list.append((''.join(result), conf))
    213.  
    214.                 result_list.append((''.join(result)))
    215.  
    216.         return result_list
    217.  
    218. class DBPostProcess():
    219.  
    220.     def __init__(self, thresh=0.3, box_thresh=0.7, max_candidates=1000, unclip_ratio=2):
    221.  
    222.         self.min_size = 3
    223.  
    224.         self.thresh = thresh
    225.  
    226.         self.box_thresh = box_thresh
    227.  
    228.         self.max_candidates = max_candidates
    229.  
    230.         self.unclip_ratio = unclip_ratio
    231.  
    232.  
    233.     def __call__(self, pred, h_w_list, is_output_polygon=False):
    234.  
    235.         pred = pred[:, 0, :, :]
    236.  
    237.         segmentation = self.binarize(pred)
    238.  
    239.         boxes_batch = []
    240.  
    241.         scores_batch = []
    242.  
    243.         for batch_index in range(pred.shape[0]):
    244.  
    245.             height, width = h_w_list[batch_index]
    246.  
    247.             boxes, scores = self.post_p(pred[batch_index], segmentation[batch_index], width, height,
    248.  
    249.                                         is_output_polygon=is_output_polygon)
    250.  
    251.             boxes_batch.append(boxes)
    252.  
    253.             scores_batch.append(scores)
    254.  
    255.         return boxes_batch, scores_batch
    256.  
    257.  
    258.     def binarize(self, pred):
    259.  
    260.         return pred > self.thresh
    261.  
    262.  
    263.     def post_p(self, pred, bitmap, dest_width, dest_height, is_output_polygon=False):
    264.  
    265.         '''
    267.         _bitmap: single map with shape (H, W),
    269.             whose values are binarized as {0, 1}
    271.         '''
    272.  
    273.         height, width = pred.shape
    274.  
    275.         boxes = []
    276.  
    277.         new_scores = []
    278.  
    279.         # bitmap = bitmap.cpu().numpy()
    280.  
    281.         if cv2.__version__.startswith('3'):
    282.  
    283.             _, contours, _ = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    284.  
    285.         if cv2.__version__.startswith('4'):
    286.  
    287.             contours, _ = cv2.findContours((bitmap * 255).astype(np.uint8), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    288.  
    289.         for contour in contours[:self.max_candidates]:
    290.  
    291.             epsilon = 0.005 * cv2.arcLength(contour, True)
    292.  
    293.             approx = cv2.approxPolyDP(contour, epsilon, True)
    294.  
    295.             points = approx.reshape((-1, 2))
    296.  
    297.             if points.shape[0] < 4:
    298.  
    299.                 continue
    300.  
    301.             score = self.box_score_fast(pred, contour.squeeze(1))
    302.  
    303.             if self.box_thresh > score:
    304.  
    305.                 continue
    306.  
    307.             if points.shape[0] > 2:
    308.  
    309.                 box = self.unclip(points, unclip_ratio=self.unclip_ratio)
    310.  
    311.                 if len(box) > 1:
    312.  
    313.                     continue
    314.  
    315.             else:
    316.  
    317.                 continue
    318.  
    319.             four_point_box, sside = self.get_mini_boxes(box.reshape((-1, 1, 2)))
    320.  
    321.             if sside < self.min_size + 2:
    322.  
    323.                 continue
    324.  
    325.             if not isinstance(dest_width, int):
    326.  
    327.                 dest_width = dest_width.item()
    328.  
    329.                 dest_height = dest_height.item()
    330.  
    331.             if not is_output_polygon:
    332.  
    333.                 box = np.array(four_point_box)
    334.  
    335.             else:
    336.  
    337.                 box = box.reshape(-1, 2)
    338.  
    339.             box[:, 0] = np.clip(np.round(box[:, 0] / width * dest_width), 0, dest_width)
    340.  
    341.             box[:, 1] = np.clip(np.round(box[:, 1] / height * dest_height), 0, dest_height)
    342.  
    343.             boxes.append(box)
    344.  
    345.             new_scores.append(score)
    346.  
    347.         return boxes, new_scores
    348.  
    349.  
    350.     def unclip(self, box, unclip_ratio=1.5):
    351.  
    352.         poly = Polygon(box)
    353.  
    354.         distance = poly.area * unclip_ratio / poly.length
    355.  
    356.         offset = pyclipper.PyclipperOffset()
    357.  
    358.         offset.AddPath(box, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)
    359.  
    360.         expanded = np.array(offset.Execute(distance))
    361.  
    362.         return expanded
    363.  
    364.  
    365.     def get_mini_boxes(self, contour):
    366.  
    367.         bounding_box = cv2.minAreaRect(contour)
    368.  
    369.         points = sorted(list(cv2.boxPoints(bounding_box)), key=lambda x: x[0])
    370.  
    371.  
    372.         index_1, index_2, index_3, index_4 = 0, 1, 2, 3
    373.  
    374.         if points[1][1] > points[0][1]:
    375.  
    376.             index_1 = 0
    377.  
    378.             index_4 = 1
    379.  
    380.         else:
    381.  
    382.             index_1 = 1
    383.  
    384.             index_4 = 0
    385.  
    386.         if points[3][1] > points[2][1]:
    387.  
    388.             index_2 = 2
    389.  
    390.             index_3 = 3
    391.  
    392.         else:
    393.  
    394.             index_2 = 3
    395.  
    396.             index_3 = 2
    397.  
    398.  
    399.         box = [points[index_1], points[index_2], points[index_3], points[index_4]]
    400.  
    401.         return box, min(bounding_box[1])
    402.  
    403.  
    404.     def box_score_fast(self, bitmap, _box):
    405.  
    406.         # bitmap = bitmap.detach().cpu().numpy()
    407.  
    408.         h, w = bitmap.shape[:2]
    409.  
    410.         box = _box.copy()
    411.  
    412.         xmin = np.clip(np.floor(box[:, 0].min()).astype(np.int), 0, w - 1)
    413.  
    414.         xmax = np.clip(np.ceil(box[:, 0].max()).astype(np.int), 0, w - 1)
    415.  
    416.         ymin = np.clip(np.floor(box[:, 1].min()).astype(np.int), 0, h - 1)
    417.  
    418.         ymax = np.clip(np.ceil(box[:, 1].max()).astype(np.int), 0, h - 1)
    419.  
    420.  
    421.         mask = np.zeros((ymax - ymin + 1, xmax - xmin + 1), dtype=np.uint8)
    422.  
    423.         box[:, 0] = box[:, 0] - xmin
    424.  
    425.         box[:, 1] = box[:, 1] - ymin
    426.  
    427.         cv2.fillPoly(mask, box.reshape(1, -1, 2).astype(np.int32), 1)
    428.  
    429.         return cv2.mean(bitmap[ymin:ymax + 1, xmin:xmax + 1], mask)[0]
    430.  
    431. class Process_Class(QWidget):
    432.  
    433.     detect_end = pyqtSignal(str)
    434.  
    435.     clear_text = pyqtSignal()
    436.  
    437.     def __init__(self):
    438.  
    439.         super().__init__()
    440.  
    441.         self.image = None
    442.  
    443.         self.img=None
    444.  
    445.         self.camera_status=False
    446.  
    447.         self.result_string=None
    448.  
    449.         self.cap = cv2.VideoCapture()
    450.  
    451.         #detect
    452.  
    453.         rknn_model_detect = DETECT_MODEL
    454.  
    455.         self.rknn_lite_detect = RKNNLite()
    456.  
    457.         self.rknn_lite_detect.load_rknn(rknn_model_detect)# load RKNN model
    458.  
    459.         self.rknn_lite_detect.init_runtime(core_mask=RKNNLite.NPU_CORE_2)# init runtime environment
    460.  
    461.         #rego
    462.  
    463.         rknn_model_rego = REGO_MODEL
    464.  
    465.         self.rknn_lite_rego = RKNNLite()
    466.  
    467.         self.rknn_lite_rego.load_rknn(rknn_model_rego)# load RKNN model
    468.  
    469.         self.rknn_lite_rego.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1)# init runtime environment
    470.  
    471.         self.detect_end.connect(self.update_text_box)
    472.  
    473.         self.clear_text.connect(self.clear_text_box)
    474.  
    475.     def cv2_to_qpixmap(self, cv_image):
    476.  
    477.         height, width, channel = cv_image.shape
    478.  
    479.         bytes_per_line = 3 * width
    480.  
    481.         q_image = QImage(cv_image.data, width, height, bytes_per_line, QImage.Format_RGB888).rgbSwapped()
    482.  
    483.         return QPixmap.fromImage(q_image)
    484.  
    485.     def show_pic(self, cv_image):
    486.  
    487.         pixmap = self.cv2_to_qpixmap(cv_image)
    488.  
    489.         if MainWindow.pic_label is not None:
    490.  
    491.             MainWindow.pic_label.setPixmap(pixmap)
    492.  
    493.             QApplication.processEvents()
    494.  
    495.         else:
    496.  
    497.             print("wrong!!!!!!!")
    498.  
    499.     def camera_open(self):
    500.  
    501.         self.camera_status = not self.camera_status
    502.  
    503.         print("------------camera status is",self.camera_status)
    504.  
    505.         if  self.camera_status:
    506.  
    507.             self.cap.open(12)
    508.  
    509.             if  self.cap.isOpened():
    510.  
    511.                 print("run camera")
    512.  
    513.                 while(True):
    514.  
    515.                     frame = self.cap.read()
    516.  
    517.                     if not frame[0]:
    518.  
    519.                         print("read frame failed!!!!")
    520.  
    521.                         exit()
    522.  
    523.                     self.image=frame[1]
    524.  
    525.                     self.detect_pic()
    526.  
    527.                     if not self.camera_status:  
    528.  
    529.                         break
    530.  
    531.             else:
    532.  
    533.                 print("Cannot open camera")
    534.  
    535.                 exit()
    536.  
    537.         else:
    538.  
    539.             self.release_camera()
    540.  
    541.     def release_camera(self):
    542.  
    543.         if self.cap.isOpened():
    544.  
    545.             self.cap.release()
    546.  
    547.             self.camera_status = False
    548.  
    549.             print("摄像头关闭")
    550.  
    551.     def open_file(self):
    552.  
    553.         # 获取图像的路径
    554.  
    555.         img_path, _ = QFileDialog.getOpenFileName()
    556.  
    557.         if img_path != '':
    558.  
    559.             self.image = cv2.imread(img_path)
    560.  
    561.         self.show_pic(self.image)
    562.  
    563.     def crop_and_save_image(self,image, box_points):
    564.  
    565.         global LABEL_SIZE_PRIVIOUS
    566.  
    567.         global LABEL_SIZE_LATTER
    568.  
    569.         i=-1
    570.  
    571.         # 将box_points转换为NumPy数组，并取整数值
    572.  
    573.         box_points = np.array(box_points, dtype=np.int32)
    574.  
    575.         mask = np.zeros_like(image)  # 创建与图像相同大小的全黑图像
    576.  
    577.         print("LABEL_SIZE_PRIVIOUS ",LABEL_SIZE_PRIVIOUS,"LABEL_SIZE_LATTER ",LABEL_SIZE_LATTER)
    578.  
    579.         if LABEL_SIZE_PRIVIOUS==LABEL_SIZE_LATTER:
    580.  
    581.             LABEL_SIZE_PRIVIOUS=len(box_points)
    582.  
    583.             for box_point in box_points:
    584.  
    585.                 i=i+1
    586.  
    587.                 cropped_image = image.copy()
    588.  
    589.                 # 使用OpenCV的函数裁剪图像
    590.  
    591.                 x, y, w, h = cv2.boundingRect(box_point)
    592.  
    593.                 cropped_image = image[y:y+h, x:x+w]
    594.  
    595.                 # 创建与图像大小相同的全黑掩码
    596.  
    597.                 mask = np.zeros_like(cropped_image)
    598.  
    599.                 # 在掩码上绘制多边形
    600.  
    601.                 cv2.fillPoly(mask, [box_point - (x, y)], (255, 255, 255))
    602.  
    603.                 # 使用 bitwise_and 进行图像裁剪
    604.  
    605.                 masked_cropped_image = cv2.bitwise_and(cropped_image, mask) 
    606.  
    607.                 # 保存裁剪后的图像
    608.  
    609.                 output_path = f"{'./crop_pic/'}img_{i}.jpg"
    610.  
    611.                 cv2.imwrite(output_path, masked_cropped_image)
    612.  
    613.         else:
    614.  
    615.             #self.clear_text.emit()
    616.  
    617.             LABEL_SIZE_LATTER=LABEL_SIZE_PRIVIOUS
    618.  
    619.             current_directory = os.getcwd()+'/crop_pic'  # Get the current directory
    620.  
    621.             for filename in os.listdir(current_directory):
    622.  
    623.                 if filename.endswith(".jpg"):
    624.  
    625.                     file_path = os.path.join(current_directory, filename)
    626.  
    627.                     os.remove(file_path)
    628.  
    629.                     print(f"Deleted: {file_path}")
    630.  
    631.     def detect_thread(self):
    632.  
    633.         #detect inference
    634.  
    635.         img0 , image= narrow_224_32(self.image,expected_size=(640,640))
    636.  
    637.         outputs =self.rknn_lite_detect.inference(inputs=[image])
    638.  
    639.         post_proess = DBPostProcess()
    640.  
    641.         is_output_polygon = False
    642.  
    643.         box_list, score_list = post_proess(outputs[0], [image.shape[:2]], is_output_polygon=is_output_polygon)
    644.  
    645.         box_list, score_list = box_list[0], score_list[0]
    646.  
    647.         if len(box_list) > 0:
    648.  
    649.             idx = [x.sum() > 0 for x in box_list]
    650.  
    651.             box_list = [box_list[i] for i, v in enumerate(idx) if v]
    652.  
    653.             score_list = [score_list[i] for i, v in enumerate(idx) if v]
    654.  
    655.         else:
    656.  
    657.             box_list, score_list = [], []
    658.  
    659.         self.image = draw_bbox(image, box_list)
    660.  
    661.         self.crop_and_save_image(image,box_list)
    662.  
    663.         self.image = self.image[0:img0.shape[0],0:img0.shape[1]]
    664.  
    665.         self.show_pic(self.image)
    666.  
    667.     def rego_thread(self):
    668.  
    669.         label_convert=CTCLabelConverter(LABEL_FILE)
    670.  
    671.         self.clear_text.emit()
    672.  
    673.         for image_file in image_files:
    674.  
    675.             if os.path.exists(image_file):
    676.  
    677.                 print('-----------image file',image_file,len(image_files))
    678.  
    679.                 self.img = cv2.imread(image_file)
    680.  
    681.                 image = resize_img_self(self.img,reszie_size=(448,32))
    682.  
    683.                 # Inference
    684.  
    685.                 outputs = self.rknn_lite_rego.inference(inputs=[image])
    686.  
    687.                 #post process
    688.  
    689.                 feat_2 = torch.tensor(outputs[0],dtype=torch.float32)
    690.  
    691.                 txt = label_convert.decode(feat_2.detach().numpy())
    692.  
    693.                 self.result_string = ' '.join(txt)
    694.  
    695.                 print(self.result_string)
    696.  
    697.                 self.detect_end.emit(self.result_string)
    698.  
    699.             else:
    700.  
    701.                 print("-----------no crop image!!!")
    702.  
    703.     def detect_pic(self):
    704.  
    705.         self.detect_thread()
    706.  
    707.         my_thread = threading.Thread(target=self.rego_thread)
    708.  
    709.         # 启动线程
    710.  
    711.         my_thread.start()
    712.  
    713.         # 等待线程结束
    714.  
    715.         my_thread.join()
    716.  
    717.     def update_text_box(self, text):
    718.  
    719.         # 在主线程中更新文本框的内容
    720.  
    721.         MainWindow.text_box.append(text)
    722.  
    723.     def clear_text_box(self):
    724.  
    725.         print("clear--------------------------------")
    726.  
    727.         # 在主线程中更新文本框的内容
    728.  
    729.         MainWindow.text_box.clear()
    730.  
    731.        
    732.  
    733.    
    734.  
    735. class MainWindow(QMainWindow):
    736.  
    737.     #pic_label = None
    738.  
    739.     def __init__(self):
    740.  
    741.         pic_label = None
    742.  
    743.         text_box  = None
    744.  
    745.         super().__init__()
    746.  
    747.         self.process_functions = Process_Class()
    748.  
    749.         self.window = QWidget()
    750.  
    751.         # 创建小部件
    752.  
    753.         self.pic_label = QLabel('Show Window!', parent=self.window)
    754.  
    755.         self.pic_label.setMinimumHeight(500)  # 设置最小高度
    756.  
    757.         self.pic_label.setMaximumHeight(500)  # 设置最大高度
    758.  
    759.         self.pic_button = QPushButton('Picture', parent=self.window)
    760.  
    761.         self.pic_button.clicked.connect(self.process_functions.open_file)
    762.  
    763.         self.camera_button = QPushButton('Camera', parent=self.window)
    764.  
    765.         self.camera_button.clicked.connect(self.process_functions.camera_open)
    766.  
    767.         self.detect_button = QPushButton('Detect', parent=self.window)
    768.  
    769.         self.detect_button.clicked.connect(self.process_functions.detect_pic)
    770.  
    771.         self.text_box = QTextEdit()
    772.  
    773.         # 创建垂直布局管理器并将小部件添加到布局中
    774.  
    775.         self.left_layout = QVBoxLayout()
    776.  
    777.         self.right_layout = QVBoxLayout()
    778.  
    779.         self.layout = QHBoxLayout()
    780.  
    781.         self.create_ui()
    782.  
    783.         self.window.closeEvent = self.closeEvent
    784.  
    785.  
    786.     def create_ui(self):
    787.  
    788.         self.window.setWindowTitle('Scene_text_rego')
    789.  
    790.         self.window.setGeometry(0, 0, 800, 600)  # 设置窗口位置和大小
    791.  
    792.         # 设置主窗口的布局
    793.  
    794.         self.pic_label.setStyleSheet('border: 2px solid black; padding: 10px;')
    795.  
    796.         self.left_layout.addWidget(self.pic_label)
    797.  
    798.         self.left_layout.addWidget(self.text_box)
    799.  
    800.         self.right_layout.addWidget(self.pic_button)
    801.  
    802.         self.right_layout.addWidget(self.camera_button)
    803.  
    804.         self.right_layout.addWidget(self.detect_button)
    805.  
    806.         self.layout.addLayout(self.left_layout)
    807.  
    808.         self.layout.addLayout(self.right_layout)
    809.  
    810.         self.window.setLayout(self.layout)
    811.  
    812.         self.window.show()
    813.  
    814.     def closeEvent(self, event):
    815.  
    816.         # 释放摄像头资源
    817.  
    818.         self.process_functions.release_camera()
    819.  
    820.         event.accept()
    821.  
    822.  
    823. def main():
    824.  
    825.     # 创建应用程序对象
    826.  
    827.     app = QApplication(sys.argv)
    828.  
    829.     win = MainWindow()
    830.  
    831.     MainWindow.pic_label = win.pic_label  # 设置类变量pic_label为MainWindow对象的pic_label
    832.  
    833.     MainWindow.text_box = win.text_box  # 设置类变量pic_label为MainWindow对象的pic_label
    834.  
    835.     # 运行应用程序
    836.  
    837.     sys.exit(app.exec_())
    838.  
    839.     rknn_lite_detect.release()
    840.  
    841.  
    842. if __name__ == '__main__':
    843.  
    844.     main()

    运行结果

     参考资料

    博文：

    【工程部署】手把手教你在RKNN上部署OCR服务（上）_rknn ocr_三叔家的猫的博客-CSDN博客

  • 相关阅读:
    图像处理Laplacian 算子
    Q-REG论文阅读
    CMake Day 7 —— option
    【Java】replace替换方法
    第63篇-解释器与编译器适配（二）
    jenkins搭建
    手动构造感知机模型Perceptron（Numpy代码手写）
    数据库timestamp遇到的小问题
    Docker高级-4.可视化工具Portainer/容器监控之CAdvisor+InfluxDB+Granfana
    Mybatis的collection三层嵌套查询（验证通过）
  • 原文地址：https://blog.csdn.net/warren103098/article/details/134392494