• 项目一：基于YOLOv7的输电线路销钉缺失检测项目

    1. YOLOv7模型介绍

    YOLOv7是目标检测算法YOLO（You Only Look Once）的第七个版本，也是目前较流行的YOLO算法版本之一。

    YOLOv8主要结构：

    1. Backbone网络：采用CSPDarknet53作为主干网络，在不增加参数数量的情况下提高了网络效果。CSPDarknet53使用多层跨层连接（Cross Stage Partial）实现，可以缓解梯度弥散，提高了特征表达能力。

    2. Neck：采用YOLONeck模块作为连接头，用于整合backbone网络输出的不同尺度特征图，提供更丰富的语义信息给下一步的处理。

    3. Head：使用YOLOv7Head模块，具有多任务同时实现物体识别和定位，且充分利用不同分辨率的特征图，使得对不同尺寸物体的检测有更好的性能。

    4. 损失函数：采用YOLOv3采用的损失函数，由于该损失函数在训练过程中可以平衡不同尺寸目标框的权重，使得算法对大、小目标框有更好的检测效果。

    相较于其他YOLO系列算法：

    1. 网络结构：YOLOv7采用了CSPDarknet53作为主干网络，使用多层跨层连接，提高了特征表达能力，同时采用了YOLONeck模块和YOLOv7Head模块，使得对不同尺寸物体的检测有更好的性能。

    2. 数据增强：YOLOv7引入新的数据增强策略，增加训练数据的难度，提高算法的鲁棒性和泛化能力。

    3. 训练策略：YOLOv7使用动态权重更新技术，可根据目标的重要性自适应地调整权重，同时使用注意力机制和最大建模平均池化等技术，提高了检测性能。

    4. 精度和速度：YOLOv7采用上述改进方案，提高了算法的精度和速度，具有更好的鲁棒性和泛化性能。

    YOLOv7在COCO数据集的评测结果为，使用YOLOv7-S模型，测试时使用图像的每个正方形区域都被分为2个子区域的方法，得到的F1值可以达到46.9%，可以满足一些低精度的检测应用。同时，使用YOLOv7母型模型在图像尺寸为608×608的情况下，在COCO最新测试集上，获得的FPS为76.5，同时平均准确率F1值为54.2%，相较于其它目标检测算法，YOLOv7的检测速度和准确度都具有一定优势。

    YOLOv7结构图来源：YOLOV7详细解读（一）网络架构解读

    YOLOv7的主干特征提取网络：

    1. from functools import wraps
    2.  
    3. from tensorflow.keras import backend as K
    4. from tensorflow.keras.initializers import RandomNormal
    5. from tensorflow.keras.layers import (Add, BatchNormalization, Concatenate, Conv2D, Layer,
    6.                           MaxPooling2D, ZeroPadding2D)
    7. from tensorflow.keras.regularizers import l2
    8. from utils.utils import compose
    9.  
    10.  
    11. class SiLU(Layer):
    12.     def __init__(self, **kwargs):
    13.         super(SiLU, self).__init__(**kwargs)
    14.         self.supports_masking = True
    15.  
    16.     def call(self, inputs):
    17.         return inputs * K.sigmoid(inputs)
    18.  
    19.     def get_config(self):
    20.         config = super(SiLU, self).get_config()
    21.         return config
    22.  
    23.     def compute_output_shape(self, input_shape):
    24.         return input_shape
    25.  
    26. @wraps(Conv2D)
    27. def DarknetConv2D(*args, **kwargs):
    28.     darknet_conv_kwargs = {'kernel_initializer' : RandomNormal(stddev=0.02), 'kernel_regularizer' : l2(kwargs.get('weight_decay', 0))}
    29.     darknet_conv_kwargs['padding'] = 'valid' if kwargs.get('strides')==(2, 2) else 'same'   
    30.     try:
    31.         del kwargs['weight_decay']
    32.     except:
    33.         pass
    34.     darknet_conv_kwargs.update(kwargs)
    35.     return Conv2D(*args, **darknet_conv_kwargs)
    36.     
    37. def DarknetConv2D_BN_SiLU(*args, **kwargs):
    38.     no_bias_kwargs = {'use_bias': False}
    39.     no_bias_kwargs.update(kwargs)
    40.     if "name" in kwargs.keys():
    41.         no_bias_kwargs['name'] = kwargs['name'] + '.conv'
    42.     return compose(
    43.         DarknetConv2D(*args, **no_bias_kwargs),
    44.         BatchNormalization(momentum = 0.97, epsilon = 0.001, name = kwargs['name'] + '.bn'),
    45.         SiLU())
    46.  
    47. def Transition_Block(x, c2, weight_decay=5e-4, name = ""):
    48.     x_1 = MaxPooling2D((2, 2), strides=(2, 2))(x)
    49.     x_1 = DarknetConv2D_BN_SiLU(c2, (1, 1), weight_decay=weight_decay, name = name + '.cv1')(x_1)
    50.     
    51.     x_2 = DarknetConv2D_BN_SiLU(c2, (1, 1), weight_decay=weight_decay, name = name + '.cv2')(x)
    52.     x_2 = ZeroPadding2D(((1, 1),(1, 1)))(x_2)
    53.     x_2 = DarknetConv2D_BN_SiLU(c2, (3, 3), strides=(2, 2), weight_decay=weight_decay, name = name + '.cv3')(x_2)
    54.     y = Concatenate(axis=-1)([x_2, x_1])
    55.     return y
    56.  
    57. def Multi_Concat_Block(x, c2, c3, n=4, e=1, ids=[0], weight_decay=5e-4, name = ""):
    58.     c_ = int(c2 * e)
    59.         
    60.     x_1 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv1')(x)
    61.     x_2 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv2')(x)
    62.     
    63.     x_all = [x_1, x_2]
    64.     for i in range(n):
    65.         x_2 = DarknetConv2D_BN_SiLU(c2, (3, 3), weight_decay=weight_decay, name = name + '.cv3.' + str(i))(x_2)
    66.         x_all.append(x_2)
    67.     y = Concatenate(axis=-1)([x_all[id] for id in ids])
    68.     y = DarknetConv2D_BN_SiLU(c3, (1, 1), weight_decay=weight_decay, name = name + '.cv4')(y)
    69.     return y
    70.  
    71. def darknet_body(x, transition_channels, block_channels, n, phi, weight_decay=5e-4):
    72.     ids = {
    73.         'l' : [-1, -3, -5, -6],
    74.         'x' : [-1, -3, -5, -7, -8], 
    75.     }[phi]
    76.  
    77.     x = DarknetConv2D_BN_SiLU(transition_channels, (3, 3), strides = (1, 1), weight_decay=weight_decay, name = 'backbone.stem.0')(x)
    78.     x = ZeroPadding2D(((1, 1),(1, 1)))(x)
    79.     x = DarknetConv2D_BN_SiLU(transition_channels * 2, (3, 3), strides = (2, 2), weight_decay=weight_decay, name = 'backbone.stem.1')(x)
    80.     x = DarknetConv2D_BN_SiLU(transition_channels * 2, (3, 3), strides = (1, 1), weight_decay=weight_decay, name = 'backbone.stem.2')(x)
    81.     
    82.     x = ZeroPadding2D(((1, 1),(1, 1)))(x)
    83.     x = DarknetConv2D_BN_SiLU(transition_channels * 4, (3, 3), strides = (2, 2), weight_decay=weight_decay, name = 'backbone.dark2.0')(x)
    84.     x = Multi_Concat_Block(x, block_channels * 2, transition_channels * 8, n=n, ids=ids, weight_decay=weight_decay, name = 'backbone.dark2.1')
    85.     
    86.     x = Transition_Block(x, transition_channels * 4, weight_decay=weight_decay, name = 'backbone.dark3.0')
    87.     x = Multi_Concat_Block(x, block_channels * 4, transition_channels * 16, n=n, ids=ids, weight_decay=weight_decay, name = 'backbone.dark3.1')
    88.     feat1 = x
    89.     
    90.     x = Transition_Block(x, transition_channels * 8, weight_decay=weight_decay, name = 'backbone.dark4.0')
    91.     x = Multi_Concat_Block(x, block_channels * 8, transition_channels * 32, n=n, ids=ids, weight_decay=weight_decay, name = 'backbone.dark4.1')
    92.     feat2 = x
    93.     
    94.     x = Transition_Block(x, transition_channels * 16, weight_decay=weight_decay, name = 'backbone.dark5.0')
    95.     x = Multi_Concat_Block(x, block_channels * 8, transition_channels * 32, n=n, ids=ids, weight_decay=weight_decay, name = 'backbone.dark5.1')
    96.     feat3 = x
    97.     return feat1, feat2, feat3
    98.

    YOLOv7特征金字塔部分：

    1. import numpy as np
    2. from tensorflow.keras.layers import (Add, BatchNormalization, Concatenate, Conv2D, Input,
    3.                                     Lambda, MaxPooling2D, UpSampling2D)
    4. from tensorflow.keras.models import Model
    5.  
    6. from nets.backbone import (DarknetConv2D, DarknetConv2D_BN_SiLU,
    7.                            Multi_Concat_Block, SiLU, Transition_Block,
    8.                            darknet_body)
    9. from nets.yolo_training import yolo_loss
    10.  
    11.  
    12. def SPPCSPC(x, c2, n=1, shortcut=False, g=1, e=0.5, k=(5, 9, 13), weight_decay=5e-4, name=""):
    13.     c_ = int(2 * c2 * e)  # hidden channels
    14.     x1 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv1')(x)
    15.     x1 = DarknetConv2D_BN_SiLU(c_, (3, 3), weight_decay=weight_decay, name = name + '.cv3')(x1)
    16.     x1 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv4')(x1)
    17.     
    18.     y1 = Concatenate(axis=-1)([x1] + [MaxPooling2D(pool_size=(m, m), strides=(1, 1), padding='same')(x1) for m in k])
    19.     y1 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv5')(y1)
    20.     y1 = DarknetConv2D_BN_SiLU(c_, (3, 3), weight_decay=weight_decay, name = name + '.cv6')(y1)
    21.     
    22.     y2 = DarknetConv2D_BN_SiLU(c_, (1, 1), weight_decay=weight_decay, name = name + '.cv2')(x)
    23.     out = Concatenate(axis=-1)([y1, y2])
    24.     out = DarknetConv2D_BN_SiLU(c2, (1, 1), weight_decay=weight_decay, name = name + '.cv7')(out)
    25.     
    26.     return out
    27.  
    28. def fusion_rep_vgg(fuse_layers, trained_model, infer_model):
    29.     for layer_name, use_bias, use_bn in fuse_layers:
    30.  
    31.         conv_kxk_weights = trained_model.get_layer(layer_name + '.rbr_dense.0').get_weights()[0]
    32.         conv_1x1_weights = trained_model.get_layer(layer_name + '.rbr_1x1.0').get_weights()[0]
    33.  
    34.         if use_bias:
    35.             conv_kxk_bias = trained_model.get_layer(layer_name + '.rbr_dense.0').get_weights()[1]
    36.             conv_1x1_bias = trained_model.get_layer(layer_name + '.rbr_1x1.0').get_weights()[1]
    37.         else:
    38.             conv_kxk_bias = np.zeros((conv_kxk_weights.shape[-1],))
    39.             conv_1x1_bias = np.zeros((conv_1x1_weights.shape[-1],))
    40.  
    41.         if use_bn:
    42.             gammas_kxk, betas_kxk, means_kxk, var_kxk = trained_model.get_layer(layer_name + '.rbr_dense.1').get_weights()
    43.             gammas_1x1, betas_1x1, means_1x1, var_1x1 = trained_model.get_layer(layer_name + '.rbr_1x1.1').get_weights()
    44.  
    45.         else:
    46.             gammas_1x1, betas_1x1, means_1x1, var_1x1 = [np.ones((conv_1x1_weights.shape[-1],)),
    47.                                                          np.zeros((conv_1x1_weights.shape[-1],)),
    48.                                                          np.zeros((conv_1x1_weights.shape[-1],)),
    49.                                                          np.ones((conv_1x1_weights.shape[-1],))]
    50.             gammas_kxk, betas_kxk, means_kxk, var_kxk = [np.ones((conv_kxk_weights.shape[-1],)),
    51.                                                          np.zeros((conv_kxk_weights.shape[-1],)),
    52.                                                          np.zeros((conv_kxk_weights.shape[-1],)),
    53.                                                          np.ones((conv_kxk_weights.shape[-1],))]
    54.         gammas_res, betas_res, means_res, var_res = [np.ones((conv_1x1_weights.shape[-1],)),
    55.                                                      np.zeros((conv_1x1_weights.shape[-1],)),
    56.                                                      np.zeros((conv_1x1_weights.shape[-1],)),
    57.                                                      np.ones((conv_1x1_weights.shape[-1],))]
    58.  
    59.         # _fuse_bn_tensor(self.rbr_dense)
    60.         w_kxk = (gammas_kxk / np.sqrt(np.add(var_kxk, 1e-3))) * conv_kxk_weights
    61.         b_kxk = (((conv_kxk_bias - means_kxk) * gammas_kxk) / np.sqrt(np.add(var_kxk, 1e-3))) + betas_kxk
    62.         
    63.         # _fuse_bn_tensor(self.rbr_dense)
    64.         kernel_size = w_kxk.shape[0]
    65.         in_channels = w_kxk.shape[2]
    66.         w_1x1 = np.zeros_like(w_kxk)
    67.         w_1x1[kernel_size // 2, kernel_size // 2, :, :] = (gammas_1x1 / np.sqrt(np.add(var_1x1, 1e-3))) * conv_1x1_weights
    68.         b_1x1 = (((conv_1x1_bias - means_1x1) * gammas_1x1) / np.sqrt(np.add(var_1x1, 1e-3))) + betas_1x1
    69.  
    70.         w_res = np.zeros_like(w_kxk)
    71.         for i in range(in_channels):
    72.             w_res[kernel_size // 2, kernel_size // 2, i % in_channels, i] = 1
    73.         w_res = ((gammas_res / np.sqrt(np.add(var_res, 1e-3))) * w_res)
    74.         b_res = (((0 - means_res) * gammas_res) / np.sqrt(np.add(var_res, 1e-3))) + betas_res
    75.  
    76.         weight = [w_res, w_1x1, w_kxk]
    77.         bias = [b_res, b_1x1, b_kxk]
    78.         
    79.         infer_model.get_layer(layer_name).set_weights([np.array(weight).sum(axis=0), np.array(bias).sum(axis=0)])
    80.  
    81. def RepConv(x, c2, mode="train", weight_decay=5e-4, name=""):
    82.     if mode == "predict":
    83.         out = DarknetConv2D(c2, (3, 3), name = name, use_bias=True, weight_decay=weight_decay, padding='same')(x)
    84.         out = SiLU()(out)
    85.     elif mode == "train":
    86.         x1 = DarknetConv2D(c2, (3, 3), name = name + '.rbr_dense.0', use_bias=False, weight_decay=weight_decay, padding='same')(x)
    87.         x1 = BatchNormalization(momentum = 0.97, epsilon = 0.001, name = name + '.rbr_dense.1')(x1)
    88.         x2 = DarknetConv2D(c2, (1, 1), name = name + '.rbr_1x1.0', use_bias=False, weight_decay=weight_decay, padding='same')(x)
    89.         x2 = BatchNormalization(momentum = 0.97, epsilon = 0.001, name = name + '.rbr_1x1.1')(x2)
    90.         
    91.         out = Add()([x1, x2])
    92.         out = SiLU()(out)
    93.     return out
    94.  
    95. def yolo_body(input_shape, anchors_mask, num_classes, phi, weight_decay=5e-4, mode="train"):
    96.  
    97.     transition_channels = {'l' : 32, 'x' : 40}[phi]
    98.     block_channels      = 32
    99.     panet_channels      = {'l' : 32, 'x' : 64}[phi]
    100.     e       = {'l' : 2, 'x' : 1}[phi]
    101.     n       = {'l' : 4, 'x' : 6}[phi]
    102.     ids     = {'l' : [-1, -2, -3, -4, -5, -6], 'x' : [-1, -3, -5, -7, -8]}[phi]
    103.  
    104.     inputs      = Input(input_shape)
    105.     feat1, feat2, feat3 = darknet_body(inputs, transition_channels, block_channels, n, phi, weight_decay)
    106.     P5          = SPPCSPC(feat3, transition_channels * 16, weight_decay=weight_decay, name="sppcspc")
    107.     P5_conv     = DarknetConv2D_BN_SiLU(transition_channels * 8, (1, 1), weight_decay=weight_decay, name="conv_for_P5")(P5)
    108.     P5_upsample = UpSampling2D()(P5_conv)
    109.     P4          = Concatenate(axis=-1)([DarknetConv2D_BN_SiLU(transition_channels * 8, (1, 1), weight_decay=weight_decay, name="conv_for_feat2")(feat2), P5_upsample])
    110.     P4          = Multi_Concat_Block(P4, panet_channels * 4, transition_channels * 8, e=e, n=n, ids=ids, weight_decay=weight_decay, name="conv3_for_upsample1")
    111.  
    112.     P4_conv     = DarknetConv2D_BN_SiLU(transition_channels * 4, (1, 1), weight_decay=weight_decay, name="conv_for_P4")(P4)
    113.     P4_upsample = UpSampling2D()(P4_conv)
    114.     P3          = Concatenate(axis=-1)([DarknetConv2D_BN_SiLU(transition_channels * 4, (1, 1), weight_decay=weight_decay, name="conv_for_feat1")(feat1), P4_upsample])
    115.     P3          = Multi_Concat_Block(P3, panet_channels * 2, transition_channels * 4, e=e, n=n, ids=ids, weight_decay=weight_decay, name="conv3_for_upsample2")
    116.         
    117.     P3_downsample = Transition_Block(P3, transition_channels * 4, weight_decay=weight_decay, name="down_sample1")
    118.     P4 = Concatenate(axis=-1)([P3_downsample, P4])
    119.     P4 = Multi_Concat_Block(P4, panet_channels * 4, transition_channels * 8, e=e, n=n, ids=ids, weight_decay=weight_decay, name="conv3_for_downsample1")
    120.  
    121.     P4_downsample = Transition_Block(P4, transition_channels * 8, weight_decay=weight_decay, name="down_sample2")
    122.     P5 = Concatenate(axis=-1)([P4_downsample, P5])
    123.     P5 = Multi_Concat_Block(P5, panet_channels * 8, transition_channels * 16, e=e, n=n, ids=ids, weight_decay=weight_decay, name="conv3_for_downsample2")
    124.     
    125.     if phi == "l":
    126.         P3 = RepConv(P3, transition_channels * 8, mode, weight_decay=weight_decay, name="rep_conv_1")
    127.         P4 = RepConv(P4, transition_channels * 16, mode, weight_decay=weight_decay, name="rep_conv_2")
    128.         P5 = RepConv(P5, transition_channels * 32, mode, weight_decay=weight_decay, name="rep_conv_3")
    129.     else:
    130.         P3 = DarknetConv2D_BN_SiLU(transition_channels * 8, (3, 3), strides=(1, 1), weight_decay=weight_decay, name="rep_conv_1")(P3)
    131.         P4 = DarknetConv2D_BN_SiLU(transition_channels * 16, (3, 3), strides=(1, 1), weight_decay=weight_decay, name="rep_conv_2")(P4)
    132.         P5 = DarknetConv2D_BN_SiLU(transition_channels * 32, (3, 3), strides=(1, 1), weight_decay=weight_decay, name="rep_conv_3")(P5)
    133.  
    134.     out2 = DarknetConv2D(len(anchors_mask[2]) * (5 + num_classes), (1, 1), weight_decay=weight_decay, strides = (1, 1), name = 'yolo_head_P3')(P3)
    135.     out1 = DarknetConv2D(len(anchors_mask[1]) * (5 + num_classes), (1, 1), weight_decay=weight_decay, strides = (1, 1), name = 'yolo_head_P4')(P4)
    136.     out0 = DarknetConv2D(len(anchors_mask[0]) * (5 + num_classes), (1, 1), weight_decay=weight_decay, strides = (1, 1), name = 'yolo_head_P5')(P5)
    137.     return Model(inputs, [out0, out1, out2])
    138.  
    139. def get_train_model(model_body, input_shape, num_classes, anchors, anchors_mask, label_smoothing):
    140.     y_true = [Input(shape = (input_shape[0] // {0:32, 1:16, 2:8}[l], input_shape[1] // {0:32, 1:16, 2:8}[l], \
    141.                                 len(anchors_mask[l]), 2)) for l in range(len(anchors_mask))] + [Input(shape = [None, 5])]
    142.     model_loss  = Lambda(
    143.         yolo_loss, 
    144.         output_shape    = (1, ), 
    145.         name            = 'yolo_loss', 
    146.         arguments       = {
    147.             'input_shape'       : input_shape, 
    148.             'anchors'           : anchors, 
    149.             'anchors_mask'      : anchors_mask, 
    150.             'num_classes'       : num_classes, 
    151.             'label_smoothing'   : label_smoothing, 
    152.             'balance'           : [0.4, 1.0, 4],
    153.             'box_ratio'         : 0.05,
    154.             'obj_ratio'         : 1 * (input_shape[0] * input_shape[1]) / (640 ** 2), 
    155.             'cls_ratio'         : 0.5 * (num_classes / 80)
    156.         }
    157.     )([*model_body.output, *y_true])
    158.     model       = Model([model_body.input, *y_true], model_loss)
    159.     return model

    2. 数据集简介

    数据集包含1200张图像，利用labelimg标注程序对图像中包含的缺陷进行标注，标注销钉正常与销钉缺失两类目标，标签格式为voc标签。两类标签分布：销钉正常789，销钉异常656。

    3. 检测模型训练

    3.1 环境准备

    训练软件环境：scipy==1.4.1；numpy==1.18.4；matplotlib==3.2.1；opencv_python==4.2.0.34

    tensorflow_gpu==2.2.0；tqdm==4.46.1；Pillow==8.2.0；h5py==2.10.0。

    硬件环境：Windows11，3060显卡。

    3.2 训练参数设置

    训练：测试：验证=8:1:1；一开始采用的是Aadm优化器，但训练过程中发现，虽模型拟合速度较快，但训练得到的模型泛化能力很差。采用冻结训练，主要训练参数设置如下：

    1.     input_shape     = [640, 640]
    2.     mosaic              = True
    3.     mosaic_prob         = 0.5
    4.     mixup               = True
    5.     mixup_prob          = 0.5
    6.     special_aug_ratio   = 0.5（前50%轮开启moasic增强）
    7.     Init_Epoch          = 180
    8.     Freeze_Epoch        = 50
    9.     Freeze_batch_size   = 16 
    10.     UnFreeze_Epoch      = 300
    11.     Unfreeze_batch_size = 4
    12.     Init_lr             = 1e-2
    13.     Min_lr              = Init_lr * 0.01
    14.     optimizer_type      = "sgd"
    15.     momentum            = 0.937
    16.     weight_decay        = 5e-4
    17.   
    18.     lr_decay_type       = 'cos'

    3.3 训练结果

    3.4 测试结果可视化

  • 相关阅读:
    还在用饼状图？来瞧瞧这些炫酷的百分比可视化新图形（附代码实现）
    （C++）复原IP地址
    Python初级练习小实例（1-20例），1个实例多个例子相互参考
    Android 基础知识3-3(九妹)图怎么用呢？
    GD32F4xx GCC编译
    K8s Pod 创建埋点处理(Mutating Admission Webhook)
    linux笔记(5):按照东山派的官方教程编译buildroot(东山哪吒,D1-H)踩坑记录
    Redis面试题(二)
    数据处理及跳转
    环境规制指数，两种测算方法，整理好的面板数据
  • 原文地址：https://blog.csdn.net/ncusz/article/details/132528863