• paddle 复现ResNet模型（只关注模型）

    ResNet（Residual Neural Network）由微软研究院的Kaiming He等四名华人提出，通过使用ResNet Unit成功训练出了152层的神经网络，并在ILSVRC2015比赛中取得冠军，在top5上的错误率为3.57%，同时参数量比VGGNet低，效果非常突出。ResNet的结构可以极快的加速神经网络的训练，模型的准确率也有比较大的提升。
    论文地址：“Deep_Residual_Learning_for_Image_Recognition”

    二、对总体网络的介绍
    1、网络亮点
    网络提出了residual结构（残差结构），可有效缓解随网络层数的加深而导致的梯度消失和梯度爆炸现象。从而可以搭建超深的网络结构(突破1000层)。

    2、网络构成
    网络大体上由以下两种block构成：


    Basicblock（用于resnet18，resnet34)


    Bottleneckblock（用于resnet50，resnet101，resnet152）

    仔细思考为什么？？

     3、网络结构

     

    三、论文复现

    3.1paddle 依赖包

    1. import paddle
    2. import paddle.nn as nn
    3. from paddle.nn import Conv2D, MaxPool2D, AdaptiveAvgPool2D, Linear, ReLU, BatchNorm2D
    4. import paddle.nn.functional as F

    3.2 建立Basicblock

     在特征矩阵的大小和通道数发生改变时，应在shortcut中对特征矩阵进行卷积，使shortcut输出的特征矩阵和主分支的大小相同，便于相加。

    1. class Basicblock(paddle.nn.Layer):
    2.     def __init__(self, in_channel, out_channel, stride=1):
    3.         super(Basicblock, self).__init__()
    4.         self.stride = stride
    5.         self.conv0 = Conv2D(in_channel, out_channel, 3, stride=stride, padding=1)
    6.         self.conv1 = Conv2D(out_channel, out_channel, 3, stride=1, padding=1)
    7.         self.conv2 = Conv2D(in_channel, out_channel, 1, stride=stride)
    8.         self.bn0 = BatchNorm2D(out_channel)
    9.         self.bn1 = BatchNorm2D(out_channel)
    10.         self.bn2 = BatchNorm2D(out_channel)
    11.  
    12.     def forward(self, inputs):
    13.         y = inputs
    14.         x = self.conv0(inputs)
    15.         x = self.bn0(x)
    16.         x = F.relu(x)
    17.         x = self.conv1(x)
    18.         x = self.bn1(x)
    19.         if self.stride == 2:
    20.             y = self.conv2(y)
    21.             y = self.bn2(y)
    22.         z = F.relu(x+y)
    23.         return z

    3.3 建立Bottleneckblock

    1. class Bottleneckblock(paddle.nn.Layer):
    2.     def __init__(self, inplane, in_channel, out_channel, stride=1, start=False):
    3.         super(Bottleneckblock, self).__init__()
    4.         self.stride = stride
    5.         self.start = start
    6.         self.conv0 = Conv2D(in_channel, inplane, 1, stride = stride)
    7.         self.conv1 = Conv2D(inplane, inplane, 3, stride=1, padding=1)
    8.         self.conv2 = Conv2D(inplane, out_channel, 1, stride=1)
    9.         self.conv3 = Conv2D(in_channel, out_channel, 1, stride = stride)
    10.         self.bn0 = BatchNorm2D(inplane)
    11.         self.bn1 = BatchNorm2D(inplane)
    12.         self.bn2 = BatchNorm2D(out_channel)
    13.         self.bn3 = BatchNorm2D(out_channel)
    14.  
    15.     def forward(self, inputs):
    16.         y = inputs
    17.         x = self.conv0(inputs)
    18.         x = self.bn0(x)
    19.         x = F.relu(x)
    20.         x = self.conv1(x)
    21.         x = self.bn1(x)
    22.         x = F.relu(x)
    23.         x = self.conv2(x)
    24.         x = self.bn2(x)
    25.         if self.start:
    26.             y = self.conv3(y)
    27.             y = self.bn3(y)
    28.         z = F.relu(x+y)
    29.         return z

    3.4 搭建Resnet的主干
    值得注意的细节

    1.在使用basic_layer的block中，第一个block保留原通道，之后的block因为承接上一个block所以通道数减半。

    2.在使用bottleneck_layer的block中，在每一块block内部，输出block时通道深度为输入通道深度乘4，第一个block保留原通道，因为和第一个卷积池化后的通道数和输入通道数一样为64.
     

    1. class Resnet(paddle.nn.Layer):
    2.     def __init__(self, num, bottlenet):
    3.         super(Resnet, self).__init__()
    4.         self.conv0 = Conv2D(3, 64, 7, stride=2)
    5.         self.bn = BatchNorm2D(64)
    6.         self.pool1 = MaxPool2D(3, stride=2)
    7.         if bottlenet:
    8.             self.layer0 = self.add_bottleneck_layer(num[0], 64, start = True)
    9.             self.layer1 = self.add_bottleneck_layer(num[1], 128)
    10.             self.layer2 = self.add_bottleneck_layer(num[2], 256)
    11.             self.layer3 = self.add_bottleneck_layer(num[3], 512)
    12.         else:
    13.             self.layer0 = self.add_basic_layer(num[0], 64, start = True)
    14.             self.layer1 = self.add_basic_layer(num[1], 128)
    15.             self.layer2 = self.add_basic_layer(num[2], 256)
    16.             self.layer3 = self.add_basic_layer(num[3], 512)
    17.         self.pool2 = AdaptiveAvgPool2D(output_size = (1, 1))
    18.  
    19.     def add_basic_layer(self, num, inplane, start = False):
    20.         layer = []
    21.         if start:
    22.             layer.append(Basicblock(inplane, inplane))
    23.         else:
    24.             layer.append(Basicblock(inplane//2, inplane, stride = 2))
    25.         for i in range(num-1):
    26.             layer.append(Basicblock(inplane, inplane))
    27.         return nn.Sequential(*layer)
    28.  
    29.     def add_bottleneck_layer(self, num, inplane, start = False):
    30.         layer = []
    31.         if start:
    32.             layer.append(Bottleneckblock(inplane, inplane, inplane*4, start = True))
    33.         else:
    34.             layer.append(Bottleneckblock(inplane, inplane*2, inplane*4, stride = 2, start = True))
    35.         for i in range(num-1):
    36.             layer.append(Bottleneckblock(inplane, inplane*4, inplane*4))
    37.         return nn.Sequential(*layer)
    38.  
    39.     def forward(self, inputs):
    40.         x = self.conv0(inputs)
    41.         x = self.bn(x)
    42.         x = self.pool1(x)
    43.         x = self.layer0(x)
    44.         x = self.layer1(x)
    45.         x = self.layer2(x)
    46.         x = self.layer3(x)
    47.         x = self.pool2(x)
    48.         x = paddle.squeeze(x)
    49.         return x

    3.5 具体化网络

    就是你想要什么样的网络，上面提出来的几种 resnet18，resnet50，resnet101 等等

    这个是resnet18

    1. def resnet18():
    2.     return Resnet([2, 2, 2, 2], bottlenet=False)

    这个是resnet34

    1. def resnet34():
    2.     return Resnet([3, 4, 6, 3], bottlenet=False)

     这个是resnet50

    1. def resnet50():
    2.     return Resnet([3, 4, 6, 3], bottlenet=True)

     这个是resnet101

    1. def resnet101():
    2.     return Resnet([3, 4, 23, 3], bottlenet=True)

    这个是resnet152

    1. def resnet152():
    2.     return Resnet([3, 8, 36, 3], bottlenet=True)

    到此  resnet的模型已经建立好了。跑一下paddle  看一下网络架构吧

    四、查看网络结构

    1. model=resnet18()
    2. paddle.summary(model, (1, 3, 224, 224))

    运行结果：

    1. -------------------------------------------------------------------------------
    2.    Layer (type)         Input Shape          Output Shape         Param #    
    3. ===============================================================================
    4.      Conv2D-1        [[1, 3, 224, 224]]   [1, 64, 109, 109]        9,472     
    5.    BatchNorm2D-1    [[1, 64, 109, 109]]   [1, 64, 109, 109]         256      
    6.     MaxPool2D-1     [[1, 64, 109, 109]]    [1, 64, 54, 54]           0       
    7.      Conv2D-2        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    8.    BatchNorm2D-2     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    9.      Conv2D-3        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    10.    BatchNorm2D-3     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    11.    Basicblock-1      [[1, 64, 54, 54]]     [1, 64, 54, 54]           0       
    12.      Conv2D-5        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    13.    BatchNorm2D-5     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    14.      Conv2D-6        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    15.    BatchNorm2D-6     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    16.    Basicblock-2      [[1, 64, 54, 54]]     [1, 64, 54, 54]           0       
    17.      Conv2D-8        [[1, 64, 54, 54]]     [1, 128, 27, 27]       73,856     
    18.    BatchNorm2D-8     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    19.      Conv2D-9        [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    20.    BatchNorm2D-9     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    21.      Conv2D-10       [[1, 64, 54, 54]]     [1, 128, 27, 27]        8,320     
    22.   BatchNorm2D-10     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    23.    Basicblock-3      [[1, 64, 54, 54]]     [1, 128, 27, 27]          0       
    24.      Conv2D-11       [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    25.   BatchNorm2D-11     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    26.      Conv2D-12       [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    27.   BatchNorm2D-12     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    28.    Basicblock-4      [[1, 128, 27, 27]]    [1, 128, 27, 27]          0       
    29.      Conv2D-14       [[1, 128, 27, 27]]    [1, 256, 14, 14]       295,168    
    30.   BatchNorm2D-14     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    31.      Conv2D-15       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    32.   BatchNorm2D-15     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    33.      Conv2D-16       [[1, 128, 27, 27]]    [1, 256, 14, 14]       33,024     
    34.   BatchNorm2D-16     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    35.    Basicblock-5      [[1, 128, 27, 27]]    [1, 256, 14, 14]          0       
    36.      Conv2D-17       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    37.   BatchNorm2D-17     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    38.      Conv2D-18       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    39.   BatchNorm2D-18     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    40.    Basicblock-6      [[1, 256, 14, 14]]    [1, 256, 14, 14]          0       
    41.      Conv2D-20       [[1, 256, 14, 14]]     [1, 512, 7, 7]       1,180,160   
    42.   BatchNorm2D-20      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    43.      Conv2D-21        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    44.   BatchNorm2D-21      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    45.      Conv2D-22       [[1, 256, 14, 14]]     [1, 512, 7, 7]        131,584    
    46.   BatchNorm2D-22      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    47.    Basicblock-7      [[1, 256, 14, 14]]     [1, 512, 7, 7]           0       
    48.      Conv2D-23        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    49.   BatchNorm2D-23      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    50.      Conv2D-24        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    51.   BatchNorm2D-24      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    52.    Basicblock-8       [[1, 512, 7, 7]]      [1, 512, 7, 7]           0       
    53. AdaptiveAvgPool2D-1   [[1, 512, 7, 7]]      [1, 512, 1, 1]           0       
    54. ===============================================================================
    55. Total params: 11,190,912
    56. Trainable params: 11,171,712
    57. Non-trainable params: 19,200
    58. -------------------------------------------------------------------------------
    59. Input size (MB): 0.57
    60. Forward/backward pass size (MB): 42.70
    61. Params size (MB): 42.69
    62. Estimated Total Size (MB): 85.97
    63. -------------------------------------------------------------------------------

    参考：

    paddle实现resnet复现_扬志九洲的博客-CSDN博客_paddle实现resnet

  • 相关阅读:
    Puppeteer+RabbitMQ：Node.js 批量加工pdf服务架构设计与落地
    为什么MySQL索引选择B+树而不使用B树？
    Java - Gson和Fastjson如何选择？
    Java IP归属地查询（离线方式+在线方式，内附查询IP方法）
    证书过期问题: https://registry.npm.taobao.org npm ERR! code CERT_HAS_EXPIRED npm ERR
    《设计模式》之策略模式
    抄写Linux源码(Day18：读取硬盘前的准备工作有哪些？)
    Mybatis-plus使用教程
    【已解决】UE5 plugin ‘xxx‘ failed to load because module ‘xxx‘ could not be found.
    Extended-csv-dataset-config_Jmeter插件推荐
  • 原文地址：https://blog.csdn.net/Vertira/article/details/126789086