• paddle 复现ResNet模型（只关注模型）

    ResNet（Residual Neural Network）由微软研究院的Kaiming He等四名华人提出，通过使用ResNet Unit成功训练出了152层的神经网络，并在ILSVRC2015比赛中取得冠军，在top5上的错误率为3.57%，同时参数量比VGGNet低，效果非常突出。ResNet的结构可以极快的加速神经网络的训练，模型的准确率也有比较大的提升。
    论文地址：“Deep_Residual_Learning_for_Image_Recognition”

    二、对总体网络的介绍
    1、网络亮点
    网络提出了residual结构（残差结构），可有效缓解随网络层数的加深而导致的梯度消失和梯度爆炸现象。从而可以搭建超深的网络结构(突破1000层)。

    2、网络构成
    网络大体上由以下两种block构成：


    Basicblock（用于resnet18，resnet34)


    Bottleneckblock（用于resnet50，resnet101，resnet152）

    仔细思考为什么？？

     3、网络结构

     

    三、论文复现

    3.1paddle 依赖包

    1. import paddle
    2. import paddle.nn as nn
    3. from paddle.nn import Conv2D, MaxPool2D, AdaptiveAvgPool2D, Linear, ReLU, BatchNorm2D
    4. import paddle.nn.functional as F

    3.2 建立Basicblock

     在特征矩阵的大小和通道数发生改变时，应在shortcut中对特征矩阵进行卷积，使shortcut输出的特征矩阵和主分支的大小相同，便于相加。

    1. class Basicblock(paddle.nn.Layer):
    2.     def __init__(self, in_channel, out_channel, stride=1):
    3.         super(Basicblock, self).__init__()
    4.         self.stride = stride
    5.         self.conv0 = Conv2D(in_channel, out_channel, 3, stride=stride, padding=1)
    6.         self.conv1 = Conv2D(out_channel, out_channel, 3, stride=1, padding=1)
    7.         self.conv2 = Conv2D(in_channel, out_channel, 1, stride=stride)
    8.         self.bn0 = BatchNorm2D(out_channel)
    9.         self.bn1 = BatchNorm2D(out_channel)
    10.         self.bn2 = BatchNorm2D(out_channel)
    11.  
    12.     def forward(self, inputs):
    13.         y = inputs
    14.         x = self.conv0(inputs)
    15.         x = self.bn0(x)
    16.         x = F.relu(x)
    17.         x = self.conv1(x)
    18.         x = self.bn1(x)
    19.         if self.stride == 2:
    20.             y = self.conv2(y)
    21.             y = self.bn2(y)
    22.         z = F.relu(x+y)
    23.         return z

    3.3 建立Bottleneckblock

    1. class Bottleneckblock(paddle.nn.Layer):
    2.     def __init__(self, inplane, in_channel, out_channel, stride=1, start=False):
    3.         super(Bottleneckblock, self).__init__()
    4.         self.stride = stride
    5.         self.start = start
    6.         self.conv0 = Conv2D(in_channel, inplane, 1, stride = stride)
    7.         self.conv1 = Conv2D(inplane, inplane, 3, stride=1, padding=1)
    8.         self.conv2 = Conv2D(inplane, out_channel, 1, stride=1)
    9.         self.conv3 = Conv2D(in_channel, out_channel, 1, stride = stride)
    10.         self.bn0 = BatchNorm2D(inplane)
    11.         self.bn1 = BatchNorm2D(inplane)
    12.         self.bn2 = BatchNorm2D(out_channel)
    13.         self.bn3 = BatchNorm2D(out_channel)
    14.  
    15.     def forward(self, inputs):
    16.         y = inputs
    17.         x = self.conv0(inputs)
    18.         x = self.bn0(x)
    19.         x = F.relu(x)
    20.         x = self.conv1(x)
    21.         x = self.bn1(x)
    22.         x = F.relu(x)
    23.         x = self.conv2(x)
    24.         x = self.bn2(x)
    25.         if self.start:
    26.             y = self.conv3(y)
    27.             y = self.bn3(y)
    28.         z = F.relu(x+y)
    29.         return z

    3.4 搭建Resnet的主干
    值得注意的细节

    1.在使用basic_layer的block中，第一个block保留原通道，之后的block因为承接上一个block所以通道数减半。

    2.在使用bottleneck_layer的block中，在每一块block内部，输出block时通道深度为输入通道深度乘4，第一个block保留原通道，因为和第一个卷积池化后的通道数和输入通道数一样为64.
     

    1. class Resnet(paddle.nn.Layer):
    2.     def __init__(self, num, bottlenet):
    3.         super(Resnet, self).__init__()
    4.         self.conv0 = Conv2D(3, 64, 7, stride=2)
    5.         self.bn = BatchNorm2D(64)
    6.         self.pool1 = MaxPool2D(3, stride=2)
    7.         if bottlenet:
    8.             self.layer0 = self.add_bottleneck_layer(num[0], 64, start = True)
    9.             self.layer1 = self.add_bottleneck_layer(num[1], 128)
    10.             self.layer2 = self.add_bottleneck_layer(num[2], 256)
    11.             self.layer3 = self.add_bottleneck_layer(num[3], 512)
    12.         else:
    13.             self.layer0 = self.add_basic_layer(num[0], 64, start = True)
    14.             self.layer1 = self.add_basic_layer(num[1], 128)
    15.             self.layer2 = self.add_basic_layer(num[2], 256)
    16.             self.layer3 = self.add_basic_layer(num[3], 512)
    17.         self.pool2 = AdaptiveAvgPool2D(output_size = (1, 1))
    18.  
    19.     def add_basic_layer(self, num, inplane, start = False):
    20.         layer = []
    21.         if start:
    22.             layer.append(Basicblock(inplane, inplane))
    23.         else:
    24.             layer.append(Basicblock(inplane//2, inplane, stride = 2))
    25.         for i in range(num-1):
    26.             layer.append(Basicblock(inplane, inplane))
    27.         return nn.Sequential(*layer)
    28.  
    29.     def add_bottleneck_layer(self, num, inplane, start = False):
    30.         layer = []
    31.         if start:
    32.             layer.append(Bottleneckblock(inplane, inplane, inplane*4, start = True))
    33.         else:
    34.             layer.append(Bottleneckblock(inplane, inplane*2, inplane*4, stride = 2, start = True))
    35.         for i in range(num-1):
    36.             layer.append(Bottleneckblock(inplane, inplane*4, inplane*4))
    37.         return nn.Sequential(*layer)
    38.  
    39.     def forward(self, inputs):
    40.         x = self.conv0(inputs)
    41.         x = self.bn(x)
    42.         x = self.pool1(x)
    43.         x = self.layer0(x)
    44.         x = self.layer1(x)
    45.         x = self.layer2(x)
    46.         x = self.layer3(x)
    47.         x = self.pool2(x)
    48.         x = paddle.squeeze(x)
    49.         return x

    3.5 具体化网络

    就是你想要什么样的网络，上面提出来的几种 resnet18，resnet50，resnet101 等等

    这个是resnet18

    1. def resnet18():
    2.     return Resnet([2, 2, 2, 2], bottlenet=False)

    这个是resnet34

    1. def resnet34():
    2.     return Resnet([3, 4, 6, 3], bottlenet=False)

     这个是resnet50

    1. def resnet50():
    2.     return Resnet([3, 4, 6, 3], bottlenet=True)

     这个是resnet101

    1. def resnet101():
    2.     return Resnet([3, 4, 23, 3], bottlenet=True)

    这个是resnet152

    1. def resnet152():
    2.     return Resnet([3, 8, 36, 3], bottlenet=True)

    到此  resnet的模型已经建立好了。跑一下paddle  看一下网络架构吧

    四、查看网络结构

    1. model=resnet18()
    2. paddle.summary(model, (1, 3, 224, 224))

    运行结果：

    1. -------------------------------------------------------------------------------
    2.    Layer (type)         Input Shape          Output Shape         Param #    
    3. ===============================================================================
    4.      Conv2D-1        [[1, 3, 224, 224]]   [1, 64, 109, 109]        9,472     
    5.    BatchNorm2D-1    [[1, 64, 109, 109]]   [1, 64, 109, 109]         256      
    6.     MaxPool2D-1     [[1, 64, 109, 109]]    [1, 64, 54, 54]           0       
    7.      Conv2D-2        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    8.    BatchNorm2D-2     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    9.      Conv2D-3        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    10.    BatchNorm2D-3     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    11.    Basicblock-1      [[1, 64, 54, 54]]     [1, 64, 54, 54]           0       
    12.      Conv2D-5        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    13.    BatchNorm2D-5     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    14.      Conv2D-6        [[1, 64, 54, 54]]     [1, 64, 54, 54]        36,928     
    15.    BatchNorm2D-6     [[1, 64, 54, 54]]     [1, 64, 54, 54]          256      
    16.    Basicblock-2      [[1, 64, 54, 54]]     [1, 64, 54, 54]           0       
    17.      Conv2D-8        [[1, 64, 54, 54]]     [1, 128, 27, 27]       73,856     
    18.    BatchNorm2D-8     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    19.      Conv2D-9        [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    20.    BatchNorm2D-9     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    21.      Conv2D-10       [[1, 64, 54, 54]]     [1, 128, 27, 27]        8,320     
    22.   BatchNorm2D-10     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    23.    Basicblock-3      [[1, 64, 54, 54]]     [1, 128, 27, 27]          0       
    24.      Conv2D-11       [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    25.   BatchNorm2D-11     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    26.      Conv2D-12       [[1, 128, 27, 27]]    [1, 128, 27, 27]       147,584    
    27.   BatchNorm2D-12     [[1, 128, 27, 27]]    [1, 128, 27, 27]         512      
    28.    Basicblock-4      [[1, 128, 27, 27]]    [1, 128, 27, 27]          0       
    29.      Conv2D-14       [[1, 128, 27, 27]]    [1, 256, 14, 14]       295,168    
    30.   BatchNorm2D-14     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    31.      Conv2D-15       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    32.   BatchNorm2D-15     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    33.      Conv2D-16       [[1, 128, 27, 27]]    [1, 256, 14, 14]       33,024     
    34.   BatchNorm2D-16     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    35.    Basicblock-5      [[1, 128, 27, 27]]    [1, 256, 14, 14]          0       
    36.      Conv2D-17       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    37.   BatchNorm2D-17     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    38.      Conv2D-18       [[1, 256, 14, 14]]    [1, 256, 14, 14]       590,080    
    39.   BatchNorm2D-18     [[1, 256, 14, 14]]    [1, 256, 14, 14]        1,024     
    40.    Basicblock-6      [[1, 256, 14, 14]]    [1, 256, 14, 14]          0       
    41.      Conv2D-20       [[1, 256, 14, 14]]     [1, 512, 7, 7]       1,180,160   
    42.   BatchNorm2D-20      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    43.      Conv2D-21        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    44.   BatchNorm2D-21      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    45.      Conv2D-22       [[1, 256, 14, 14]]     [1, 512, 7, 7]        131,584    
    46.   BatchNorm2D-22      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    47.    Basicblock-7      [[1, 256, 14, 14]]     [1, 512, 7, 7]           0       
    48.      Conv2D-23        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    49.   BatchNorm2D-23      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    50.      Conv2D-24        [[1, 512, 7, 7]]      [1, 512, 7, 7]       2,359,808   
    51.   BatchNorm2D-24      [[1, 512, 7, 7]]      [1, 512, 7, 7]         2,048     
    52.    Basicblock-8       [[1, 512, 7, 7]]      [1, 512, 7, 7]           0       
    53. AdaptiveAvgPool2D-1   [[1, 512, 7, 7]]      [1, 512, 1, 1]           0       
    54. ===============================================================================
    55. Total params: 11,190,912
    56. Trainable params: 11,171,712
    57. Non-trainable params: 19,200
    58. -------------------------------------------------------------------------------
    59. Input size (MB): 0.57
    60. Forward/backward pass size (MB): 42.70
    61. Params size (MB): 42.69
    62. Estimated Total Size (MB): 85.97
    63. -------------------------------------------------------------------------------

    参考：

    paddle实现resnet复现_扬志九洲的博客-CSDN博客_paddle实现resnet

  • 相关阅读:
    基于grpc从零开始搭建一个准生产分布式应用(6) - 03 - MapStruct高级映射
    【proteus】8086 写一个汇编程序并调试
    Java包装类
    文件拷贝python脚本
    NIO总结文——一篇读懂NIO整个流程
    人才早已过剩，计算机专业还值得报吗？
    测试进阶必备，这5款http接口自动化测试工具不要太香~
    JAVA计算机毕业设计读书网络社区设计Mybatis+系统+数据库+调试部署
    豪越智慧后勤管理平台为企业带来哪些转变？
    流量卡可以自己选地区吗？看完你就明白了！
  • 原文地址：https://blog.csdn.net/Vertira/article/details/126789086