paddlepaddle 复现论文LeNet-5网络结构（仅仅关注模型）

选用LeNet-5网络结构，论文地址：Gradient-based learning applied to document recognition | IEEE Journals & Magazine | IEEE Xplore

每个阶段用到的Layer

原论文模型复现(方法一)


import paddle.nn as nn
 
network = nn.Sequential(
    nn.Conv2D(in_channels=1, out_channels=6, kernel_size=5, stride=1, padding=0),  # C1 卷积层
    nn.Tanh(),
    nn.AvgPool2D(kernel_size=2, stride=2),  # S2 平局池化层
    nn.Sigmoid(),   # Sigmoid激活函数
    nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0),  # C3 卷积层
    nn.Tanh(),
    nn.AvgPool2D(kernel_size=2, stride=2),  # S4 平均池化层
    nn.Sigmoid(),  # Sigmoid激活函数
    nn.Conv2D(in_channels=16, out_channels=120, kernel_size=5, stride=1, padding=0), # C5 卷积层
    nn.Tanh(),
    nn.Flatten(),
    nn.Linear(in_features=120, out_features=84), # F6 全连接层
    nn.Tanh(),
    nn.Linear(in_features=84, out_features=10) # OUTPUT 全连接层
)

模型可视化

paddle.summary(network, (1, 1, 32, 32))

输出模型结果


--------------------------------------------------------------------------
 Layer (type)       Input Shape          Output Shape         Param #    
===========================================================================
   Conv2D-1       [[1, 1, 32, 32]]      [1, 6, 28, 28]          156      
    Tanh-1        [[1, 6, 28, 28]]      [1, 6, 28, 28]           0       
  AvgPool2D-1     [[1, 6, 28, 28]]      [1, 6, 14, 14]           0       
   Sigmoid-1      [[1, 6, 14, 14]]      [1, 6, 14, 14]           0       
   Conv2D-2       [[1, 6, 14, 14]]     [1, 16, 10, 10]         2,416     
    Tanh-2       [[1, 16, 10, 10]]     [1, 16, 10, 10]           0       
  AvgPool2D-2    [[1, 16, 10, 10]]      [1, 16, 5, 5]            0       
   Sigmoid-2      [[1, 16, 5, 5]]       [1, 16, 5, 5]            0       
   Conv2D-3       [[1, 16, 5, 5]]       [1, 120, 1, 1]        48,120     
    Tanh-3        [[1, 120, 1, 1]]      [1, 120, 1, 1]           0       
   Flatten-1      [[1, 120, 1, 1]]         [1, 120]              0       
   Linear-1          [[1, 120]]            [1, 84]            10,164     
    Tanh-4           [[1, 84]]             [1, 84]               0       
   Linear-2          [[1, 84]]             [1, 10]              850      
===========================================================================
Total params: 61,706
Trainable params: 61,706
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.12
Params size (MB): 0.24
Estimated Total Size (MB): 0.36
---------------------------------------------------------------------------

对模型进行简单修改（方法二）

改进模型

Sequential写法


import paddle.nn as nn
 
network_2 = nn.Sequential(
    nn.Conv2D(in_channels=1, out_channels=6, kernel_size=3, stride=1, padding=1),
    nn.ReLU(),
    nn.MaxPool2D(kernel_size=2, stride=2),
    nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0),
    nn.ReLU(),
    nn.MaxPool2D(kernel_size=2, stride=2),
    nn.Flatten(),
    nn.Linear(in_features=400, out_features=120),  # 400 = 5x5x16，输入形状为32x32， 输入形状为28x28时调整为256
    nn.Linear(in_features=120, out_features=84),
    nn.Linear(in_features=84, out_features=10)
)

模型可视化

paddle.summary(network_2, (1, 1, 28, 28))

输出结果


---------------------------------------------------------------------------
 Layer (type)       Input Shape          Output Shape         Param #    
===========================================================================
   Conv2D-4       [[1, 1, 28, 28]]      [1, 6, 28, 28]          60       
    ReLU-1        [[1, 6, 28, 28]]      [1, 6, 28, 28]           0       
  MaxPool2D-1     [[1, 6, 28, 28]]      [1, 6, 14, 14]           0       
   Conv2D-5       [[1, 6, 14, 14]]     [1, 16, 10, 10]         2,416     
    ReLU-2       [[1, 16, 10, 10]]     [1, 16, 10, 10]           0       
  MaxPool2D-2    [[1, 16, 10, 10]]      [1, 16, 5, 5]            0       
   Flatten-2      [[1, 16, 5, 5]]          [1, 400]              0       
   Linear-3          [[1, 400]]            [1, 120]           48,120     
   Linear-4          [[1, 120]]            [1, 84]            10,164     
   Linear-5          [[1, 84]]             [1, 10]              850      
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------

方法三

Sub Class写法


class LeNet(nn.Layer):
    """
    继承paddle.nn.Layer定义网络结构
    """
 
    def __init__(self, num_classes=10):
        """
        初始化函数
        """
        super(LeNet, self).__init__()
 
        self.features = nn.Sequential(
            nn.Conv2D(in_channels=1, out_channels=6, kernel_size=3, stride=1, padding=1),  # 第一层卷积
            nn.ReLU(), # 激活函数
            nn.MaxPool2D(kernel_size=2, stride=2),  # 最大池化，下采样
            nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0), # 第二层卷积
            nn.ReLU(), # 激活函数
            nn.MaxPool2D(kernel_size=2, stride=2) # 最大池化，下采样
        )
 
        self.fc = nn.Sequential(
            nn.Linear(400, 120),  # 全连接
            nn.Linear(120, 84),   # 全连接
            nn.Linear(84, num_classes) # 输出层
        )
 
    def forward(self, inputs):
        """
        前向计算
        """
        y = self.features(inputs)
        y = paddle.flatten(y, 1)
        out = self.fc(y)
 
        return out
 
network_3 = LeNet()

模型可视化

paddle.summary(network_3, (1, 1, 28, 28))


---------------------------------------------------------------------------
 Layer (type)       Input Shape          Output Shape         Param #    
===========================================================================
   Conv2D-6       [[1, 1, 28, 28]]      [1, 6, 28, 28]          60       
    ReLU-3        [[1, 6, 28, 28]]      [1, 6, 28, 28]           0       
  MaxPool2D-3     [[1, 6, 28, 28]]      [1, 6, 14, 14]           0       
   Conv2D-7       [[1, 6, 14, 14]]     [1, 16, 10, 10]         2,416     
    ReLU-4       [[1, 16, 10, 10]]     [1, 16, 10, 10]           0       
  MaxPool2D-4    [[1, 16, 10, 10]]      [1, 16, 5, 5]            0       
   Linear-6          [[1, 400]]            [1, 120]           48,120     
   Linear-7          [[1, 120]]            [1, 84]            10,164     
   Linear-8          [[1, 84]]             [1, 10]              850      
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------

PaddlePaddle高层API写法（方法四）

高层API ，最简单，只需一行就可以。

network_4 = paddle.vision.models.LeNet(num_classes=10)

模型可视化

paddle.summary(network_4, (1, 1, 28, 28))

输出结果


---------------------------------------------------------------------------
 Layer (type)       Input Shape          Output Shape         Param #    
===========================================================================
   Conv2D-8       [[1, 1, 28, 28]]      [1, 6, 28, 28]          60       
    ReLU-5        [[1, 6, 28, 28]]      [1, 6, 28, 28]           0       
  MaxPool2D-5     [[1, 6, 28, 28]]      [1, 6, 14, 14]           0       
   Conv2D-9       [[1, 6, 14, 14]]     [1, 16, 10, 10]         2,416     
    ReLU-6       [[1, 16, 10, 10]]     [1, 16, 10, 10]           0       
  MaxPool2D-6    [[1, 16, 10, 10]]      [1, 16, 5, 5]            0       
   Linear-9          [[1, 400]]            [1, 120]           48,120     
   Linear-10         [[1, 120]]            [1, 84]            10,164     
   Linear-11         [[1, 84]]             [1, 10]              850      
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------

总结一下方法二、方法三、方法四。他们是同一种网络结构，不同的表现形式，网络的模型结构是一样的。

最后是重要的模型训练与调优

选方法二的网络模型结构进行训练与调优


# 模型封装
model = paddle.Model(network_2)
 
# 模型配置
model.prepare(paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters()), # 优化器
              paddle.nn.CrossEntropyLoss(), # 损失函数
              paddle.metric.Accuracy()) # 评估指标
 
# 启动全流程训练
model.fit(train_dataset,  # 训练数据集
          eval_dataset,   # 评估数据集
          epochs=5,       # 训练轮次
          batch_size=64,  # 单次计算数据样本量
          verbose=1)      # 日志展示形式

模型评估


result = model.evaluate(eval_dataset, verbose=1)
 
print(result)

批量预测


# 进行预测操作
result = model.predict(eval_dataset)
 
# 定义画图方法
def show_img(img, predict):
    plt.figure()
    plt.title('predict: {}'.format(predict))
    plt.imshow(img.reshape([28, 28]), cmap=plt.cm.binary)
    plt.show()
 
# 抽样展示
indexs = [2, 15, 38, 211]
 
for idx in indexs:
    show_img(eval_dataset[idx][0], np.argmax(result[0][idx]))

参考链接：

(79条消息) PaddlePaddle入门实践——初识卷积神经网络_面包猎人的博客-CSDN博客_paddle 卷积

(117条消息) 深度学习之paddle之卷积神经网络定义，自定义数据集_小码农--向日葵的博客-CSDN博客_paddle 卷积

(119条消息) 模型量化（2）：Paddle 模型的静态量化和动态量化_AI Studio的博客-CSDN博客_paddle模型量化

#Json 数据集加载和训练，适用于缺陷检测

(130条消息) 飞桨深度学习零基础入门（二）——使用飞浆（Paddle）实现卷积神经网络的手写数字（mnist）识别_AiXing-w的博客-CSDN博客_mnist 飞桨

相关阅读:
LeetCode二分查找系列（704，34，35，69，367）
小程序的赛道越来越拥挤
 python机器学习库中Scikit-learn和TensorFlow如何选择？
使用imx 8m 测试matter协议功能
 想用macbook录制视频？这几个技巧让你事半功倍！
这就是艺术，优雅的二维码生成器「GitHub 热点速览」
ffmpeg深度学习滤镜
 【Linux】：Kafka基础命令
 基于JSP技术的校园餐厅管理系统
 Vue源码总结
原文地址：https://blog.csdn.net/Vertira/article/details/126661523