卷积神经网络基础概念理解（二）

一 背景说明

后学末进，多为参考前人经验所得，文中如有谬误，请大家不吝促教，及时留言指正，以免不当之处误人子弟

1.数据集

利用 MNIST 数据集实现一个手写数字识别的卷积神经网络实现，堪称人工智能界的"Hello World"
MNIST 包含的是尺寸为 28 x 28 的数字灰度图，以及数字标签，分为一个训练集和一个测试集
1
2

MINST 数据地址

2.框架 Pytorch

PyTorch是一个开源的Python机器学习库
已兼容 Windows(CUDA,CPU)、MacOS(CPU)、Linux(CUDA,ROCm,CPU)
其他深度学习框架还有诸如：PaddlePaddle、Tensorflow、 Caffe等
1
2
3

Pytorch 文档地址

二 准备工作

1.数据集处理

数据标准化：对输入数据做归一化处理，使输入值尽可能平滑分区，缩小值域区间
所以标准化又称为：特征缩放
为什么要进行数据标准化呢？我们试想，红苹果、绿苹果，于我们人类而言，都是苹果，但是于机器而言，
输入图就是一组矩阵值，在卷积计算后，可能突出红色特征或者绿色特征，然后我们给它的参考值是苹果，
机器就可能反复修正网络权重去兼容红、绿特征，如果不影响收敛还好，如果有影响网络训练再久也是没
结果，收敛曲线反复横跳
除此外，亮度、不同度量单位的特征数据导致数据分布高度离散，都不利于学习收敛
包括 R G B 通道值，可能差异很大，不加以处理会导致无法使训练输出值理想化
怎么归一化？
举个例子：  一组正整数集合，归一化到 (0,1] 的区间，怎么处理？
1
2
3
4
5
6
7
8
9
10

2.网络

前面我们说了很多关于卷积的概念，但是到底该如何基于理论，去做一个实现呢？
首先，我们应该思考一下，卷积能提取特征，但这些特征值是怎么使用的呢；我们说了池化、全连接、激
活，但是其实还不全面，如何根据卷积结果，和预期结果，来调整入参，重复计算，直至得到我们要的值，才是完整的学习方案
1
2
3

三 按步实现

环境版本

程序已添加详细注释，关于 Python 语法或者 Torch 函数，不理解的可以分别查看网络资料与

上述 Torch 官网查询函数名称

1.下载数据集

可以去网站下载并放到程序目录，或者我们通过程序自动首次下载
代码如下
1
2

import torch
import torchvision

## 如果数据不存在 则下载数据到当前目录下的 data 下
train_set=torchvision.datasets.MNIST('./data/', train=True, download=True,
           transform=torchvision.transforms.Compose([
               torchvision.transforms.ToTensor()
           ]))

test_set=torchvision.datasets.MNIST('./data/', train=False, download=True,
           transform=torchvision.transforms.Compose([
               torchvision.transforms.ToTensor()
           ]))
1
2
3
4
5
6
7
8
9
10
11
12
13

2.标准化参数

使用 z-score 标准化算法，Pytorch 能直接计算出参数值

import torch
import torchvision
from torch.utils.data import DataLoader

train_set = torchvision.datasets.MNIST('./data/', train=True, download=True,
                               transform=torchvision.transforms.Compose([
                                   torchvision.transforms.ToTensor()
                               ]))

loader=torch.utils.data.DataLoader(
    train_set,batch_size=len(train_set),num_workers=1
    )

data=next(iter(loader))

print(data[0].mean(),",",data[0].std())
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

对于当前数据集，这里取了全量数据进行计算；实际处理时数据是分批操作的

3.数据集查看

import torch
import torchvision
import cv2
import numpy as np

## 超参数（不随迭代更新）
batch_size_train=64

## 加载数据
train_loader=torch.utils.data.DataLoader(
    torchvision.datasets.MNIST('./data/', train=True, download=True,
                               transform=torchvision.transforms.Compose([
                                   torchvision.transforms.ToTensor(),
                                   torchvision.transforms.Normalize(
                                       (0.1307,), (0.3081,))
                               ])),
    batch_size=batch_size_train, shuffle=True)

examples = enumerate(train_loader)
batch_idx, (example_data, example_val) = next(examples)


## 打印批次、数字值数据结构、灰度图像素数据结构
print("batch_idx:",batch_idx)
print("val_shape",example_val.shape)
print("data shape",example_data.shape)

## 将数字值拆分合并一下，按 8 * 8 打印出来 
val_temp=torch.zeros(8,8)
for x in range(8):
    for y in range(8):
        val_temp[x][y] = example_val[ x*8 + y ]
    
print("val_8_8",val_temp)

## 遍历像素数据矩阵 64 * 1* 28 * 28 将每次矩阵转为 np 在拼接
for x in range(8):
    start = x * 8
    end = (x + 1) * 8
    np_temp = np.squeeze(torch.sigmoid(example_data[start]).cpu().data.numpy())
    for y in range(start + 1,end):
        temp=np.squeeze(torch.sigmoid(example_data[y]).cpu().data.numpy())
        np_temp = np.hstack((np_temp,temp))
    if x == 0:
        np_result = np_temp
    else:
        np_result = np.vstack((np_result,np_temp))

cv2.imshow('Result Image',np_result)
cv2.waitKey(0)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

执行上述程序，我们将第一批次 64 个数据转为图片，查看效果

4.网络结构设计

继承 torch 的 module 定义网络

## 定义一个网络类,并继承 torch.nn.Module
class Net(nn.Module):

    ## 显示定义构造方法 __init__ 时,除额外参数,第一个参必须为对象自身关键字 self
    def __init__(self):

        ## 调用父类初始化方法
        super(Net, self).__init__()
        ## 定义卷积核 输入通道数 输出通道数 卷积核尺寸 nn.Conv2d(in_cahnnels=1, out_channels=10, kernel_size=5) 默认步长为 1 不填充
        self.conv1 = nn.Conv2d(1, 10, 5)
        self.conv2 = nn.Conv2d(10, 20, 5)
        ## 定义通道值随机清零处理方法 在多参数、小样本的场景下,网络容易过拟合,现象为训练集里表现很好,但在测试集并不友好,原因如下
        ## 1.多参数、小样本情况下,由于不同特征间的复杂依赖关系,使得我们的网络难以厘出细粒度的特征区别,比如1000个特征,有600个一致,100不同,300无参考价值
        ##   实际起到作用的100特征在所有特整里面所占比例太低,虽然在训练集里面有一个良好表现,但换到测试集完全可能走向另一种可能
        ## 2.Dropout2d 实际就是随机丢弃一些特征,以干扰当前训练,抑制不同特征的依赖关系,提升最终结果可靠性
        self.conv2_drop = nn.Dropout2d()
        ## 全连接层 二维图像处理时全连接输入、输出一般重置为二维张量,不同于卷积计算时是四维张量
        ## 第一层全连接 320 来源下面计算说明
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    ## 前向传播
    def forward(self, x):
        ## 卷积 self.conv1(x) 
        ## 最大池化 Fun.max_pool2d(self.conv1(x), kernel_size=2) 池化核大小为 2 **stride default kernel_size** 下面计算输出大小过程步长应取 2 
        ## 此方法可在上面提到的 Pytorch 官方查到 API 说明,里面有更多、更详细的参数说明 
        x = Fun.relu(Fun.max_pool2d(self.conv1(x), 2))
        ## 卷积 self.conv2(x) 
        ## 让某个神经元的激活值以一定的概率停止工作 self.conv2_drop 具体实现会做比字面上更多的处理和优化
        ## 最大池化 Fun.max_pool2d
        x = Fun.relu(Fun.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        ## 将 x 转为二维张量 -1 表示其他维度,此时 tensor 会根据自身总量和输出列自动计算行数
        ## 第一次卷积输出：28 x 28 填充 0 步长 1 => (28 + 2 * 0 - 5)/1 + 1 = 24 => 24 * 10 = 240
        ## 第一次池化输出：24 x 24 填充 0 步长 2 => (24 + 2 * 0 - 2)/2 + 1 = 12
        ## 第二次卷积输出：12 x 12 填充 0 步长 1 => (12 + 2 * 0 - 5)/1 + 1 = 8
        ## 第二次池化输出：8 x 8 填充 0 步长 1 => (8 + 2 * 0 - 2)/2 + 1 = 4
        ## 最终输出20通道：4 x 4 共 20 * 4 * 4 = 320 个点
        x = x.view(-1, 320)
        ## 全连接并激活
        x = Fun.relu(self.fc1(x))
        ## 随机丢弃
        x = Fun.dropout(x, training=self.training)
        ## 再次全连接
        x = self.fc2(x)
        ## 计算损失 log_softmax 为 log(softmax(x)) 的变形,官方说法是：单独执行 softmax 和 log 这两个操作速度较慢且数值不稳定
        ## softmax 是概率分布计算 比如我们的模型预测结果 A:0.256 B:0.687 C:0.057 但是实际我们输入是B 所以结果应为 A:0.000 B:1.000 C:0.000
        ## crossEntropy 是交叉熵 用来做似然估计,即损失 计算如下：-(0.0 * ln(0.256) + 1.0 * ln(0.687 + 0.0 * ln(0.025)) => [0,+ ∞)
        ## log_softmax 将 crossEntropy 的操作提前了,直接对概率分布求对数, 将 (0,1] 映射到了 (- ∞,0]
        return Fun.log_softmax(x,dim = 1)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

5.完整训练程序

所有语句均添加了详细注释说明
1.train方法，可单独调用一次，查看随机卷积核情况下损失情况
2.训练 3 个循环，并每次结束时通过测试集验证精度
3.测试方法，可以自己手写一些数字图片进行验证，最好尺寸直接控制在 28 x 28 否则情况可能不太好
1
2
3
4

import cv2
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as Fun
import torch.optim as optim
import torchvision
from torch.utils.data import DataLoader

## 超参数定义
## 训练和验证时每批次处理的数据量
batch_size_train = 64
batch_size_test = 1000
## 参数打印间隔
log_interval = 10
## SGD 定义优化器参数 学习率 冲量
## p′ = p − lr∗dp (p - 学习率 乘以自己的一阶导数)
## v′ = −dx∗lr+v∗momemtum 相对于旧的把 −dx∗lr 作为梯度更新量的方式，引入v∗momemtum后 则如果上次更新方向与本次一致，产生加速效果，反之不一致就是减速效果
learning_rate = 0.01
momentum = 0.5
## 损失值记录
train_losses = []
train_counter = []
test_losses = []
# test_counter = [i * len(train_loader.dataset) for i in range(n_epochs + 1)]
## 平均数 和 标准差 前面已经计算获得
μ=0.1307
σ=0.3081
## 模型参数保存
module_w='./weight/model.pth'
optimizer_w='./weight/optimizer.pth'

## 设置随机种子
torch.manual_seed(1)

## ----------------------------------------------------------------------------------------------------------------------
## 加载数据
train_loader=DataLoader(
    torchvision.datasets.MNIST('./data/', train=True, download=True,
                               transform=torchvision.transforms.Compose([
                                   torchvision.transforms.ToTensor(),
                                   torchvision.transforms.Normalize((μ,), (σ,))
                               ])),
    ## shuffle = true 对输入数据进行打乱处理，先打乱，再batch
    batch_size=batch_size_train, shuffle=True)

test_loader=DataLoader(
    torchvision.datasets.MNIST('./data/', train=False, download=True,
                               transform=torchvision.transforms.Compose([
                                   torchvision.transforms.ToTensor(),
                                   torchvision.transforms.Normalize((μ,), (σ,))
                               ])),
    batch_size=batch_size_test, shuffle=True)

## ----------------------------------------------------------------------------------------------------------------------
## 定义一个网络类,并继承 torch.nn.Module
class Net(nn.Module):

    ## 显示定义构造方法 __init__ 时,除额外参数,第一个参必须为对象自身关键字 self
    def __init__(self):

        ## 调用父类初始化方法
        super(Net, self).__init__()
        ## 定义卷积核 输入通道数 输出通道数 卷积核尺寸 nn.Conv2d(in_cahnnels=1, out_channels=10, kernel_size=5) 默认步长为 1 不填充
        self.conv1 = nn.Conv2d(1, 10, 5)
        self.conv2 = nn.Conv2d(10, 20, 5)
        ## 定义通道值随机清零处理方法 在多参数、小样本的场景下,网络容易过拟合,现象为训练集里表现很好,但在测试集并不友好,原因如下
        ## 1.多参数、小样本情况下,由于不同特征间的复杂依赖关系,使得我们的网络难以厘出细粒度的特征区别,比如1000个特征,有600个一致,100不同,300无参考价值
        ##   实际起到作用的100特征在所有特整里面所占比例太低,虽然在训练集里面有一个良好表现,但换到测试集完全可能走向另一种可能
        ## 2.Dropout2d 实际就是随机丢弃一些特征,以干扰当前训练,抑制不同特征的依赖关系,提升最终结果可靠性
        self.conv2_drop = nn.Dropout2d()
        ## 全连接层 二维图像处理时全连接输入、输出一般重置为二维张量,不同于卷积计算时是四维张量
        ## 第一层全连接 320 来源下面计算说明
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    ## 前向传播
    def forward(self, x):
        ## 卷积 self.conv1(x) 
        ## 最大池化 Fun.max_pool2d(self.conv1(x), kernel_size=2) 池化核大小为 2 **stride default kernel_size** 下面计算输出大小过程步长应取 2 
        ## 此方法可在上面提到的 Pytorch 官方查到 API 说明,里面有更多、更详细的参数说明 
        x = Fun.relu(Fun.max_pool2d(self.conv1(x), 2))
        ## 卷积 self.conv2(x) 
        ## 让某个神经元的激活值以一定的概率停止工作 self.conv2_drop 具体实现会做比字面上更多的处理和优化
        ## 最大池化 Fun.max_pool2d
        x = Fun.relu(Fun.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        ## 将 x 转为二维张量 -1 表示其他维度,此时 tensor 会根据自身总量和输出列自动计算行数
        ## 第一次卷积输出：28 x 28 填充 0 步长 1 => (28 + 2 * 0 - 5)/1 + 1 = 24 => 24 * 10 = 240
        ## 第一次池化输出：24 x 24 填充 0 步长 2 => (24 + 2 * 0 - 2)/2 + 1 = 12
        ## 第二次卷积输出：12 x 12 填充 0 步长 1 => (12 + 2 * 0 - 5)/1 + 1 = 8
        ## 第二次池化输出：8 x 8 填充 0 步长 1 => (8 + 2 * 0 - 2)/2 + 1 = 4
        ## 最终输出20通道：4 x 4 共 20 * 4 * 4 = 320 个点
        x = x.view(-1, 320)
        ## 全连接并激活
        x = Fun.relu(self.fc1(x))
        ## 随机丢弃
        x = Fun.dropout(x, training=self.training)
        ## 再次全连接
        x = self.fc2(x)
        ## 计算损失 log_softmax 为 log(softmax(x)) 的变形,官方说法是：单独执行 softmax 和 log 这两个操作速度较慢且数值不稳定
        ## softmax 是概率分布计算 比如我们的模型预测结果 A:0.256 B:0.687 C:0.057 但是实际我们输入是B 所以结果应为 A:0.000 B:1.000 C:0.000
        ## crossEntropy 是交叉熵 用来做似然估计,即损失 计算如下：-(0.0 * ln(0.256) + 1.0 * ln(0.687 + 0.0 * ln(0.025)) => [0,+ ∞)
        ## log_softmax 将 crossEntropy 的操作提前了,直接对概率分布求对数, 将 (0,1] 映射到了 (- ∞,0]
        return Fun.log_softmax(x,dim = 1)

## ----------------------------------------------------------------------------------------------------------------------
## 定义训练方法 epoch 迭代次数
def train(epoch):
  ## 加载已有模型参数
  network
  ## train 表示启用：批次标准化和随机丢弃 
  ## Batch Normalization 保障 bn 层能够用到每一批次数据的平均值和标准差
  ## Dropout 随机取一部分网络连接来训练更新参数
  network.train()
  ## enumerate 取出索引、值 即批次 batch_idx 和数据 (data, target)
  for batch_idx, (data, target) in enumerate(train_loader):
    ## 将模型的参数梯度初始化为 0 避免不同批次的梯度被累计使用
    optimizer.zero_grad()
    ## 传入数据
    output = network(data)
    ## 交叉熵函数 值越小则预测结果越接近真实结果
    loss = Fun.nll_loss(output, target)
    ## 反向传播计算梯度
    loss.backward()
    ## 更新所有参数 
    optimizer.step()
    ## 每 10 批打印一次，一批处理 64 张，所以每次打印实际处理了 640 张
    if batch_idx % log_interval == 0:
      print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
        epoch, batch_idx * len(data), len(train_loader.dataset),
        100. * batch_idx / len(train_loader), loss.item()))
      train_losses.append(loss.item())
      train_counter.append((batch_idx*64) + ((epoch-1)*len(train_loader.dataset)))
      torch.save(network.state_dict(), module_w)
      torch.save(optimizer.state_dict(), optimizer_w)
          
## ----------------------------------------------------------------------------------------------------------------------
## 定义验证方法
def check():
    ## eval 表示不启用：标准化和随机丢弃
    ## Batch Normalization 保障BN层能够用全部训练数据的均值和标准差
    ## Dropout 利用所有网络连接（不进行随机舍弃神经元）
    ## 训练完成后 在模型测试时执行 eval 避免因为有新的输入导致权值更新
    network.eval()
    ## 
    test_loss = 0
    correct = 0
    ## tensor 有一个参数 requires_grad 默认为 false 设置为 true 以后则反向传播时会自动求导
    ## with torch.no_grad 语法使得当前模块计算所得的新的 tensor 的 requires_grad 都自动设置为False 可以在验证阶段节省GPU或CPU内存
    with torch.no_grad():
        ## 遍历测试数据
        for data, target in test_loader:
            ## 传入数据
            output = network(data)
            ## reduction='sum'  对N个样本损失求和返回
            ## reduction='none' 对N个样本损失求直接返回
            ## reduction='mean' 对N个样本损失求平均返回（已弃用）
            test_loss += Fun.nll_loss(output, target, reduction='sum').item()
            ## 取最大值 max(input, dim, keepdim=False)
            ## input 输入 tensor dim 维度 0 按列 1 按行 keepdim true 不压缩 false 压缩（输出结果比输入低一个维度）
            ## max 返回两个 tensor 前者为值、后者为索引 此处我们取索引即恰好为分析所得结果
            pred = output.data.max(1, keepdim=True)[1]
            ## 逐元素比较 返回一个 boolean 类型张量 再求和 => 输出即识别结果和真实结果一致的数量
            correct += pred.eq(target.data.view_as(pred)).sum()
    ## 平均损失 = 损失和/数据量
    test_loss /= len(test_loader.dataset)
    test_losses.append(test_loss)
    print('\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
        test_loss, correct, len(test_loader.dataset),
        100. * correct / len(test_loader.dataset)))

## ----------------------------------------------------------------------------------------------------------------------
## 功能测试
def test():
    ## 加载图片
    original_img=cv2.imdecode(np.fromfile('88.png',dtype=np.uint8), 1)
    ## 转换
    new_img = cv2.cvtColor(cv2.resize(original_img,(28,28)),cv2.COLOR_RGB2GRAY)
    ## 下面两行注释取消后，可以查看转化出的灰度图
    ## cv2.imshow('Result Image',new_img)
    ## cv2.waitKey(0)
    ## 定义一个四维张量
    data = torch.randn(1,1,28,28)
    ## 填充灰度化的图片数据
    data[0][0] = torch.from_numpy(new_img)
    ## print(data.shape)
    ## 标准化
    μ=data.mean()
    σ=data.std()
    inputdata=data.sub(μ).div_(σ)
    ## 加载权重
    network.load_state_dict(torch.load(module_w))
    optimizer.load_state_dict(torch.load(optimizer_w))
    network.eval()
    with torch.no_grad():
      output = network(inputdata)
    ## 取出最大值索引(对于数字来说，无需二次映射，直接就是输出结果了)
    pred = output.data.max(1, keepdim=True)[1]
    ## print(output.shape)
    ## print(output)
    print(pred)

## ----------------------------------------------------------------------------------------------------------------------
## 初始化网络
network = Net()
## 定义优化器 保存当前的参数状态并且基于计算梯度更新参数
optimizer = optim.SGD(network.parameters(), lr=learning_rate, momentum=momentum)

## 加载已有模型数据
# network.load_state_dict(torch.load(module_w))
# optimizer.load_state_dict(torch.load(optimizer_w))

## 1.执行一次训练
#  train(1)

## 2.训练
# for epoch in range(1, 1 + 3):
#     train(epoch)
#     check()

## 3.测试
## test()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223

6.结果打印

训练随机输入训练 1 次的效果

Train Epoch: 1 [0/60000 (0%)]   Loss: 2.371851
Train Epoch: 1 [640/60000 (1%)] Loss: 2.366941
Train Epoch: 1 [1280/60000 (2%)]        Loss: 2.253085
Train Epoch: 1 [1920/60000 (3%)]        Loss: 2.258119
Train Epoch: 1 [2560/60000 (4%)]        Loss: 2.279200
Train Epoch: 1 [3200/60000 (5%)]        Loss: 2.284118
Train Epoch: 1 [3840/60000 (6%)]        Loss: 2.224427
Train Epoch: 1 [4480/60000 (7%)]        Loss: 2.182901
Train Epoch: 1 [5120/60000 (9%)]        Loss: 2.092957
Train Epoch: 1 [5760/60000 (10%)]       Loss: 2.014003
Train Epoch: 1 [6400/60000 (11%)]       Loss: 2.066881
Train Epoch: 1 [7040/60000 (12%)]       Loss: 1.937319
Train Epoch: 1 [7680/60000 (13%)]       Loss: 1.780319
Train Epoch: 1 [8320/60000 (14%)]       Loss: 1.727874
Train Epoch: 1 [8960/60000 (15%)]       Loss: 1.722512
Train Epoch: 1 [9600/60000 (16%)]       Loss: 1.627026
Train Epoch: 1 [10240/60000 (17%)]      Loss: 1.649031
Train Epoch: 1 [10880/60000 (18%)]      Loss: 1.317975
Train Epoch: 1 [11520/60000 (19%)]      Loss: 1.215969
Train Epoch: 1 [12160/60000 (20%)]      Loss: 1.311500
Train Epoch: 1 [12800/60000 (21%)]      Loss: 1.205322
Train Epoch: 1 [13440/60000 (22%)]      Loss: 1.249098
Train Epoch: 1 [14080/60000 (23%)]      Loss: 1.102184
Train Epoch: 1 [14720/60000 (25%)]      Loss: 1.101858
Train Epoch: 1 [15360/60000 (26%)]      Loss: 1.056550
Train Epoch: 1 [16000/60000 (27%)]      Loss: 0.966378
Train Epoch: 1 [16640/60000 (28%)]      Loss: 0.898759
Train Epoch: 1 [17280/60000 (29%)]      Loss: 1.141907
Train Epoch: 1 [17920/60000 (30%)]      Loss: 0.744676
Train Epoch: 1 [18560/60000 (31%)]      Loss: 0.865845
Train Epoch: 1 [19200/60000 (32%)]      Loss: 0.743363
Train Epoch: 1 [19840/60000 (33%)]      Loss: 0.896839
Train Epoch: 1 [20480/60000 (34%)]      Loss: 0.797301
Train Epoch: 1 [21120/60000 (35%)]      Loss: 0.760073
Train Epoch: 1 [21760/60000 (36%)]      Loss: 0.796578
Train Epoch: 1 [22400/60000 (37%)]      Loss: 0.632670
Train Epoch: 1 [23040/60000 (38%)]      Loss: 0.671789
Train Epoch: 1 [23680/60000 (39%)]      Loss: 0.852199
Train Epoch: 1 [24320/60000 (41%)]      Loss: 0.684957
Train Epoch: 1 [24960/60000 (42%)]      Loss: 0.658827
Train Epoch: 1 [25600/60000 (43%)]      Loss: 0.599369
Train Epoch: 1 [26240/60000 (44%)]      Loss: 0.925203
Train Epoch: 1 [26880/60000 (45%)]      Loss: 0.569247
Train Epoch: 1 [27520/60000 (46%)]      Loss: 0.595495
Train Epoch: 1 [28160/60000 (47%)]      Loss: 0.787057
Train Epoch: 1 [28800/60000 (48%)]      Loss: 0.709622
Train Epoch: 1 [29440/60000 (49%)]      Loss: 0.771295
Train Epoch: 1 [30080/60000 (50%)]      Loss: 0.459200
Train Epoch: 1 [30720/60000 (51%)]      Loss: 0.515915
Train Epoch: 1 [31360/60000 (52%)]      Loss: 0.510923
Train Epoch: 1 [32000/60000 (53%)]      Loss: 0.789831
Train Epoch: 1 [32640/60000 (54%)]      Loss: 0.668127
Train Epoch: 1 [33280/60000 (55%)]      Loss: 0.596397
Train Epoch: 1 [33920/60000 (57%)]      Loss: 0.545272
Train Epoch: 1 [34560/60000 (58%)]      Loss: 0.791866
Train Epoch: 1 [35200/60000 (59%)]      Loss: 0.623321
Train Epoch: 1 [35840/60000 (60%)]      Loss: 0.501244
Train Epoch: 1 [36480/60000 (61%)]      Loss: 0.753823
Train Epoch: 1 [37120/60000 (62%)]      Loss: 0.610182
Train Epoch: 1 [37760/60000 (63%)]      Loss: 0.472686
Train Epoch: 1 [38400/60000 (64%)]      Loss: 0.742567
Train Epoch: 1 [39040/60000 (65%)]      Loss: 0.577439
Train Epoch: 1 [39680/60000 (66%)]      Loss: 0.659224
Train Epoch: 1 [40320/60000 (67%)]      Loss: 0.758697
Train Epoch: 1 [40960/60000 (68%)]      Loss: 0.801238
Train Epoch: 1 [41600/60000 (69%)]      Loss: 0.456015
Train Epoch: 1 [42240/60000 (70%)]      Loss: 0.584893
Train Epoch: 1 [42880/60000 (71%)]      Loss: 0.763128
Train Epoch: 1 [43520/60000 (72%)]      Loss: 0.408243
Train Epoch: 1 [44160/60000 (74%)]      Loss: 0.462214
Train Epoch: 1 [44800/60000 (75%)]      Loss: 0.511222
Train Epoch: 1 [45440/60000 (76%)]      Loss: 0.520635
Train Epoch: 1 [46080/60000 (77%)]      Loss: 0.404983
Train Epoch: 1 [46720/60000 (78%)]      Loss: 0.551706
Train Epoch: 1 [47360/60000 (79%)]      Loss: 0.539516
Train Epoch: 1 [48000/60000 (80%)]      Loss: 0.649675
Train Epoch: 1 [48640/60000 (81%)]      Loss: 0.387086
Train Epoch: 1 [49280/60000 (82%)]      Loss: 0.531161
Train Epoch: 1 [49920/60000 (83%)]      Loss: 0.468416
Train Epoch: 1 [50560/60000 (84%)]      Loss: 0.313190
Train Epoch: 1 [51200/60000 (85%)]      Loss: 0.473787
Train Epoch: 1 [51840/60000 (86%)]      Loss: 0.313375
Train Epoch: 1 [52480/60000 (87%)]      Loss: 0.563074
Train Epoch: 1 [53120/60000 (88%)]      Loss: 0.639663
Train Epoch: 1 [53760/60000 (90%)]      Loss: 0.598785
Train Epoch: 1 [54400/60000 (91%)]      Loss: 0.604863
Train Epoch: 1 [55040/60000 (92%)]      Loss: 0.433123
Train Epoch: 1 [55680/60000 (93%)]      Loss: 0.479933
Train Epoch: 1 [56320/60000 (94%)]      Loss: 0.384417
Train Epoch: 1 [56960/60000 (95%)]      Loss: 0.572573
Train Epoch: 1 [57600/60000 (96%)]      Loss: 0.474651
Train Epoch: 1 [58240/60000 (97%)]      Loss: 0.420517
Train Epoch: 1 [58880/60000 (98%)]      Loss: 0.468604
Train Epoch: 1 [59520/60000 (99%)]      Loss: 0.532335
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94

训练 3 个循环的效果

Train Epoch: 1 [0/60000 (0%)]   Loss: 2.371851
Train Epoch: 1 [640/60000 (1%)] Loss: 2.366941
Train Epoch: 1 [1280/60000 (2%)]        Loss: 2.253085
Train Epoch: 1 [1920/60000 (3%)]        Loss: 2.258119
Train Epoch: 1 [2560/60000 (4%)]        Loss: 2.279200
Train Epoch: 1 [3200/60000 (5%)]        Loss: 2.284118
Train Epoch: 1 [3840/60000 (6%)]        Loss: 2.224427
Train Epoch: 1 [4480/60000 (7%)]        Loss: 2.182901
Train Epoch: 1 [5120/60000 (9%)]        Loss: 2.092957
Train Epoch: 1 [5760/60000 (10%)]       Loss: 2.014003
Train Epoch: 1 [6400/60000 (11%)]       Loss: 2.066881
Train Epoch: 1 [7040/60000 (12%)]       Loss: 1.937319
Train Epoch: 1 [7680/60000 (13%)]       Loss: 1.780319
Train Epoch: 1 [8320/60000 (14%)]       Loss: 1.727874
Train Epoch: 1 [8960/60000 (15%)]       Loss: 1.722512
Train Epoch: 1 [9600/60000 (16%)]       Loss: 1.627026
Train Epoch: 1 [10240/60000 (17%)]      Loss: 1.649031
Train Epoch: 1 [10880/60000 (18%)]      Loss: 1.317975
Train Epoch: 1 [11520/60000 (19%)]      Loss: 1.215969
Train Epoch: 1 [12160/60000 (20%)]      Loss: 1.311500
Train Epoch: 1 [12800/60000 (21%)]      Loss: 1.205322
Train Epoch: 1 [13440/60000 (22%)]      Loss: 1.249098
Train Epoch: 1 [14080/60000 (23%)]      Loss: 1.102184
Train Epoch: 1 [14720/60000 (25%)]      Loss: 1.101858
Train Epoch: 1 [15360/60000 (26%)]      Loss: 1.056550
Train Epoch: 1 [16000/60000 (27%)]      Loss: 0.966378
Train Epoch: 1 [16640/60000 (28%)]      Loss: 0.898759
Train Epoch: 1 [17280/60000 (29%)]      Loss: 1.141907
Train Epoch: 1 [17920/60000 (30%)]      Loss: 0.744676
Train Epoch: 1 [18560/60000 (31%)]      Loss: 0.865845
Train Epoch: 1 [19200/60000 (32%)]      Loss: 0.743363
Train Epoch: 1 [19840/60000 (33%)]      Loss: 0.896839
Train Epoch: 1 [20480/60000 (34%)]      Loss: 0.797301
Train Epoch: 1 [21120/60000 (35%)]      Loss: 0.760073
Train Epoch: 1 [21760/60000 (36%)]      Loss: 0.796578
Train Epoch: 1 [22400/60000 (37%)]      Loss: 0.632670
Train Epoch: 1 [23040/60000 (38%)]      Loss: 0.671789
Train Epoch: 1 [23680/60000 (39%)]      Loss: 0.852199
Train Epoch: 1 [24320/60000 (41%)]      Loss: 0.684957
Train Epoch: 1 [24960/60000 (42%)]      Loss: 0.658827
Train Epoch: 1 [25600/60000 (43%)]      Loss: 0.599369
Train Epoch: 1 [26240/60000 (44%)]      Loss: 0.925203
Train Epoch: 1 [26880/60000 (45%)]      Loss: 0.569247
Train Epoch: 1 [27520/60000 (46%)]      Loss: 0.595495
Train Epoch: 1 [28160/60000 (47%)]      Loss: 0.787057
Train Epoch: 1 [28800/60000 (48%)]      Loss: 0.709622
Train Epoch: 1 [29440/60000 (49%)]      Loss: 0.771295
Train Epoch: 1 [30080/60000 (50%)]      Loss: 0.459200
Train Epoch: 1 [30720/60000 (51%)]      Loss: 0.515915
Train Epoch: 1 [31360/60000 (52%)]      Loss: 0.510923
Train Epoch: 1 [32000/60000 (53%)]      Loss: 0.789831
Train Epoch: 1 [32640/60000 (54%)]      Loss: 0.668127
Train Epoch: 1 [33280/60000 (55%)]      Loss: 0.596397
Train Epoch: 1 [33920/60000 (57%)]      Loss: 0.545272
Train Epoch: 1 [34560/60000 (58%)]      Loss: 0.791866
Train Epoch: 1 [35200/60000 (59%)]      Loss: 0.623321
Train Epoch: 1 [35840/60000 (60%)]      Loss: 0.501244
Train Epoch: 1 [36480/60000 (61%)]      Loss: 0.753823
Train Epoch: 1 [37120/60000 (62%)]      Loss: 0.610182
Train Epoch: 1 [37760/60000 (63%)]      Loss: 0.472686
Train Epoch: 1 [38400/60000 (64%)]      Loss: 0.742567
Train Epoch: 1 [39040/60000 (65%)]      Loss: 0.577439
Train Epoch: 1 [39680/60000 (66%)]      Loss: 0.659224
Train Epoch: 1 [40320/60000 (67%)]      Loss: 0.758697
Train Epoch: 1 [40960/60000 (68%)]      Loss: 0.801238
Train Epoch: 1 [41600/60000 (69%)]      Loss: 0.456015
Train Epoch: 1 [42240/60000 (70%)]      Loss: 0.584893
Train Epoch: 1 [42880/60000 (71%)]      Loss: 0.763128
Train Epoch: 1 [43520/60000 (72%)]      Loss: 0.408243
Train Epoch: 1 [44160/60000 (74%)]      Loss: 0.462214
Train Epoch: 1 [44800/60000 (75%)]      Loss: 0.511222
Train Epoch: 1 [45440/60000 (76%)]      Loss: 0.520635
Train Epoch: 1 [46080/60000 (77%)]      Loss: 0.404983
Train Epoch: 1 [46720/60000 (78%)]      Loss: 0.551706
Train Epoch: 1 [47360/60000 (79%)]      Loss: 0.539516
Train Epoch: 1 [48000/60000 (80%)]      Loss: 0.649675
Train Epoch: 1 [48640/60000 (81%)]      Loss: 0.387086
Train Epoch: 1 [49280/60000 (82%)]      Loss: 0.531161
Train Epoch: 1 [49920/60000 (83%)]      Loss: 0.468416
Train Epoch: 1 [50560/60000 (84%)]      Loss: 0.313190
Train Epoch: 1 [51200/60000 (85%)]      Loss: 0.473787
Train Epoch: 1 [51840/60000 (86%)]      Loss: 0.313375
Train Epoch: 1 [52480/60000 (87%)]      Loss: 0.563074
Train Epoch: 1 [53120/60000 (88%)]      Loss: 0.639663
Train Epoch: 1 [53760/60000 (90%)]      Loss: 0.598785
Train Epoch: 1 [54400/60000 (91%)]      Loss: 0.604863
Train Epoch: 1 [55040/60000 (92%)]      Loss: 0.433123
Train Epoch: 1 [55680/60000 (93%)]      Loss: 0.479933
Train Epoch: 1 [56320/60000 (94%)]      Loss: 0.384417
Train Epoch: 1 [56960/60000 (95%)]      Loss: 0.572573
Train Epoch: 1 [57600/60000 (96%)]      Loss: 0.474651
Train Epoch: 1 [58240/60000 (97%)]      Loss: 0.420517
Train Epoch: 1 [58880/60000 (98%)]      Loss: 0.468604
Train Epoch: 1 [59520/60000 (99%)]      Loss: 0.532335

Test set: Avg. loss: 0.2090, Accuracy: 9399/10000 (94%)

Train Epoch: 2 [0/60000 (0%)]   Loss: 0.366734
Train Epoch: 2 [640/60000 (1%)] Loss: 0.722454
Train Epoch: 2 [1280/60000 (2%)]        Loss: 0.336176
Train Epoch: 2 [1920/60000 (3%)]        Loss: 0.748185
Train Epoch: 2 [2560/60000 (4%)]        Loss: 0.519431
Train Epoch: 2 [3200/60000 (5%)]        Loss: 0.469481
Train Epoch: 2 [3840/60000 (6%)]        Loss: 0.484743
Train Epoch: 2 [4480/60000 (7%)]        Loss: 0.242931
Train Epoch: 2 [5120/60000 (9%)]        Loss: 0.713406
Train Epoch: 2 [5760/60000 (10%)]       Loss: 0.832393
Train Epoch: 2 [6400/60000 (11%)]       Loss: 0.454071
Train Epoch: 2 [7040/60000 (12%)]       Loss: 0.420207
Train Epoch: 2 [7680/60000 (13%)]       Loss: 0.510488
Train Epoch: 2 [8320/60000 (14%)]       Loss: 0.472568
Train Epoch: 2 [8960/60000 (15%)]       Loss: 0.380424
Train Epoch: 2 [9600/60000 (16%)]       Loss: 0.344593
Train Epoch: 2 [10240/60000 (17%)]      Loss: 0.400106
Train Epoch: 2 [10880/60000 (18%)]      Loss: 0.357579
Train Epoch: 2 [11520/60000 (19%)]      Loss: 0.244068
Train Epoch: 2 [12160/60000 (20%)]      Loss: 0.329613
Train Epoch: 2 [12800/60000 (21%)]      Loss: 0.395200
Train Epoch: 2 [13440/60000 (22%)]      Loss: 0.680711
Train Epoch: 2 [14080/60000 (23%)]      Loss: 0.523370
Train Epoch: 2 [14720/60000 (25%)]      Loss: 0.399706
Train Epoch: 2 [15360/60000 (26%)]      Loss: 0.374576
Train Epoch: 2 [16000/60000 (27%)]      Loss: 0.401027
Train Epoch: 2 [16640/60000 (28%)]      Loss: 0.667782
Train Epoch: 2 [17280/60000 (29%)]      Loss: 0.325726
Train Epoch: 2 [17920/60000 (30%)]      Loss: 0.424247
Train Epoch: 2 [18560/60000 (31%)]      Loss: 0.499896
Train Epoch: 2 [19200/60000 (32%)]      Loss: 0.451446
Train Epoch: 2 [19840/60000 (33%)]      Loss: 0.439135
Train Epoch: 2 [20480/60000 (34%)]      Loss: 0.487537
Train Epoch: 2 [21120/60000 (35%)]      Loss: 0.441139
Train Epoch: 2 [21760/60000 (36%)]      Loss: 0.402886
Train Epoch: 2 [22400/60000 (37%)]      Loss: 0.291915
Train Epoch: 2 [23040/60000 (38%)]      Loss: 0.636113
Train Epoch: 2 [23680/60000 (39%)]      Loss: 0.269374
Train Epoch: 2 [24320/60000 (41%)]      Loss: 0.392922
Train Epoch: 2 [24960/60000 (42%)]      Loss: 0.266148
Train Epoch: 2 [25600/60000 (43%)]      Loss: 0.380597
Train Epoch: 2 [26240/60000 (44%)]      Loss: 0.344070
Train Epoch: 2 [26880/60000 (45%)]      Loss: 0.477857
Train Epoch: 2 [27520/60000 (46%)]      Loss: 0.606165
Train Epoch: 2 [28160/60000 (47%)]      Loss: 0.551639
Train Epoch: 2 [28800/60000 (48%)]      Loss: 0.558916
Train Epoch: 2 [29440/60000 (49%)]      Loss: 0.749660
Train Epoch: 2 [30080/60000 (50%)]      Loss: 0.332171
Train Epoch: 2 [30720/60000 (51%)]      Loss: 0.375500
Train Epoch: 2 [31360/60000 (52%)]      Loss: 0.362925
Train Epoch: 2 [32000/60000 (53%)]      Loss: 0.306489
Train Epoch: 2 [32640/60000 (54%)]      Loss: 0.299393
Train Epoch: 2 [33280/60000 (55%)]      Loss: 0.208521
Train Epoch: 2 [33920/60000 (57%)]      Loss: 0.348526
Train Epoch: 2 [34560/60000 (58%)]      Loss: 0.508665
Train Epoch: 2 [35200/60000 (59%)]      Loss: 0.190389
Train Epoch: 2 [35840/60000 (60%)]      Loss: 0.291090
Train Epoch: 2 [36480/60000 (61%)]      Loss: 0.357614
Train Epoch: 2 [37120/60000 (62%)]      Loss: 0.316668
Train Epoch: 2 [37760/60000 (63%)]      Loss: 0.311942
Train Epoch: 2 [38400/60000 (64%)]      Loss: 0.323030
Train Epoch: 2 [39040/60000 (65%)]      Loss: 0.327689
Train Epoch: 2 [39680/60000 (66%)]      Loss: 0.471894
Train Epoch: 2 [40320/60000 (67%)]      Loss: 0.242763
Train Epoch: 2 [40960/60000 (68%)]      Loss: 0.430212
Train Epoch: 2 [41600/60000 (69%)]      Loss: 0.180888
Train Epoch: 2 [42240/60000 (70%)]      Loss: 0.276546
Train Epoch: 2 [42880/60000 (71%)]      Loss: 0.266703
Train Epoch: 2 [43520/60000 (72%)]      Loss: 0.253300
Train Epoch: 2 [44160/60000 (74%)]      Loss: 0.492046
Train Epoch: 2 [44800/60000 (75%)]      Loss: 0.615155
Train Epoch: 2 [45440/60000 (76%)]      Loss: 0.344643
Train Epoch: 2 [46080/60000 (77%)]      Loss: 0.308926
Train Epoch: 2 [46720/60000 (78%)]      Loss: 0.338160
Train Epoch: 2 [47360/60000 (79%)]      Loss: 0.440104
Train Epoch: 2 [48000/60000 (80%)]      Loss: 0.481084
Train Epoch: 2 [48640/60000 (81%)]      Loss: 0.326619
Train Epoch: 2 [49280/60000 (82%)]      Loss: 0.298519
Train Epoch: 2 [49920/60000 (83%)]      Loss: 0.357107
Train Epoch: 2 [50560/60000 (84%)]      Loss: 0.274719
Train Epoch: 2 [51200/60000 (85%)]      Loss: 0.345895
Train Epoch: 2 [51840/60000 (86%)]      Loss: 0.241214
Train Epoch: 2 [52480/60000 (87%)]      Loss: 0.243374
Train Epoch: 2 [53120/60000 (88%)]      Loss: 0.191087
Train Epoch: 2 [53760/60000 (90%)]      Loss: 0.287821
Train Epoch: 2 [54400/60000 (91%)]      Loss: 0.237997
Train Epoch: 2 [55040/60000 (92%)]      Loss: 0.274627
Train Epoch: 2 [55680/60000 (93%)]      Loss: 0.253250
Train Epoch: 2 [56320/60000 (94%)]      Loss: 0.309018
Train Epoch: 2 [56960/60000 (95%)]      Loss: 0.277265
Train Epoch: 2 [57600/60000 (96%)]      Loss: 0.276998
Train Epoch: 2 [58240/60000 (97%)]      Loss: 0.459611
Train Epoch: 2 [58880/60000 (98%)]      Loss: 0.516984
Train Epoch: 2 [59520/60000 (99%)]      Loss: 0.311498

Test set: Avg. loss: 0.1252, Accuracy: 9623/10000 (96%)

Train Epoch: 3 [0/60000 (0%)]   Loss: 0.454000
Train Epoch: 3 [640/60000 (1%)] Loss: 0.274891
Train Epoch: 3 [1280/60000 (2%)]        Loss: 0.390851
Train Epoch: 3 [1920/60000 (3%)]        Loss: 0.156189
Train Epoch: 3 [2560/60000 (4%)]        Loss: 0.419210
Train Epoch: 3 [3200/60000 (5%)]        Loss: 0.300367
Train Epoch: 3 [3840/60000 (6%)]        Loss: 0.256845
Train Epoch: 3 [4480/60000 (7%)]        Loss: 0.418629
Train Epoch: 3 [5120/60000 (9%)]        Loss: 0.171152
Train Epoch: 3 [5760/60000 (10%)]       Loss: 0.321221
Train Epoch: 3 [6400/60000 (11%)]       Loss: 0.424148
Train Epoch: 3 [7040/60000 (12%)]       Loss: 0.143154
Train Epoch: 3 [7680/60000 (13%)]       Loss: 0.488087
Train Epoch: 3 [8320/60000 (14%)]       Loss: 0.402844
Train Epoch: 3 [8960/60000 (15%)]       Loss: 0.417593
Train Epoch: 3 [9600/60000 (16%)]       Loss: 0.276311
Train Epoch: 3 [10240/60000 (17%)]      Loss: 0.503877
Train Epoch: 3 [10880/60000 (18%)]      Loss: 0.387446
Train Epoch: 3 [11520/60000 (19%)]      Loss: 0.612089
Train Epoch: 3 [12160/60000 (20%)]      Loss: 0.430010
Train Epoch: 3 [12800/60000 (21%)]      Loss: 0.300433
Train Epoch: 3 [13440/60000 (22%)]      Loss: 0.346322
Train Epoch: 3 [14080/60000 (23%)]      Loss: 0.187743
Train Epoch: 3 [14720/60000 (25%)]      Loss: 0.330318
Train Epoch: 3 [15360/60000 (26%)]      Loss: 0.201379
Train Epoch: 3 [16000/60000 (27%)]      Loss: 0.264247
Train Epoch: 3 [16640/60000 (28%)]      Loss: 0.305077
Train Epoch: 3 [17280/60000 (29%)]      Loss: 0.386015
Train Epoch: 3 [17920/60000 (30%)]      Loss: 0.267599
Train Epoch: 3 [18560/60000 (31%)]      Loss: 0.245534
Train Epoch: 3 [19200/60000 (32%)]      Loss: 0.407366
Train Epoch: 3 [19840/60000 (33%)]      Loss: 0.231558
Train Epoch: 3 [20480/60000 (34%)]      Loss: 0.134505
Train Epoch: 3 [21120/60000 (35%)]      Loss: 0.332575
Train Epoch: 3 [21760/60000 (36%)]      Loss: 0.298820
Train Epoch: 3 [22400/60000 (37%)]      Loss: 0.283264
Train Epoch: 3 [23040/60000 (38%)]      Loss: 0.267990
Train Epoch: 3 [23680/60000 (39%)]      Loss: 0.274185
Train Epoch: 3 [24320/60000 (41%)]      Loss: 0.258080
Train Epoch: 3 [24960/60000 (42%)]      Loss: 0.235735
Train Epoch: 3 [25600/60000 (43%)]      Loss: 0.313570
Train Epoch: 3 [26240/60000 (44%)]      Loss: 0.248482
Train Epoch: 3 [26880/60000 (45%)]      Loss: 0.292335
Train Epoch: 3 [27520/60000 (46%)]      Loss: 0.327348
Train Epoch: 3 [28160/60000 (47%)]      Loss: 0.283135
Train Epoch: 3 [28800/60000 (48%)]      Loss: 0.441134
Train Epoch: 3 [29440/60000 (49%)]      Loss: 0.311160
Train Epoch: 3 [30080/60000 (50%)]      Loss: 0.424866
Train Epoch: 3 [30720/60000 (51%)]      Loss: 0.355469
Train Epoch: 3 [31360/60000 (52%)]      Loss: 0.434249
Train Epoch: 3 [32000/60000 (53%)]      Loss: 0.205223
Train Epoch: 3 [32640/60000 (54%)]      Loss: 0.270712
Train Epoch: 3 [33280/60000 (55%)]      Loss: 0.189404
Train Epoch: 3 [33920/60000 (57%)]      Loss: 0.182121
Train Epoch: 3 [34560/60000 (58%)]      Loss: 0.214576
Train Epoch: 3 [35200/60000 (59%)]      Loss: 0.315236
Train Epoch: 3 [35840/60000 (60%)]      Loss: 0.367852
Train Epoch: 3 [36480/60000 (61%)]      Loss: 0.402626
Train Epoch: 3 [37120/60000 (62%)]      Loss: 0.494649
Train Epoch: 3 [37760/60000 (63%)]      Loss: 0.473171
Train Epoch: 3 [38400/60000 (64%)]      Loss: 0.465688
Train Epoch: 3 [39040/60000 (65%)]      Loss: 0.401424
Train Epoch: 3 [39680/60000 (66%)]      Loss: 0.252164
Train Epoch: 3 [40320/60000 (67%)]      Loss: 0.283355
Train Epoch: 3 [40960/60000 (68%)]      Loss: 0.214266
Train Epoch: 3 [41600/60000 (69%)]      Loss: 0.197134
Train Epoch: 3 [42240/60000 (70%)]      Loss: 0.411207
Train Epoch: 3 [42880/60000 (71%)]      Loss: 0.330771
Train Epoch: 3 [43520/60000 (72%)]      Loss: 0.207687
Train Epoch: 3 [44160/60000 (74%)]      Loss: 0.296396
Train Epoch: 3 [44800/60000 (75%)]      Loss: 0.447102
Train Epoch: 3 [45440/60000 (76%)]      Loss: 0.151577
Train Epoch: 3 [46080/60000 (77%)]      Loss: 0.134489
Train Epoch: 3 [46720/60000 (78%)]      Loss: 0.292283
Train Epoch: 3 [47360/60000 (79%)]      Loss: 0.256751
Train Epoch: 3 [48000/60000 (80%)]      Loss: 0.130260
Train Epoch: 3 [48640/60000 (81%)]      Loss: 0.217153
Train Epoch: 3 [49280/60000 (82%)]      Loss: 0.496362
Train Epoch: 3 [49920/60000 (83%)]      Loss: 0.567312
Train Epoch: 3 [50560/60000 (84%)]      Loss: 0.246096
Train Epoch: 3 [51200/60000 (85%)]      Loss: 0.368138
Train Epoch: 3 [51840/60000 (86%)]      Loss: 0.300753
Train Epoch: 3 [52480/60000 (87%)]      Loss: 0.341426
Train Epoch: 3 [53120/60000 (88%)]      Loss: 0.091202
Train Epoch: 3 [53760/60000 (90%)]      Loss: 0.310372
Train Epoch: 3 [54400/60000 (91%)]      Loss: 0.188257
Train Epoch: 3 [55040/60000 (92%)]      Loss: 0.238355
Train Epoch: 3 [55680/60000 (93%)]      Loss: 0.366855
Train Epoch: 3 [56320/60000 (94%)]      Loss: 0.340705
Train Epoch: 3 [56960/60000 (95%)]      Loss: 0.322747
Train Epoch: 3 [57600/60000 (96%)]      Loss: 0.388191
Train Epoch: 3 [58240/60000 (97%)]      Loss: 0.154319
Train Epoch: 3 [58880/60000 (98%)]      Loss: 0.207342
Train Epoch: 3 [59520/60000 (99%)]      Loss: 0.458232

Test set: Avg. loss: 0.1009, Accuracy: 9685/10000 (97%)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290

7.手写图片验证

1.进行多次训练

注释掉训练日志打印，基于已有参数训练，多调用几次脚本
训练次数已经没什么效果了
1
2

2.测试

通过上面查看数据方法，去测试集截一张图用来验证

第一个 8 输出了 5

第三行 第三列的 8 输出了 8

自己手写数字，截图 8 进行识别

自己手写数字，截图 2 进行识别

自己手写数字，截图 5 进行识别

数字 8 的效果不太理想，感觉可能和图片尺寸有关，有时间可以在细究一下
1