• 深度学习10——卷积神经网络

    目录

    1.全连接网络复习 

     2.卷积

     2.1 卷积核​

     2.2 卷积层的基本实现

     2.3 padding填充

     2.4 stride步长

     2.5 池化层

     3. CNN实例

    3.1 完整代码：

    3.1.1 cpu训练

    3.1.2 GPU上训练

    4. 卷积神经网络进阶

     4.1 GoogLeNet

    4.2 inception Module

    4.2.1  1*1卷积及其作用

     4.2.2 implemetaion of inception Module​编辑

    4.2.3 Inception Module 以及模型构建的实现代码

     完整代码：

    4.3 residual Network 

    4.3.1 Residual Block的代码实现

    参考文章： 

    PyTorch 深度学习实践 第10讲_错错莫的博客-CSDN博客

    1.全连接网络复习 

     2.卷积

     

     

     2.1 卷积核

     

     

     

     

     2.2 卷积层的基本实现

    1. import torch
    2.  
    3. in_channels, out_channels = 5, 10 #定义输入输出的通道数
    4. width, hight = 100, 100  #定义图片的大小
    5. kernel_size = 3 # 定义的卷积核是一个3*3的
    6. batch_size = 1
    7.  
    8. #输入层
    9. input = torch.randn(batch_size,
    10.                     in_channels,
    11.                     width,
    12.                     hight
    13.                     )
    14.  
    15. # 定义卷积层
    16. conv_layer = torch.nn.Conv2d(
    17.     in_channels,
    18.     out_channels,
    19.     kernel_size=kernel_size
    20. )
    21.  
    22. #定义输出层
    23. output = conv_layer(input)
    24.  
    25. print(input.shape)
    26. print(output.shape)
    27. print(conv_layer.weight.shape)
    1. torch.Size([1, 5, 100, 100])
    2. torch.Size([1, 10, 98, 98])
    3. torch.Size([10, 5, 3, 3])

     2.3 padding填充

     

     

     

    1. import torch
    2.  
    3. #以下数据假设是输入的图像像素点
    4. input =[
    5.      3,4,6,5,7,
    6.      2,4,6,8,2,
    7.      1,6,7,8,4,
    8.      9,7,4,6,2,
    9.      3,7,5,4,1
    10.  ]
    11.  
    12. input = torch.Tensor(input).view(1,1,5,5) #将input的数据转化为tensor类型，并设置B,C,W,H为1，1，5，5
    13.  
    14. #padding填充
    15. conv_layer = torch.nn.Conv2d(1,1,kernel_size=3,padding=1,bias=False) #前两个数字对应的是输入与输出对应的通道数
    16.  
    17. kernel = torch.Tensor([1,2,3,4,5,6,7,8,9]).view(1,1,3,3) # 构造卷积核，view（输出的通道(卷积核的个数)，输入的通道数（卷积核的通道数），3*3）
    18. conv_layer.weight.data = kernel.data # 将卷积核的权重赋给卷积层
    19.  
    20. output = conv_layer(input) #做卷积运算
    21. print(output)
    22.  
    23.

     2.4 stride步长

     

    1. import torch
    2.  
    3. #以下数据假设是输入的图像像素点
    4. input =[
    5.      3,4,6,5,7,
    6.      2,4,6,8,2,
    7.      1,6,7,8,4,
    8.      9,7,4,6,2,
    9.      3,7,5,4,1
    10.  ]
    11.  
    12. input = torch.Tensor(input).view(1,1,5,5) #将input的数据转化为tensor类型，并设置B,C,W,H为1，1，5，5
    13.  
    14. #stride步长
    15. conv_layer = torch.nn.Conv2d(1,1,kernel_size=3,stride=2,bias=False)
    16.  
    17. kernel = torch.Tensor([1,2,3,4,5,6,7,8,9]).view(1,1,3,3) # 构造卷积核，view（输出的通道(卷积核的个数)，输入的通道数（卷积核的通道数），3*3）
    18. conv_layer.weight.data = kernel.data # 将卷积核的权重赋给卷积层
    19.  
    20. output = conv_layer(input) #做卷积运算
    21. print(output)
    22.  
    23.

     

     2.5 池化层

     

    1. import torch
    2.  
    3. #以下数据假设是输入的图像像素点
    4. input =[
    5.      3,4,6,5,7,
    6.      2,4,6,8,2,
    7.      1,6,7,8,4,
    8.      9,7,4,6,2,
    9.      3,7,5,4,1
    10.  ]
    11.  
    12. input = torch.Tensor(input).view(1,1,5,5) #将input的数据转化为tensor类型，并设置B,C,W,H为1，1，5，5
    13. #池化层
    14. maxpooling_layer = torch.nn.MaxPool2d(kernel_size=2)
    15. output = maxpooling_layer(input) #做卷积运算
    16. print(output)
    17.  
    18.

     3. CNN实例

     

     

     

     

     

                                                                                                

     

     

     

    3.1 完整代码：

    代码说明：

    1、torch.nn.Conv2d(1,10,kernel_size=3,stride=2,bias=False)

          1是指输入的Channel，灰色图像是1维的；10是指输出的Channel，也可以说第一个卷积层需要10个卷积核；kernel_size=3,卷积核大小是3x3；stride=2进行卷积运算时的步长，默认为1；bias=False卷积运算是否需要偏置bias，默认为False。padding = 0，卷积操作是否补0。

    2、self.fc = torch.nn.Linear(320, 10)，这个320获取的方式，可以通过x = x.view(batch_size, -1) # print(x.shape)可得到(64,320),64指的是batch，320就是指要进行全连接操作时，输入的特征维度。


    3.1.1 cpu训练

    1. import torch
    2. from torchvision import transforms
    3. from torchvision import datasets
    4. from torch.utils.data import DataLoader
    5. import torch.nn.functional as F
    6. import torch.optim as optim
    7.  
    8. # prepare dataset
    9.  
    10. batch_size = 64
    11. transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
    12.  
    13. train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
    14. train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
    15. test_dataset = datasets.MNIST(root='../dataset/mnist/', train=False, download=True, transform=transform)
    16. test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)
    17.  
    18.  
    19. # design model using class 构建网络模型
    20. class Net(torch.nn.Module):
    21.     def __init__(self):
    22.         super(Net, self).__init__()
    23.         self.conv1 = torch.nn.Conv2d(1, 10, kernel_size=5) #卷积核为5*5
    24.         self.conv2 = torch.nn.Conv2d(10, 20, kernel_size=5)
    25.         self.pooling = torch.nn.MaxPool2d(2)
    26.         self.fc = torch.nn.Linear(320, 10) #全连接层,320 = 20*4*4
    27.  
    28.     def forward(self, x):
    29.         # flatten data from (n,1,28,28) to (n, 784)
    30.         batch_size = x.size(0) #样本的数据量
    31.         x = F.relu(self.pooling(self.conv1(x))) #执行第一层卷积运算
    32.         x = F.relu(self.pooling(self.conv2(x)))
    33.         # 以下两步用于全连接运算
    34.         x = x.view(batch_size, -1)  # -1 此处自动算出的是320
    35.         x = self.fc(x)
    36.  
    37.         return x
    38.  
    39.  
    40. model = Net()
    41.  
    42. # construct loss and optimizer
    43. criterion = torch.nn.CrossEntropyLoss()
    44. optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
    45.  
    46.  
    47. # training cycle forward, backward, update
    48.  
    49.  
    50. def train(epoch):
    51.     running_loss = 0.0
    52.     for batch_idx, data in enumerate(train_loader, 0):
    53.         inputs, target = data
    54.         optimizer.zero_grad()
    55.  
    56.         outputs = model(inputs)
    57.         loss = criterion(outputs, target)
    58.         loss.backward()
    59.         optimizer.step()
    60.  
    61.         running_loss += loss.item()
    62.         if batch_idx % 300 == 299:
    63.             print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
    64.             running_loss = 0.0
    65.  
    66.  
    67. def test():
    68.     correct = 0
    69.     total = 0
    70.     with torch.no_grad():
    71.         for data in test_loader:
    72.             images, labels = data
    73.             outputs = model(images)
    74.             _, predicted = torch.max(outputs.data, dim=1)
    75.             total += labels.size(0)
    76.             correct += (predicted == labels).sum().item()
    77.     print('accuracy on test set: %d %% ' % (100 * correct / total))
    78.  
    79.  
    80. if __name__ == '__main__':
    81.     for epoch in range(10):
    82.         train(epoch)
    83.         test()
    1. C:\Users\ZARD\anaconda3\envs\PyTorch\python.exe C:/Users/ZARD/PycharmProjects/pythonProject/机器学习库/PyTorch实战课程内容/lecture10/CNN01.py
    2. Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
    3. Using downloaded and verified file: ../dataset/mnist/MNIST\raw\train-images-idx3-ubyte.gz
    4. Extracting ../dataset/mnist/MNIST\raw\train-images-idx3-ubyte.gz to ../dataset/mnist/MNIST\raw
    5.  
    6. Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
    7. Using downloaded and verified file: ../dataset/mnist/MNIST\raw\train-labels-idx1-ubyte.gz
    8. Extracting ../dataset/mnist/MNIST\raw\train-labels-idx1-ubyte.gz to ../dataset/mnist/MNIST\raw
    9.  
    10. Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
    11. Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../dataset/mnist/MNIST\raw\t10k-images-idx3-ubyte.gz
    12. 100%|██████████| 1648877/1648877 [00:30<00:00, 53892.68it/s]
    13. Extracting ../dataset/mnist/MNIST\raw\t10k-images-idx3-ubyte.gz to ../dataset/mnist/MNIST\raw
    14.  
    15. Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
    16. Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../dataset/mnist/MNIST\raw\t10k-labels-idx1-ubyte.gz
    17. 100%|██████████| 4542/4542 [00:03<00:00, 1153.06it/s]
    18. Extracting ../dataset/mnist/MNIST\raw\t10k-labels-idx1-ubyte.gz to ../dataset/mnist/MNIST\raw
    19.  
    20. [1,   300] loss: 0.737
    21. [1,   600] loss: 0.213
    22. [1,   900] loss: 0.147
    23. accuracy on test set: 96 % 
    24. [2,   300] loss: 0.118
    25. [2,   600] loss: 0.105
    26. [2,   900] loss: 0.084
    27. accuracy on test set: 97 % 
    28. [3,   300] loss: 0.080
    29. [3,   600] loss: 0.071
    30. [3,   900] loss: 0.075
    31. accuracy on test set: 98 % 
    32. [4,   300] loss: 0.065
    33. [4,   600] loss: 0.060
    34. [4,   900] loss: 0.064
    35. accuracy on test set: 98 % 
    36. [5,   300] loss: 0.058
    37. [5,   600] loss: 0.054
    38. [5,   900] loss: 0.050
    39. accuracy on test set: 98 % 
    40. [6,   300] loss: 0.046
    41. [6,   600] loss: 0.051
    42. [6,   900] loss: 0.049
    43. accuracy on test set: 98 % 
    44. [7,   300] loss: 0.044
    45. [7,   600] loss: 0.046
    46. [7,   900] loss: 0.043
    47. accuracy on test set: 98 % 
    48. [8,   300] loss: 0.040
    49. [8,   600] loss: 0.044
    50. [8,   900] loss: 0.038
    51. accuracy on test set: 98 % 
    52. [9,   300] loss: 0.039
    53. [9,   600] loss: 0.037
    54. [9,   900] loss: 0.038
    55. accuracy on test set: 98 % 
    56. [10,   300] loss: 0.038
    57. [10,   600] loss: 0.035
    58. [10,   900] loss: 0.034
    59. accuracy on test set: 98 % 
    60.  
    61. Process finished with exit code 0

    3.1.2 GPU上训练

    1. import torch
    2. from torchvision import transforms
    3. from torchvision import datasets
    4. from torch.utils.data import DataLoader
    5. import torch.nn.functional as F
    6. import torch.optim as optim
    7. import matplotlib.pyplot as plt
    8.  
    9. # prepare dataset
    10.  
    11. batch_size = 64
    12. transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
    13.  
    14. train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
    15. train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
    16. test_dataset = datasets.MNIST(root='../dataset/mnist/', train=False, download=True, transform=transform)
    17. test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)
    18.  
    19. # design model using class
    20.  
    21.  
    22. class Net(torch.nn.Module):
    23.     def __init__(self):
    24.         super(Net, self).__init__()
    25.         self.conv1 = torch.nn.Conv2d(1, 10, kernel_size=5)
    26.         self.conv2 = torch.nn.Conv2d(10, 20, kernel_size=5)
    27.         self.pooling = torch.nn.MaxPool2d(2)
    28.         self.fc = torch.nn.Linear(320, 10)
    29.  
    30.  
    31.     def forward(self, x):
    32.         # flatten data from (n,1,28,28) to (n, 784)
    33.         
    34.         batch_size = x.size(0)
    35.         x = F.relu(self.pooling(self.conv1(x)))
    36.         x = F.relu(self.pooling(self.conv2(x)))
    37.         x = x.view(batch_size, -1) # -1 此处自动算出的是320
    38.         # print("x.shape",x.shape)
    39.         x = self.fc(x)
    40.  
    41.         return x
    42.  
    43.  
    44. model = Net()
    45. device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    46. model.to(device)
    47.  
    48. # construct loss and optimizer
    49. criterion = torch.nn.CrossEntropyLoss()
    50. optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
    51.  
    52. # training cycle forward, backward, update
    53.  
    54.  
    55. def train(epoch):
    56.     running_loss = 0.0
    57.     for batch_idx, data in enumerate(train_loader, 0):
    58.         inputs, target = data
    59.         inputs, target = inputs.to(device), target.to(device)
    60.         optimizer.zero_grad()
    61.  
    62.         outputs = model(inputs)
    63.         loss = criterion(outputs, target)
    64.         loss.backward()
    65.         optimizer.step()
    66.  
    67.         running_loss += loss.item()
    68.         if batch_idx % 300 == 299:
    69.             print('[%d, %5d] loss: %.3f' % (epoch+1, batch_idx+1, running_loss/300))
    70.             running_loss = 0.0
    71.  
    72.  
    73. def test():
    74.     correct = 0
    75.     total = 0
    76.     with torch.no_grad():
    77.         for data in test_loader:
    78.             images, labels = data
    79.             images, labels = images.to(device), labels.to(device)
    80.             outputs = model(images)
    81.             _, predicted = torch.max(outputs.data, dim=1)
    82.             total += labels.size(0)
    83.             correct += (predicted == labels).sum().item()
    84.     print('accuracy on test set: %d %% ' % (100*correct/total))
    85.     return correct/total
    86.  
    87.  
    88. if __name__ == '__main__':
    89.     epoch_list = []
    90.     acc_list = []
    91.     
    92.     for epoch in range(10):
    93.         train(epoch)
    94.         acc = test()
    95.         epoch_list.append(epoch)
    96.         acc_list.append(acc)
    97.     
    98.     plt.plot(epoch_list,acc_list)
    99.     plt.ylabel('accuracy')
    100.     plt.xlabel('epoch')
    101.     plt.show()
    102.

    4. 卷积神经网络进阶

     4.1 GoogLeNet

    4.2 inception Module

    4.2.1  1*1卷积及其作用

     

     

     4.2.2 implemetaion of inception Module

     

     

     

    如上class Net网络结构的第二层卷积的输入为88，是因为在其 froward()函数中，在调用第二层卷积self.conv2之前已经调用了InceptionA类的实例，而InceptionA类实例返回的结果是各网络分支所得的通道数之和，即24*3+16=88.

     

    4.2.3 Inception Module 以及模型构建的实现代码

    1. #将某分支网络抽象为一个Inception类
    2. class Inception(nn.Module):
    3.     def __init__(self,in_channels):
    4.         super(Inception,self).__init__()
    5.         #定义如下4个分支结构
    6.         self.branch1x1 = nn.Conv2d(in_channels,16,kernel_size=1) # 该分支为一层卷积核为1*1的16通道的卷积层
    7.  
    8.         #该分支为由两层构成，第一层卷积核1*1，输出通道为16；第二层卷积核为5*5，输出通道为24
    9.         self.branch5x5_1 = nn.Conv2d(in_channels,16,kernel_size=1)
    10.         self.branch5x5_2 = nn.Conv2d(16,24,kernel_size=5,padding=2)
    11.  
    12.         #第三分支
    13.         self.branch3x3_1 = nn.Conv2d(in_channels,16,kernel_size=1)
    14.         self.branch3x3_2 = nn.Conv2d(16,24,kernel_size=3,padding=1)
    15.         self.branch3x3_3 = nn.Conv2d(24,24,kernel_size=3,padding=1)
    16.  
    17.         #第四分支：池化分支
    18.         self.branch_pool = nn.Conv2d(in_channels,24,kernel_size=1)
    19.  
    20.     #计算各层分支的卷积和
    21.     def forward(self,x):
    22.         branch1x1 = self.branch1x1(x)
    23.  
    24.         branch5x5 = self.branch5x5_1(x)
    25.         branch5x5 = self.branch5x5_2(branch5x5)
    26.  
    27.         branch3x3 = self.branch3x3_1(x)
    28.         branch3x3 = self.branch3x3_3(branch3x3)
    29.         branch3x3 = self.branch3x3_3(branch3x3)
    30.  
    31.         branch_pool = F.avg_pool2d(x,kernel_size=3,stride=1,padding=1)
    32.         branch_pool = self.branch_pool(branch_pool)
    33.         
    34.         output = [branch1x1,branch5x5,branch3x3,branch_pool]
    35.         return torch.cat(output,dim=1)
    36.  
    37. class Net(nn.Module):
    38.     def __init__(self):
    39.         super(Net, self).__init__()
    40.         self.conv1 = nn.Conv2d(1,10,kernel_size=5)
    41.         self.conv2 = nn.Conv2d(88,20,kernel_size=5)
    42.         
    43.         #实例化分支网络结构Inception
    44.         self.incep1 = Inception(in_channels=10)
    45.         self.incep2 = Inception(in_channels=20)
    46.         
    47.         self.mp = nn.MaxPool2d(2) #池化
    48.         self.fc = nn.Linear(1408,10) #实例化全连接层，全连接层的实质就是一个线性层
    49.     
    50.     def forward(self,x):
    51.         in_side = x.size(0)
    52.         x = F.relu(self.mp(self.conv1(x)))
    53.         x = self.incep1(x) 
    54.         x = F.relu(self.mp(self.conv2(x)))
    55.         x = self.incep2(x) 
    56.         #全连接层
    57.         x = x.view(in_side,-1)
    58.         x = self.fc(x)
    59.         return x

     完整代码：

    1. import torch
    2. import torch.nn as nn
    3. from torchvision import transforms
    4. from torchvision import datasets
    5. from torch.utils.data import DataLoader
    6. import torch.nn.functional as F
    7. import torch.optim as optim
    8.  
    9. # prepare datasets 数据集的准备
    10.  
    11. batch_size = 64
    12. transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])  # 归一化,均值和方差
    13.  
    14. train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
    15. train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
    16.  
    17. test_dataset = datasets.MNIST(root='../dataset/mnist/', train=False, download=True, transform=transform)
    18. test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)
    19.  
    20. #将某分支网络抽象为一个Inception类
    21. class Inception(nn.Module):
    22.     def __init__(self,in_channels):
    23.         super(Inception,self).__init__()
    24.         #定义如下4个分支结构
    25.         self.branch1x1 = nn.Conv2d(in_channels, 16, kernel_size=1) # 该分支为一层卷积核为1*1的16通道的卷积层
    26.  
    27.         #该分支为由两层构成，第一层卷积核1*1，输出通道为16；第二层卷积核为5*5，输出通道为24
    28.         self.branch5x5_1 = nn.Conv2d(in_channels, 16, kernel_size=1)
    29.         self.branch5x5_2 = nn.Conv2d(16, 24, kernel_size=5, padding=2)
    30.  
    31.         #第三分支
    32.         self.branch3x3_1 = nn.Conv2d(in_channels, 16, kernel_size=1)
    33.         self.branch3x3_2 = nn.Conv2d(16, 24, kernel_size=3, padding=1)
    34.         self.branch3x3_3 = nn.Conv2d(24, 24, kernel_size=3, padding=1)
    35.  
    36.         #第四分支：池化分支
    37.         self.branch_pool = nn.Conv2d(in_channels,24,kernel_size=1)
    38.  
    39.     #计算各层分支的卷积和
    40.     def forward(self,x):
    41.         branch1x1 = self.branch1x1(x)
    42.  
    43.         branch5x5 = self.branch5x5_1(x)
    44.         branch5x5 = self.branch5x5_2(branch5x5)
    45.  
    46.         branch3x3 = self.branch3x3_1(x)
    47.         branch3x3 = self.branch3x3_2(branch3x3)
    48.         branch3x3 = self.branch3x3_3(branch3x3)
    49.  
    50.         branch_pool = F.avg_pool2d(x,kernel_size=3,stride=1,padding=1)
    51.         branch_pool = self.branch_pool(branch_pool)
    52.  
    53.         output = [branch1x1,branch5x5,branch3x3,branch_pool]
    54.         return torch.cat(output,dim=1)
    55.  
    56. class Net(nn.Module):
    57.     def __init__(self):
    58.         super(Net, self).__init__()
    59.         self.conv1 = nn.Conv2d(1,10,kernel_size=5)
    60.         self.conv2 = nn.Conv2d(88,20,kernel_size=5)
    61.  
    62.         #实例化分支网络结构Inception
    63.         self.incep1 = Inception(in_channels=10)
    64.         self.incep2 = Inception(in_channels=20)
    65.  
    66.         self.mp = nn.MaxPool2d(2) #池化
    67.         self.fc = nn.Linear(1408,10) #实例化全连接层，全连接层的实质就是一个线性层
    68.  
    69.     def forward(self,x):
    70.         in_side = x.size(0)
    71.         x = F.relu(self.mp(self.conv1(x)))
    72.         x = self.incep1(x)
    73.         x = F.relu(self.mp(self.conv2(x)))
    74.         x = self.incep2(x)
    75.         #全连接层
    76.         x = x.view(in_side,-1)
    77.         x = self.fc(x)
    78.         return x
    79.  
    80.  
    81. model = Net()
    82.  
    83. # construct loss and optimizer
    84. criterion = torch.nn.CrossEntropyLoss()
    85. optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
    86.  
    87.  
    88. # training cycle forward, backward, update
    89.  
    90.  
    91. def train(epoch):
    92.     running_loss = 0.0
    93.     for batch_idx, data in enumerate(train_loader, 0):
    94.         inputs, target = data
    95.         optimizer.zero_grad()
    96.  
    97.         outputs = model(inputs)
    98.         loss = criterion(outputs, target)
    99.         loss.backward()
    100.         optimizer.step()
    101.  
    102.         running_loss += loss.item()
    103.         if batch_idx % 300 == 299:
    104.             print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
    105.             running_loss = 0.0
    106.  
    107.  
    108. def test():
    109.     correct = 0
    110.     total = 0
    111.     with torch.no_grad():
    112.         for data in test_loader:
    113.             images, labels = data
    114.             outputs = model(images)
    115.             _, predicted = torch.max(outputs.data, dim=1)
    116.             total += labels.size(0)
    117.             correct += (predicted == labels).sum().item()
    118.     print('accuracy on test set: %d %% ' % (100 * correct / total))
    119.  
    120.  
    121. if __name__ == '__main__':
    122.     for epoch in range(10):
    123.         train(epoch)
    124.         test()
    125.  
    126.  
    127.  
    128.  
    129.  
    130.

    结果：

    1. C:\Users\ZARD\anaconda3\envs\PyTorch\python.exe C:/Users/ZARD/PycharmProjects/pythonProject/机器学习库/PyTorch实战课程内容/lecture10/CNN_plus.py
    2. [1,   300] loss: 0.884
    3. [1,   600] loss: 0.224
    4. [1,   900] loss: 0.144
    5. accuracy on test set: 96 % 
    6. [2,   300] loss: 0.108
    7. [2,   600] loss: 0.095
    8. [2,   900] loss: 0.087
    9. accuracy on test set: 97 % 
    10. [3,   300] loss: 0.077
    11. [3,   600] loss: 0.068
    12. [3,   900] loss: 0.069
    13. accuracy on test set: 98 % 
    14. [4,   300] loss: 0.060
    15. [4,   600] loss: 0.060
    16. [4,   900] loss: 0.062
    17. accuracy on test set: 98 %

     

    4.3 residual Network 

     

     

     

    4.3.1 Residual Block的代码实现

     

     代码：
     

    1.  
    2. # design model using class
    3. class ResidualBlock(nn.Module):
    4.     def __init__(self, channels):
    5.         super(ResidualBlock, self).__init__()
    6.         self.channels = channels
    7.         self.conv1 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
    8.         self.conv2 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
    9.  
    10.     def forward(self, x):
    11.         y = F.relu(self.conv1(x))
    12.         y = self.conv2(y)
    13.         return F.relu(x + y)
    14.  
    15. class Net(nn.Module):
    16.     def __init__(self):
    17.         super(Net, self).__init__()
    18.         self.conv1 = nn.Conv2d(1, 16, kernel_size=5)
    19.         self.conv2 = nn.Conv2d(16, 32, kernel_size=5) # 88 = 24x3 + 16
    20.  
    21.         self.rblock1 = ResidualBlock(16)
    22.         self.rblock2 = ResidualBlock(32)
    23.  
    24.         self.mp = nn.MaxPool2d(2)
    25.         self.fc = nn.Linear(512, 10) # 暂时不知道1408咋能自动出来的
    26.  
    27.  
    28.     def forward(self, x):
    29.         in_size = x.size(0)
    30.  
    31.         x = self.mp(F.relu(self.conv1(x)))
    32.         x = self.rblock1(x)
    33.         x = self.mp(F.relu(self.conv2(x)))
    34.         x = self.rblock2(x)
    35.  
    36.         x = x.view(in_size, -1)
    37.         x = self.fc(x)
    38.         return x
    39.

     完整代码：
     

    1. import torch
    2. import torch.nn as nn
    3. from torchvision import transforms
    4. from torchvision import datasets
    5. from torch.utils.data import DataLoader
    6. import torch.nn.functional as F
    7. import torch.optim as optim
    8.  
    9. # prepare dataset
    10.  
    11. batch_size = 64
    12. transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])  # 归一化,均值和方差
    13.  
    14. train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
    15. train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
    16. test_dataset = datasets.MNIST(root='../dataset/mnist/', train=False, download=True, transform=transform)
    17. test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)
    18.  
    19.  
    20. # design model using class
    21. class ResidualBlock(nn.Module):
    22.     def __init__(self, channels):
    23.         super(ResidualBlock, self).__init__()
    24.         self.channels = channels
    25.         self.conv1 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
    26.         self.conv2 = nn.Conv2d(channels, channels, kernel_size=3, padding=1)
    27.  
    28.     def forward(self, x):
    29.         y = F.relu(self.conv1(x))
    30.         y = self.conv2(y)
    31.         return F.relu(x + y)
    32.  
    33.  
    34. class Net(nn.Module):
    35.     def __init__(self):
    36.         super(Net, self).__init__()
    37.         self.conv1 = nn.Conv2d(1, 16, kernel_size=5)
    38.         self.conv2 = nn.Conv2d(16, 32, kernel_size=5)  # 88 = 24x3 + 16
    39.  
    40.         self.rblock1 = ResidualBlock(16)
    41.         self.rblock2 = ResidualBlock(32)
    42.  
    43.         self.mp = nn.MaxPool2d(2)
    44.         self.fc = nn.Linear(512, 10)  # 暂时不知道1408咋能自动出来的
    45.  
    46.     def forward(self, x):
    47.         in_size = x.size(0)
    48.  
    49.         x = self.mp(F.relu(self.conv1(x)))
    50.         x = self.rblock1(x)
    51.         x = self.mp(F.relu(self.conv2(x)))
    52.         x = self.rblock2(x)
    53.  
    54.         x = x.view(in_size, -1)
    55.         x = self.fc(x)
    56.         return x
    57.  
    58.  
    59. model = Net()
    60.  
    61. # construct loss and optimizer
    62. criterion = torch.nn.CrossEntropyLoss()
    63. optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
    64.  
    65.  
    66. # training cycle forward, backward, update
    67.  
    68.  
    69. def train(epoch):
    70.     running_loss = 0.0
    71.     for batch_idx, data in enumerate(train_loader, 0):
    72.         inputs, target = data
    73.         optimizer.zero_grad()
    74.  
    75.         outputs = model(inputs)
    76.         loss = criterion(outputs, target)
    77.         loss.backward()
    78.         optimizer.step()
    79.  
    80.         running_loss += loss.item()
    81.         if batch_idx % 300 == 299:
    82.             print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
    83.             running_loss = 0.0
    84.  
    85.  
    86. def test():
    87.     correct = 0
    88.     total = 0
    89.     with torch.no_grad():
    90.         for data in test_loader:
    91.             images, labels = data
    92.             outputs = model(images)
    93.             _, predicted = torch.max(outputs.data, dim=1)
    94.             total += labels.size(0)
    95.             correct += (predicted == labels).sum().item()
    96.     print('accuracy on test set: %d %% ' % (100 * correct / total))
    97.  
    98.  
    99. if __name__ == '__main__':
    100.     for epoch in range(10):
    101.         train(epoch)
    102.         test()

     运行结果：
     

    1. C:\Users\ZARD\anaconda3\envs\PyTorch\python.exe C:/Users/ZARD/PycharmProjects/pythonProject/机器学习库/PyTorch实战课程内容/lecture10/aaa.py
    2. [1,   300] loss: 0.509
    3. [1,   600] loss: 0.155
    4. [1,   900] loss: 0.107
    5. accuracy on test set: 97 % 
    6. [2,   300] loss: 0.087
    7. [2,   600] loss: 0.076
    8. [2,   900] loss: 0.073
    9. accuracy on test set: 98 % 
    1. import  torch
    2. from torch.utils.data import DataLoader #我们要加载数据集的
    3. from torchvision import transforms #数据的原始处理
    4. from torchvision import datasets #pytorch十分贴心的为我们直接准备了这个数据集
    5. import  torch.nn.functional as F#激活函数
    6. import torch.optim as optim
    7.  
    8. batch_size = 64
    9.  #我们拿到的图片是pillow,我们要把他转换成模型里能训练的tensor也就是张量的格式
    10. transform = transforms.Compose([transforms.ToTensor()])
    11.  
    12. #加载训练集，pytorch十分贴心的为我们直接准备了这个数据集，注意，即使你没有下载这个数据集
    13. #在函数中输入download=True，他在运行到这里的时候发现你给的路径没有，就自动下载
    14. train_dataset = datasets.MNIST(root='../data', train=True, download=True, transform=transform)
    15. train_loader = DataLoader(dataset=train_dataset, shuffle=True, batch_size=batch_size)
    16. #同样的方式加载一下测试集
    17. test_dataset = datasets.MNIST(root='../data',  train=False, download=True, transform=transform)
    18. test_loader = DataLoader(dataset=test_dataset, shuffle=False,  batch_size=batch_size)
    19.  
    20. #接下来我们看一下模型是怎么做的
    21. class Net(torch.nn.Module):
    22.     def __init__(self):
    23.         super(Net, self).__init__()
    24.          #定义了我们第一个要用到的卷积层，因为图片输入通道为1，第一个参数就是1
    25.          #输出的通道为10，kernel_size是卷积核的大小，这里定义的是5x5的
    26.         self.conv1 = torch.nn.Conv2d(1, 10, kernel_size=5)
    27.         #看懂了上面的定义，下面这个你肯定也能看懂
    28.         self.conv2 = torch.nn.Conv2d(10, 20, kernel_size=5)
    29.         #再定义一个池化层
    30.         self.pooling = torch.nn.MaxPool2d(2)
    31.         #最后是我们做分类用的线性层
    32.         self.fc = torch.nn.Linear(320, 10)
    33.        
    34.     #下面就是计算的过程
    35.     def forward(self, x):
    36.         # Flatten data from (n, 1, 28, 28) to (n, 784)
    37.         batch_size = x.size(0) #这里面的0是x大小第1个参数，自动获取batch大小
    38.         #输入x经过一个卷积层，之后经历一个池化层，最后用relu做激活
    39.         x = F.relu(self.pooling(self.conv1(x)))
    40.         #再经历上面的过程
    41.         x = F.relu(self.pooling(self.conv2(x)))
    42.         #为了给我们最后一个全连接的线性层用
    43.         #我们要把一个二维的图片（实际上这里已经是处理过的）20x4x4张量变成一维的
    44.         x = x.view(batch_size, -1) # flatten
    45.         #经过线性层，确定他是0~9每一个数的概率
    46.         x = self.fc(x)
    47.         return x
    48.  
    49. model = Net()#实例化模型
    50. #把计算迁移到GPU
    51. device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
    52. model.to(device)
    53.  
    54. #定义一个损失函数，来计算我们模型输出的值和标准值的差距
    55. criterion = torch.nn.CrossEntropyLoss()
    56. #定义一个优化器，训练模型咋训练的，就靠这个，他会反向的更改相应层的权重
    57. optimizer = optim.SGD(model.parameters(),lr=0.1,momentum=0.5)#lr为学习率
    58.  
    59.  
    60. def train(epoch):
    61.     running_loss = 0.0
    62.     for batch_idx, data in enumerate(train_loader, 0):#每次取一个样本
    63.         inputs, target = data
    64.         inputs, target = inputs.to(device), target.to(device)
    65.         #优化器清零
    66.         optimizer.zero_grad()
    67.         # 正向计算一下
    68.         outputs = model(inputs)
    69.         #计算损失
    70.         loss = criterion(outputs, target)
    71.         #反向求梯度
    72.         loss.backward()
    73.         #更新权重
    74.         optimizer.step()
    75.         #把损失加起来
    76.         running_loss += loss.item()
    77.         #每300次输出一下数据
    78.         if batch_idx % 300 == 299:
    79.             print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 2000))
    80.             running_loss = 0.0
    81.  
    82. def test():
    83.     correct = 0
    84.     total = 0
    85.     with torch.no_grad():#不用算梯度
    86.         for data in test_loader:
    87.             inputs, target = data
    88.             inputs, target = inputs.to(device), target.to(device)
    89.             outputs = model(inputs)
    90.             #我们取概率最大的那个数作为输出
    91.             _, predicted = torch.max(outputs.data, dim=1)
    92.             total += target.size(0)
    93.             #计算正确率
    94.             correct += (predicted == target).sum().item()
    95.     print('Accuracy on test set: %d %% [%d/%d]' % (100 * correct / total, correct, total))
    96.  
    97.  
    98. if __name__=='__main__':
    99.     for epoch in range(10):
    100.         train(epoch)
    101.         if epoch % 10 == 9:
    102.             test()
    103.  
    104.  
    105.

     

    深度学习10—卷积神经网络（复盘总结）

    1. 先要掌握全连接网络。定义一个全连接网络：class Net(torch.nn.Module)，其中包含def __init__()和def forward()的设置；

    卷积：
    （1） 辨析“一个卷积核”和“一个卷积核的一个通道”。一个卷积核可能包含多个通道。卷积核的通道数cahnnel是由被其处理的数据决定的。比如，被做卷积处理的数据的channel=3，那么就需要一个3通道的卷积核来进行卷积操作。将这3通道处理的数据对应相加，就变成了1通道，这样就实现了降维操作。

    1. 若要将一个输入为n通道的数据，转化为m通道的数据，则需要通道为n的m个卷积核；
    2. 在程序中实现输入层、卷积层torch.nn.Conv2d(输入通道n,输出通道m,卷积核大小)、输出层;
    3. 对卷积核的定义：kernel = torch.Tensor([1,2,3,4,5,6,7,8,9]).view(1,1,3,3)；将卷积核作为权重赋值给卷积层中对应的数据：conv_layer.weight.data = kernel.data
    4. Padding填充：比如，输入一个5*5的数据，通过3*3的卷积核，得到的是3*3的输出。如果想要得到也是5*5的输出，就得在原输入数据的基础上做一个填充处理。关于在原输入数据外围填充多少圈的计算方法是：若卷积核是a*a的，那么就填充(a/2得到的整数部分数值)的圈数。
    5. Stride步长
    6. 池化层，下采样。通道数不变。Torch.nn.MaxPool2d(kernel_size=n)。其中分最大池化和平均池化两类；
    7. 最后以全连接层作为输出；

    卷积神经网络进阶：
    （1）1*1卷积的作用在于改变输入数据的通道。其好处在于，使用1*1卷积后，会极大降低计算周期。

    （2）将不同的网络结构抽象为Inception类，即将不同的网络分支定义在一个class Inception(nn.Module)中，并将计算得到的值通过torch.cat()进行拼接，并作为forward（）函数的返回值。（将某分支网络抽象为一个Inception类，然后计算各层分支的卷积和）

    （3）在构造的网络class Net(nn.Module)中调用如上定义的Inception类，要注意的是，调用Inception类的实例后，数据的通道数就变成了Inception类实例返回的各个网络分支结构的通道数之和，因此在调用Inception类的实例后，下一网络层的输入通道数就应该是各个网络分支结构的通道数之和。

    （4）residual Network：在训练网络的过程中，在回传时可能会存在梯度消失的问题。解决的方法是先将F（x）与x相加，再通过激活函数去激活。（其实这一部分了解得还还不够丰富，后续学习需要注意）

      


     

  • 相关阅读:
    齐岳|近红外染料CY7.5标记PCL聚已内酯纳米载体CY7.5-PCL|PCL-CY7.5|CY7.5-PEG-PCL
    字符串函数
    Vue axios和vue-axios的关系及使用区别
    Linux 完整 Redis 安装配置教程（可用远程连接）
    面试官：说一下网络数据传输的具体流程
    c++------类和对象（下）包含了this指针、构造函数、析构函数、拷贝构造等
    【shell学习】企业运维工作中常用的shell脚本
    异构AI算力操作平台的架构设计与优化策略
    马达步进机工作原理
    自定义springboot-start
  • 原文地址：https://blog.csdn.net/m0_55196097/article/details/127137067