ShuffleNet V1 & V2

ShuffleNet V1框架：卷积层、最大池化下采样、Stage2、Stage3、Stage4、全局池化、全连接层

ResNeXt第二步group卷积因此/g，ShuffleNet同样，第二步DW卷积g=m，即约去一个m。

ShuffleNet V2：
主要拥有两个创新点：
（1）由于逐点卷积占了很大的计算量，所以使用pointwise group convolution 逐点分组卷积
（2）由于存在不同组之间特征通信问题，所以采用channel shuffle


网络四个设计原则：
（1）使用平衡convoluntions，让输入特征矩阵channel和输出特征矩阵channel比值为1
（2）注意group convolution的计算成本，不能一味增加group数
（3）降低网络训练程度
（4）尽可能减少使用element-wise opeartions张量操作

EfficientNet网络

EfficientNet与其他网络的对比：


采用不同的d,r组合，然后不断改变网络的width得到如下图所示的4条曲线，通过分析可以发现在相同的FLOPs下，同时增加d和r的效果最好。

下表为EfficientNet-B0的网络框架（B1-B7就是在B0的基础上修改Resolution，Channels以及Layers），可以看出网络总共分成了9个Stage，第一个Stage就是一个卷积核大小为3x3步距为2的普通卷积层（包含BN和激活函数Swish），Stage2～Stage8都是在重复堆叠MBConv结构（最后一列的Layers表示该Stage重复MBConv结构多少次），而Stage9由一个普通的1x1的卷积层（包含BN和激活函数Swish）一个平均池化层和一个全连接层组成。表格中每个MBConv后会跟一个数字1或6，这里的1或6就是倍率因子n即MBConv中第一个1x1的卷积层会将输入特征矩阵的channels扩充为n倍，其中k3x3或k5x5表示MBConv中Depthwise Conv所采用的卷积核大小。Channels表示通过该Stage后输出特征矩阵的Channels。

Transformer里的 multi-head self-attention

首先了解了self-attention，在一般任务的Encoder-Decoder框架中，输入Source和输出Target内容是不一样的，比如对于英-中机器翻译来说，Source是英文句子，Target是对应的翻译出的中文句子，Attention机制发生在Target的元素和Source中的所有元素之间。而Self Attention顾名思义，指的不是Target和Source之间的Attention机制，而是Source内部元素之间或者Target内部元素之间发生的Attention机制，也可以理解为Target=Source这种特殊情况下的注意力计算机制。其具体计算过程是一样的，只是计算对象发生了变化而已，相当于是Query=Key=Value，计算过程与attention一样。
基于Encoder-Decoder框架的模型Transformer结构图：

Multi-Head Attention模块和Self-Attention模块一样将$a_i$分别通过$W^q$，$W^k$，$W^v$得到对应的$q^i$，$k^i$，$v^i$，然后再根据使用的head数目h进一步把得到的$q^i$，$k^i$，$v^i$均分成h份。比如下图中假设h=2然后$q^1$拆分成$q^{1,1}$和$q^{1,2}$，那么$q^{1,1}$就属于head1，$q^{1,2}$属于head2。
也可以将$W_i^Q$，$W_i^K$，$W_i^V$设置成对应值来实现均分，比如下图中的Q通过$W_1^Q$就能得到均分后的$Q_1$。

通过上述方法就能得到每个$hea{d}_i$对应的$Q_i$，$K_i$，$V_i$参数，接下来针对每个head使用和Self-Attention中相同的方法即可得到对应的结果。


接着将每个head得到的结果进行concat拼接，比如下图中$b_{1,1}$（$hea{d}_1$得到的$b_1$）和$b_{1,2}$（$hea{d}_2$得到的$b_1$）拼接在一起，$b_{2,1}$（$hea{d}_1$得到的$b_2$）和$b_{2,2}$（$hea{d}_2$得到的$b_2$）拼接在一起。

接着将拼接后的结果通过$W^O$（可学习的参数）进行融合，如下图所示，就可以得到最终的结果$b_1$和$b_2$。
Multi-Head Attention中的两个重要公式：

ShuffleNet & EfficientNet & 迁移学习

Step 1：下载数据

import numpy as np
import matplotlib.pyplot as plt
import os
import torch
import torch.nn as nn
import torchvision
from torchvision import models,transforms,datasets
import time
import json
# 判断是否存在GPU设备
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('Using gpu: %s ' % torch.cuda.is_available())
1
2
3
4
5
6
7
8
9
10
11
12

训练集包含1800张图(猫的图片900张，狗的图片900张)，测试集包含2000张图

! wget http://fenggao-image.stor.sinaapp.com/dogscats.zip
! unzip dogscats.zip
1
2

step 2：数据处理

torchvision 支持对输入数据进行一些复杂的预处理/变换 （normalization, cropping, flipping, jittering 等，datasets 是 torchvision 中的一个包，可以用做加载图像数据。它可以以多线程（multi-thread）的形式从硬盘中读取数据，使用 mini-batch 的形式，在网络训练中向 GPU 输送。在使用CNN处理图像时，需要进行预处理。图片将被整理成 224×224×3的大小，同时还将进行归一化处理。

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

vgg_format = transforms.Compose([
                transforms.CenterCrop(224),
                transforms.ToTensor(),
                normalize,
            ])

data_dir = './dogscats'

dsets = {x: datasets.ImageFolder(os.path.join(data_dir, x), vgg_format)
         for x in ['train', 'valid']}

dset_sizes = {x: len(dsets[x]) for x in ['train', 'valid']}
dset_classes = dsets['train'].classes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

通过下面代码可以查看 dsets 的一些属性：

print(dsets['train'].classes)
print(dsets['train'].class_to_idx)
print(dsets['train'].imgs[:5])
print('dset_sizes: ', dset_sizes)
1
2
3
4

加载数据，设定循环次数：

loader_train = torch.utils.data.DataLoader(dsets['train'], batch_size=64, shuffle=True, num_workers=6)
loader_valid = torch.utils.data.DataLoader(dsets['valid'], batch_size=5, shuffle=False, num_workers=6)
'''
valid 数据一共有2000张图，每个batch是5张，因此，下面进行遍历一共会输出到 400
同时，把第一个 batch 保存到 inputs_try, labels_try，分别查看
'''
count = 1
for data in loader_valid:
    print(count, end='\n')
    if count == 1:
        inputs_try,labels_try = data
    count +=1

print(labels_try)
print(inputs_try.shape)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

显示图片的小程序：

def imshow(inp, title=None):
#   Imshow for Tensor.
    inp = inp.numpy().transpose((1, 2, 0))
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    inp = np.clip(std * inp + mean, 0,1)
    plt.imshow(inp)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # pause a bit so that plots are updated
1
2
3
4
5
6
7
8
9
10

#显示 labels_try 的5张图片，即valid里第一个batch的5张图片：

out = torchvision.utils.make_grid(inputs_try)
imshow(out, title=[dset_classes[x] for x in labels_try])
1
2

step 3：创建VGG Model

直接使用预训练好的 VGG 模型，同时，为了展示 VGG 模型对本数据的预测结果，还下载了 ImageNet 1000 个类的 JSON 文件。对输入的5个图片利用VGG模型进行预测，同时，使用softmax对结果进行处理，随后展示了识别结果：

!wget https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json
1

model_vgg = models.vgg16(pretrained=True)

with open('./imagenet_class_index.json') as f:
    class_dict = json.load(f)
dic_imagenet = [class_dict[str(i)][1] for i in range(len(class_dict))]

inputs_try , labels_try = inputs_try.to(device), labels_try.to(device)
model_vgg = model_vgg.to(device)

outputs_try = model_vgg(inputs_try)

print(outputs_try)
print(outputs_try.shape)

'''
可以看到结果为5行，1000列的数据，每一列代表对每一种目标识别的结果。
但是我也可以观察到，结果非常奇葩，有负数，有正数，
为了将VGG网络输出的结果转化为对每一类的预测概率，我们把结果输入到 Softmax 函数
'''
m_softm = nn.Softmax(dim=1)
probs = m_softm(outputs_try)
vals_try,pred_try = torch.max(probs,dim=1)

print( 'prob sum: ', torch.sum(probs,1))
print( 'vals_try: ', vals_try)
print( 'pred_try: ', pred_try)

print([dic_imagenet[i] for i in pred_try.data])
imshow(torchvision.utils.make_grid(inputs_try.data.cpu()), 
       title=[dset_classes[x] for x in labels_try.data.cpu()])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

可以看到预测结果，五张猫猫都正确预测。

step 4：修改最后一层，冻结前面层的参数

VGG 模型如下图所示，网络由三种元素组成：

我们的目标是使用预训练好的模型，因此，需要把最后的 nn.Linear 层由1000类，替换为2类。为了在训练中冻结前面层的参数，需要设置 required_grad=False。这样，反向传播训练梯度时，前面层的权重就不会自动更新了。训练中，只会更新最后一层的参数。

print(model_vgg)

model_vgg_new = model_vgg;

for param in model_vgg_new.parameters():
    param.requires_grad = False
model_vgg_new.classifier._modules['6'] = nn.Linear(4096, 2)
model_vgg_new.classifier._modules['7'] = torch.nn.LogSoftmax(dim = 1)

model_vgg_new = model_vgg_new.to(device)

print(model_vgg_new.classifier)
1
2
3
4
5
6
7
8
9
10
11
12

step 5：训练并测试全连接层

创建损失函数和优化器 + 训练模型 + 测试模型

'''
第一步：创建损失函数和优化器

损失函数 NLLLoss() 的 输入 是一个对数概率向量和一个目标标签. 
它不会为我们计算对数概率，适合最后一层是log_softmax()的网络. 
'''
criterion = nn.NLLLoss()

# 学习率
lr = 0.001

# 随机梯度下降
optimizer_vgg = torch.optim.SGD(model_vgg_new.classifier[6].parameters(),lr = lr)

'''
第二步：训练模型
'''

def train_model(model,dataloader,size,epochs=1,optimizer=None):
    model.train()
    
    for epoch in range(epochs):
        running_loss = 0.0
        running_corrects = 0
        count = 0
        for inputs,classes in dataloader:
            inputs = inputs.to(device)
            classes = classes.to(device)
            outputs = model(inputs)
            loss = criterion(outputs,classes)           
            optimizer = optimizer
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            _,preds = torch.max(outputs.data,1)
            # statistics
            running_loss += loss.data.item()
            running_corrects += torch.sum(preds == classes.data)
            count += len(inputs)
            print('Training: No. ', count, ' process ... total: ', size)
        epoch_loss = running_loss / size
        epoch_acc = running_corrects.data.item() / size
        print('Loss: {:.4f} Acc: {:.4f}'.format(
                     epoch_loss, epoch_acc))
        
        
# 模型训练
train_model(model_vgg_new,loader_train,size=dset_sizes['train'], epochs=1, 
            optimizer=optimizer_vgg)  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

训练集测试准确率79.5%

def test_model(model,dataloader,size):
    model.eval()
    predictions = np.zeros(size)
    all_classes = np.zeros(size)
    all_proba = np.zeros((size,2))
    i = 0
    running_loss = 0.0
    running_corrects = 0
    for inputs,classes in dataloader:
        inputs = inputs.to(device)
        classes = classes.to(device)
        outputs = model(inputs)
        loss = criterion(outputs,classes)           
        _,preds = torch.max(outputs.data,1)
        # statistics
        running_loss += loss.data.item()
        running_corrects += torch.sum(preds == classes.data)
        predictions[i:i+len(classes)] = preds.to('cpu').numpy()
        all_classes[i:i+len(classes)] = classes.to('cpu').numpy()
        all_proba[i:i+len(classes),:] = outputs.data.to('cpu').numpy()
        i += len(classes)
        print('Testing: No. ', i, ' process ... total: ', size)        
    epoch_loss = running_loss / size
    epoch_acc = running_corrects.data.item() / size
    print('Loss: {:.4f} Acc: {:.4f}'.format(
                     epoch_loss, epoch_acc))
    return predictions, all_proba, all_classes
  
predictions, all_proba, all_classes = test_model(model_vgg_new,loader_valid,size=dset_sizes['valid'])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

测试集测试准确率95.65%

step 6：可视化模型预测结果（主观分析）

主观分析就是把预测的结果和相对应的测试图像输出出来看看，一般有四种方式：
（1）随机查看一些预测正确的图片
（2）随机查看一些预测错误的图片
（3）预测正确，同时具有较大的probability的图片
（4）预测错误，同时具有较大的probability的图片
（5）最不确定的图片，比如说预测概率接近0.5的图片

# 单次可视化显示的图片个数
n_view = 8
# 随即查看了预测正确的图片
correct = np.where(predictions==all_classes)[0]
from numpy.random import random, permutation
idx = permutation(correct)[:n_view]
print('random correct idx: ', idx)
loader_correct = torch.utils.data.DataLoader([dsets['valid'][x] for x in idx],
                  batch_size = n_view,shuffle=True)
for data in loader_correct:
    inputs_cor,labels_cor = data
# Make a grid from batch
out = torchvision.utils.make_grid(inputs_cor)
imshow(out, title=[l.item() for l in labels_cor])

# 类似的思路，可以显示错误分类的图片，这里不再重复
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

AI艺术鉴赏挑战赛 - 看画猜作者

季军代码：基于Resnext50，eff-b3训练图像尺寸448，512，600的模型，取得分最高的4组结果进行投票

data_count.py

划分数据集，30张以下的数据没有被划分验证集，class_cnt：label，idx，idx。with open() as 用于读写文件

import os
import pandas as pd
from sklearn.model_selection import train_test_split
import numpy as np


df = pd.read_csv('train.csv').values
print(df[:5])

class_cnt = {}


for idx, label in df:
    print(idx, label)
    if label not in class_cnt:
        class_cnt[label] = []
    class_cnt[label].append(idx)


for k, v in class_cnt.items():
    print(k, len(v))


big_x = []
big_y = []
small_x = []
small_y = []
for k, v in class_cnt.items():
    if len(v) < 30:
        small_x.extend(v)
        small_y.extend(np.ones(len(v), dtype=np.int16) * k)
    else:
        big_x.extend(v)
        big_y.extend(np.ones(len(v), dtype=np.int16) * k)

print(big_x)
print(big_y)

train_x, test_x, train_y, test_y = train_test_split(big_x, big_y, random_state=999, test_size=0.2)
train_x.extend(small_x)
train_y.extend(small_y)

with open('train.txt', 'w')as f:
    for fn, label in zip(train_x, train_y):
        f.write('./data/Art/data/train/{}.jpg,{}\n'.format(fn, label))

with open('val.txt', 'w')as f:
    for fn, label in zip(test_x, test_y):
        f.write('./data/Art/data/train/{}.jpg,{}\n'.format(fn, label))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

dataload.py

torchvision是pytorch的一个图形库，它服务于PyTorch深度学习框架的，主要用来构建计算机视觉模型。torchvision.transforms主要是用于常见的一些图形变换。主要由以下四部分构成：
（1）torchvision.datasets: 一些加载数据的函数及常用的数据集接口
（2）torchvision.models: 包含常用的模型结构（含预训练模型），例如AlexNet、VGG、ResNet等
（3）torchvision.transforms: 常用的图片变换，例如裁剪、旋转等
（4）torchvision.utils: 其他的一些有用的方法
torchvision.transforms.Compose()的主要作用是串联多个图片变换的操作，进行数据增强

from torch.utils.data import dataset
from PIL import Image
from torchvision import transforms, models
import random
import numpy as np
import torch

size = 512
trans = {
        'train':
            transforms.Compose([
            #以0.5的概率水平翻转给定的PIL图像
                transforms.RandomHorizontalFlip(),
                # transforms.RandomVerticalFlip(),
                # transforms.ColorJitter(brightness=0.126, saturation=0.5),
                # transforms.RandomAffine(degrees=30, translate=(0.2, 0.2), fillcolor=0, scale=(0.8, 1.2), shear=None),
                transforms.Resize((int(size / 0.875), int(size / 0.875))),
                 #在一个随机的位置进行裁剪
                transforms.RandomCrop((size, size)),
                transforms.ToTensor(),
                transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
                #随机选择图像中的矩形区域并删除其像素
                transforms.RandomErasing(p=0.5, scale=(0.02, 0.33), ratio=(0.3, 3.3))
            ]),
        'val':
            transforms.Compose([
                transforms.Resize((int(size / 0.875), int(size / 0.875))),
                transforms.CenterCrop((size, size)),
                transforms.ToTensor(),
                transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
            ])
        }


class Dataset(dataset.Dataset):
    def __init__(self, mode):
        assert mode in ['train', 'val']
        txt = './data/Art/data/%s.txt' % mode

        fpath = []
        labels = []
        with open(txt, 'r')as f:
            for i in f.readlines():
                fp, label = i.strip().split(',')
                fpath.append(fp)
                labels.append(int(label))

        self.fpath = fpath
        self.labels = labels
        self.mode = mode
        self.trans = trans[mode]
        
    def __getitem__(self, index):
        fp = self.fpath[index]
        label = self.labels[index]
        img = Image.open(fp).convert('RGB')
        if self.trans is not None:
            img = self.trans(img)

        return img, label

    def __len__(self):
        return len(self.labels)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

train.py

导入数据进行训练，记录最好结果

from torch.utils.data import DataLoader
from ArtModel import BaseModel
import time
import numpy as np
import random
from torch.optim import lr_scheduler
from torch.backends import cudnn
import argparse
import os
import torch
import torch.nn as nn
from dataload import Dataset

#参数
parser = argparse.ArgumentParser()
parser.add_argument('--model_name', default='resnext50', type=str)
parser.add_argument('--savepath', default='./Art/', type=str)
parser.add_argument('--loss', default='ce', type=str)
parser.add_argument('--num_classes', default=49, type=int)
parser.add_argument('--pool_type', default='avg', type=str)
parser.add_argument('--metric', default='linear', type=str)
parser.add_argument('--down', default=0, type=int)
parser.add_argument('--lr', default=0.01, type=float)
parser.add_argument('--weight_decay', default=5e-4, type=float)
parser.add_argument('--momentum', default=0.9, type=float)
parser.add_argument('--scheduler', default='cos', type=str)
parser.add_argument('--resume', default=None, type=str)
parser.add_argument('--lr_step', default=25, type=int)
parser.add_argument('--lr_gamma', default=0.1, type=float)
parser.add_argument('--total_epoch', default=60, type=int)
parser.add_argument('--batch_size', default=32, type=int)
parser.add_argument('--num_workers', default=8, type=int)
parser.add_argument('--multi-gpus', default=0, type=int)
parser.add_argument('--gpu', default=0, type=int)
parser.add_argument('--seed', default=2020, type=int)
parser.add_argument('--pretrained', default=1, type=int)
parser.add_argument('--gray', default=0, type=int)

args = parser.parse_args()


def train():
    model.train()

    epoch_loss = 0
    correct = 0.
    total = 0.
    t1 = time.time()
    for idx, (data, labels) in enumerate(trainloader):
        data, labels = data.to(device), labels.long().to(device)

        out, se, feat_flat = model(data)
       
        loss = criterion(out, labels)
        optimizer.zero_grad()
        
        loss.backward()
        optimizer.step()

        epoch_loss += loss.item() * data.size(0)
        total += data.size(0)
        _, pred = torch.max(out, 1)
        correct += pred.eq(labels).sum().item()

    acc = correct / total
    loss = epoch_loss / total

    print(f'loss:{loss:.4f} acc@1:{acc:.4f} time:{time.time() - t1:.2f}s', end=' --> ')
    
    with open(os.path.join(savepath, 'log.txt'), 'a+')as f:
        f.write('loss:{:.4f}, acc:{:.4f} ->'.format(loss, acc))
    
    return {'loss': loss, 'acc': acc}


def test(epoch):
    model.eval()

    epoch_loss = 0
    correct = 0.
    total = 0.
    with torch.no_grad():
        for idx, (data, labels) in enumerate(valloader):
            data, labels = data.to(device), labels.long().to(device)
            
            out = model(data)

            loss = criterion(out, labels)

            epoch_loss += loss.item() * data.size(0)
            total += data.size(0)
            _, pred = torch.max(out, 1)
            correct += pred.eq(labels).sum().item()

        acc = correct / total
        loss = epoch_loss / total

        print(f'test loss:{loss:.4f} acc@1:{acc:.4f}', end=' ')

    global best_acc, best_epoch

    state = {
        'net': model.state_dict(),
        'acc': acc,
        'epoch': epoch
    }

    if acc > best_acc:
        best_acc = acc
        best_epoch = epoch

        torch.save(state, os.path.join(savepath, 'best.pth'))
        print('*')
    else:
        print()

    torch.save(state, os.path.join(savepath, 'last.pth'))


    with open(os.path.join(savepath, 'log.txt'), 'a+')as f:
        f.write('epoch:{}, loss:{:.4f}, acc:{:.4f}\n'.format(epoch, loss, acc))

    return {'loss': loss, 'acc': acc}


def plot(d, mode='train', best_acc_=None):
    import matplotlib.pyplot as plt
    plt.figure(figsize=(10, 4))
    plt.suptitle('%s_curve' % mode)
    plt.subplots_adjust(wspace=0.2, hspace=0.2)
    epochs = len(d['acc'])

    plt.subplot(1, 2, 1)
    plt.plot(np.arange(epochs), d['loss'], label='loss')
    plt.xlabel('epoch')
    plt.ylabel('loss')
    plt.legend(loc='upper left')

    plt.subplot(1, 2, 2)
    plt.plot(np.arange(epochs), d['acc'], label='acc')
    if best_acc_ is not None:
        plt.scatter(best_acc_[0], best_acc_[1], c='r')
    plt.xlabel('epoch')
    plt.ylabel('acc')
    plt.legend(loc='upper left')

    plt.savefig(os.path.join(savepath, '%s.jpg' % mode), bbox_inches='tight')
    plt.close()


if __name__ == '__main__':
    best_epoch = 0
    best_acc = 0.
    use_gpu = False

    if args.seed is not None:
        print('use random seed:', args.seed)
        torch.manual_seed(args.seed)
        torch.cuda.manual_seed(args.seed)
        torch.cuda.manual_seed_all(args.seed)
        np.random.seed(args.seed)
        random.seed(args.seed)
        cudnn.deterministic = False

    if torch.cuda.is_available():
        use_gpu = True
        cudnn.benchmark = True

    # loss交叉熵损失
    criterion = nn.CrossEntropyLoss()
    # dataloader
    trainset = Dataset(mode='train')
    valset = Dataset(mode='val')
    
    trainloader = DataLoader(dataset=trainset, batch_size=args.batch_size, shuffle=True, \
                             num_workers=args.num_workers, pin_memory=True, drop_last=True)
    
    valloader = DataLoader(dataset=valset, batch_size=128, shuffle=False, num_workers=args.num_workers, \
                           pin_memory=True)

    # model
    model = BaseModel(model_name=args.model_name, num_classes=args.num_classes, pretrained=args.pretrained, pool_type=args.pool_type, down=args.down, metric=args.metric)
    if args.resume:
        state = torch.load(args.resume)
        print('best_epoch:{}, best_acc:{}'.format(state['epoch'], state['acc']))
        model.load_state_dict(state['net'])

    if torch.cuda.device_count() > 1 and args.multi_gpus:
        print('use multi-gpus...')
        os.environ['CUDA_VISIBLE_DEVICES'] = '0,1'
        device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        torch.distributed.init_process_group(backend="nccl", init_method='tcp://localhost:23456', rank=0, world_size=1)
        model = model.to(device)
        model = nn.parallel.DistributedDataParallel(model)
    else:
        device = ('cuda:%d'%args.gpu if torch.cuda.is_available() else 'cpu')
        model = model.to(device)
    print('device:', device)

    # optim优化
    optimizer = torch.optim.SGD(
            [{'params': filter(lambda p: p.requires_grad, model.parameters()), 'lr': args.lr}],
            weight_decay=args.weight_decay, momentum=args.momentum)

    print('init_lr={}, weight_decay={}, momentum={}'.format(args.lr, args.weight_decay, args.momentum))

    if args.scheduler == 'step':
		#动态调整学习率
        scheduler = lr_scheduler.StepLR(optimizer, step_size=args.lr_step, gamma=args.lr_gamma, last_epoch=-1)
    elif args.scheduler == 'multi':
        scheduler = lr_scheduler.MultiStepLR(optimizer, milestones=[150, 225], gamma=args.lr_gamma, last_epoch=-1)
    elif args.scheduler == 'cos':
        warm_up_step = 10
        lambda_ = lambda epoch: (epoch + 1) / warm_up_step if epoch < warm_up_step else 0.5 * (
                    np.cos((epoch - warm_up_step) / (args.total_epoch - warm_up_step) * np.pi) + 1)
        scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lambda_)
    
    # savepath
    savepath = os.path.join(args.savepath, args.model_name+args.pool_type+args.metric+'_'+str(args.down))

    print('savepath:', savepath)

    if not os.path.exists(savepath):
        os.makedirs(savepath)

    with open(os.path.join(savepath, 'setting.txt'), 'w')as f:
        for k, v in vars(args).items():
            f.write('{}:{}\n'.format(k, v))

    f = open(os.path.join(savepath, 'log.txt'), 'w')
    f.close()

    total = args.total_epoch
    start = time.time()

    train_info = {'loss': [], 'acc': []}
    test_info = {'loss': [], 'acc': []}

    for epoch in range(total):
        print('epoch[{:>3}/{:>3}]'.format(epoch, total), end=' ')
        d_train = train()
        scheduler.step()
        d_test = test(epoch)

        for k in train_info.keys():
            train_info[k].append(d_train[k])
            test_info[k].append(d_test[k])

        plot(train_info, mode='train')
        plot(test_info, mode='test', best_acc_=[best_epoch, best_acc])

    end = time.time()
    print('total time:{}m{:.2f}s'.format((end - start) // 60, (end - start) % 60))
    print('best_epoch:', best_epoch)
    print('best_acc:', best_acc)
	#记录最好结果
    with open(os.path.join(savepath, 'log.txt'), 'a+')as f:
        f.write('# best_acc:{:.4f}, best_epoch:{}'.format(best_acc, best_epoch))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259

test.py

进行预测并记录预测结果

import torch
from ArtModel import BaseModel
import os
import pandas as pd
from PIL import Image
from torchvision import transforms
import numpy as np
import argparse


def get_setting(path):
    args = {}
    with open(os.path.join(path, 'setting.txt'), 'r')as f:
        for i in f.readlines():
            k, v = i.strip().split(':')
            args[k] = v
    return args


def load_pretrained_model(path, model, mode='best'):
    print('load pretrained model...')
    state = torch.load(os.path.join(path, '%s.pth' % mode))
    print('best_epoch:{}, best_acc:{}'.format(state['epoch'], state['acc']))
    model.load_state_dict(state['net'])


if __name__ == '__main__':
    mode = 'best'
    
    parser = argparse.ArgumentParser()
    parser.add_argument('--savepath', default='./Base224L2/eff-b3', type=str)
    parser.add_argument('--last', action='store_true')
    args = parser.parse_args()
    
    path = args.savepath
    if args.last:
        mode = 'last'
    
    args = get_setting(path)
    # print(args)

    # model
    model = BaseModel(model_name=args['model_name'], num_classes=int(args['num_classes']), \
        pretrained=int(args['pretrained']), pool_type=args['pool_type'], down=int(args['down']), metric=args['metric'])

    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    # device = torch.device('cpu')
    model = model.to(device)
    #加载预训练模型
    load_pretrained_model(path, model, mode=mode)

    size = 255
    trans = transforms.Compose([
        transforms.Resize((int(size / 0.875), int(size / 0.875))),
        transforms.RandomHorizontalFlip(),
        transforms.RandomCrop((size, size)),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    ])

    submit = {'uuid': [], 'label': []}
    TTA_times = 7

    model.eval()
    with torch.no_grad():
        for i in range(0, 800):
            img_path = 'test/%d.jpg' % i
            raw_img = Image.open(img_path).convert('RGB')
            results = np.zeros(49)

            for j in range(TTA_times):
                img = trans(raw_img)
                img = img.unsqueeze(0).to(device)
                out = model(img)
                out = torch.softmax(out, dim=1)
                _, pred = torch.max(out.cpu(), dim=1)

                results[pred] += 1
            pred = np.argmax(results)
            print(i, ',', pred)
            submit['uuid'].append(i)
            submit['label'].append(pred)

    #保存结果
    df = pd.DataFrame(submit)
    df.to_csv(os.path.join(path, 'result.csv'), encoding='utf-8', index=False, header=False)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

ArtModel.py

搭建分类模型：resnext50/eff-b3

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import models
import math
import numpy as np
from efficientnet_pytorch import EfficientNet
import random


class SELayer(nn.Module):
    def __init__(self, channel, reduction=16):
        super(SELayer, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Sequential(
            nn.Linear(channel, channel // reduction, bias=False),
            nn.ReLU(inplace=True),
            nn.Linear(channel // reduction, channel, bias=False),
            nn.Sigmoid()
        )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x).view(b, c)
        y = self.fc(y).view(b, c, 1, 1)
        return y


class AdaptiveConcatPool2d(nn.Module):
    def __init__(self, sz=(1,1)):
        super().__init__()
        self.ap = nn.AdaptiveAvgPool2d(sz)
        self.mp = nn.AdaptiveMaxPool2d(sz)
        
    def forward(self, x):
        return torch.cat([self.mp(x), self.ap(x)], 1)


class GeneralizedMeanPooling(nn.Module):
    def __init__(self, norm=3, output_size=1, eps=1e-6):
        super().__init__()
        assert norm > 0
        self.p = float(norm)
        self.output_size = output_size
        self.eps = eps

    def forward(self, x):
        x = x.clamp(min=self.eps).pow(self.p)
        
        return torch.nn.functional.adaptive_avg_pool2d(x, self.output_size).pow(1. / self.p)

    def __repr__(self):
        return self.__class__.__name__ + '(' \
               + str(self.p) + ', ' \
               + 'output_size=' + str(self.output_size) + ')'


#模型框架
class BaseModel(nn.Module):
    def __init__(self, model_name, num_classes=2, pretrained=True, pool_type='max', down=True, metric='linear'):
        super().__init__()
        self.model_name = model_name
        
        #eff-b3/resnext50
        if model_name == 'eff-b3':
            backbone = EfficientNet.from_pretrained('efficientnet-b3')
            plane = 1536
        elif model_name == 'resnext50':
            backbone = nn.Sequential(*list(models.resnext50_32x4d(pretrained=pretrained).children())[:-2])
            plane = 2048
        else:
            backbone = None
            plane = None

        self.backbone = backbone
        
        #pool
        if pool_type == 'avg':
            self.pool = nn.AdaptiveAvgPool2d((1, 1))
        elif pool_type == 'cat':
            self.pool = AdaptiveConcatPool2d()
            down = 1
        elif pool_type == 'max':
            self.pool = nn.AdaptiveMaxPool2d((1, 1))
        elif pool_type == 'gem':
            self.pool = GeneralizedMeanPooling()
        else:
            self.pool = None
        
        if down:
            if pool_type == 'cat':
                self.down = nn.Sequential(
                    nn.Linear(plane * 2, plane),
                    nn.BatchNorm1d(plane),
                    nn.Dropout(0.2),
                    nn.ReLU(True)
                    )
            else:
                self.down = nn.Sequential(
                    nn.Linear(plane, plane),
                    nn.BatchNorm1d(plane),
                    nn.Dropout(0.2),
                    nn.ReLU(True)
                )
        else:
            self.down = nn.Identity()
        
        self.se = SELayer(plane)
        self.hidden = nn.Linear(plane, plane)
        self.relu = nn.ReLU(True)
        
        if metric == 'linear':
            self.metric = nn.Linear(plane, num_classes)
        elif metric == 'am':
            self.metric = AddMarginProduct(plane, num_classes)
        else:
            self.metric = None

    def forward(self, x):
        if self.model_name == 'eff-b3':
            feat = self.backbone.extract_features(x)
        else:
            feat = self.backbone(x)
        
        feat = self.pool(feat)
        se = self.se(feat).view(feat.size(0), -1)
        feat_flat = feat.view(feat.size(0), -1)
        feat_flat = self.relu(self.hidden(feat_flat) * se)

        out = self.metric(feat_flat)
        return out


if __name__ == '__main__':
    model = BaseModel(model_name='eff-b3').eval()
    x = torch.randn((1, 3, 224, 224))
    out = model(x)
    print(out.size())
    print(model)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139

vote.py

主干网络resnest200，输入448尺寸，在不同loss下取得5组最好效果，最后进行投票，得到最后分数。

import pandas as pd
import numpy as np


files = ['1.csv', '2.csv', '3.csv', '4.csv']
weights = [1, 1, 1, 1]

results = np.zeros((800, 49))
for file, w in zip(files, weights):
    print(w)
    df = pd.read_csv(file, header=None).values
    for x, y in df:
        # print(x, y)
        results[x, y] += w
        # break

print(results[0])

submit = {
    'name': np.arange(800).tolist(),
    'pred': np.argmax(results, axis=1).tolist()
    }

for k, v in submit.items():
    print(k, v)

df = pd.DataFrame(submit)
df.to_csv('vote.csv', header=False, index=False)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29