参考论文：Squeeze-and-Excitation Networks

作者：Jie Hu，Li Shen，Samuel Albanie，Gang Sun，Enhua Wu

论文中给出的源码链接：https://github.com/hujie-frank/SENet

1、SeNet简介

  SENet 是 ImageNet Challenge 图像识别比赛 2017 年的冠军，是来自 Momenta 公司 的团队完成。他们提出了 Squeeze-and-Excitation Networks（简称 SENet）。SENet 不是独立的模型设计，只对模型的一种优化。一般 SENet 都会结合其它模型一起使用，比如 SENet 用于 ResNet-50 中我们就把这个模型称为 SE-ResNet-50，比如 SENet 用于 Inception-ResNet-v2 中我们就把这个模型称为 SE- Inception-ResNet-v2。最早提出 SENet 的论文是《Squeeze-and-Excitation Networks》。

  SENet 的模型优化思路很有意思，主要是针对特征的 channel 进行优化。

  我们可以想象在进行图像识别的时候，卷积计算后生成了很多特征图，不同的滤波器会 得到不同的特征图，不同的特征图代表从图像中提取的不同的特征。我们得到了这么多的特征图，按理来说某些特征图的应该更重要，某些特征图应该没这么重要，并不是所有特征图 都一样的重要。所以 SENet 的核心思想就是给特征图增加注意力和门控机制，增强重要的特征图的信息，减弱不重要的特征图的信息。

2、注意力和门控机制

  我们看一下 SENet 的名字 Squeeze-and-Excitation Networks。其中的“Squeeze”中文意思是“挤压”，在模型中 的实际操作其实是压缩特征图的特征，作者使用的压缩特征图的特征的方式是 avg pooling 平均池化。这个大家应该很熟悉了，求一个特征图所有值的平均值，把 avg pooling 计算后的结果作为这个特征图压缩后的特征。比如一共有 64 个特征图，“Squeeze”计算后我们 就会得到 64 个值，代表 64 个特征图压缩后的特征。

  “Excitation”中文意思是“激发”，在模型中的实际操作是调节特征图信号强弱，作者 使用的方式是给“Squeeze”计算后的结果加上两个全连接层，最终输出每个特征图对应的 激活值，激活值可以改变特征图信号的强弱。每个特征图乘以它所对应的激活值，得到特征 图的输出，然后再传给下一层。

2.1 SENet block

原论文中给出的图如下所示：

具体实现如下：

• 对输入进来的特征层进行全局平均池化。
• 然后进行两次全连接(这两个全连接可用1*1卷积代替)，第一次全连接神经元个数较少，第二次全连接神经元个数和输入特征层个数相同。
• 在完成两次全连接之后，再取一次sigmoid讲值固定到0-1之间，此时我们获得了输入特征层每一个通道的权值(0-1之间)。
• 在获得这个权值之后，讲这个权值与原输入特征层相乘即可。

2.2 SENet block的Tensorflow实现

import math

import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras.layers import (Activation, Add, Concatenate, Conv1D, Conv2D, Dense,
                                     GlobalAveragePooling2D, GlobalMaxPooling2D, Lambda, BatchNormalization,
                                     Reshape, multiply, Input)
from tensorflow.keras.models import Model
from plot_model import plot_model
1
2
3
4
5
6
7
8
9

def se_block(input_feature, ratio=16, name=""):
    # 获得通道数[h,w,c]
    channel = input_feature.shape[-1]
    # 全局平均池化 [c]
    se_feature = GlobalAveragePooling2D()(input_feature)
    # [1,1,c]
    se_feature = Reshape((1, 1, channel))(se_feature)
    # 接下来两次全连接(也可以使用两个1*1卷积)
    # 使用两个全连接
    se_feature = Dense(channel // ratio,
                       activation='relu',
                       kernel_initializer='he_normal',
                       use_bias=False,
                       name="se_block_one_" + str(name))(se_feature)

    se_feature = Dense(channel,
                       kernel_initializer='he_normal',
                       use_bias=False,
                       name="se_block_two_" + str(name))(se_feature)
    # 使用两个1*1卷积
    # se_feature = Conv2D(channel // ratio,
    #                    activation='relu',
    #                    kernel_size=[1, 1],
    #                    kernel_initializer='he_normal',
    #                    use_bias=False,
    #                    name="se_block_one_" + str(name))(se_feature)
    #
    # se_feature = Conv2D(channel,
    #                     kernel_size=[1,1],
    #                    kernel_initializer='he_normal',
    #                    use_bias=False,
    #                    name="se_block_two_" + str(name))(se_feature)

    # 使用sigmoid获得输入特征层每一个通道的权值(0-1之间)
    se_feature = Activation('sigmoid')(se_feature)
    # 将权值乘上原输入特征层即可。
    out = multiply([input_feature, se_feature])
    return out
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

这里可以将两个全连接层替换为两个1*1卷积，就是注释掉的那部分代码。

if __name__ == '__main__':
    inputs = Input([26, 26, 512])
    x = se_block(inputs)
    model = Model(inputs, x)
    model.summary()
    plot_model(model,to_file='img/SeNet.png',show_shapes=True)
1
2
3
4
5
6

3、Se-Inception模块与SE-ResNet模块

图 2. 原始 Inception 模块（左）和 SEInception 模块（右）的架构。

  图中的 Global pooling 表示全局池化；W 表示图片宽度；H 表示图片高度；C 表示图片 通道数；FC 表示全连接层；r 表示缩减率，意思是通道数在第一个全连接层缩减多少，总之 就是一个超参数，不用细究，一般取值为 16。

图 3. 原始残差模块（左）和 SEResNet 模块（右）的架构。

  图中的 Global pooling 表示全局池化；W 表示图片宽度；H 表示图片高度；C 表示图片 通道数；FC 表示全连接层；r 表示缩减率，意思是通道数在第一个全连接层缩减多少，总之 就是一个超参数，不用细究，一般取值为 16。

4、SE-残差结构的理解

图片来源：《深度学习从0到1》

普通的 ResNet 中的残差结构如下图所示

  图中的 Conv 表示卷积；Batch Norm 表示批量标准化；Identity mapping 表示恒等映射；batch 表示批次。

  普通的残差结构我们就不需要多说了，我前面也有论文的精读博客。下面我们看一下加上了 Squeeze-and-Excitation 模块后的残差结构如下图所示。

  加上 Squeeze-and-Excitation 模块后的残差结构主要变化是在原来的残差结构最后一个 卷积层后面进行 Squeeze-and-Excitation 的操作。Squeeze 就是先做平均池化，得到每一个特征图的压缩特征。图中特征图大小为 56×56，所以池化的窗口大小也是 56×56。池化 过后就是 Excitation 操作，前面我们有提到 Excitation 操作有两个全连接层，这是 SENet 原始论文中的做法，实际我们在写程序的时候也可以用两个窗口大小 1×1 的卷积层的替代，效果跟全连接是一样的。

  Excitation 操作部分最后的激活函数是 Sigmoid 函数，作者在这里 使用 Sigmoid 函数主要是利用 Sigmoid 函数输出范围是 0-1 这个特性，让 Excitation 的输 出激活值可以起到一个门控的作用。Excitation 的输出的激活值会乘以原始残差结构最后一 个卷积层的输出结果，对特征图的数值大小进行控制。如果是重要的特征图，会保持比较大 的数值；如果是不重要的特征图，特征图的数值就会变小。

5、ResNet-50，SE-ResNet-50，SE-ResNeXt-50(32×4d)模型结构

  （左）ResNet-50 [13]。 （中）SE-ResNet-50。 （右）带有 32×4d 模板的 SE-ResNeXt-50。括号内列出了残差构建块的特定参数设置的形状和操作，外部显示了阶段中堆叠块的数量。 fc后面的内括号表示SE模块中两个全连接层的输出维度。

   图中的 Output size 表示输出大小；conv 表示卷积；max pool 表示最大池化；stride 表示步长；fc 表示全连接层；global average pool 表示全局平均池化；C 表示分组数。

6、SeNet结果分析

  基础模型增加 SE 模块后会使得整体模型的参数增加 10%左右，计算量增加不多，一般 来说模型的效果也会有所提升。作者使用多个模型在 ImageNet 数据集上进行了测试，下图为多个模型在 ImageNet 验证集测试结果。

  图中 original 表示模型原始论文中的结果；re-implementation 表示 SENet 作者重新训 练模型的结果；SENet 表示给这些模型加上 SE 模块后的结果；top-1 err.表示 top1 错误 率；top-5 err.表示 top5 错误率；GFLOPs 表示计算量。

  图中结果可以看出，图中测试的所有模型只要加上 SE 模块，错误率都能降低，并且模型 浮点计算量没有太大变化。

  下图 也能看出加上 SE 模块后模型效果可以变得更好：

   图中的 epochs 表示周期；Top-1 error 表示 Top1 错误率；train 表示训练集；val 表示 验证集。

  SENet 论文的最后，作者还给了一组很有意思的图。作者用 ImageNet 数据集训练了一 个 SE-ResNet-50，然后选出 4 个种类(goldfish,pug,plane,cliff)的图片，统计这 4 个种类在 SE-ResNet-50 模型的每个 SE 模块的特征图的激活情况，如下图所示。

  图中的 all 表示所有 1000 个种类的平均值；goldfish 表示金鱼；pug 表示哈巴狗； plane 表示飞机；cliff 表示悬崖；channel index 表示通道；activation 表示激活值

作者观察实验结果得到 3 个结论：

• 第一，不同种类的物体在浅层激活分布情况是类似的，如图中的 SE_2_3 和 SE_3_4。也 就是不管是识别哪种物体，浅层的卷积层中，重要的特征图总是比较固定的那些。

• 第二，在更深层一些的位置，不同种类在不同的特征图激活分布不同，因为不同类别对 特征有不同的偏好，如图中的 SE_4_6 和 SE_5_1。低层特征通常更普遍，识别不同种类物体可以使用类似的滤波器，而高层特征通常包含更多细节，识别不同种类物体需要使用不同的滤波器。

• 第三，在模型的最后阶段，SE_5_2 呈现出饱和状态，其中大部分激活值都接近于 1，也 有一些接近于 0。对于激活值为 1 的特征图，相当于 SE 模块不存在。在网络的最后一个 SE 模块 SE_5_3，不同种类有着类似的分布，只是尺度不同。也就是说 SE_5_2 和 SE_5_3 相对 来说没有前面的一些 SE 模块重要，作者通过实验发现删除最后一个阶段的 SE 模块，总体参 数可以显著减少，性能只有一点损失(<0.1%的 Top1 错误率)。

7、SE-ResNet50图像识别

import numpy as np
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input,Dense,Dropout,Conv2D,MaxPool2D,Flatten,GlobalAvgPool2D,BatchNormalization,Activation,Add,ZeroPadding2D,Multiply
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt
from tensorflow.keras.callbacks import LearningRateScheduler
from tensorflow.keras.models import Model
from tensorflow.keras.utils import plot_model
1
2
3
4
5
6
7
8
9
10

# 类别数
num_classes = 17
# 批次大小
batch_size = 32
# 周期数
epochs = 100
# 图片大小
image_size = 224
1
2
3
4
5
6
7
8

# 训练集数据进行数据增强
train_datagen = ImageDataGenerator(
    rotation_range = 20,     # 随机旋转度数
    width_shift_range = 0.1, # 随机水平平移
    height_shift_range = 0.1,# 随机竖直平移
    rescale = 1/255,         # 数据归一化
    shear_range = 10,       # 随机错切变换
    zoom_range = 0.1,        # 随机放大
    horizontal_flip = True,  # 水平翻转
    brightness_range=(0.7, 1.3), # 亮度变化
    fill_mode = 'nearest',   # 填充方式
) 
# 测试集数据只需要归一化就可以
test_datagen = ImageDataGenerator(
    rescale = 1/255,         # 数据归一化
) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# 训练集数据生成器，可以在训练时自动产生数据进行训练
# 从'data/train'获得训练集数据
# 获得数据后会把图片resize为image_size×image_size的大小
# generator每次会产生batch_size个数据
train_generator = train_datagen.flow_from_directory(
    'data/train',
    target_size=(image_size,image_size),
    batch_size=batch_size,
    )

# 测试集数据生成器
test_generator = test_datagen.flow_from_directory(
    'data/test',
    target_size=(image_size,image_size),
    batch_size=batch_size,
    )
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# 字典的键为17个文件夹的名字，值为对应的分类编号
train_generator.class_indices
1
2

# SE模块
def ChannelSE(input_tensor, reduction=16):
    # 获得信号通道数
    channels = input_tensor.shape[-1]
    # SE模块
    x = GlobalAvgPool2D()(input_tensor)
    # 把2维数据再变成4维(?,1,1,?)
    x = x[:, None, None, :]
    # 卷积替代全连接层
    x = Conv2D(filters=channels//reduction,kernel_size=1,strides=1)(x)
    x = Activation('relu')(x)
    x = Conv2D(filters=channels,kernel_size=1,strides=1)(x)
    x = Activation('sigmoid')(x)
    x = Multiply()([input_tensor, x])
    return x


# 定义残差单元
def block(x, filters, strides=1, conv_shortcut=True, reduction=16): 
    # projection shortcut
    if conv_shortcut == True:
        shortcut = Conv2D(filters*4,kernel_size=1,strides=strides,padding='valid')(x)
        # epsilon为BN公式中防止分母为零的值
        shortcut = BatchNormalization(epsilon=1.001e-5)(shortcut)
    else:
        # identity_shortcut
        shortcut = x
    # 3个卷积层
    x = Conv2D(filters=filters,kernel_size=1,strides=strides,padding='valid')(x)
    x = BatchNormalization(epsilon=1.001e-5)(x)
    x = Activation('relu')(x)
 
    x = Conv2D(filters=filters,kernel_size=3,strides=1,padding='same')(x)
    x = BatchNormalization(epsilon=1.001e-5)(x)
    x = Activation('relu')(x)
 
    x = Conv2D(filters=filters*4,kernel_size=1,strides=1,padding='valid')(x)
    x = BatchNormalization(epsilon=1.001e-5)(x)
    
    # SE模块
    x = ChannelSE(x, reduction=reduction)
 
    x = Add()([x, shortcut])
    x = Activation('relu')(x)
    return x

# 堆叠残差单元
def stack(x, filters, blocks, strides):
    x = block(x, filters, strides=strides)
    for i in range(blocks-1):
        x = block(x, filters, conv_shortcut=False)
    return x
    
# 定义SE-ResNet50
inputs = Input(shape=(image_size,image_size,3))
# 填充3圈0，填充后图像从224×224变成230×230
x = ZeroPadding2D((3, 3))(inputs)
x= Conv2D(filters=64,kernel_size=7,strides=2,padding='valid')(x)
x = BatchNormalization(epsilon=1.001e-5)(x)
x = Activation('relu')(x)
# 填充1圈0
x = ZeroPadding2D((1, 1))(x)
x = MaxPool2D(pool_size=3,strides=2,padding='valid')(x)
# 堆叠残差结构
# blocks表示堆叠数量
x = stack(x, filters=64, blocks=3, strides=1)
x = stack(x, filters=128, blocks=4, strides=2)
x = stack(x, filters=256, blocks=6, strides=2)
x = stack(x, filters=512, blocks=3, strides=2)
# 根据特征图大小进行平均池化，池化后得到2维数据
x = GlobalAvgPool2D()(x)
x = Dense(num_classes, activation='softmax')(x)
# 定义模型
model = Model(inputs=inputs,outputs=x)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

# 学习率调节函数，逐渐减小学习率
def adjust_learning_rate(epoch):
    # 前40周期
    if epoch<=40:
        lr = 1e-4
    # 前40到80周期
    elif epoch>40 and epoch<=80:
        lr = 1e-5
    # 80到100周期
    else:
        lr = 1e-6
    return lr
1
2
3
4
5
6
7
8
9
10
11
12

# 定义优化器
adam = Adam(lr=1e-4)

# 定义学习率衰减策略
callbacks = []
callbacks.append(LearningRateScheduler(adjust_learning_rate))

# 定义优化器，loss function，训练过程中计算准确率
model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])

# Tensorflow2.1版本之前可以使用fit_generator训练模型
# history = model.fit_generator(train_generator,steps_per_epoch=len(train_generator),epochs=epochs,validation_data=test_generator,validation_steps=len(test_generator))

# Tensorflow2.1版本(包括2.1)之后可以直接使用fit训练模型
history = model.fit(x=train_generator,epochs=epochs,validation_data=test_generator,callbacks=callbacks)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Epoch 1/100
34/34 [==============================] - 19s 549ms/step - loss: 2.5603 - accuracy: 0.1930 - val_loss: 2.8569 - val_accuracy: 0.0588
Epoch 2/100
34/34 [==============================] - 17s 501ms/step - loss: 1.9323 - accuracy: 0.3603 - val_loss: 3.0023 - val_accuracy: 0.0588
Epoch 3/100
34/34 [==============================] - 18s 519ms/step - loss: 1.5821 - accuracy: 0.4614 - val_loss: 3.3108 - val_accuracy: 0.0588
Epoch 4/100
34/34 [==============================] - 17s 508ms/step - loss: 1.4403 - accuracy: 0.5285 - val_loss: 3.5824 - val_accuracy: 0.0588
Epoch 5/100
34/34 [==============================] - 17s 510ms/step - loss: 1.2837 - accuracy: 0.5781 - val_loss: 4.0358 - val_accuracy: 0.0625
Epoch 6/100
34/34 [==============================] - 17s 507ms/step - loss: 1.2065 - accuracy: 0.6075 - val_loss: 4.3552 - val_accuracy: 0.0588
Epoch 7/100
34/34 [==============================] - 18s 515ms/step - loss: 1.1144 - accuracy: 0.6259 - val_loss: 4.0170 - val_accuracy: 0.0846
Epoch 8/100
34/34 [==============================] - 17s 507ms/step - loss: 1.0141 - accuracy: 0.6572 - val_loss: 4.4925 - val_accuracy: 0.0625
Epoch 9/100
34/34 [==============================] - 17s 510ms/step - loss: 0.9487 - accuracy: 0.6857 - val_loss: 4.7573 - val_accuracy: 0.0993
Epoch 10/100
34/34 [==============================] - 17s 503ms/step - loss: 0.8503 - accuracy: 0.7132 - val_loss: 4.5246 - val_accuracy: 0.0919
Epoch 11/100
34/34 [==============================] - 17s 499ms/step - loss: 0.8037 - accuracy: 0.7298 - val_loss: 4.4767 - val_accuracy: 0.1213
Epoch 12/100
34/34 [==============================] - 17s 507ms/step - loss: 0.7193 - accuracy: 0.7518 - val_loss: 4.1744 - val_accuracy: 0.2243
Epoch 13/100
34/34 [==============================] - 17s 506ms/step - loss: 0.6467 - accuracy: 0.7904 - val_loss: 3.3856 - val_accuracy: 0.2537
Epoch 14/100
34/34 [==============================] - 17s 511ms/step - loss: 0.6490 - accuracy: 0.7748 - val_loss: 3.3443 - val_accuracy: 0.2941
Epoch 15/100
34/34 [==============================] - 17s 501ms/step - loss: 0.6287 - accuracy: 0.7914 - val_loss: 2.8897 - val_accuracy: 0.3603
Epoch 16/100
34/34 [==============================] - 17s 510ms/step - loss: 0.5383 - accuracy: 0.8088 - val_loss: 2.5651 - val_accuracy: 0.4338
Epoch 17/100
34/34 [==============================] - 18s 515ms/step - loss: 0.5573 - accuracy: 0.8033 - val_loss: 2.0221 - val_accuracy: 0.5000
Epoch 18/100
34/34 [==============================] - 17s 506ms/step - loss: 0.5236 - accuracy: 0.8217 - val_loss: 2.5051 - val_accuracy: 0.4485
Epoch 19/100
34/34 [==============================] - 17s 511ms/step - loss: 0.5003 - accuracy: 0.8355 - val_loss: 2.3877 - val_accuracy: 0.5000
Epoch 20/100
34/34 [==============================] - 17s 513ms/step - loss: 0.4758 - accuracy: 0.8290 - val_loss: 1.5135 - val_accuracy: 0.6728
Epoch 21/100
34/34 [==============================] - 17s 511ms/step - loss: 0.4111 - accuracy: 0.8621 - val_loss: 1.4329 - val_accuracy: 0.6176
Epoch 22/100
34/34 [==============================] - 17s 509ms/step - loss: 0.3885 - accuracy: 0.8768 - val_loss: 1.5033 - val_accuracy: 0.6434
Epoch 23/100
34/34 [==============================] - 17s 507ms/step - loss: 0.3462 - accuracy: 0.8851 - val_loss: 2.2281 - val_accuracy: 0.5000
Epoch 24/100
34/34 [==============================] - 17s 510ms/step - loss: 0.3518 - accuracy: 0.8824 - val_loss: 2.0414 - val_accuracy: 0.6176
Epoch 25/100
34/34 [==============================] - 17s 506ms/step - loss: 0.3619 - accuracy: 0.8704 - val_loss: 1.8381 - val_accuracy: 0.6287
Epoch 26/100
34/34 [==============================] - 17s 513ms/step - loss: 0.3551 - accuracy: 0.8814 - val_loss: 1.4278 - val_accuracy: 0.6507
Epoch 27/100
34/34 [==============================] - 18s 516ms/step - loss: 0.3022 - accuracy: 0.9017 - val_loss: 1.3334 - val_accuracy: 0.6838
Epoch 28/100
34/34 [==============================] - 17s 508ms/step - loss: 0.2860 - accuracy: 0.9072 - val_loss: 1.3219 - val_accuracy: 0.6875
Epoch 29/100
34/34 [==============================] - 17s 510ms/step - loss: 0.2219 - accuracy: 0.9219 - val_loss: 0.9909 - val_accuracy: 0.7132
Epoch 30/100
34/34 [==============================] - 17s 507ms/step - loss: 0.2143 - accuracy: 0.9228 - val_loss: 1.1023 - val_accuracy: 0.7353
Epoch 31/100
34/34 [==============================] - 17s 514ms/step - loss: 0.2513 - accuracy: 0.9136 - val_loss: 1.4077 - val_accuracy: 0.6691
Epoch 32/100
34/34 [==============================] - 17s 511ms/step - loss: 0.2759 - accuracy: 0.8980 - val_loss: 2.3026 - val_accuracy: 0.5772
Epoch 33/100
34/34 [==============================] - 17s 511ms/step - loss: 0.2204 - accuracy: 0.9219 - val_loss: 1.7835 - val_accuracy: 0.6213
Epoch 34/100
34/34 [==============================] - 17s 506ms/step - loss: 0.2390 - accuracy: 0.9228 - val_loss: 1.9852 - val_accuracy: 0.6103
Epoch 35/100
34/34 [==============================] - 17s 504ms/step - loss: 0.1942 - accuracy: 0.9301 - val_loss: 1.1830 - val_accuracy: 0.6765
Epoch 36/100
34/34 [==============================] - 18s 516ms/step - loss: 0.1756 - accuracy: 0.9403 - val_loss: 1.2185 - val_accuracy: 0.6985
Epoch 37/100
34/34 [==============================] - 17s 510ms/step - loss: 0.1713 - accuracy: 0.9403 - val_loss: 1.2769 - val_accuracy: 0.7243
Epoch 38/100
34/34 [==============================] - 17s 509ms/step - loss: 0.1709 - accuracy: 0.9430 - val_loss: 1.6684 - val_accuracy: 0.6654
Epoch 39/100
34/34 [==============================] - 17s 506ms/step - loss: 0.1908 - accuracy: 0.9467 - val_loss: 1.4263 - val_accuracy: 0.7096
Epoch 40/100
34/34 [==============================] - 17s 502ms/step - loss: 0.1768 - accuracy: 0.9338 - val_loss: 1.2879 - val_accuracy: 0.7096
Epoch 41/100
34/34 [==============================] - 17s 513ms/step - loss: 0.1483 - accuracy: 0.9504 - val_loss: 1.0808 - val_accuracy: 0.7390
Epoch 42/100
34/34 [==============================] - 17s 511ms/step - loss: 0.1057 - accuracy: 0.9623 - val_loss: 0.8448 - val_accuracy: 0.7868
Epoch 43/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0670 - accuracy: 0.9807 - val_loss: 0.7289 - val_accuracy: 0.8162
Epoch 44/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0768 - accuracy: 0.9779 - val_loss: 0.6560 - val_accuracy: 0.8162
Epoch 45/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0568 - accuracy: 0.9871 - val_loss: 0.6800 - val_accuracy: 0.8199
Epoch 46/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0612 - accuracy: 0.9871 - val_loss: 0.6611 - val_accuracy: 0.8309
Epoch 47/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0499 - accuracy: 0.9899 - val_loss: 0.6868 - val_accuracy: 0.8015
Epoch 48/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0561 - accuracy: 0.9816 - val_loss: 0.6607 - val_accuracy: 0.8051
Epoch 49/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0418 - accuracy: 0.9926 - val_loss: 0.6790 - val_accuracy: 0.8015
Epoch 50/100
34/34 [==============================] - 18s 516ms/step - loss: 0.0383 - accuracy: 0.9917 - val_loss: 0.6684 - val_accuracy: 0.8162
Epoch 51/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0416 - accuracy: 0.9917 - val_loss: 0.6851 - val_accuracy: 0.8051
Epoch 52/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0435 - accuracy: 0.9881 - val_loss: 0.6635 - val_accuracy: 0.8051
Epoch 53/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0457 - accuracy: 0.9917 - val_loss: 0.6653 - val_accuracy: 0.8199
Epoch 54/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0444 - accuracy: 0.9862 - val_loss: 0.7060 - val_accuracy: 0.8125
Epoch 55/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0497 - accuracy: 0.9862 - val_loss: 0.7086 - val_accuracy: 0.8051
Epoch 56/100
34/34 [==============================] - 18s 516ms/step - loss: 0.0355 - accuracy: 0.9926 - val_loss: 0.6972 - val_accuracy: 0.8162
Epoch 57/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0407 - accuracy: 0.9917 - val_loss: 0.7006 - val_accuracy: 0.8088
Epoch 58/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0429 - accuracy: 0.9890 - val_loss: 0.6793 - val_accuracy: 0.8162
Epoch 59/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0327 - accuracy: 0.9936 - val_loss: 0.6627 - val_accuracy: 0.8199
Epoch 60/100
34/34 [==============================] - 17s 504ms/step - loss: 0.0449 - accuracy: 0.9890 - val_loss: 0.6477 - val_accuracy: 0.8088
Epoch 61/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0349 - accuracy: 0.9926 - val_loss: 0.6539 - val_accuracy: 0.8125
Epoch 62/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0273 - accuracy: 0.9963 - val_loss: 0.6503 - val_accuracy: 0.8235
Epoch 63/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0287 - accuracy: 0.9972 - val_loss: 0.6243 - val_accuracy: 0.8346
Epoch 64/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0306 - accuracy: 0.9954 - val_loss: 0.6159 - val_accuracy: 0.8272
Epoch 65/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0272 - accuracy: 0.9936 - val_loss: 0.6473 - val_accuracy: 0.8125
Epoch 66/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0257 - accuracy: 0.9972 - val_loss: 0.6518 - val_accuracy: 0.8088
Epoch 67/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0325 - accuracy: 0.9954 - val_loss: 0.6649 - val_accuracy: 0.8015
Epoch 68/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0269 - accuracy: 0.9963 - val_loss: 0.6461 - val_accuracy: 0.8088
Epoch 69/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0278 - accuracy: 0.9954 - val_loss: 0.6525 - val_accuracy: 0.8088
Epoch 70/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0347 - accuracy: 0.9899 - val_loss: 0.6667 - val_accuracy: 0.8088
Epoch 71/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0296 - accuracy: 0.9936 - val_loss: 0.6512 - val_accuracy: 0.8235
Epoch 72/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0373 - accuracy: 0.9899 - val_loss: 0.6657 - val_accuracy: 0.8125
Epoch 73/100
34/34 [==============================] - 17s 500ms/step - loss: 0.0307 - accuracy: 0.9954 - val_loss: 0.6822 - val_accuracy: 0.8051
Epoch 74/100
34/34 [==============================] - 17s 499ms/step - loss: 0.0332 - accuracy: 0.9899 - val_loss: 0.6935 - val_accuracy: 0.8162
Epoch 75/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0187 - accuracy: 0.9991 - val_loss: 0.6491 - val_accuracy: 0.8162
Epoch 76/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0243 - accuracy: 0.9963 - val_loss: 0.6576 - val_accuracy: 0.8272
Epoch 77/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0227 - accuracy: 0.9945 - val_loss: 0.6988 - val_accuracy: 0.8162
Epoch 78/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0205 - accuracy: 0.9954 - val_loss: 0.7424 - val_accuracy: 0.8162
Epoch 79/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0298 - accuracy: 0.9936 - val_loss: 0.6804 - val_accuracy: 0.8309
Epoch 80/100
34/34 [==============================] - 17s 514ms/step - loss: 0.0246 - accuracy: 0.9936 - val_loss: 0.7717 - val_accuracy: 0.8015
Epoch 81/100
34/34 [==============================] - 17s 505ms/step - loss: 0.0294 - accuracy: 0.9945 - val_loss: 0.6915 - val_accuracy: 0.7978
Epoch 82/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0223 - accuracy: 0.9954 - val_loss: 0.6843 - val_accuracy: 0.8015
Epoch 83/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0260 - accuracy: 0.9945 - val_loss: 0.6777 - val_accuracy: 0.8051
Epoch 84/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0250 - accuracy: 0.9926 - val_loss: 0.6703 - val_accuracy: 0.8051
Epoch 85/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0263 - accuracy: 0.9954 - val_loss: 0.6659 - val_accuracy: 0.8088
Epoch 86/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0189 - accuracy: 0.9963 - val_loss: 0.6651 - val_accuracy: 0.8088
Epoch 87/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0250 - accuracy: 0.9954 - val_loss: 0.6708 - val_accuracy: 0.8199
Epoch 88/100
34/34 [==============================] - 17s 506ms/step - loss: 0.0247 - accuracy: 0.9945 - val_loss: 0.6672 - val_accuracy: 0.8199
Epoch 89/100
34/34 [==============================] - 17s 508ms/step - loss: 0.0237 - accuracy: 0.9936 - val_loss: 0.6719 - val_accuracy: 0.8235
Epoch 90/100
34/34 [==============================] - 17s 507ms/step - loss: 0.0268 - accuracy: 0.9936 - val_loss: 0.6678 - val_accuracy: 0.8235
Epoch 91/100
34/34 [==============================] - 17s 510ms/step - loss: 0.0228 - accuracy: 0.9936 - val_loss: 0.6671 - val_accuracy: 0.8199
Epoch 92/100
34/34 [==============================] - 18s 515ms/step - loss: 0.0154 - accuracy: 1.0000 - val_loss: 0.6638 - val_accuracy: 0.8235
Epoch 93/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0206 - accuracy: 0.9991 - val_loss: 0.6617 - val_accuracy: 0.8199
Epoch 94/100
34/34 [==============================] - 17s 501ms/step - loss: 0.0249 - accuracy: 0.9954 - val_loss: 0.6695 - val_accuracy: 0.8235
Epoch 95/100
34/34 [==============================] - 17s 509ms/step - loss: 0.0282 - accuracy: 0.9945 - val_loss: 0.6846 - val_accuracy: 0.8235
Epoch 96/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0196 - accuracy: 0.9963 - val_loss: 0.6784 - val_accuracy: 0.8272
Epoch 97/100
34/34 [==============================] - 18s 523ms/step - loss: 0.0255 - accuracy: 0.9917 - val_loss: 0.6732 - val_accuracy: 0.8346
Epoch 98/100
34/34 [==============================] - 17s 513ms/step - loss: 0.0278 - accuracy: 0.9899 - val_loss: 0.6697 - val_accuracy: 0.8309
Epoch 99/100
34/34 [==============================] - 17s 512ms/step - loss: 0.0163 - accuracy: 0.9963 - val_loss: 0.6711 - val_accuracy: 0.8309
Epoch 100/100
34/34 [==============================] - 17s 503ms/step - loss: 0.0238 - accuracy: 0.9945 - val_loss: 0.6632 - val_accuracy: 0.8272
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

可视化acc

# 画出训练集准确率曲线图
plt.plot(np.arange(epochs),history.history['accuracy'],c='b',label='train_accuracy')
# 画出验证集准确率曲线图
plt.plot(np.arange(epochs),history.history['val_accuracy'],c='y',label='val_accuracy')
# 图例
plt.legend()
# x坐标描述
plt.xlabel('epochs')
# y坐标描述
plt.ylabel('accuracy')
# 显示图像
plt.show()
# 模型保存
model.save('SE-ResNet50.h5')
1
2
3
4
5
6
7
8
9
10
11
12
13
14

模型结构如下所示：

Model: "functional_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, 230, 230, 3)  0           input_1[0][0]                    
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 112, 112, 64) 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 112, 112, 64) 256         conv2d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 112, 112, 64) 0           batch_normalization[0][0]        
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, 114, 114, 64) 0           activation[0][0]                 
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 56, 56, 64)   0           zero_padding2d_1[0][0]           
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 56, 56, 64)   4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 56, 56, 64)   256         conv2d_2[0][0]                   
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 56, 56, 64)   0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 56, 56, 64)   36928       activation_1[0][0]               
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 56, 56, 64)   256         conv2d_3[0][0]                   
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 56, 56, 64)   0           batch_normalization_3[0][0]      
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 56, 56, 256)  16640       activation_2[0][0]               
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 56, 56, 256)  1024        conv2d_4[0][0]                   
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 256)          0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
tf_op_layer_strided_slice (Tens (None, 1, 1, 256)    0           global_average_pooling2d[0][0]   
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 1, 1, 16)     4112        tf_op_layer_strided_slice[0][0]  
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 1, 1, 16)     0           conv2d_5[0][0]                   
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 1, 1, 256)    4352        activation_3[0][0]               
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 1, 1, 256)    0           conv2d_6[0][0]                   
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 56, 56, 256)  16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
multiply (Multiply)             (None, 56, 56, 256)  0           batch_normalization_4[0][0]      
                                                                 activation_4[0][0]               
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 56, 56, 256)  1024        conv2d_1[0][0]                   
__________________________________________________________________________________________________
add (Add)                       (None, 56, 56, 256)  0           multiply[0][0]                   
                                                                 batch_normalization_1[0][0]      
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 56, 56, 256)  0           add[0][0]                        
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 56, 56, 64)   16448       activation_5[0][0]               
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 56, 56, 64)   256         conv2d_7[0][0]                   
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 56, 56, 64)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 56, 56, 64)   36928       activation_6[0][0]               
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 56, 56, 64)   256         conv2d_8[0][0]                   
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 56, 56, 64)   0           batch_normalization_6[0][0]      
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 56, 56, 256)  16640       activation_7[0][0]               
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 56, 56, 256)  1024        conv2d_9[0][0]                   
__________________________________________________________________________________________________
global_average_pooling2d_1 (Glo (None, 256)          0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
tf_op_layer_strided_slice_1 (Te (None, 1, 1, 256)    0           global_average_pooling2d_1[0][0] 
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 1, 1, 16)     4112        tf_op_layer_strided_slice_1[0][0]
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 1, 1, 16)     0           conv2d_10[0][0]                  
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 1, 1, 256)    4352        activation_8[0][0]               
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 1, 1, 256)    0           conv2d_11[0][0]                  
__________________________________________________________________________________________________
multiply_1 (Multiply)           (None, 56, 56, 256)  0           batch_normalization_7[0][0]      
                                                                 activation_9[0][0]               
__________________________________________________________________________________________________
add_1 (Add)                     (None, 56, 56, 256)  0           multiply_1[0][0]                 
                                                                 activation_5[0][0]               
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 56, 56, 256)  0           add_1[0][0]                      
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 56, 56, 64)   16448       activation_10[0][0]              
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 56, 56, 64)   256         conv2d_12[0][0]                  
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 56, 56, 64)   0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 56, 56, 64)   36928       activation_11[0][0]              
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 56, 56, 64)   256         conv2d_13[0][0]                  
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 56, 56, 64)   0           batch_normalization_9[0][0]      
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, 56, 56, 256)  16640       activation_12[0][0]              
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 56, 56, 256)  1024        conv2d_14[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 256)          0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_2 (Te (None, 1, 1, 256)    0           global_average_pooling2d_2[0][0] 
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, 1, 1, 16)     4112        tf_op_layer_strided_slice_2[0][0]
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 1, 1, 16)     0           conv2d_15[0][0]                  
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, 1, 1, 256)    4352        activation_13[0][0]              
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 1, 1, 256)    0           conv2d_16[0][0]                  
__________________________________________________________________________________________________
multiply_2 (Multiply)           (None, 56, 56, 256)  0           batch_normalization_10[0][0]     
                                                                 activation_14[0][0]              
__________________________________________________________________________________________________
add_2 (Add)                     (None, 56, 56, 256)  0           multiply_2[0][0]                 
                                                                 activation_10[0][0]              
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 56, 56, 256)  0           add_2[0][0]                      
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 28, 28, 128)  32896       activation_15[0][0]              
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 128)  512         conv2d_18[0][0]                  
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 28, 28, 128)  0           batch_normalization_12[0][0]     
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, 28, 28, 128)  147584      activation_16[0][0]              
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 28, 28, 128)  512         conv2d_19[0][0]                  
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 28, 28, 128)  0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, 28, 28, 512)  66048       activation_17[0][0]              
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 28, 28, 512)  2048        conv2d_20[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_3 (Glo (None, 512)          0           batch_normalization_14[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_3 (Te (None, 1, 1, 512)    0           global_average_pooling2d_3[0][0] 
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 1, 1, 32)     16416       tf_op_layer_strided_slice_3[0][0]
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 1, 1, 32)     0           conv2d_21[0][0]                  
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 1, 1, 512)    16896       activation_18[0][0]              
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 1, 1, 512)    0           conv2d_22[0][0]                  
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, 28, 28, 512)  131584      activation_15[0][0]              
__________________________________________________________________________________________________
multiply_3 (Multiply)           (None, 28, 28, 512)  0           batch_normalization_14[0][0]     
                                                                 activation_19[0][0]              
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 512)  2048        conv2d_17[0][0]                  
__________________________________________________________________________________________________
add_3 (Add)                     (None, 28, 28, 512)  0           multiply_3[0][0]                 
                                                                 batch_normalization_11[0][0]     
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 28, 28, 512)  0           add_3[0][0]                      
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 28, 28, 128)  65664       activation_20[0][0]              
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 28, 28, 128)  512         conv2d_23[0][0]                  
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 28, 28, 128)  0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 28, 28, 128)  147584      activation_21[0][0]              
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 28, 28, 128)  512         conv2d_24[0][0]                  
__________________________________________________________________________________________________
activation_22 (Activation)      (None, 28, 28, 128)  0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 28, 28, 512)  66048       activation_22[0][0]              
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 28, 28, 512)  2048        conv2d_25[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_4 (Glo (None, 512)          0           batch_normalization_17[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_4 (Te (None, 1, 1, 512)    0           global_average_pooling2d_4[0][0] 
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 1, 1, 32)     16416       tf_op_layer_strided_slice_4[0][0]
__________________________________________________________________________________________________
activation_23 (Activation)      (None, 1, 1, 32)     0           conv2d_26[0][0]                  
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 1, 1, 512)    16896       activation_23[0][0]              
__________________________________________________________________________________________________
activation_24 (Activation)      (None, 1, 1, 512)    0           conv2d_27[0][0]                  
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 28, 28, 512)  0           batch_normalization_17[0][0]     
                                                                 activation_24[0][0]              
__________________________________________________________________________________________________
add_4 (Add)                     (None, 28, 28, 512)  0           multiply_4[0][0]                 
                                                                 activation_20[0][0]              
__________________________________________________________________________________________________
activation_25 (Activation)      (None, 28, 28, 512)  0           add_4[0][0]                      
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 28, 28, 128)  65664       activation_25[0][0]              
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 28, 28, 128)  512         conv2d_28[0][0]                  
__________________________________________________________________________________________________
activation_26 (Activation)      (None, 28, 28, 128)  0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 28, 28, 128)  147584      activation_26[0][0]              
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 28, 28, 128)  512         conv2d_29[0][0]                  
__________________________________________________________________________________________________
activation_27 (Activation)      (None, 28, 28, 128)  0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 28, 28, 512)  66048       activation_27[0][0]              
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 28, 28, 512)  2048        conv2d_30[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_5 (Glo (None, 512)          0           batch_normalization_20[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_5 (Te (None, 1, 1, 512)    0           global_average_pooling2d_5[0][0] 
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 1, 1, 32)     16416       tf_op_layer_strided_slice_5[0][0]
__________________________________________________________________________________________________
activation_28 (Activation)      (None, 1, 1, 32)     0           conv2d_31[0][0]                  
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 1, 1, 512)    16896       activation_28[0][0]              
__________________________________________________________________________________________________
activation_29 (Activation)      (None, 1, 1, 512)    0           conv2d_32[0][0]                  
__________________________________________________________________________________________________
multiply_5 (Multiply)           (None, 28, 28, 512)  0           batch_normalization_20[0][0]     
                                                                 activation_29[0][0]              
__________________________________________________________________________________________________
add_5 (Add)                     (None, 28, 28, 512)  0           multiply_5[0][0]                 
                                                                 activation_25[0][0]              
__________________________________________________________________________________________________
activation_30 (Activation)      (None, 28, 28, 512)  0           add_5[0][0]                      
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 28, 28, 128)  65664       activation_30[0][0]              
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 28, 28, 128)  512         conv2d_33[0][0]                  
__________________________________________________________________________________________________
activation_31 (Activation)      (None, 28, 28, 128)  0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 28, 28, 128)  147584      activation_31[0][0]              
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 28, 28, 128)  512         conv2d_34[0][0]                  
__________________________________________________________________________________________________
activation_32 (Activation)      (None, 28, 28, 128)  0           batch_normalization_22[0][0]     
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 28, 28, 512)  66048       activation_32[0][0]              
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 28, 28, 512)  2048        conv2d_35[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_6 (Glo (None, 512)          0           batch_normalization_23[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_6 (Te (None, 1, 1, 512)    0           global_average_pooling2d_6[0][0] 
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 1, 1, 32)     16416       tf_op_layer_strided_slice_6[0][0]
__________________________________________________________________________________________________
activation_33 (Activation)      (None, 1, 1, 32)     0           conv2d_36[0][0]                  
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 1, 1, 512)    16896       activation_33[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (None, 1, 1, 512)    0           conv2d_37[0][0]                  
__________________________________________________________________________________________________
multiply_6 (Multiply)           (None, 28, 28, 512)  0           batch_normalization_23[0][0]     
                                                                 activation_34[0][0]              
__________________________________________________________________________________________________
add_6 (Add)                     (None, 28, 28, 512)  0           multiply_6[0][0]                 
                                                                 activation_30[0][0]              
__________________________________________________________________________________________________
activation_35 (Activation)      (None, 28, 28, 512)  0           add_6[0][0]                      
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 14, 14, 256)  131328      activation_35[0][0]              
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 256)  1024        conv2d_39[0][0]                  
__________________________________________________________________________________________________
activation_36 (Activation)      (None, 14, 14, 256)  0           batch_normalization_25[0][0]     
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 14, 14, 256)  590080      activation_36[0][0]              
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 256)  1024        conv2d_40[0][0]                  
__________________________________________________________________________________________________
activation_37 (Activation)      (None, 14, 14, 256)  0           batch_normalization_26[0][0]     
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 14, 14, 1024) 263168      activation_37[0][0]              
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 1024) 4096        conv2d_41[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_7 (Glo (None, 1024)         0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_7 (Te (None, 1, 1, 1024)   0           global_average_pooling2d_7[0][0] 
__________________________________________________________________________________________________
conv2d_42 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_7[0][0]
__________________________________________________________________________________________________
activation_38 (Activation)      (None, 1, 1, 64)     0           conv2d_42[0][0]                  
__________________________________________________________________________________________________
conv2d_43 (Conv2D)              (None, 1, 1, 1024)   66560       activation_38[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (None, 1, 1, 1024)   0           conv2d_43[0][0]                  
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 14, 14, 1024) 525312      activation_35[0][0]              
__________________________________________________________________________________________________
multiply_7 (Multiply)           (None, 14, 14, 1024) 0           batch_normalization_27[0][0]     
                                                                 activation_39[0][0]              
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 1024) 4096        conv2d_38[0][0]                  
__________________________________________________________________________________________________
add_7 (Add)                     (None, 14, 14, 1024) 0           multiply_7[0][0]                 
                                                                 batch_normalization_24[0][0]     
__________________________________________________________________________________________________
activation_40 (Activation)      (None, 14, 14, 1024) 0           add_7[0][0]                      
__________________________________________________________________________________________________
conv2d_44 (Conv2D)              (None, 14, 14, 256)  262400      activation_40[0][0]              
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 256)  1024        conv2d_44[0][0]                  
__________________________________________________________________________________________________
activation_41 (Activation)      (None, 14, 14, 256)  0           batch_normalization_28[0][0]     
__________________________________________________________________________________________________
conv2d_45 (Conv2D)              (None, 14, 14, 256)  590080      activation_41[0][0]              
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 256)  1024        conv2d_45[0][0]                  
__________________________________________________________________________________________________
activation_42 (Activation)      (None, 14, 14, 256)  0           batch_normalization_29[0][0]     
__________________________________________________________________________________________________
conv2d_46 (Conv2D)              (None, 14, 14, 1024) 263168      activation_42[0][0]              
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 1024) 4096        conv2d_46[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_8 (Glo (None, 1024)         0           batch_normalization_30[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_8 (Te (None, 1, 1, 1024)   0           global_average_pooling2d_8[0][0] 
__________________________________________________________________________________________________
conv2d_47 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_8[0][0]
__________________________________________________________________________________________________
activation_43 (Activation)      (None, 1, 1, 64)     0           conv2d_47[0][0]                  
__________________________________________________________________________________________________
conv2d_48 (Conv2D)              (None, 1, 1, 1024)   66560       activation_43[0][0]              
__________________________________________________________________________________________________
activation_44 (Activation)      (None, 1, 1, 1024)   0           conv2d_48[0][0]                  
__________________________________________________________________________________________________
multiply_8 (Multiply)           (None, 14, 14, 1024) 0           batch_normalization_30[0][0]     
                                                                 activation_44[0][0]              
__________________________________________________________________________________________________
add_8 (Add)                     (None, 14, 14, 1024) 0           multiply_8[0][0]                 
                                                                 activation_40[0][0]              
__________________________________________________________________________________________________
activation_45 (Activation)      (None, 14, 14, 1024) 0           add_8[0][0]                      
__________________________________________________________________________________________________
conv2d_49 (Conv2D)              (None, 14, 14, 256)  262400      activation_45[0][0]              
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 256)  1024        conv2d_49[0][0]                  
__________________________________________________________________________________________________
activation_46 (Activation)      (None, 14, 14, 256)  0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
conv2d_50 (Conv2D)              (None, 14, 14, 256)  590080      activation_46[0][0]              
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 256)  1024        conv2d_50[0][0]                  
__________________________________________________________________________________________________
activation_47 (Activation)      (None, 14, 14, 256)  0           batch_normalization_32[0][0]     
__________________________________________________________________________________________________
conv2d_51 (Conv2D)              (None, 14, 14, 1024) 263168      activation_47[0][0]              
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 1024) 4096        conv2d_51[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_9 (Glo (None, 1024)         0           batch_normalization_33[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_9 (Te (None, 1, 1, 1024)   0           global_average_pooling2d_9[0][0] 
__________________________________________________________________________________________________
conv2d_52 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_9[0][0]
__________________________________________________________________________________________________
activation_48 (Activation)      (None, 1, 1, 64)     0           conv2d_52[0][0]                  
__________________________________________________________________________________________________
conv2d_53 (Conv2D)              (None, 1, 1, 1024)   66560       activation_48[0][0]              
__________________________________________________________________________________________________
activation_49 (Activation)      (None, 1, 1, 1024)   0           conv2d_53[0][0]                  
__________________________________________________________________________________________________
multiply_9 (Multiply)           (None, 14, 14, 1024) 0           batch_normalization_33[0][0]     
                                                                 activation_49[0][0]              
__________________________________________________________________________________________________
add_9 (Add)                     (None, 14, 14, 1024) 0           multiply_9[0][0]                 
                                                                 activation_45[0][0]              
__________________________________________________________________________________________________
activation_50 (Activation)      (None, 14, 14, 1024) 0           add_9[0][0]                      
__________________________________________________________________________________________________
conv2d_54 (Conv2D)              (None, 14, 14, 256)  262400      activation_50[0][0]              
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 256)  1024        conv2d_54[0][0]                  
__________________________________________________________________________________________________
activation_51 (Activation)      (None, 14, 14, 256)  0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
conv2d_55 (Conv2D)              (None, 14, 14, 256)  590080      activation_51[0][0]              
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 256)  1024        conv2d_55[0][0]                  
__________________________________________________________________________________________________
activation_52 (Activation)      (None, 14, 14, 256)  0           batch_normalization_35[0][0]     
__________________________________________________________________________________________________
conv2d_56 (Conv2D)              (None, 14, 14, 1024) 263168      activation_52[0][0]              
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 1024) 4096        conv2d_56[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_10 (Gl (None, 1024)         0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_10 (T (None, 1, 1, 1024)   0           global_average_pooling2d_10[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_10[0][0
__________________________________________________________________________________________________
activation_53 (Activation)      (None, 1, 1, 64)     0           conv2d_57[0][0]                  
__________________________________________________________________________________________________
conv2d_58 (Conv2D)              (None, 1, 1, 1024)   66560       activation_53[0][0]              
__________________________________________________________________________________________________
activation_54 (Activation)      (None, 1, 1, 1024)   0           conv2d_58[0][0]                  
__________________________________________________________________________________________________
multiply_10 (Multiply)          (None, 14, 14, 1024) 0           batch_normalization_36[0][0]     
                                                                 activation_54[0][0]              
__________________________________________________________________________________________________
add_10 (Add)                    (None, 14, 14, 1024) 0           multiply_10[0][0]                
                                                                 activation_50[0][0]              
__________________________________________________________________________________________________
activation_55 (Activation)      (None, 14, 14, 1024) 0           add_10[0][0]                     
__________________________________________________________________________________________________
conv2d_59 (Conv2D)              (None, 14, 14, 256)  262400      activation_55[0][0]              
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 256)  1024        conv2d_59[0][0]                  
__________________________________________________________________________________________________
activation_56 (Activation)      (None, 14, 14, 256)  0           batch_normalization_37[0][0]     
__________________________________________________________________________________________________
conv2d_60 (Conv2D)              (None, 14, 14, 256)  590080      activation_56[0][0]              
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 256)  1024        conv2d_60[0][0]                  
__________________________________________________________________________________________________
activation_57 (Activation)      (None, 14, 14, 256)  0           batch_normalization_38[0][0]     
__________________________________________________________________________________________________
conv2d_61 (Conv2D)              (None, 14, 14, 1024) 263168      activation_57[0][0]              
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 1024) 4096        conv2d_61[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_11 (Gl (None, 1024)         0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_11 (T (None, 1, 1, 1024)   0           global_average_pooling2d_11[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_11[0][0
__________________________________________________________________________________________________
activation_58 (Activation)      (None, 1, 1, 64)     0           conv2d_62[0][0]                  
__________________________________________________________________________________________________
conv2d_63 (Conv2D)              (None, 1, 1, 1024)   66560       activation_58[0][0]              
__________________________________________________________________________________________________
activation_59 (Activation)      (None, 1, 1, 1024)   0           conv2d_63[0][0]                  
__________________________________________________________________________________________________
multiply_11 (Multiply)          (None, 14, 14, 1024) 0           batch_normalization_39[0][0]     
                                                                 activation_59[0][0]              
__________________________________________________________________________________________________
add_11 (Add)                    (None, 14, 14, 1024) 0           multiply_11[0][0]                
                                                                 activation_55[0][0]              
__________________________________________________________________________________________________
activation_60 (Activation)      (None, 14, 14, 1024) 0           add_11[0][0]                     
__________________________________________________________________________________________________
conv2d_64 (Conv2D)              (None, 14, 14, 256)  262400      activation_60[0][0]              
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 14, 14, 256)  1024        conv2d_64[0][0]                  
__________________________________________________________________________________________________
activation_61 (Activation)      (None, 14, 14, 256)  0           batch_normalization_40[0][0]     
__________________________________________________________________________________________________
conv2d_65 (Conv2D)              (None, 14, 14, 256)  590080      activation_61[0][0]              
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 14, 14, 256)  1024        conv2d_65[0][0]                  
__________________________________________________________________________________________________
activation_62 (Activation)      (None, 14, 14, 256)  0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
conv2d_66 (Conv2D)              (None, 14, 14, 1024) 263168      activation_62[0][0]              
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 1024) 4096        conv2d_66[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_12 (Gl (None, 1024)         0           batch_normalization_42[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_12 (T (None, 1, 1, 1024)   0           global_average_pooling2d_12[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D)              (None, 1, 1, 64)     65600       tf_op_layer_strided_slice_12[0][0
__________________________________________________________________________________________________
activation_63 (Activation)      (None, 1, 1, 64)     0           conv2d_67[0][0]                  
__________________________________________________________________________________________________
conv2d_68 (Conv2D)              (None, 1, 1, 1024)   66560       activation_63[0][0]              
__________________________________________________________________________________________________
activation_64 (Activation)      (None, 1, 1, 1024)   0           conv2d_68[0][0]                  
__________________________________________________________________________________________________
multiply_12 (Multiply)          (None, 14, 14, 1024) 0           batch_normalization_42[0][0]     
                                                                 activation_64[0][0]              
__________________________________________________________________________________________________
add_12 (Add)                    (None, 14, 14, 1024) 0           multiply_12[0][0]                
                                                                 activation_60[0][0]              
__________________________________________________________________________________________________
activation_65 (Activation)      (None, 14, 14, 1024) 0           add_12[0][0]                     
__________________________________________________________________________________________________
conv2d_70 (Conv2D)              (None, 7, 7, 512)    524800      activation_65[0][0]              
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 512)    2048        conv2d_70[0][0]                  
__________________________________________________________________________________________________
activation_66 (Activation)      (None, 7, 7, 512)    0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
conv2d_71 (Conv2D)              (None, 7, 7, 512)    2359808     activation_66[0][0]              
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 512)    2048        conv2d_71[0][0]                  
__________________________________________________________________________________________________
activation_67 (Activation)      (None, 7, 7, 512)    0           batch_normalization_45[0][0]     
__________________________________________________________________________________________________
conv2d_72 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_67[0][0]              
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 7, 7, 2048)   8192        conv2d_72[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_13 (Gl (None, 2048)         0           batch_normalization_46[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_13 (T (None, 1, 1, 2048)   0           global_average_pooling2d_13[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D)              (None, 1, 1, 128)    262272      tf_op_layer_strided_slice_13[0][0
__________________________________________________________________________________________________
activation_68 (Activation)      (None, 1, 1, 128)    0           conv2d_73[0][0]                  
__________________________________________________________________________________________________
conv2d_74 (Conv2D)              (None, 1, 1, 2048)   264192      activation_68[0][0]              
__________________________________________________________________________________________________
activation_69 (Activation)      (None, 1, 1, 2048)   0           conv2d_74[0][0]                  
__________________________________________________________________________________________________
conv2d_69 (Conv2D)              (None, 7, 7, 2048)   2099200     activation_65[0][0]              
__________________________________________________________________________________________________
multiply_13 (Multiply)          (None, 7, 7, 2048)   0           batch_normalization_46[0][0]     
                                                                 activation_69[0][0]              
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 2048)   8192        conv2d_69[0][0]                  
__________________________________________________________________________________________________
add_13 (Add)                    (None, 7, 7, 2048)   0           multiply_13[0][0]                
                                                                 batch_normalization_43[0][0]     
__________________________________________________________________________________________________
activation_70 (Activation)      (None, 7, 7, 2048)   0           add_13[0][0]                     
__________________________________________________________________________________________________
conv2d_75 (Conv2D)              (None, 7, 7, 512)    1049088     activation_70[0][0]              
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 7, 7, 512)    2048        conv2d_75[0][0]                  
__________________________________________________________________________________________________
activation_71 (Activation)      (None, 7, 7, 512)    0           batch_normalization_47[0][0]     
__________________________________________________________________________________________________
conv2d_76 (Conv2D)              (None, 7, 7, 512)    2359808     activation_71[0][0]              
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 7, 7, 512)    2048        conv2d_76[0][0]                  
__________________________________________________________________________________________________
activation_72 (Activation)      (None, 7, 7, 512)    0           batch_normalization_48[0][0]     
__________________________________________________________________________________________________
conv2d_77 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_72[0][0]              
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 7, 7, 2048)   8192        conv2d_77[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_14 (Gl (None, 2048)         0           batch_normalization_49[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_14 (T (None, 1, 1, 2048)   0           global_average_pooling2d_14[0][0]
__________________________________________________________________________________________________
conv2d_78 (Conv2D)              (None, 1, 1, 128)    262272      tf_op_layer_strided_slice_14[0][0
__________________________________________________________________________________________________
activation_73 (Activation)      (None, 1, 1, 128)    0           conv2d_78[0][0]                  
__________________________________________________________________________________________________
conv2d_79 (Conv2D)              (None, 1, 1, 2048)   264192      activation_73[0][0]              
__________________________________________________________________________________________________
activation_74 (Activation)      (None, 1, 1, 2048)   0           conv2d_79[0][0]                  
__________________________________________________________________________________________________
multiply_14 (Multiply)          (None, 7, 7, 2048)   0           batch_normalization_49[0][0]     
                                                                 activation_74[0][0]              
__________________________________________________________________________________________________
add_14 (Add)                    (None, 7, 7, 2048)   0           multiply_14[0][0]                
                                                                 activation_70[0][0]              
__________________________________________________________________________________________________
activation_75 (Activation)      (None, 7, 7, 2048)   0           add_14[0][0]                     
__________________________________________________________________________________________________
conv2d_80 (Conv2D)              (None, 7, 7, 512)    1049088     activation_75[0][0]              
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 7, 7, 512)    2048        conv2d_80[0][0]                  
__________________________________________________________________________________________________
activation_76 (Activation)      (None, 7, 7, 512)    0           batch_normalization_50[0][0]     
__________________________________________________________________________________________________
conv2d_81 (Conv2D)              (None, 7, 7, 512)    2359808     activation_76[0][0]              
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 7, 7, 512)    2048        conv2d_81[0][0]                  
__________________________________________________________________________________________________
activation_77 (Activation)      (None, 7, 7, 512)    0           batch_normalization_51[0][0]     
__________________________________________________________________________________________________
conv2d_82 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_77[0][0]              
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 7, 7, 2048)   8192        conv2d_82[0][0]                  
__________________________________________________________________________________________________
global_average_pooling2d_15 (Gl (None, 2048)         0           batch_normalization_52[0][0]     
__________________________________________________________________________________________________
tf_op_layer_strided_slice_15 (T (None, 1, 1, 2048)   0           global_average_pooling2d_15[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D)              (None, 1, 1, 128)    262272      tf_op_layer_strided_slice_15[0][0
__________________________________________________________________________________________________
activation_78 (Activation)      (None, 1, 1, 128)    0           conv2d_83[0][0]                  
__________________________________________________________________________________________________
conv2d_84 (Conv2D)              (None, 1, 1, 2048)   264192      activation_78[0][0]              
__________________________________________________________________________________________________
activation_79 (Activation)      (None, 1, 1, 2048)   0           conv2d_84[0][0]                  
__________________________________________________________________________________________________
multiply_15 (Multiply)          (None, 7, 7, 2048)   0           batch_normalization_52[0][0]     
                                                                 activation_79[0][0]              
__________________________________________________________________________________________________
add_15 (Add)                    (None, 7, 7, 2048)   0           multiply_15[0][0]                
                                                                 activation_75[0][0]              
__________________________________________________________________________________________________
activation_80 (Activation)      (None, 7, 7, 2048)   0           add_15[0][0]                     
__________________________________________________________________________________________________
global_average_pooling2d_16 (Gl (None, 2048)         0           activation_80[0][0]              
__________________________________________________________________________________________________
dense (Dense)                   (None, 17)           34833       global_average_pooling2d_16[0][0]
==================================================================================================
Total params: 26,153,537
Trainable params: 26,100,417
Non-trainable params: 53,120
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617

References

Jie Hu, Li Shen, Samuel Albanie, Gang Sun, & Enhua Wu (2018). Squeeze-and-Excitation Networks computer vision and pattern recognition.

Tensorflow2 图像处理中注意力机制的解析与代码详解

https://github.com/hujie-frank/SENet

Tensorflow2 图像处理中注意力机制的代码详解与应用（Bubbliiiing 深度学习 教程）