• 基于LSTM+FCN处理多变量时间序列问题记录(二)


    Long Short Term Memory Fully Convolutional Network (LSTM-FCN) and
    Attention LSTM-FCN (ALSTM-FCN) have been successful in classifying
    univariate time series . However, they have never been applied to on a
    multivariate time series classification problem. The models we
    propose, Multivariate LSTM-FCN (MLSTM-FCN) and Multivariate.Attention
    LSTM-FCN (MALSTM-FCN), converts their respective univariate models
    into multivariate variants. We extend the squeeze-and-excite block to
    the case of 1D sequence models and augment the fully convolutional
    blocks of the LSTM-FCN and ALSTM-FCN models to enhance classification
    accuracy.

    LSTM-FCN与ALSTM-FCN已经在单变量时间序列分类问题上取得了成功,但它们还没有应用到一个多变量时间序列的分类问题中去。我们提出的MLSTM-FCN和MALSTM-FCN模型,即转变它们各自的单变量模型为多变量模型。我们延展了挤压-激活模块到一维卷积层以及增强LSTM-FCN与ALSTM-FCN层的全连接块取提高分类精度。

    As the datasets now consist of multivariate time series, we can define
    a time series dataset as a tensor of shape (N, Q, M ), where N is the
    number of samples in the dataset, Q is the maximum number of time
    steps amongst all variables and M is the number of variables processed
    per time step. Therefore a univariate time series dataset is a special
    case of the above definition, where M is 1. The alteration required to
    the input of the LSTM-FCN and ALSTM-FCN models is to accept M inputs
    per time step, rather than a single input per time step.

    目前该数据集由多变量时间序列组成,我们可以将一个时间序列定义为(N,Q,M)的tensor类型,其中N代表数据集中样例的数量,Q代表

    下面是
    在这里插入图片描述

    模型介绍

    在这里插入图片描述

    Similar to LSTM-FCN and ALSTM-FCN, the proposed models comprise a
    fully convolutional block and a LSTM block, as depicted in Fig. 1.
    The fully convolutional block contains three temporal convolutional
    blocks, used as a feature extractor, which is replicated from the
    original fully convolutional block by Wang et al [34]. The
    convolutional blocks contain a convolutional layer with a number of
    filters (128, 256, and 128) and a kernel size of 8, 5, and 3
    respectively. Each convolutional layer is succeeded by batch
    normalization, with a momentum of 0.99 and epsilon of 0.001. The batch
    normalization layer is succeeded by the ReLU activation function. In
    addition, the first two convolutional blocks conclude 6 with a
    squeeze-and-excite block, which sets the proposed model apart from
    LSTM-FCN and ALSTM-FCN. Fig. 2 summarizes the process of how the
    squeeze-and-excite block is computed in our architecture. For all
    squeeze and excitation blocks, we set the reduction ratio r to 16. The
    final temporal convolutional block is followed by a global average
    pooling layer.The squeeze-and-excite block is an addition to the FCN block which adaptively recalibrates the input feature maps. Due to the reduction ratio r set to 16, the number of parameters required to learn these self-attention maps is reduced such that the overall model
    size increases by just 3-10 %.

    与LSTM-FCN和ALSTM-FCN相同,该模型包含一个全卷积块和一个LSTM块,如下图所描述的。
    全卷积块包含三个时间卷积块,作为一个特征提取器,它最早作为初始全卷积块被人提出。这个卷积块包含一个卷积层,其过滤器数量分别为128,256,128,其卷积核大小分别为8,5,3。每个卷积层后紧跟着为批次归一化,归一化完成后使用ReLU激活函数。此外,在前两个卷积块中还紧跟一个挤压-激活块,这是该模型之所以区别于LSTM-FCN和ALSTM-FCN。
    下图总结了激活-挤压块如何在我们的结构中计算,对于所有的挤压-激活块,我们都设置减速比r为16,最后时间卷积块再紧跟一个平均池化层。
    这个挤压-激活块是作为FCN块的补充,用于自适应重新校准feture map的权重值。由于监所比设置为16,学习这些自我注意图所需的参数数量减少,使得整个模型大小仅增加3-10%。
    在这里插入图片描述
    过滤器即卷积核,在卷积神经网络中根据其要提取的信息不同,可以有多个卷积和,卷积核大小即为过滤器大小
    在这里插入图片描述
    ReLU激活函数
    线性整流函数(Linear rectification function),又称修正线性单元,是一种人工神经网络中常用的激活函数(activation function),通常指代以斜坡函数及其变种为代表的非线性函数。
    在这里插入图片描述

    This adaptive rescaling of the filter maps is of utmost importance to
    the improved performance of the MLSTM-FCN model compared to LSTM-FCN,
    as it incorporates learned self-attention to the inter-correlations
    between multiple variables at each time step, which was inadequate
    with the LSTM-FCN

    滤波器图的自适应重新缩放对MLSTM-FCN模型的性能提升至关重要,相比于LSTM-FCN,因为其在每个时间步长中整合自注意力学习到多变量到多变量的内在联系中,这在LSTM中是不充分的。

  • 相关阅读:
    某网站视频播放花屏解密
    Mybatis-Plus之单表操作和分表查询
    软件定义卫星:数字卫星实践
    Git分布式版本控制工具
    【C++】string类模拟实现下篇(附完整源码)
    【Leetcode刷题Python】191. 位1的个数
    vr虚拟现实技术融入司法办案实操培训中的优势
    Yolov8小目标检测(26):多尺度空洞注意力(MSDA) | 中科院一区顶刊 DilateFormer 2023.9
    Kaggle Feedback Prize 3比赛总结:两种模型设计思路
    Apinto 网关: Go语言实现 HTTP 转 gRPC
  • 原文地址:https://blog.csdn.net/pengxiang1998/article/details/126784936