基于递归长短期记忆网络和镜头序列注意网络的视频摘要生成

为解决基于长短期记忆网络LSTM的视频摘要生成方法当输入序列过长时LSTM网络中的记忆单元不能集中在长时间序列的跨度上。通过深度学习的方法研究了一种基于递归长短期记忆网络(ReLSTM)和序列注意(SSA)的视频摘要生成模型用以提高深度学习网络学习时序特征的能力。该模型使用ReLSTM网络提取时间特征。同时，利用SSA动态调整每个视频序列输入到ReLSTM网络中的特征权重。结果表明：在数据集TVSum上F1-score平均提高2.5%，最高提高0.2%。在数据集SumMe上F1-score平均提高7.8%，最高提高3.4%。可见该方法能有效地学习镜头之间的时序特征。

Research on Video Summarization Generation based on ReLSTM and Shot-Sequence-Attention

In the current LSTM-based video summary generation methods, memory units in the LSTM network cannot focus on the span of long time series when the input sequence is too long. A video summary generation model based on Recurrent Long short-term memory network (ReLSTM) and sequential attention (SSA) is proposed to improve the ability of deep learning networks to learn temporal features. This model uses the ReLSTM network to extract temporal features. Meanwhile, SSA is used to dynamically adjust the feature weights of each video sequence input into the ReLSTM network. The experimental results show that F1-score is increased by 2.5% on average and 0.2% at the highest on the dataset TVSum. On the dataset SumMe, F1-score was improved by 7.8% on average and 3.4% at the highest. Experimental results show that this method can effectively learn the temporal features between shots.