基于改进Transformer的生成式文本摘要模型

基于循环神经网络(recurrent neural network，RNN)注意力机制的序列到序列模型在摘要信息提取服务中已经取得了较好的应用，但RNN不能较好地捕捉长时序信息，这使现有模型受限。为此，提出了基于改进Transformer的生成式文本摘要模型。该模型利用Transformer提取全局语义，使用局部卷积提取器提取原文细粒度特征，并设计全局门控单元以防止信息冗余和弥补语义不足，筛选出利于摘要生成的文本关键信息。实验结果表明，所提模型在大规模中文短文本摘要数据集(large scale Chinese short text summarization，LCSTS)上的效果有明显的提升，在电力运维摘要数据集上，也有良好效果，具有可扩展性。

A generative abstractive summarization method based on the improved Transformer

The sequence-to-sequence model based on recurrent neural network attention mechanism has been well applied in summary information extraction service. But the existing models are limited by the fact that RNN cannot capture long time sequence information. Therefore, this paper proposes a generative text summary method based on the improved Transformer. Firstly, the model uses Transformer to extract global semantics, uses local convolution extractor to extract fine-grained features of the original text, and designs a global gating unit to prevent information redundancy and make up for semantic deficiencies, and screens out key text information that is conducive to summary generation. The experimental results show that the proposed model has significantly improved the effect on large scale Chinese short text summarization (LCSTS), and also has good effect and scalability on power operation and maintenance summary dataset.