联合自编码任务的多机制融合复述生成模型

基于神经网络编码–解码框架的复述生成模型存在两方面的问题: 1) 生成的复述句中存在实体词不准确、未登录词和词汇重复生成; 2) 复述平行语料的有限规模限制了编码器的语义学习能力。针对第一个问题, 本文提出在解码过程中融合注意力机制、复制机制和覆盖机制的多机制复述生成模型, 利用复制机制从原句复制词语来解决实体词和未登录词生成问题; 利用覆盖机制建模学习注意力机制历史决策信息来规避词汇重复生成。针对第二个问题, 基于多任务学习框架, 提出在复述生成任务中联合自编码任务, 两个任务共享一个编码器, 同时利用平行复述语料和原句子数据, 共同增强复述生成编码器的语义学习能力。在Quora复述数据集上的实验结果表明, 提出的联合自编码的多机制融合复述生成模型有效地解决了复述生成的问题, 并提高了复述句的生成质量。

A Multi-Mechanism Fused Paraphrase Generation Model with Joint Auto-Encoding Learning

Neural network encoder-decoder framework has become the popular method for paraphrase generation, but there are still two problems. On the one hand, there are such issues as inaccurate entity words, unknown words and word repetition in the generated paraphrase sentences. To solve the first problem, we proposed a multimechanism fused paraphrase generation model to improve the decoder. The copy mechanism was used to copy words form input sentence for improving the generation of entity and unknown words. The coverage mechanism was used to model historical attention information to avoid word repetition. On the other hand, the limited-scale parallel paraphrase corpus limits the learning ability of the encoder. We proposed to jointly learn auto-encoding task, which shares one encoder with paraphrase generation task. The joint auto-encoding task enhances the learning ability of the encoder. Experimental results on Quora paraphrase dataset show that the multi-mechanism fused paraphrase generation model with joint auto-encoding task can effectively improve the performance of paraphrase generation.