基于预训练模型和多视角循环神经网络的电力文本匹配模型

针对传统方法未能考虑词向量的动态性及句子间交互不充分等问题，提出基于BERT预训练模型及多视角循环神经网络的文本匹配模型。通过BERT-whitening方法对BERT输出的句向量进行线性变换优化，并利用多视角循环神经网络将两句子不同位置的BERT动态词向量进行双向交互计算;将句向量与词粒度交互向量进行融合后计算结果。实验结果表明，提出的模型相较于对比模型有明显性能提升，实用性良好。

Electric power text matching model based on pre-training model and multi-view recurrent neural network

Aiming at the problem that traditional text matching methods fail to consider the dynamics of word vectors and insufficient interaction between sentences, a text matching model based on the BERT pre-training model and multi-view recurrent neural network is proposed. The model uses the BERT-whitening linear transformation method to optimize the sentence vector of the BERT, and uses the multi-view recurrent neural network to perform two-way interactive calculation of the BERT dynamic word vector at different positions of the two sentences. Finally, the sentence vector and the interaction vector are fused by highway network to obtain the calculation results. Experimental results on two power operation and maintenance data sets show that this model has significant performance improvement and good practicality compared with the comparison model.