基于BERT-BiGRU-Attention的在线健康社区用户意图识别方法

针对传统用户意图识别主要使用基于模板匹配或人工特征集合方法导致成本高、扩展性低的问题,提出了一种基于BERT词向量和BiGRU-Attention的混合神经网络意图识别模型。首先使用BERT预训练的词向量作为输入,通过BiGRU对问句进行特征提取,再引入Attention机制提取对句子含义有重要影响力的词的信息以及分配相应的权重,获得融合了词级权重的句子向量,并输入到softmax分类器,实现意图分类。爬取语料实验结果表明,BERT-BiGRU-Attention方法性能均优于传统的模板匹配、SVM和目前效果较好的CNN-LSTM深度学习组合模型。提出的新方法能有效提升意图识别模型的性能,提高在线健康信息服务质量、为在线健康社区问答系统提供技术支撑。

An online health community user intention identification method based on BERT-BiGRU-Attention

Aiming at the problem of high cost and low expansibility of traditional user intention recognition, which mainly uses template matching or artificial feature set, a hybrid neural network intention recognition model based on BERT word embedding and BiGRU-Attention was proposed. First, the word embedding pre-trained by BERT was used as the input, and the features of the interrogative sentences were extracted by BiGRU. Then, the attention mechanism was introduced to extract the information of words that have important influence on the meaning of sentences and allocate the corresponding weights, so as to obtain the sentence embedding that integrates the word-level weights and input it into the softmax classifier to realize intention classification. According to the experiment on the crawling corpus, it shows that the performance of BERT-BiGRU-Attention method is better than that of traditional template matching, SVM and lately popular CNN-LSTM deep learning combined model. The proposed method can effectively improve the performance of intention recognition model and the quality of online health information service, which provide technical support for the online health community question answering system.