融合BERT中间隐藏层的方面级情感分析模型

现有的基于BERT(bidirectional encoder representations from transformers)的方面级情感分析模型仅使用BERT最后一层隐藏层的输出，忽略BERT中间隐藏层的语义信息，存在信息利用不充分的问题，提出一种融合BERT中间隐藏层的方面级情感分析模型。首先，将评论和方面信息拼接为句子对输入BERT模型，通过BERT的自注意力机制建立评论与方面信息的联系；其次，构建门控卷积网络(gated convolutional neural network, GCNN)对BERT所有隐藏层输出的词向量矩阵进行特征提取，并将提取的特征进行最大池化、拼接得到特征序列；然后，使用双向门控循环单元(bidirectional gated recurrent unit, BiGRU)网络对特征序列进行融合，编码BERT不同隐藏层的信息；最后，引入注意力机制，根据特征与方面信息的相关程度赋予权值。在公开的SemEval2014 Task4评论数据集上的实验结果表明：所提模型在准确率和F₁值两种评价指标上均优于BERT、CapsBERT(ca...

Aspect-based Sentiment Analysis Model Incorporating the BERT Intermediate Hidden Layer

The existing Aspect-based sentiment analysis models based on BERT only use the output of the last hidden layer of BERT and ignore the semantic knowledge in the intermediate layers, which have the problem of insufficient information utilization, this paper proposes an Aspect-level sentiment analysis model that incorporates the intermediate hidden layer of BERT. Firstly, the comments and aspect information are spliced into sentence pairs and input to BERT model, and the connection between comments and aspect information is established through the self-attention mechanism of BERT. Secondly, the Gated Convolutional network is constructed to extract features from the word vector matrices of the outputs of all hidden layers of BERT, and the extracted features are added to the max pooling layer and spliced into feature sequences. In addition, the feature sequences are fused using BiGRU network to encode the information of different hidden layers of BERT. Finally, the attention mechanism is introduced to assign weights to features based on their relevance to aspect information. The experimental results on the public SemEval2014 Task4 review datasets show that the proposed model outperforms the comparative models such as BERT, CapsBERT, BERT-PT, and BERT-LSTM in terms of accuracy and F1 value. It has a good effect on sentiment classification.