基于Stacking-Bert集成学习的中文短文本分类算法

由于word2vec、Glove等静态词向量表示方法存在无法完整表示文本语义等问题,且当前主流神经网络模型在做文本分类问题时,其预测效果往往依赖于具体问题,场景适应性差,泛化能力弱.针对上述问题,提出一种多基模型框架(Stacking-Bert)的中文短文本分类方法.模型采用BERT预训练语言模型进行文本字向量表示,输...

Chinese Short Text Classification Algorithm Based on STACKING-BERT Ensemble Learning

Duo to the static word vector representation methods such as word2vec and Glove have problems such as incomplete representation of text semantics, and when the current mainstream neural network model is doing text classification problems, its prediction effect often depends on specific problems, the scene adaptability is poor, and the generalization ability is weak. To solve the above problems, a Chinese short text classification method based on multi-base model framework named Stacking-Bert is proposed. The model uses the BERT pre-trained language model to represent text word vectors, outputs the deep feature information vector of the text, and uses neural network models such as TextCNN, DPCNN, TextRNN, TextRCNN to construct a heterogeneous multi-base classifier, and obtains the text vector through Stacking integration learning Different feature information is expressed to improve the generalization ability of the model, and finally SVM is used as a meta-classifier model for training and prediction. Comparing experiments with text classification algorithms such as word2vec-CNN, word2vec-BiLSTM, BERT-texCNN, BERT-DPCNN, BERT-RNN, BERT-RCNN, etc. on three Chinese data sets published on the Internet, the results show that Stacking-Bert integrated learning The model has the highest accuracy rate, precision rate, recall rate and F1 value, which can effectively improve the classification performance of Chinese short texts.