基于信息增益的LDA模型的短文本分类

在基于LDA的短文本分类基础上进行改进,提出信息增益结合LDA的短文本分类方法.该方法采用信息增益计算词汇对于文本分类的贡献度,提高＂作用词＂的权重,过滤掉＂非作用词＂,最后对过滤后的短文本进行LDA主题建模,并采用中心向量法建立文本类别模型.实验证明,该方法随着作用词比例的减少,分类性能有较大的提高.

The classification of LDA model essay based on information gain

In this paper the classification of short essay was improved based on LDA.The information gain of the essay with LDA classification method was put forward.Using the information gain calculation to calculate the text classification vocabulary contribution,to improve ＂function word＂ weight,and to filter out ＂the function word＂,at last the passage of the filtered was in the LDA theme modeling,and the center vector method was used to establish the text category model.The experimental results prove that with the reducing of function word ratio,classification performance is distinctly improved in the method.