一种改进型TF-IDF文本聚类方法

针对传统词频 逆文档频率(TF-IDF)算法对具有特定属性的文本分类存在的不足, 尤其是词汇在特定分类中具有特殊意义情形下准确率较低的问题, 提出一种改进的TF-IDF文本聚类算法. 采用2015—2019年吉林省科研机构发表论文数据进行对比实验, 分别用改进TF-IDF算法和传统TF-IDF算法先统计论文中的关键词词频, 再通过K-means++算法进行聚类, 最后使用随机森林算法分别评估聚类的准确性. 实验结果表明, 改进TF-IDF算法提高了分类的准确率.

An Improved TF-IDF Text Clustering Method

Aiming at the shortcomings of traditional term frequency-inverse document frequency (TF-IDF) algorithm for text classification with specific attributes, especially the low accuracy of words with specific meaning under specific classification, we proposed an improved TF-IDF text clustering algorithm. Comparative experiments were carried out through the papers published by scientific research institutions in Jilin Province from 2015 to 2019. The improved TF-IDF algorithm and the traditional TF-IDF algorithm were used to calculate the frequency of keywords in the papers, then K-means++ method was used to cluster. Finally, random forest algorithm was used to evaluate the accuracy of clustering. The experimental results show that the improved TF-IDF algorithm improves the accuracy of classification.