中文文本分类相关算法的研究与实现

通过对分词歧义处理情况的分析, 提出一种基于上下文的双向扫描分词算法, 对分词词典进行改进, 将词组短语的固定搭配引入词典中. 讨论了特征项的选择及权重的设定, 并引进χ²统计量参与项的权值计算, 解决了目前通用TF-IDF加权法的不足, 同时提出了项打分分类算法, 提高了特征项对于文本分类的有效性.
 实验结果表明, 改进后的权重计算方法性能更优越.

Research and Implementation of Related Algorithm of Chinese Text Categorization

On the basis of the analysis of the process of dealing with the Chinese word segmentation ambiguity,this paper covers bidirectional scan word segmentation algorithm based on the context.In order to improve the word segmentation dictionary,the authors put the fixed phrase into the dictionary and discussed the feature selection and the weighting schema enactment in detail.In order to solve the problem of general TF-IDF weighting schema at present,we took statistics into consideration,and meanwhile put up the ...