基于文本频谱的中文文本聚类方法

本文提出了利用文本频谱进行中文文本轮廓分析的表征方式.该方法基于不同时代、体裁和领域的文本在文字使用方面具有偏好性的假说,以文本中单个字符为单位,通过文本频谱刻画方法统计所有单字符在文本中出现的频率,并使用刻画出的文本频谱对文本进行表征;利用频谱比对分析技术,可计算出任意文本间的距离,并以此距离为基础进行聚类分析.进一步的实验证实了该方法的有效性.

Text profiling based chinese text analysis and clustering

In this paper, a Chinese text representation method by using text profiling approach was presented. This method is based on the hypothesis that, texts in different eras, styles and fields have biases in the use of characters. Text profiling was construct by counting the frequency of each character in the text in character level, and use the constructed text profiling to present the whole text. In addition, the distance between the two given texts was calculated by using text profiling alignment method. Finally, an effective text clustering method was developed on the basis of the distance. The experimental results validated the efficiency of the method.