中国古诗统计建模与宏观分析

利用自然语言处理技术处理文学文本是计算语言学领域近年来的热门话题.该文结合点态互信息量与频率阈值,自动发现中国古诗词汇.基于构建的诗歌词典,利用启发式的正向最大匹配算法,对中国古诗作分词处理.采用主题模型对分词后的诗歌文本进行统计建模,并在此基础上进行了主题演变和诗人群体风格网络的探索性分析.基于全唐诗语料的实验结果表明:主题模型可以给出具有较好解释力的中国古诗统计模型,验证已有的文学史研究,并在传统的文本细读的研究范式之外,对中国诗学提供了全新视角的宏观刻画、描述与阐释.

The Statistical Modeling and Macro-Analysis of Chinese Classical Poetry

Modeling literary texts with natural language processing technology has become a popular topic of computational linguistics in recent years.the vocabulary of Chinese classical poetry by combining point-wise mutual information(PMI)method and frequency threshold has been extracted.Based on the extracted poetic vocabulary,a heuristic forward maximum matching algorithm to segment the poems has been used.In order to model the poetry,latent dirichlet allocation(topic model), based on which we also put forward explorative analysis of the literature evolution and poet network has been used.The experiments on the corpus of All-Tang poetry indicate that topic model is an explanatory statistical model of the Chinese classical poetry.While proving the existed evolution theory of Chinese literature, the statistical model also provides insightful macro-analysis from a new perspective, in addition to the traditional methodology of Chinese literature research.