针对Web论坛的一种结构化数据自动抽取方法

由于网页布局设计的复杂性和用户发表帖子的灵活性,从论坛网页中抽取结构化的数据是一项未能很好解决并非常具有挑战性的任务。本文提出了一种从任意的论坛站点中自动抽取结构化数据的通用解决方案,通过分析网页结构发现列表页和帖子页中的数据记录,并利用一组产生式规则从发现的数据记录中抽取结构化的数据。实验结果表明该方法在抽取论坛数据记录方面明显优于已有的方法,对论坛帖子的标题、作者、发表时间和内容文本块等元数据的抽取达到了较高的准确率。

Automatic structured data extraction from Web forums

Because of both complex page layout designs and unrestricted user created posts, extracting structured data from Web forum pages is a very challenging task and not easily solved.  A general solution to automatically extract structured data from any forum site was proposed. By analyzing page structure, a group of data records were found from both list page and post page, and then a set of production rules was used to extract structured data from these data records. Experimental results showed that the proposed approach significantly outperformed some existing methods in extracting data records and achieved high accuracy in extracting some metadata of Web forums such as title, author, time and content.