一种增量倒排索引结构的设计与实现

针对主题爬行器获取网页更新速度快的特点, 提出一种用于网络搜索引擎的增量索引结构. 在建立倒排索引时, 每个词项的记录表以链接块的形式存放于倒排索引文件中, 每次新分配的块大小递增. 该索引结构解决了倒排索引连续存储所带来的难以更新问题. 实验结果表明, 与支持实时更新的传统链表式存储方式相比, 这种索引结构能提供更高效的检索, 采用以空间换时间的方法有效地提高了索引的更新效率.

Design and Implementation of an Incremental Inverted Index Framework

In the present paper is proposesd an incremental index structure used in web search engine in order to deal with the high update frequency of the web pages crawled by domain-specific crawler. In the inverted index structure, the posting list of each term is partitioned into linked blocks, whose sizes form an arithmetical series. The incremental index structure resolves the problem of document update, which is expensive in inverted index of continuous storage, and experimental results show that it provides much higher retrieval efficiency than naive linked list structure, which also supports real-time update. The space-for-time approach effectively raises the update rate of index.