| 高性能文本索引系统的设计与实现简 |
| |
| 引用本文： | 路炜,张宇,周美孜,刘燕兵,王春露.高性能文本索引系统的设计与实现简[J].中国科技论文在线,2014(1):92-95,107. |
| |
| 作者姓名： | 路炜 张宇 周美孜 刘燕兵 王春露 |
| |
| 作者单位： | [1]中国科学院信息工程研究所,北京100093 [2]河北师范大学附属民族学院,石家庄050091 [3]北京邮电大学计算机学院,北京100876 [4]中国人民大学信息学院,北京100872 |
| |
| 基金项目： | 国家高技术研究发展计划（863计划）资助项目（2011AA010703）;国家自然科学基金资助项目（61202477）;中国科学院战略性先导科技专项资助项目（XD06030602）;河北省教育厅资助科研项目（QN20131164） |
| |
| 摘 要： | 针对传统文本索引技术空间消耗大、分词不准确等问题，设计并实现了高性能文本索引系统。该系统采用压缩的全文自索引算法，节省了空间开销，避免了自然语言分词方法的影响，配合通配符搜索算法扩展了模糊搜索的功能，在众核CPU高性能处理器上可实现多线程并行处理，提高了处理速度，整个系统的实现是基于Web方式的，可以跨平台运行。实验结果表明，该系统将文本索引的空间消耗降为原文本的50％左右，具有较高的实用价值。
|
| 关 键 词： | 计算机应用 文本索引 全文索引 自索引 通配符搜索 |
Design and implementation of high-performance text index system |
| |
| Lu Wei,Zhang Yu,Zhou Meizi,Liu Yanbing,Wang Chunlu.Design and implementation of high-performance text index system[J].Sciencepaper Online,2014(1):92-95,107. |
| |
| Authors: | Lu Wei Zhang Yu Zhou Meizi Liu Yanbing Wang Chunlu |
| |
| Institution: | 1. Institute of Information Engineering, Chinese Academy of Science, Beijing 100093, China ; 2. School of Nationalities, Hebei Normal University, Shijiazhuang 050091, China; 3. School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China; 4. School of Information, Renmin University of China, Beijing 100872, China) |
| |
| Abstract: | The traditional text index technology has problems like oversize space consumption,inaccurate word segmentation,etc. In order to solve these problems,a high-performance text index system is designed and implemented.This system employs com-pressed full-text self-index algorithm,which helps save space and avoid influence from natural language word segmentation.Com-bined with wildcard search algorithm,its search function is enriched.And applied on high-performance processors with many-core CPU,this system can execute in a parallel multi-thread way,which increases processing speed.The entire system is Web-based,so that it can be run in different operating systems.Experimental results show that this systemhas high practical value in thatit reduces space consumption to about 50%of primary text. |
| |
| Keywords: | computer application text index full-text index self-index wildcard search |
| 本文献已被 CNKI 维普 等数据库收录！ |
|
