| 基于语义分析的网络爬虫模型的研究 |
| |
| 引用本文： | 刘炜,张红云,熊前兴.基于语义分析的网络爬虫模型的研究[J].科技信息,2008(35). |
| |
| 作者姓名： | 刘炜 张红云 熊前兴 |
| |
| 作者单位： | 武汉理工大学计算机科学与技术系; |
| |
| 基金项目： | 湖北省交通科技专项(20073H0400) |
| |
| 摘 要： | 针对传统通用网络信息采集系统自身固有的缺陷,根据语义分析的相关理论,本文提出了基于语义的网络爬虫的相关模型,该模型构建知识概念集合,并对其进行关键字切割与划分,生成能表达主题的主题关键词集合。结合中国《知网》的相关理论与技术,对传统抓取的网页在语义的角度进行分析,对已抓取的网页页面内容及其中链接的扩展元数据等相关信息进行分词及语法语义等相关处理,获取网页内容关键词集合及超链接的关键词集合,然后分别对获取的网页内容关键词与链接关键词进行与主体关键词集合采用语义分析算法进行语义相关性的判定,保存需要的网页,并预测及提取与主题相关的URL,从而提高网络资源信息采集相关率。
|
| 关 键 词： | 领域知识集 网络爬虫 扩展元数据 知网 |
Research Of Web Crawler Based On Semantic Analysis |
| |
| LiuWei,ZhangHong-yun,XiongQian-Xing.Research Of Web Crawler Based On Semantic Analysis[J].Science,2008(35). |
| |
| Authors: | LiuWei ZhangHong-yun XiongQian-Xing |
| |
| Institution: | LiuWei,ZhangHong-yun,XiongQian-Xing(Department Of Computer Science , Technology,WuHan University Of Technology,WuHan HuBei 430063,China) |
| |
| Abstract: | According to inborn defects of the traditional general web crawler,besides relevant theories of semantic design,we put forward a new model of web crawler which is based on semantic technology.The model make full use of the technology to generate a collection of domain knowledge topics which will be processed in key words and divided into several segments.The model will finally produced a collection containing whole key words related with topics.Also,It will deal with pages and URLs which are fetched by our ... |
| |
| Keywords: | semantic web crawler extended metadata HowNet |
| 本文献已被 CNKI 等数据库收录！ |
|
