基于Web Services的数据采集

以异构的、自治的、分布的数据库系统构建数据仓库是个挑战, 必须解决两个问题:一是采取有效的措施从各分布的异构数据源采集数据,二是对收集来的数据进行清理和格式转换.笔者分析了Web services的开放性和互操作性,提出了以SOAP协议和HTTP协议连接Internet的异构数据源,实现基于Web Services的数据采集系统.基于组件技术,提出了一种通用的数据采集器结构,用于数据清理和数据转换.并探讨了数据采集器以XML方式实现数据转换和数据装载的核心技术.

Data Collection Based on Web Services

To construct data warehouse with heterogeneous, self-governing and distributed database system is a challenge. Two problems have to be solved: one is how to collect data from heterogeneous data sources by an effective method, the other is how to clear up and change the format of the data collected from data sources. The opening and the interact-operation property of Web Services is Analyzed. A Data Collection System based on Web Services, which communicates with heterogeneous data sources with SOAP and HTTP, has been designed. A Data Collector is introduced to clear up and to transform data with composition technology. The pivotal XML technology of transforming and loading up the data are discussed.