基于集群的并行分布式聚类及其应用

海量和高维大数据集的聚类对计算机性能提出了很高的要求.基于具有层次聚类特性的RSOM树方法提供了一种有效的手段以实现对高维大数据集的聚类索引,这种RSOM树可支持最近邻搜索且不需要对数据进行线性搜索.注意到RSOM模型具有内在的层次化、分布式结构特点,并可进行增量的训练,研究了基于高效并行集群的增量、分布式RSOM并行算法,并通过视频图像特征集实例证实了算法的可行性.

Cluster-PC Based Parallel Distributed Data Clustering and Its Applications

Clustering data with high dimensionalities requires high-performance computers to get results in a reasonable amount of time,particularly for extremely large-scale databases.Thus,the recursive SOM(RSOM) tree method is proposed.RSOM tree is a hierarchy of clusters and sub-clusters which incorporates the cluster representation into the index structure.It provides a practical solution to index clustered data set,and it supports the retrieval of the nearest-neighbors effectively and efficiently without having to linearly search a high-dimensional large database.Meanwhile,an incremental RSOM tree-based clustering algorithm is proposed;and because of the RSOM tree is of the nature of parallelism,and can be implemented on scalable parallel computers. Thus a cluster-system based distributed parallel algorithm of incremental RSOM tree is proposed.The performance of the method has been tested with high dimensional feature sets extracted from large image database.