一种基于Hadoop的布鲁姆过滤器联结算法 |
| |
引用本文: | 钟杰卓,杜文才. 一种基于Hadoop的布鲁姆过滤器联结算法[J]. 海南大学学报(自然科学版), 2014, 0(1): 45-50,87 |
| |
作者姓名: | 钟杰卓 杜文才 |
| |
作者单位: | [1]海南大学应用科技学院,海南海口571101; [2]海南大学信息学院,海南海口570228 |
| |
基金项目: | 国家自然科学基金(61162010);海南大学青年基金(qnjj118) |
| |
摘 要: | 通过对Hadoop平台下MapReduce作业处理方式及布鲁姆过滤器算法的深入研究,将优化的压缩型布鲁姆过滤器算法用于节点间数据联结操作,解决了基于Hadoop平台同时处理多个大规模数据集时的数据关联问题.实验证明,压缩型布鲁姆过滤器算法在MapReduce作业中的应用,使得大数据集之间的联结效率显著提高.
|
关 键 词: | Hadoop 大数据集 联结 布鲁姆过滤器 |
A Bloom Filter Join Algorithm Based on Hadoop |
| |
Affiliation: | ZHONG Jie-zhuo1, DU Wen-cai2 (1. College of Applied Science and Technology, Hainan University, Haikou $71101, China; 2. College of Information Science & Technology, Hainan University, Haikou 570228, China) |
| |
Abstract: | Based on intensive research on Bloom Filter algorithm and the processing mode of MapReduce job on Hadoop platform, the problems of data association when simultaneous processing multiple large-scale dataset on Hadoop platform were solved. The results indicated that the application of Compressed Bloom Filter in the pro- cessing mode of MapReduce job can improve the efficiency of joining among large datasets obviously. |
| |
Keywords: | Hadoop large dataset join bloom filter |
本文献已被 CNKI 维普 等数据库收录! |
|