| 一种基于Hadoop的布鲁姆过滤器联结算法 |
| |
| 引用本文： | 钟杰卓,杜文才.一种基于Hadoop的布鲁姆过滤器联结算法[J].海南大学学报(自然科学版),2014(1):45-50,87. |
| |
| 作者姓名： | 钟杰卓 杜文才 |
| |
| 作者单位： | [1]海南大学应用科技学院,海南海口571101; [2]海南大学信息学院,海南海口570228 |
| |
| 基金项目： | 国家自然科学基金（61162010）;海南大学青年基金（qnjj118） |
| |
| 摘 要： | 通过对Hadoop平台下MapReduce作业处理方式及布鲁姆过滤器算法的深入研究,将优化的压缩型布鲁姆过滤器算法用于节点间数据联结操作,解决了基于Hadoop平台同时处理多个大规模数据集时的数据关联问题.实验证明,压缩型布鲁姆过滤器算法在MapReduce作业中的应用,使得大数据集之间的联结效率显著提高.
|
| 关 键 词： | Hadoop 大数据集 联结 布鲁姆过滤器 |
A Bloom Filter Join Algorithm Based on Hadoop |
| |
| Institution: | ZHONG Jie-zhuo1, DU Wen-cai2 (1. College of Applied Science and Technology, Hainan University, Haikou $71101, China; 2. College of Information Science ＆ Technology, Hainan University, Haikou 570228, China) |
| |
| Abstract: | Based on intensive research on Bloom Filter algorithm and the processing mode of MapReduce job on Hadoop platform, the problems of data association when simultaneous processing multiple large-scale dataset on Hadoop platform were solved. The results indicated that the application of Compressed Bloom Filter in the pro- cessing mode of MapReduce job can improve the efficiency of joining among large datasets obviously. |
| |
| Keywords: | Hadoop large dataset join bloom filter |
| 本文献已被 CNKI 维普 等数据库收录！ |
|
