基于MapReduce的关联规则并行增量更新算法

针对在关联规则的实际挖掘中,由数据快速增加所造成的大数据问题和增量更新问题?在快速更新频繁模式树算法(fast updated frequent pattern tree,FUFP - tree)的基础上,引入MapReduce 编程模型,提出了一个面向大数据的并行的关联规则增量更新算法(parallel fast updated frequent pattern tree,PFUFP - tree)?该算法通过构建原始事务数据的分块索引,从而使得在每次增量更新时,能够最小化地扫描原始事务数据库,提高了挖掘效率;同时采用动态负载均衡的项目分组策略来优化并行计算过程中的项集分组问题,从而保证分布式集群中节点之间的负载均衡;实验结果证明,提出 的算法是有效的和高效的,适用于动态增长的大数据环境?

Parallel and incremental updating algorithm for association rules based on mapReduce

In the actual mining of association rules, aiming at the big data problem and incremental updating problem caused by the rapidly increasing of data, in this paper,a parallel incremental updating algorithm of association rules is proposed based on the MapReduce parallel programming model and the FUFP-tree algorithm. At first, the block index of the original transactions would be built.Based on the index, the number of scanning the original transaction database can be reduced.Therefore,the mining efficiency would be improved.Secondly, the grouping strategy of dynamic load-balancing is adopted to solve the item grouping problem in the process of parallel computing, so as to ensure the load-balancing between nodes of the distributed clusters.Finally,according to the compared experiment results,it is demonstrated that the proposed algorithm is effective and efficient,and can be used to incremental big data environment.