基于二叉级联结构的并行极速学习机算法

为解决因庞大的矩阵存储和计算,ELM(Extreme Learning Machines)难以应用到大规模、高维数据集的问题,提出一种基于“分而治之”策略的并行极速学习机算法.该算法利用二叉级联结构,将大规模数据集分派到多个计算节点上,并行地更新单隐层前馈网络的输出权值,且能有限步地单调收敛到最小二乘解.实验结果表明,该算法不仅泛化性能优异,并且具有非常高的加速比和并行效率.

Parallel Extreme Learning Machines Based on Binary Cascade Architecture

ELM(Extreme Learning Machines) always works inefficiently on large-scale and high-dimension datasets for huge memory and computation costs.We prepose a novel parallel algorithm for training ELM quickly based on divide and conquer strategy.It dispatches large-scale datasets to a cluster of computing nodes by utilizing special binary cascade architecture,and then updates weights of SLFN(Single Hidden Layer Feed-forward Neural Network) in parallel.Theoretical analysis proves that the new algorithm converges to the best least-square solution monotonously with finite steps.Preliminary experimental results show that the new algorithm has good generalization ability,excellent speedup ratio and parallel efficiency.