基于GPU的并行拟牛顿神经网络训练算法设计

针对人工神经网络训练需要极强的计算能力和高效的最优解搜寻方法的问题,提出基于GPU的BFGS拟牛顿神经网络训练算法的并行实现。该并行实现将BFGS算法划分为不同的功能模块,针对不同模块特点采用混合的数据并行模式,充分利用GPU的处理和存储资源,取得较好的加速效果。试验结果显示:在复杂的神经网络结构下,基于GPU的并行神经网络的训练速度相比于基于CPU的实现方法最高提升了80倍;在微波器件的建模测试中,基于GPU的并行神经网络的速度相比于Neuro Modeler软件提升了430倍,训练误差在1%左右。

Parallel BFGS quasi-Newton algorithm of neural network training based on GPU

One application challenge of the neural networks is the training, the essence of which is an iterative optimization process involving numerous data. The training process requires significant computing power and efficient searching methods for optimal solutions. To meet the requirement, a parallel BFGS quasi-Newton training algorithm based on GPU is proposed in this paper. To maximize the parallelism, the BFGS quasi-Newton algorithm is divided into different function modules, and each module is designed with a specific parallelism regarding its characteristics. In addition, the processing and memory resources of GPU are fully utilized to achieve a better parallelization. Experimental results show that the GPU implementation accelerates the neural network training by up to 80 times compared to the CPU implementation for the complicated neural network structure, while the speed up ratio is up to 430 for the modelling test of microwave device, where the training error is about 1%.