前馈网络构造性设计中基于GP实现神经元激活函数类型优化

讨论了在前馈网络构造性设计中如何基于遗传编程(GP)实现神经元激活函数类型自动优化的问题.首先,提出了典型前馈网络的一种构造性设计方法框架,将整个网络的设计分解为单个神经元的逐个设计;然后,在此框架下提出了基于GP的单个神经元的设计方法,该方法可实现对激活函数类型的优化.仿真实验显示,本文的前馈网络构造性设计方案是可行的,与其他几种不优化激活函数类型的网络设计方法相比,本方法更有效,能用较小的网络规模获得更满意的泛化特性.

Optimizing neuronal activation function types based on GP in constructive FNN design

Aiming at typical feedforward neural networks (FNN), a constructive FNN designing algorithm with the auto-optimization of neuronal activation function types based on genetic programming (GP) is investigated. First, a frame of the constructive FNN design is given, in which the design of the whole FNN breaks down to the design of single neurons one by one. Then, based on GP the design algorithm of single neuron, which realizes the auto-optimization of neuronal activation function types, is proposed. Finally, with many function approximation experiments, it is shown that the proposed constructive FNN design scheme is feasible. Compared with some other designing algorithms without activation function type optimization, it is more effective, being able to achieve better FNN generalization with smaller network size.