基于凸规划观点的神经网络学习

人工神经网络最重要的特征就是网络的学习能力,通过对神经网络内部权值的调整,学习外部环境结构的特征和信息的表示.从信息几何的角度,对神经网络的学习过程进行了几何描述,并将神经网络的学习问题转化为,最小化外部环境表示的真实分布与网络模型表示的逼近分布之间的"距离",并证明了对于可编码成指数簇流形的神经网络是凸规划问题.为神经网络学习的研究提出了一个新的思路,也为在高维非线性空间中解决网络学习问题的有效性,从信息几何的角度给出了一种解释.

Research of Neural Network Learning Based on the View of Convex Programming

The learning capacity is an important feature of neural network which means learning the structural features and information representation of the environment by adapting the parameters of the network. This paper interprets geometrically the learning process and transforms the learning problem in order to minimize the divergence between the true probability distribution of the environment and the approximating distribution of the network model in view of information geometry. And further more it gives a proof that this optimization problem can be seen as a convex programming in exponential neural manifold. This paper gives a new viewpoint about the learning of neural networksr: and a kind of information geometric explanation about the validity of researching learning in nonlinear high dimensional space is given.