AN INFORMATION THEORETICAL APPROACH TO NEURAL NETWORKS

The purpose of this paper is to present a unified theory of several differentneural networks that have been proposed for solving various computation, pattern recog-nition, imaging, optimization, and other problems. The functioning of these networks ischaracterized by Lyapunov energy functions. The relationship between the deterministicand stochastic neural networks is examined. The simulated annealing methods for findingthe global optimum of an objective function as well as their generalization by injectingnoise into deterministic neural networks are discussed. A statistical interpretation of thedynamic evolution of the different neural networks is presented. The problem of trainingdifferent neural networks is investigated in this general framework. It is shown how thisapproach can be used not only for analyzing various neural networks, but also for the choiceof the proper neural network for solving any given problem and the design of a trainingalgorithm for the particular neural network.

AN INFORMATION THEORETICAL APPROACH TO NEURAL NETWORKS

The purpose of this paper is to present a unified theory of several different neural networks that have been proposed for solving various computation, pattern recog- nition, imaging, optimization, and other problems. The functioning of these networks is characterized by Lyapunov energy functions. The relationship between the deterministic and stochastic neural networks is examined. The simulated annealing methods for finding the global optimum of an objective function as well as their generalization by injecting noise into deterministic neural networks are discussed. A statistical interpretation of the dynamic evolution of the different neural networks is presented. The problem of training different neural networks is investigated in this general framework. It is shown how this approach can be used not only for analyzing various neural networks, but also for the choice of the proper neural network for solving any given problem and the design of a training algorithm for the particular neural network.