Neural Networks in the Modern Age

Abstract

The idea to apply representations about the work of the human brain as mathematical models, which can be used in a variety of applied tasks is shown. It is shown that mathematical and physiological models have quite little in common, but the basic idea - to apply neurons as some independent computational nodes and combine them into layers - has been developed to the current state of technology. The historical background of the development of neural network science is given and the main researchers, whose works have influenced the vector and the pace of technology development the most, are mentioned. It is shown that neural networks had different support from investors during their development and the peak of mass interest in them depended on the emergence of necessary computing power or a breakthrough network architecture. Such networks were in its time perceptrons, networks with feedback, networks applying convolution operation for image analysis and classification. It is shown that the so-called deep learning has developed based on weight optimization methods by gradient descent. A review of known solutions of supervised learning, feedback, and language-based learning networks is conducted. Generative models look like the most promising direction in the development of scientific thought and the creation of interpretive solutions based on neural networks. It is shown that in learning with a teacher, which is typical for deep neural networks, the optimization for it is performed by regularization procedures, which help to avoid diversity and provide error minimization during error back propagation