Shadow zones of an artificial neuron
Abstract
The extremely widespread use of artificial neural networks in the diverse areas of application makes the study of their fundamental properties highly relevant. Such studies can be used to improve the properties of neural networks.
The key goal of the work: to determine the general properties of artificial neurons and detect the presence of zones where the field of output signals has a complex fractal structure in the space of all input signals.
Research methods: to explore the space of all input signals, a software that allows modelling the neuron's response to all possible input signals with a certain length in the given alphabet has been developed. With the help of the developed application the space of all input signals can be modulated and the field of output signals in this space is graphically determined. By using the capability of the software to change the scale of the input signal space, zones with a self-similar, fractal structure have been found.
Results: it has been established that when considering the overall arrangement of the neuron’s input signal space, specific areas – shadow zones – are present, which exhibit a complex fractal structure of output signal field. The impact of modifying theneuron’s weights and threshold on the presence and location of such zones has been established. The changes that follow an increase in the length of the input signals have been described. The fractal dimension of the structures within shadow zones has been determined.
Conclusions: the obtained general properties of neurons should significantly impact the properties of neural networks in the form of shadow zones in which the "response" of the network is extremely sensitive even to minute alterations in input signals. The presence of such zones is an extremely important factor that needs to be considered while developing neural networks.
Downloads
References
/References
W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115–133, 1943.https://doi.org/10.1007/BF02478259
F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, no. 6, pp. 386–408, 1958.https://doi.org/10.1037/h0042519
F. Rosenblatt, Principles of Neurodynamics. Washington, DC: Spartan Books, 1962.https://doi.org/10.2307/1419730
B. Widrow and M. E. Hoff, "Adaptive switching circuits," in 1960 IRE WESTCON Conference Record. New York, 1960.https://doi.org/10.7551/mitpress/4943.003.0012
D. O. Hebb, The organization of behavior; a neuropsychological theory. New York: Wiley, 1949.
M. L. Minsky and S. A. Papert, Perceptrons. Cambridge, MA: MIT Press, 1969.https://doi.org/10.1016/s0361-9230(99)00182-3
J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings of National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, April 1982.https://doi.org/10.1073%2Fpnas.79.8.2554
J. J. Hopfield, "Neural with graded response have collective computational properties like those of two-state neurons," Proceedings of the National Academy of Sciences, vol. 81, no. 10, pp. 3088-3092, May 1984.https://doi.org/10.1073/pnas.81.10.3088
J. J. Hopfield, "Learning algorithms and probability distributions in feed-forward and feed-back networks,"Proceedings of the National Academy of Sciences, vol. 84, no. 23, pp. 8429-8433, December 1, 1987.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing, vol. 1, pp. 318–362. Cambridge, MA: MIT Press, 1986.https://doi.org/10.7551/mitpress/5236.003.0012
Y. Wu and J. Feng, "Development and application of artificial neural network," Review of Wireless Personal Communications, vol. 102, no. 2, pp. 1645-56, 2018.https://doi.org/10.1007/s11277-017-5224-x
A. Sharma and A. Chopra, "Artificial neural networks: Applications in management," Review of Journal of Business and Management, vol. 12, no. 5, pp. 32-40, 2013.https://doi.org/10.1371%2Fjournal.pone.0212356
J. Jiang, P. Trundle, and J. Ren, "Medical image analysis with artificial neural networks," Review of Computerized Medical Imaging and Graphics, vol. 34, no. 8, pp. 617-31, 2010.https://doi.org/10.1016/j.compmedimag.2010.07.003
A. Abraham, "Artificial Neural Networks," in Handbook of Measuring System Design, 1st ed., vol. 3, John Wiley & Sons, 2005. http://dx.doi.org/10.1002/0471497398.mm421
G. Cantor, "Über eine elementare Frage der Mannigfaltigskeitslehre," Jahresbericht der Deutschen Mathematiker-Vereinigung, vol. 1, pp. 75–78, 1891.[in German]http://eudml.org/doc/144383
P. Halmos, Naive Set Theory. Princeton, NJ: D. Van Nostrand Company, 1960. Reprinted by Springer-Verlag, New York, 1974.
B. B. Mandelbrot, The Fractal Geometry of Nature. New York: W. H. Freeman and Co., 1982.
M. F. Barnsley and H. Rising, Fractals Everywhere. Boston: Academic Press Professional, 1993.
R. M. Crownover, Introduction to Fractals and Chaos. University of Missouri-Columbia: Jones and Bartlett Publishers, Boston, London, 1995.
H. G. Schuster, Deterministic Chaos: An Introduction. Weinheim: Physik-Verlag, 1984.
W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115–133, 1943.https://doi.org/10.1007/BF02478259
F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, no. 6, pp. 386–408, 1958.https://doi.org/10.1037/h0042519
F. Rosenblatt, Principles of Neurodynamics. Washington, DC: Spartan Books, 1962.https://doi.org/10.2307/1419730
B. Widrow and M. E. Hoff, "Adaptive switching circuits," in 1960 IRE WESTCON Conference Record. New York, 1960.https://doi.org/10.7551/mitpress/4943.003.0012
D. O. Hebb, The organization of behavior; a neuropsychological theory. New York: Wiley, 1949.
M. L. Minsky and S. A. Papert, Perceptrons. Cambridge, MA: MIT Press, 1969.https://doi.org/10.1016/s0361-9230(99)00182-3
J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings of National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, April 1982.https://doi.org/10.1073%2Fpnas.79.8.2554
J. J. Hopfield, "Neural with graded response have collective computational properties like those of two-state neurons," Proceedings of the National Academy of Sciences, vol. 81, no. 10, pp. 3088-3092, May 1984.https://doi.org/10.1073/pnas.81.10.3088
J. J. Hopfield, "Learning algorithms and probability distributions in feed-forward and feed-back networks,"Proceedings of the National Academy of Sciences, vol. 84, no. 23, pp. 8429-8433, December 1, 1987.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing, vol. 1, pp. 318–362. Cambridge, MA: MIT Press, 1986.https://doi.org/10.7551/mitpress/5236.003.0012
Y. Wu and J. Feng, "Development and application of artificial neural network," Review of Wireless Personal Communications, vol. 102, no. 2, pp. 1645-56, 2018.https://doi.org/10.1007/s11277-017-5224-x
A. Sharma and A. Chopra, "Artificial neural networks: Applications in management," Review of Journal of Business and Management, vol. 12, no. 5, pp. 32-40, 2013.https://doi.org/10.1371%2Fjournal.pone.0212356
J. Jiang, P. Trundle, and J. Ren, "Medical image analysis with artificial neural networks," Review of Computerized Medical Imaging and Graphics, vol. 34, no. 8, pp. 617-31, 2010.https://doi.org/10.1016/j.compmedimag.2010.07.003
A. Abraham, "Artificial Neural Networks," in Handbook of Measuring System Design, 1st ed., vol. 3, John Wiley & Sons, 2005. http://dx.doi.org/10.1002/0471497398.mm421
G. Cantor, "Über eine elementare Frage der Mannigfaltigskeitslehre," Jahresbericht der Deutschen Mathematiker-Vereinigung, vol. 1, pp. 75–78, 1891.[in German]http://eudml.org/doc/144383
P. Halmos, Naive Set Theory. Princeton, NJ: D. Van Nostrand Company, 1960. Reprinted by Springer-Verlag, New York, 1974.
B. B. Mandelbrot, The Fractal Geometry of Nature. New York: W. H. Freeman and Co., 1982.
M. F. Barnsley and H. Rising, Fractals Everywhere. Boston: Academic Press Professional, 1993.
R. M. Crownover, Introduction to Fractals and Chaos. University of Missouri-Columbia: Jones and Bartlett Publishers, Boston, London, 1995.
H. G. Schuster, Deterministic Chaos: An Introduction. Weinheim: Physik-Verlag, 1984.
