V.I. Terekhov - Ph.D. (Eng.), Associate Professor, Department - Information Processing and Control Systems?, Bauman Moscow State Technical University E-mail: terekchow@bmstu.ru I.M. Chernenkiy - Post-graduate Student, Department - Information Processing and Control Systems?, Bauman Moscow State Technical University E-mail: cheivan@mail.ru S.V. Minakova - magister, Department - Information Processing and Control Systems?, Bauman Moscow State Technical University E-mail: morgana_93@mail.ru

This article is devoted to research and development of methods for determining the number of neurons and connections in the hidden layer of fully connected neural network (perceptron) used in deep neural networks as a classifier. The purpose of this article is to examine methods for adjustment the number of neurons in a single-layer perceptron network during its training, which can be used as a tool to accelerate the training process of neural network and to choose the optimal number of neurons in the hidden layer of it. The first part of the article deals with the heuristic rules for determining the number of neurons in the hidden layers of fully connected neural network and methods of its correction during the training process. It also involves theoretical comparison of these methods and rules, as well as analysis of their benefits and drawbacks. The second part of the article describes the method of adding neurons in the hidden layer of perceptron network during its training, proposed by author. This method is based on the Falmans cascade correlation algorithm and a trick called «Jog of weights». The background of this methods development and justification of its applicability to the perceptron with one hidden layer are also provided. Then were performed analytical calculations, reflecting pros and cons of the method and proposed its modification with taking into account the deficiencies identified in the analytical calculations. In the experimental part of article the method and its modification were compared with the method of training the network without the neurons addition in terms of the classification accuracy on a test set, the learning rate and the optimal number of neurons in the hidden layer of the neural network. Summary of the article provides conclusions, from which it follows that the methods of adding neurons and connections in single-layer perceptron network during its training can be used as a tool for accelerating training of a neural network and selecting the optimal number of neurons in its hidden layer.

 

1. Osovskijj S. Nejjronnye seti dlja obrabotki informacii: Per. s polsk. I.D. Rudinskogo. M.: Biblio-Globus. 2016.
2. KHash D., KHorn B. Progress v upravljaemykh nejjronnykh setjakh // Obrabotka signalov. 1993. S. 8-39.
3. KHekht-Nilsen. Otobrazhenie teoremy Kolmogorova v terminakh iskusstvennykh nejjronnykh setejj // Dokl. na Mezhdunar. konf. po nejjronnym setjam. San-Diego. 1987. Vyp. 3. S. 11-13.
4. Pupkov K.A., Konkov V.G. Intellektualnye sistemy. Issledovanie i sozdanie. M.: MGTU im. N. EH. Baumana. 2003.
5. Makarov I.M., Lokhin V.M., Manko S.V. Iskusstvennyjj intellekt i intellektualnye sistemy upravlenija. Otdelenie informacionnykh tekhnologijj i vychislitelnykh sistem RAN. M.: Nauka. 2006. S. 30-31.
6. Kruglov V.V., Borisov V.V. Iskusstvennye nejjronnye seti. Teorija i praktika. Izd. 2-e. M.: Gorjachaja linija-Telekom. 2002.
7. LeKun JA., Denker Dzh., Solla S. Optimalnoe prorezhivanie nejjronnykh setejj // Dostizhenija v NIPS2 // Pod red. D. Tureckogo. 1990. C. 598-605.
8. Mirkes E.M. Nejjrokompjuter. Proekt standarta. Novosibirsk: Nauka. Sibirskaja izdatelskaja firma RAN. 1999. 337 s.
9. Gerc A., Anders S.K., Richard G.P. Vvedenie v teoriju nejjronnykh vychislenijj. T.2. Pressa Uehstvju. 1995.
10. Li K., Tifts D. Sintez nejjronnykh setejj posledovatelnym dobavleniem skrytykh modelejj // dokl. na mezhdunarodnojj konferencii po nejjronnym setjam. Orlando. 1994.
11. Anderson, Dzhejjms A. Diskretnaja matematika i kombinatorika. M.: Izdatelskijj dom «Viljams». 2004. 960 s.
12. Falman S.E., Leber S. Arkhitektura seti kaskadnojj korreljacii // Dostizhenija v NIPS2 // Pod red. D. Tureckogo. 1990. C. 524-532.
13. Falman S.E. Metody uskorenija obuchenija na obratnom rasprostranenii oshibok: ehmpiricheskoe issledovanie // Pod red. Morgana Kaufmana. Los-Altos. SSHA. 1998. S. 38-51.
14. Klimauskas G. Neural Ware - instrukcija dlja polzovatelejj, kompanija Neural Ware, SSHA, 1992.
15. Ryndin A.A., Ulev V.P. Issledovanie skorosti obuchenija nejjronnykh setejj // Vestnik VGU. T.8. Vyp. 5. 2012.
16. Duda O., KHart P., Stork D. Klassifikacija patternov. Izd. 2-e. Wiley-Interscience. 2000.
17. Sedzhvik R., Flazholet F. Vvedenie v analiz algoritmov. Izd. 2-e. Addison-Wesley. 2013. S. 3-26.
18. Igel K., Vigand S, Fridrikhs F. EHvoljucionnaja optimizacija nejjronnykh sistem: ispolzovanie strategii adaptacii // Mezhdunarodnyjj zhurnal vychislitelnojj matematiki. 2005. Vyp. 1.
19. LeKun JA., Korinna K., Berdzhes Dzh.K. Baza dannykh rukopisnykh simvolov MNIST, rezhim dostupa [http://yann.lecun.com/exdb/mnist/] (data obrashhenija 20.01.17).