N.S. Konnova – PhD (Eng.), Associate Professor, Bauman Moscow State Technical University

E-mail: nkonnova@bmstu.ru

The article is devoted to the review of artificial neural networks: they are examined from the point of view of the mathematical apparatus that they represent, and cases of their application in various large well-known IT projects. The basics of neural networks are described: the structural unit of the structure of all networks is a neuron, the connections between neurons are synapses, their characteristics, the laws of building networks from structural units, the types and procedures for training artificial neural networks, and the modes of their functioning. The main available architectures of neural networks and their applications are given and briefly characterized. A perceptron, multilayer perceptron, recurrent neural networks, convolutional neural networks, self-organizing neural networks are mentioned. Particular attention is paid to convolutional networks and LSTM (Long short-term memory), as type of recurrent neural networks, since networks of these architectures are currently the most popular in various fields of application of artificial intelligence and machine learning, in particular.

Details are described of the construction and features of recurrent networks with long short-term memory, which, due to the presence of feedbacks and the ability to delay the signal transmitted back, has so-called memory. The presence of "memory" gives the network the ability to maintain context. This feature of this class of neural networks is particularly useful in word processing tasks, which has led to an increase in the popularity of LSTM due to the boom in various chat bots and other services requiring sematic text analysis. Therefore, further on the example of services that have become indispensable daily helpers for us – Yandex.Alice and Google.Translate – we consider the use of these deep LSTM neural networks.

Image recognition, which is widely used, for example, both in matters of information security at enterprises and in household devices: in your iPhone that recognizes you by your face, has also become a popular trend. The article discusses the architecture and application of convolutional neural networks that show the best results in tasks of this kind, using the example of Google Photos and Google Image Search.

1. Pribram K. Languages of the brain. Prentice-Hall. 1971. 432 p.
2. Vlasov A., Yudin A. Distributed control system in mobile robot application: general approach. realization and usage. Communications in Computer and Information Science. 2011. T. 156 CCIS. P. 180–192.
3. Yakovlev V.L., Yakovleva G.L., Vlasov A.I. Neyrosetevyye metody i modeli pri prognozirovanii kratkostrochnykh i dolgosrochnykh tendentsiy finansovykh rynkov. Materialy VI Vseross. konf. «Neyrokompyutery i ikh primeneniye». 2000. S. 372–377. (in Russian)
4. Rozenblatt F. Printsipy neyrodinamiki. Pertseptrony i teoriya mekhanizmov mozga. M.: Mir. 1965. 82 s. (in Russian)
5. Soldatova O.P. Neyroinformatika. Kurs lektsiy. SGAU. 2013. 130 s. (in Russian)
6. LeCun Y., Bengio Y. Convolutional Networks for Images. Speech. and Time-Series. The Handbook of Brain Theory and Neural Networks. MIT Press. 1995.
7. Kak rabotayet neyroset Google Translate [Elektronnyy resurs] URL: https://www.cossa.ru/152/196086 (Data obrashcheniya 26.03.2020). (in Russian)
8. Understanding LSTM Networks [Elektronnyy resurs] URL: https://colah.github.io/posts/2015-08-Understanding-LSTMs/ (Data obrashcheniya 26.03.2020).
9. Kak ustroyena Alisa. Lektsiya Yandeksa [Elektronnyy resurs] URL: https://habr.com/ru/company/yandex/blog/349372/ (Data obrashcheniya 26.03.2020). (in Russian)
10. Basarab M.A., Konnova N.S. Intellektualnyye tekhnologii na osnove iskusstvennykh neyronnykh setey. M.: Izd-vo MGTU im. N.E. Baumana. 2017. 53 s. (in Russian).
11. LSTM – seti dolgoy kratkosrochnoy pamyati [Elektronnyy resurs] URL: https://habr.com/ru/company/wunderfund/blog/331310/ (Data obrashcheniya 30.03.2020). (in Russian)
12. LeCun Y., Bottou L., Bengio Y., Haffner P. GradientBased Learning Applied to Document Recognition. Proceedings of the IEEE. 1998. 46 p.
13. Krizhevsky A., Sutskever I., Hinton G. ImageNet Classification with Deep Convolutional Neural Networks. Advances in neural information processing systems. 2012. V. 25. № 2.
14. Hubel D., Wiesel T. Brain mechanisms of vision. Scientific American. 1979. 241(3).
15. Nayak S. Understanding AlexNet [Elektronnyy resurs] URL: https://www.learnopencv.com/understanding-alexnet/ (Data obrashcheniya 30.03.2020).
16. Dumoulin V., Visin F. A guide to convolution arithmetic for deep learning [Elektronnyy resurs] URL: https://arxiv.org/pdf/1603.07285.pdf (Data obrashcheniya 30.03.2020).
17. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. P. 1–10.
18. Szegedy C., Vanhoucke V., Ioffe S., Shlens J. Rethinking the Inception Architecture for Computer. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016 [Elektronnyy resurs] URL: https://arxiv.org/pdf/1512.00567.pdf (Data obrashcheniya 30.03.2020).
19. Goodfellow I.J., Shlens J., Szegedy C. Explaining and harnessing adversarial examples. ICLR. 2015 [Elektronnyy resurs] URL: https://arxiv.org/pdf/1412.6572.pdf (Data obrashcheniya 30.03.2020).