O.A. Stepanova – Post-graduate Student, Infocommunication and Radiophysics Department, 

P.G. Demidov Yaroslavl State University E-mail: olga1stepanova@yandex.ru

L.I. Ivanovsky – Post-graduate Student, Computer Network Department, 

P.G. Demidov Yaroslavl State University E-mail: leonel-unknown@yandex.ru

V.V. Khryashchev – Ph.D. (Eng.), Infocommunication and Radiophysics Department,  P.G. Demidov Yaroslavl State University

E-mail: vhr@yandex.ru

A.L. Priorov – Dr.Sc. (Eng.), Infocommunication and Radiophysics Department, 

P.G. Demidov Yaroslavl State University

E-mail: andcat@yandex.ru

This paper presents an algorithm for emotion recognition on facial image. The developed algorithm is based on the implementation of convolutional neural network. The aim of this network is to classify facial images into one of the six types of emotions: neutral, smile, surprise, squint, disgust and scream. The neural network algorithm was trained and tested on the NVIDIA DGX-1 supercomputer using images from the Multi-PIE test database. In the framework of the study, the following was obtained: confusion matrix, the dependence of the accuracy and the loss function on the number of iterations in the training of a neural network, the values of the quality metrics of the final algorithm. The developed algorithm can be used in the real time audience analysis systems based on the use of face image.

1. Ekman P., Friesen W.V. Manual for the Facial Action Coding System. Consulting Psychologists Press. 1977.
2. Berns K., Hirth J. Control of Facial expressions of the humanoid robot head roman // In: IEEE International Conference on Intelligent Robots and Systems. 2006. P. 3119–3124.
3. Bartlett M.S., Littlewort G., Fasel I., Movellan J.R. Real time face detection and facial expression recognition: development and applications to human computer interaction // In: IEEE Conference on Computer Vision and Pattern Recognition, 2003. Р. 53–59.
4. Николенко С.И., Кадурин А.А., Архангельская Е.О. Глубокое обучение. СПб: Питер. 2018. 480 с. 
5. Гудфеллоу Я., Бенджио И., Курвилль А. Глубокое обучение. ДМК-Пресс. 2017. 652 с.
6. Ivanovsky L., Khryashchev V., Lebedev A., Kosterin I. Facial Expression Recognition Algorithm Based on Deep Convolution Neural Network // Proceedings of the 21th Conference of Open Innovations Association FRUCT'21. Helsinki, Finland. 2017. Р. 141–147.
7. Ucar A. Deep Convolutional Neural Networks for facial expression recognition // INnovations in Intelligent SysTems and Applications (INISTA). IEEE International Conference on. 2017. Р. 371–375.
8. Singh H., Patel R. Facial Expression Analysis using Deep Learning // International Research Journal of Engineering and Technology. 2017. V. 4. Is. 10. Р. 66–69.
9. The Cohn–Kanade          AU-Coded                Face         Expression               Database [Электронный        ресурс].   Режим     доступа: http://www.pitt.edu/~emotion/ck-spread.htm.
10. The Japanese Female    Facial       Expression               (JAFFE) Database [Электронный        ресурс]. Режим     доступа: http://www.kasrl.org/jaffe.html.
11. The Binghamton University 3D Facial Expression (BU-3DFE) Database [Электронный ресурс]. Режим доступа: http://www.cs.binghamton.edu/~lijun/Research/3DFE/3DFE_Analysis.html.
12. The MMI Facial Expression Database [Электронный ресурс]. Режим доступа:https://mmifacedb.eu.
13. The Multimedia Understanding Group (MUG) Facial Expression Database [Электронный ресурс]. Режим доступа: https://mug.ee.auth.gr/fed.
14. The Radboud Faces Database [Электронный ресурс]. Режим доступа: http://www.socsci.ru.nl:8180/RaFD2/RaFD?p=main.
15. The CMU        Multi-PIE                 Face         Database [Электронный        ресурс].   Режим     доступа: http://www.cs.cmu.edu/afs/cs/project/PIE/MultiPie/Multi-Pie/Home.html.
16. Ивановский Л.И., Степанова О.А., Хрящев В.В., Приоров А.Л. Алгоритм распознавания эмоций по изображению лица на основе сверточной нейронной сети // Сб. тез. докл. научно-техн. конф. «Техническое зрение в системах управления-2018». М. ИКИ РАН. 2018. С. 55–56.
17. Степанова О.А., Ивановский Л.И., Хрящев В.В. Использование глубокого обучения и сверточных нейронных сетей для анализа выражения лица // Сб. докл. 20-й междунар. конф. «Цифровая обработка сигналов и ее применение (DSPA-2018)». М. 2018. Т. 2. С. 817–821.
18. Caffe Framework [Электронный ресурс]. Режим доступа: http://caffe.berkleyvision.org.
19. Lebedev A., Pavlov V., Khryashchev V., Stepanova O. Face Detection Algorithm Based on a Cascade of Ensembles of Decision Trees // Proceedings of the FRUCT’18. Saint-Petersburg, Russia. 18–22 April 2016. Р. 161–166.