O.V. Melnik1, V.A. Sablina2, N.V. Yakovlev3

1–3 FSBEI HE “Ryazan State Radio Engineering University named after V.F. Utkin” (Ryazan, Russia)

1 omela111@yandex.ru, 2 sablina.v.a@evm.rsreu.ru, 3 nikgutendorf@gmail.com

At the present time the development of investigations in the field of the human facial micro-expressions analysis is restricted by the complexity of obtaining datasets for experiments. As a simple and easy realizable approach to create the required datasets, it is proposed to use musical compositions as stimuli eliciting different human emotions. However, in this case it is necessary to describe clearly the processes of preparing the participants, selecting the music, recording the videos, and automated spotting and recognition of facial expressions using software tools. Also it is necessary to detect separately the macro-expressions, the micro-expressions, and the subtle expressions. It will be possible to use the created with the help of listening to music datasets in investigations in the directions of the hidden emotion recognition and the human psycho-emotional state diagnostics.

Aim of the work is the development of approaches to the creation of the datasets of the human facial expressions corresponding to the manifestations of the different emotions under the influence of music.

During the study, a small experimental dataset of facial expressions for five participants is created via listening by them to three musical compositions and simultaneous executing video recording using the high-speed camera. The graphs of different emotions probabilities dependencies on the time for the participants of the experiment are constructed and analyzed. In so doing, the macro-expressions are recognized in general as neutral. Additionally, four micro-expressions for each of the two participants of the experiment and also one subtle expression are spotted. The analysis is performed using different software tools. The possibility to create facial macro-expressions, micro-expressions, and subtle expressions datasets using listening to music is shown. The general recommendations for carrying out such experiments are given.

The carried out investigation will make it possible to create facial expressions datasets for the further analysis of the macro-expressions, micro-expressions, and subtle expressions.

Melnik O.V., Sablina V.A., Yakovlev N.V. An investigation of the possibilities of the facial expressions datasets creation using listening to music. Biomedicine Radioengineering. 2025. V. 28. № 4. Р. 62-71. DOI: https://doi.org/10.18127/j15604136-202504-07 (In Russian).

1. Ekman P. Emotion in the Human Face, 2nd Edition. Malor Books. 2013. 456 p.
2. Plutchik R. Emotions and Life: Perspectives from Psychology, Biology, and Evolution, 1st Edition, American Psychological Association. 2003. 592 p.
3. Three Types of Facial Expressions of Emotion [Электронный ресурс]. – URL: https://www.humintell.com/macroexpressions-microexpressions-and-subtle-expressions/ (дата обращения: 02.05.2025).
4. Västfjäll D. Emotion Induction through Music: A Review of the Musical Mood Induction Procedure. Musicae Scientiae Journal. 2001–2002. V. 5. Special Issue. P. 173–211.
5. Ribeiro F.S., Santos F.H., Albuquerque P.B., Oliveira-Silva P. Emotional Induction Through Music: Measuring Cardiac and Electrodermal Responses of Emotional States and Their Persistence. Frontiers in Psychology Journal. 2019. V. 10. № 451. 13 p.
6. FFmpeg. A complete, cross-platform solution to record, convert and stream audio and video [Elektronnyj resurs]. – URL: https://www.ffmpeg.org/ (data obrashcheniya: 02.05.2025).
7. Face++. Emotion Recognition [Elektronnyj resurs]. – URL: https://www.faceplusplus.com/emotion-recognition/ (data obrashcheniya: 02.05.2025).
8. Avidemux. What is Avidemux? [Elektronnyj resurs]. – URL: https://avidemux.sourceforge.net/ (data obrashcheniya: 02.05.2025).
9. Mel'nik O.V., Sablina V.A., Chernenko A.D. Programmnyj kompleks dlya issledovaniya prostranstvenno-vremennyh deskriptorov priznakov. Biomedicinskaya radioelektronika. 2024. T. 27. № 4. S. 48–55. (in Russian).
10. Burresi G., Sablina V.A. Micro-Facial Movement Detection Using LBP-TOP Descriptors for Landmark Based Regions, 10th Mediterranean Conference on Embedded Computing (MECO) Proceedings. Budva. Montenegro. 2021. P. 401–404.
11. Davison A.K., Lansley C., Costen N. et al. SAMM: A Spontaneous Micro-Facial Movement Dataset. IEEE Transactions on Affective Computing. 2018. V. 9. № 1. P. 116–129.