O.V. Melnik1, M.B. Nikiforov2, V.A. Sablina3, A.D. Chernenko4

1–4 Ryazan State Radio Engineering University named after V.F. Utkin (Ryazan, Russia)
1 omela111@yandex.ru, 2 nikiforov.m.b@evm.rsreu.ru, 3 flyingvictory@mail.ru, 4 anuta201294@yandex.ru

The study of facial microexpressions that are not subject to conscious control is of significant interest due to the emerging opportunities for objective identification of a person’s hidden emotions. One of the stages in the procedure for automatically assessing emotions from microexpressions based on video sequence processing is finding spatiotemporal descriptors of facial micromovement features in the image.

The goal of the work is to study ways to improve the efficiency of microfacial motion recognition through the use of spatiotemporal descriptors based on optical flow as part of a software pipeline for facial microexpression analysis.

The problem of detecting microfacial movements using spatiotemporal descriptors based on optical flow is considered. Reviews of datasets used for microfacial motion analysis and spatiotemporal feature descriptors are presented. The experimental methodology for recognizing microfacial movements is described. Based on the experimental results obtained, a comparative analysis of the spatio-temporal feature descriptors FHOOF and FHOFO is presented.

The results obtained can be used to improve the efficiency of recognition of microfacial movements in systems for objective assessment of a person’s emotional state.

Melnik O.V., Nikiforov M.B., Sablina V.A., Chernenko A.D. Microfacial motion detection using optical flow-based spatiotemporal descriptors. Biomedical radioelectronics. 2024. T. 27. № 2. P. 60–68. DOI: https://doi.org/10.18127/j15604136-202402-08 (in Russian)

1. Chernenko A.D., Ashapkina M.S., Sablina V.A., Alpatov A.V. Physical Activity Set Selection for Emotional State Harmonization Based on Facial Micro-Expression Analysis. Proceedings of the 32nd International Conference on Computer Graphics and Vision “GraphiCon”. 2022. Р. 682–691.
2. Melnik O.V., Sablina V.A., Chernenko A.D. Raspoznavaniye mikrovyrazheniy litsa s ispol'zovaniyem klassifikatorov na osnove metodov mashinnogo obucheniya. Modeli, sistemy, seti v ekonomike, tekhnike, prirode i obshchestve. 2023. № 1. S. 125–135.
3. Davison A.K., Lansley C., Costen N., Tan K., Yap M.H. SAMM: A Spontaneous Micro-Facial Movement Dataset. IEEE Transactions on Affective Computing. 2018. V. 9. No. 1. Р. 116–129.
4. Hast A., Sablina V.A., Sintorn I.M., Kylberg G. A Fast Fourier based Feature Descriptor and a Cascade Nearest Neighbour Search with an Efficient Matching Pipeline for Mosaicing of Microscopy Images. International Journal «Pattern Recognition and Image Analysis». Pleiades Publishing. 2018. V. 28. No. 2. Р. 261–272.
5. Ekman P., Friesen W.V., Hager J.C. Facial Action Coding System Investigator’s Guide. Research Nexus. 2002. 197 p.
6. Chaudhry R., Ravichandran A., Hager G., Vidal R. Histograms of Oriented Optical Flow and Binet-Cauchy Kernels on Nonlinear Dynamical Systems for the Recognition of Human Actions. IEEE Conference on Computer Vision and Pattern Recognition. Miami. FL. 2009. Р. 1932–1939.
7. Happy S.L., Routray A. Fuzzy Histogram of Optical Flow Orientations for Micro-Expression Recognition. IEEE Transactions on Affective Computing. 2017. V. 10. No. 3. Р. 394–406.
8. Burresi G., Sablina V.A. Micro-Facial Movement Detection Using LBP-TOP Descriptors for Landmark Based Regions. 10th Mediterranean Conference on Embedded Computing (MECO) Proceedings. Budva, Montenegro. 2021. Р. 401–404.