350 rub
Journal Technologies of Living Systems №8 for 2016 г.
Article in number:
Automated assessment of the psycho-emotional state of a speaker: approaches and technologies
Keywords: psycho-emotional state stress tension psychophysical research psychophysiological research
Authors:
A.U. Kalabin - Ph. D. (Politics), Senior Research Scientist, Scientific and Technical Center of Scientific and Industrial Holding «InformInvest Group», Moscow, Russia E-mail: psy-res@yandex.ru
Abstract:
The assessment of the psycho-emotional state of a person is a problem of the scientific theoretical and applied nature, re-levant to a number of areas of research and practice. The aim of the proposed work is to comprehensively examine the current and future approaches and technologies that allow by non-contact way to make an assessment of psycho-emotional state of the speaker through the use of the automated technical solutions. On the basis of the proposed criteria the approaches and technologies of the assessment of the psycho-emotional state are classified in the following way: (1) instrumental methods due to their nature used during contact procedure; (2) instrumental methods allowing to carry out an assessment of the psycho-emotional state remotely; (3) non-instrumental methods that assess the psycho-emotional state through expert analysis of verbal speech characteristics and non-verbal behavior, as well as visually observed psychological and psychophysiological reactions; (4) non-instrumental and instrumental methods of assessing psycho-emotional state of the performer\'s (author\'s) hand written document. Briefly addressed the following current and future methods of automated assessment of the psycho-emotional state of a speaker: voice stress analysis, remote lie detector (remote personnel assessment device); facial expression and expressive movement analysis; thermal imaging registration of the psycho-emotional state; VibraImage analysis; analysis of the nature of eye movements and pupil size dynamics; special mathematical method of data processing through the transition to a color display.
Pages: 14-25
References

 

  1. EHnciklopedicheskijj slovar. Izdateli: F.A. Brokgauz, I.A. Efron. S.‑Peterburg: Tipografija Akc. Obshh. «Izdatelskoe Delo», Brokgauz‑Efron. 1901. Tom XXXII. S. 2.
  2. Rajjt G. Svidetel koldovstva. M.: Radio i svjaz. 1991. Rezhim dostupa: http://royallib.ru/ book/rayt_garri/svidetel_ koldovstva.html (data obrashhenija: 08.08.2014).
  3. Veb-sajjt «Velikie istoricheskie lichnosti». Veb-stranica «Bessmertnyjj Avicenna». Rezhim dostupa: http://history-persons.ru/ 2012/06/ bessmertnyj-avicenna/ (data obrashhenija: 20.05.2013).
  4. Kalabin A.JU. Poligraf Rudolfa Rota (1881 g.) // Biomedicinskaja radioehlektronika. 2016. № 2. S. 67‑74.
  5. Lurija A.R. Priroda chelovecheskikh konfliktov. Obektivnoe izuchenie dezorganizovannogo povedenija cheloveka. M. 2002. 527 s.
  6. KHomskaja E.D. Soprjazhennaja motornaja metodika // Klinicheskaja psikhologija: Slovar / Pod red. N.D. Tvorogova. M.: PER SEH. 2007.
  7. Sreenivasa R.K. et al. Emotion Recognition from Speech // International Journal of Computer Science and Information Technologies. 2012. V. 3 (2). P. 3603-3607.
  8. Vartanov A.V. Antropomorfnyjj metod raspoznavanija ehmocijj v zvuchashhejj rechi // Nacionalnyjj psikhologicheskijj zhurnal. 2013. №2 (10). R. 69-79.
  9. Galunov V.I. O vozmozhnosti opredelenija ehmocionalnogo sostojanija govorjashhego po rechi // Rechevye tekhnologii. 2008. № 1. S. 60-66.
  10. Gorshkov JU.G. Ocenka ehmocionalnogo sostojanija cheloveka na osnove mnogourovnevogo vejjvlet-analiza rechi // Biomedicinskaja radioehlektronika. 2014. № 10. S. 64-70.
  11. Davydov A.G., Kiseljov V.V., Kochetkov D.S. Klassifikacija ehmocio­nalnogo sostojanija diktora po golosu: problemy i reshenija // Po mater. ezhegodnojj Mezhdunar. konf. «Dialog 2011». Bekasovo, 25-29 maja 2011 g.). M.: Izd-vo RGGU. 2011. Vyp. 10(17). S. 178-185.
  12. Lukjanica A.A., SHishkin A.G. Avtomaticheskoe opredelenie izmenenijj ehmocionalnogo sostojanija po rechevomu signalu // Rechevye tekhnologii. 2009. № 3. S. 60-67.
  13. Mikhajjlov V.G., Zlatoustova L.V. Izmerenie parametrov rechi (p. 6.3. Vlijanie ehmocijj na rechevye parametry) / Pod red. M.A. Sapozhkova. M.: Radio i svjaz. 1987. 168 s.
  14. Pastukhov E.S. Opredelenie psikhoehmocionalnogo sostojanija po golosu i osobennostjam leksiki. [EHlektronnaja publikacija.] Rezhim dostupa: http://www.elitarium.ru/opredelenie-jemocionalnogo-sostojanija-chelovek-golos-osobennosti-leksiki/ (data obrashhenija: 01.07.2016).
  15. Rozaliev V.L., Zaboleeva-Zotova A.V. Raspoznavanie ehmocionalnogo sostojanija cheloveka po ego rechi // Nauchnaja sessija MIFI-2007. Sb. nauch. trudov. M. 2007. T. 2. S. 149-150.
  16. KHejjdorov I.EH., JAn Czinbin, U SHi, Soroka A.M., Trus A.A. Klassifikacija ehmocionalno okrashennojj rechi s ispolzovaniem metoda opornykh vektorov // Rechevye tekhnologii. 2008. № 3. S. 63-71.
  17. KHitrov M.V., Davydov A.G., Kiseljov V.V., Romashkin JU.N., Tkachenja A.V. Avtomaticheskoe raspoznavanie ehmocijj po rechi s ispolzovaniem metoda opornykh vektorov i kriterija dzhina // Rechevye tekhnologii. 2012. № 4. S. 34-43.
  18. KHromatidi A.F. Issledovanie psikhofiziologicheskogo sostojanija cheloveka na osnove ehmocionalnykh priznakov rechi: Dis. ... kand. tekhn. nauk. Taganrog, 2005. 154 s.
  19. CHanina M.A. Razrabotka BTS raspozna- vanija izmenjonnogo psikhoehmocionalnogo sostojanija diktora po rechevomu signalu // EHlektronnyjj zhurnal «Molodjozhnyjj nauchno-tekhnicheskijj vestnik». 2012, ijul. № 07. FGBOU VPO «MGTU im. N.EH. Baumana». EHl. № FS77‑51038/467655.
  20. Patent № 2510955 (RF). Sposob obnaruzhenija ehmocijj po golosu / I.A. Saitov, O.O. Basov, V.A. JAgupov.
  21. Potapova R.K., Potapov V.V., Lebedeva N.N., Agibalova T.V. Mezhdisciplinarnost v issledovanii rechevojj poliinformativnosti / Pod red. R.K. Potapovojj. M.: JAzyki slavjanskojj kultury. 2015. 352 s.
  22. Veb-sajjt Amerikanskojj associacii poligrafa. Veb-stranica «Pozicija Amerikanskojj associacii poligrafa v otnoshenii analizatorov stressa po golosu». Rezhim dostupa: http://­www.polygraph.org/­section/­press-room/­position-voice-stress (data obrashhenija: 18.10.2011).
  23. Pelenicyn A.B., Stepanov A.A. CHto nado znat o tak nazyvaemykh «analizatorakh stressa v golose». Rezhim dostupa: http://www.poligraf.­sp.ru/znan.html (data obrashhenija: 11.12.2011).
  24. Veb-sajjt Amerikanskojj associacii poligrafa. Veb-stranica «Issledovanija stressa po golosu». Rezhim dostupa: http://www.polygraph.org/voice-stress-studies (data obrashhenija: 01.07.2016).
  25. Veb-sajjt «Centr Innovacionnojj Psikhofiziologii Alekseeva L.G.». Veb-stranica «Poligraf "Delta-Optima"». Rezhim dostupa: http://pelengator‑info.ru/ (data dostupa: 11.08.2015).
  26. Geisheimer J., Greneker E.F. Remote detection of deception using radar vital signs monitor technology // Proceedings IEEE 34th Annual 2000 International Carnahan Conference on Security Technology, 23‑25 Oct. Security Technology, 2000. Rezhim dostupa: http://ieeexplore.ieee. org/articleDetails.jsp-arnumber=891183 (data obrashhenija: 21.02.2014).
  27. Greneker G. Radar Technology For Acquiring Biological Signals // The Journal of Credibility Assessment and Witness Psychology. 2006. V. 7. № 2. R. 127‑134. Rezhim dostupa: http://truth. charleshontsphd.com/JCAAWP/2006_No_2/2006_127-134.pdf (data obrashhenija: 21.02.2014).
  28. Patent № 2216269 (RF). Sposob kontrolja psikhofizicheskojj reakcii cheloveka i ustrojjstvo dlja ego osushhestvlenija / V.I. Gubajjdullin, EH.V. Zimin.
  29. Kashapova L.KH., KHrulev A.A., Spiridonov I.N. Ispolzovanie videoizobrazhenijj dlja avtomatizirovannojj ocenki mimicheskikh izmenenijj / Mediko-tekhnicheskie tekhnologii na strazhe zdorovja. Sb. dokladov X RNTK, g. Monastir (Tunis), 28 sentjabrja - 5 oktjabrja 2008 g. M.: NII RL MGTU im. N.EH. Baumana, 2008. S. 76-78.
  30. Kashapova L.KH., Krylov JU.V., Spiridonov I.N. Razrabotka algoritma ocenki ehmocionalnogo sostojanija po videoizobrazheniju lica / IX MNTK «Fizika i radioehlektronika v medicine i ehkologii FREHMEH-2010». Materialy NMSH po biometricheskim tekhnologijam dlja mediko-biologicheskikh analizov i funkcionalnojj diagnostiki. Vladimir. 2010. S. 83‑88.
  31. Kashapova L.KH., Latysheva E.JU., Spiridonov I.N. Algoritm raspoznavanija ehmocionalnogo sostojanija po izobrazhenijam lica s ispolzovaniem diskriminantnogo analiza i filtrov Gabora // Medicinskaja tekhnika. 2012. № 3(273). S. 1‑4. [EHlektronnaja publikacija.] Rezhim dostupa: http://www.mtjournal.ru/ upload/iblock/8f6/8f676ce617053447ed8105f4ec4c1b01.pdf (data obrashhenija: 01.07.2016).
  32. Bojjko A.A. Podkhody k avtomatizi­rovannomu opredeleniju sklonnosti cheloveka k protivopravnym dejjstvijam po mimicheskim projavlenijam // Fizika i radioehlektronika v medicine i ehkologii: Doklady XI MNTK s ehlementami nauchnojj molodezhnojj shkoly. Vladimir. 1-3 ijulja 2014 g. Vladimir. 2014. Kn. 2. S. 36-39.
  33. Sharma N. et al. Thermal spatio-temporal data for stress recognition // EURASIP Journal on Image and Video Processing. 2014. V. 2014. № 1. P. 1-12.
  34. Das S. Emotion Detection from Facial Expression and Its Use in the Evaluation of Stress :Diss. - doctor of engineering. Nagaoka: Nagaoka University of Technology. 2013. 175 p.
  35. Latysheva E.JU., Kuraeva L.KH., Spiridonov I.N. Metodika kontrolja psikhofiziologicheskogo sostojanija obuchaemogo // Inzhenernyjj zhurnal: nauka i innovacii. 2015. № 8(34). S. 8.
  36. Bulgakov V.G. Metodicheskie osnovy kriminalisticheskojj identifikacii i diagnostiki cheloveka po ego diagnosticheskim priznakam. M.: JUrlitinform. 2014.
  37. Valstar M. Automatic Behaviour Understanding in Medicine // Proceedings of the 2014 Workshop on Roadmapping the Future of Multimodal Interaction Research including Business Opportunities and Challenges. ICMI\'14 International conference on multimodal interaction. Istanbul, Turkey - November 12‑16. 2014. P. 57‑60.
  38. Alvino C., Kohler C., Barrett F., Gur R.E., Gur R.C., Verma R. Computerized measurement of facial expression of emotions in schizophrenia // Journal of Neuroscience Methods. 2007.  № 163 (2). P.350‑361. (DOI: 10.1016/j.jneumeth. 2007.03.002)/
  39. Ekman P., Friesen W.V. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978.
  40. Ekman P., Friesen W.V., Hager J.C. Facial Action Coding System: The Manual on CD ROM. A Human Face. Salt Lake City. 2002.
  41. Cohn J.F., Ambadar Z., Ekman P. Observer-Based Measurement of Facial Expression With the Facial Action Coding System [EHlektronnaja publikacija.] Rezhim dostupa: http://www.pitt.edu/~jeffcohn/ biblio/Coan%20013%20chap13.pdf (data obrashhenija: 01.07.2016).
  42. Patent № 2415645 (RF). Sposob diagnostiki lozhnosti soobshhaemojj informacii po dinamike parametrov neverbalnogo povedenija cheloveka / N.M. Romanova, M.A. Samokhina, V.V. Semjonov, L.N. Ivanov.
  43. Romanova N.M., Samokhina M.A. Izmenenie parametrov neverbalnogo povedenija pri soobshhenii istinnojj i lozhnojj informacii // Vestnik Tambovskogo universiteta. Ser. Gumanitarnye nauki. 2007. Vyp. 12 (56). S. 140-145.
  44. Romanova N.M., Samokhina M.A. Neverbalnoe povedenie cheloveka pri soobshhenii istinnojj i lozhnojj informacii: parametry, tipologija, model diagnostiki // ZHurnal prikladnojj psikhologii. 2007. № 45. S. 81-90.
  45. Pavlidis I., Eberhardt N.L., Levine J.A. Seeing through the face of deception // Nature. 2002. Jan., V. 2. P. 35‑36. Erratum in: Nature. 2002. Feb., 7. P. 602
  46. Diakides N.A., Bronzino J.D. Medical Infrared Imaging. CRC Press. 2007.
  47. Knight M. Thermal imaging turns up heat on liars. CNN Labs, 14.09.2011. Rezhim dostupa: http://edition.cnn.com/2011/TECH/innovation/09/13/lie.detector.thermal.imaging (data dostupa: 17.02.2014).
  48. Sumriddetchkajorn S., Somboonkaew A. Thermal analyzer enables improved lie detection in criminal-suspect interrogations. SPIE Newsroom. 15.02.2011. DOI: 10.1117/2.1201101.003452 Rezhim dostupa: http://spie.org/x44341.xml (data obrashhenija: 21.02.2014).
  49. Latif M.H., Md. Yusof H., Sidek S.N., Rusli N., Sado Fatai. Emotion detection from thermal facial imprint based on GLCM features // ARPN Journal of Engineering and Applied Sciences. 2016, January. V. 11. № 1. P. 345-350.
  50. Corneanu C.A., Oliu M., Cohn J.F., Escalera S. Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016. V. 38. Is. 8. P. 1548-1568. (DOI 10.1109/TPAMI.2016. 2515606).
  51. Materialy Centra infrakrasnykh informacionnykh tekhnologijj FGUP «Alfa». M.: CIIT FGUP «Alfa». 2011.
  52. Minkin V.A. Vibroizobrazhenie. SPb.: Renome. 2007.
  53. Patent № 2289310 (RF). Cposob poluchenija informacii o psikhofiziologicheskom sostojanii zhivogo obekta / V.A. Minkin, A.I. SHtam.
  54. Sozdanie sistemy distancionnogo beskontaktnogo skanirovanija i identifikacii psikhofiziologicheskogo sostojanija cheloveka: Otchjot o nauchno-issledovatelskojj rabote. SHifr: lot № 2005‑BT‑13.2/003. Gosudarstvennyjj kontrakt ot 23.06.2005 № 02.435.11.6002. SPb.: OOO «Mnogoprofilnoe predprijatie «EHdsis», 2006.
  55. Behndler R., Grinder D. Iz ljagushek - v princy. Nejjro-lingvisticheskoe programmirovanie / Pod red. S. Andreasa. Novosibirsk: Izd-vo Novosib. un-ta. 1992. 275 s
  56. Patent № 2337607 (RF). Sposob ocenki psikhofizicheskogo sostojanija cheloveka / D.A. Usanov, N.M. Romanova, A.V. Skripal, A.P. Rytik, A.JU. Vagarin, M.A. Samokhina.
  57. Romanova N.M. i dr. Osobennosti glazodvigatelnykh reakcijj chelove­ka pri proiznesenii istinnojj i lozhnojj informacii // Izvestija Saratovskogo universiteta. Ser. Filosofija. Psikhologija. Pedagogika. 2008. T. 8. Vyp. 1. S. 65-73.
  58. Simonov P.V. «JA» i mozg // Nauka v Rossii. 1992. № 5-6. S. 79-81.
  59. Lüscher M. The Lüscher Color Test / transl. and ed. by Ian A. Scott. N.Y.: Pocket Books. 1971.
  60. Sobchik L.N. Metod cvetovykh vyborov - modifikacija vosmicvetovogo testa Ljushera: Prakt. rukovodstvo. SPb.: Rech. 2007. 128 s.
  61. Izmajjlov CH.A. Cvetovaja kharakteristika ehmocijj // Vestnik Mosk. Un-ta. Ser. 14. Psikhologija. 1993. № 4. S. 27-35.
  62. Gusev A.N., Matveeva L.V., Lobkaev JU.JA. O novom metode ocenki aktivacii cheloveka // Materialy mezhdunar. nauchno-tekhnich. konf. «Sostojanie i problemy izmerenijj». M.: MGTU im. N.EH. Baumana. 2000. S. 96-99.
  63. Izmajjlov CH.A., Lobkaev JU.JA., Matveeva L.V., Izmajjlova N.CH., Koshmarov A.JU. Sootnoshenie zvukov not s bazovymi ehmocijami // Psikhologicheskijj zhurnal. 2006. T. 27. № 4. S. 59‑66.
  64. Lobkaev JU.JA. Analiz fundamen­talnykh uravnenijj psikhofiziki kak funkcijj modemnogo processa obobshhjonnogo biologicheskogo i informacionnogo kanala // CHelovek i magnitnye polja. Sb. materialov dokladov II Mezhdunar. konf. Sarov: RFJAC-VNIIEHF. 2008. S. 165-181.
  65. Lobkaev JU.JA. Analiz processa obrabotki informacii biologicheskojj sistemojj v svete novejjshikh dostizhenijj psikhofiziologii // Vestnik Nizhegorodskogo universiteta im. N.I. Lobachevskogo. 2011. Vyp. 2(2). S. 277-282.
  66. Matveeva L.V., Mochalova Yu.V., Anikeeva T.Ya., Petrakova E.E., Lobkayev Yu.Ya. Identification of Basic Emotional States in Informational Communication // Procedia - social and behavioral sciences. 2013. October. V. 86. P. 584-590.