B.S. Goryachkin1, A.R. Yakubov2, F.A. Anikin3

1–3 Bauman Moscow State Technical University (Moscow, Russia)

1 bsgor@mail.ru, 2 yakubovart337@gmail.com, 3 filippanikin@mail.ru

Problem statement. Nowadays, with the development of technology, there are more and more ways of human-machine interaction, including non-mechanical ones. One of them is the “machine-eye” system, when a person receives information from a source through the organ of vision with its subsequent analysis. In the same way, an “eye-machine” system can be implemented when the data source is the human visual organ itself, and the analysis is performed directly by the machine itself. This article discusses the creation of a data collection system for the user's view of the computer screen to create a heat map.

Target. To develop a system for collecting data about a person's gaze using a webcam and a tool for working with the obtained data to create a heat map in order to visualize the user's areas of interest.

Result. Three scripts were developed in Python version 3.10 with the connection of the libraries "MediaPipe", "OpenCV" and "Matplotlib". The first script calibration.py calibrates the entire system for correct calculations of the user's "vision coordinates"; the second script coords.py collects information about the position of the pupils during operation; the third heatmap script.py creates a file of a visual representation of a person's gaze for the entire period of working with the tool.

Practical significance. Using the developed system allows you to identify areas of user interest for subsequent analysis of the collected data, which can further help in designing the structure of the user interface.

Goryachkin B.S., Yakubov A.R., Anikin F.A. Development of eye tracking tools. Neurocomputers. 2024. V. 26. № 2. Р. 5-12. DOI: https://doi.org/10.18127/j19998554-202402-01 (In Russian)

1. Khryashchev V.V., Priorov A.L., Matveev D.V., Lukashevich Yu.A. Using deep machine learning techniques to detect and track athletes in the video data stream. Neurocomputers. 2023. V. 25. № 6. Р. 37–46. DOI 10.18127/j19998554-202306-04. (In Russian)
2. The concept of a heat map. [Electronic resource] – Access mode: https://cloud.yandex.ru/ru/docs/datalens/visualization-ref/heat-map-chart, date of reference 22.11.2023. (In Russian)
3. The official Python website. [Electronic resource] – Access mode: https://www.python.org/, date of reference 23.11.2023. (In Russian)
4. Documentation for the mediapipe module. [Electronic resource] – Access mode: https://developers.google.com/mediapipe/, date of reference 24.11.2023. (In Russian)
5. Documentation for the OpenCV module. [Electronic resource] – Access mode: https://docs.opencv.org/4.x/index.html, date of reference 25.11.2023. (In Russian)
6. Documentation for the "matplotlib" module. [Electronic resource] – Access mode: https://matplotlib.org/, date of reference 25.11.2023. (In Russian)