Vladislav I. Yankovskiy  − Master-student, 

Computer Science and Control Systems Department, 

Bauman Moscow State Technical University (Moscow, Russia)

E-mail: vlyankov@mail.ru

Nikita D. Todosiev − Master-student, 

Computer Science and Control Systems Department, 

Bauman Moscow State Technical University (Moscow, Russia)

E-mail: todosievnik@gmail.com

Alexander S. Shchukin − Master-student, 

Computer Science and Control Systems Department, 

Bauman Moscow State Technical University (Moscow, Russia)

E-mail: a.shch.2@yandex.com

Yuriy E. Gapanyuk - Ph.D. (Computer Sciences), Associate Professor,

Computer Science and Control Systems Department, 

Bauman Moscow State Technical University (Moscow, Russia)

E-mail: gapyu@bmstu.ru

With the development of machine learning methods, it became possible to abandon sensors as the primary source of information about human movement, replacing them with a video camera. Using this approach completely solves the problem of restricting human movement, and also reduces financial costs.

For motion recognition, we will use a HIIS-based approach. The frames of the video stream, which are analyzed by the subconsciousness module, act as an environment. The pre-trained neural network model PoseNet is used as a subconsciousness module.

Movement is the process of changing posture over time. The movement is represented as a sequence of point coordinates. The speed of movement is determined by differentiating the coordinates. Based on the analysis of experimental data, 11 evenly spaced poses for one squat were selected. The absolute coordinates of the characteristic points take into account the position of a person in the frame relative to the corners of the image. Since the coordinate changes are analyzed for several unique points, the output of the model is 510 indicators for the classification of one squat.

The boundary model of consciousness and subconsciousness represents the coordinates of the postures by which the exercise is determined. Pose coordinates are PoseNet output and are further processed.

The model of a decision tree with additional processing of the results acts as a module of consciousness. When training the decision tree, rules are formed that perform the task of classifying movements based on the coordinates of the poses.

During the development of the system, attempts were made to replace the decision tree with a second neural network based on the LSTM architecture. Experimental results indicate that the decision tree model shows better quality and, at the same time, performs predictions in significantly less time compared to the LSTM network.

Thus, the HIIS-based approach makes it possible to implement a system for physical exercise motion recognition in the form of separate independent machine learning models. This makes it possible to replace models independently of each other, provided that the interface between them is preserved in the form of a boundary model of consciousness and subconsciousness.

1. O'Reilly M.A., Whelan D.F., Ward T.E., Delahunt E., Caulfield B. Technology in Strength and Conditioning: Assessing Bodyweight Squat Technique with Wearable Sensors. Journal of Strength and Conditioning Research. 2017. V. 31. Р. 2303–2312.
2. Chernen'kij V.M., Terehov V.I., Gapanjuk Ju.E. Struktura gibridnoj intellektual'noj informacionnoj sistemy na osnove metagrafov. Nejrokomp'jutery: razrabotka, primenenie. 2016. № 9. S. 3-14 (In Russian).
3. Chernen'kij V.M., Gapanjuk Ju.E., Revunkov G.I., Terehov V.I., Kaganov Ju.T. Metagrafovyj podhod dlja opisanija gibridnyh intellektual'nyh informacionnyh sistem. Prikladnaja informatika. 2017. T. 12. № 3(69). S. 57–79 (In Russian).
4. Shavit Y., Ferens R. Introduction to Camera Pose Estimation with Deep Learning. ArXiv, 2019. URL: https://arxiv.org/abs/1907.05272.
5. Kritz M., Cronin J., Hume P.A. The Bodyweight Squat: A Movement Screen for the Squat Pattern. Strength and Conditioning Journal. 2009. V. 31. Р. 76-85.
6. Rokach L., Maimon O. Data Mining with Decision Trees – Theory and Applications. 2nd Edition. Series in Machine Perception and Artificial Intelligence. 2014.
7. Stikharnyi A., Orekhov A., Andreev A., Gapanyuk Y. The Hybrid Intelligent Information System for Music Classification. In: Kryzhanovsky B., Dunin-Barkowski W., Redko V., Tiumentsev Y. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research III. NEUROINFORMATICS 2019. Studies in Computational Intelligence. V. 856. Р. 71-77. Springer, Cham.
8. Wu Z., Pan S., Chen F., Long G., Zhang C., Yu P.S. A Comprehensive Survey on Graph Neural Networks. IEEE Transactions onNeural Networks and Learning Systems. March 2020. Р. 1-21. doi: 10.1109/TNNLS.2020.2978386.
9. Sherstinsky A. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network. Physica D: Nonlinear Phenomena. 2020. V. 404, doi: 10.1016/j.physd.2019.132306.
10. Geron A. Hands-on machine learning with Scikit-Learn and Tensor Flow: Concepts, Tools, and Techniques to Build Intelligent Systems. 2nd Edition. O'Reilly Media. 2019. 856 p.