V.S. Lobanova1, L.N. Anishchenko2, V.V. Slizov3, E.S. Smirnova4

1–4 Bauman Moscow State Technical University (Moscow, Russia)

Age is one of key risk factors of falls. Among people over the age of 60 death rates arising from a fall are the highest. Additionally, in the developed countries there is a common trend of population ageing. Therefore, fall detection in humans is critical for prevention of life threatening conditions.

Nowadays, there are multiple methods of automatic fall detection including ambient device-based, wearable sensor-based and fusion technologies. All these methods have their advantages and drawbacks. Ambient devices are usually dependent on surrounding conditions like lighting variations, noise and reflecting objects. Wearable sensors are prone to false alarms and become useless in case of a person forgets to take on the device. Fusion technologies are the most perspective, however, there is an issue of trade-off between the cost and performance. Therefore, in this paper we have chosen video surveillance as a low-cost sensor that is easy in installation and use.

As a base model for the image processing we used AlexNet that is a convolutional neural network designed for classification task on 1000 categories. However, in this work frames of videos are to be classified into two classes (fall/non-fall). Therefore, the last fully connected layer of the neural network was replaced with another one having an appropriate number of outputs. Then the weights of the model were fine-tuned.

For model training and validation Le2i Fall Detection dataset was chosen. It consists of 191 videos captured in realistic conditions (variations of light intensity, shadows, reflections and textured background) in participation of nine volunteers, who were shot one at a time. To check the robustness of the model we validated it using leave-one-out technique, i.e. a training sample included videos shot in some distinct locations whereas frames from the remaining location were used as a test sample. As far as there were four different locations, four pairs of training and test samples were constructed.

Finally, an empirical rule was applied at the model’s predictions. The heuristic allowed us to improve specificity, i.e. to lower the rate of false alarms, meanwhile sensitivity remained unchanged that is critical in the fall detection task.

In the future work, we suppose our model to be fine-tuned using videos without falls shot in other conditions and validate the model with videos either with falls or without them. This will allow us to check whether such an approach is applicable for the design of a fall detection system based on our model in real life. Additionally, the question of further performance improvement with other empirical rules should be addressed.

Lobanova V.S., Anishchenko L.N., Slizov V.V., Smirnova E.S. Fall detection in humans using video surveillance
and transfer learning. Biomedicine Radioengineering. 2022. V. 25. № 5. Р. 39-48. DOI: https://doi.org/10.18127/j15604136-202205-05 (In Russian)

1. 10 facts on ageing and health. World Health Organization: [site]. 2017. 1 мая. URL: https://www.who.int/news-room/fact-sheets/detail/10-facts-on-ageing-and-health (data obrashcheniya 27.03.2022).
2. Falls. World Health Organization: [site]. 2021. 26 апр. URL:  https://www.who.int/news-room/fact-sheets/detail/falls (data obrashcheniya 01.06.2022).
3. World Population Prospects 2019: Highlights / United Nations, Department of Economic and Social Affairs, Population Division. The United Nations: New York. USA. 2019.
4. Singh A., Rehman S.U., Yongchareon S., Chong P.H.J. Sensor technologies for fall detection systems: a review. IEEE Sensors Journal. 2020. V. 20. P. 6889–6919. DOI: 10.1109/JSEN.2020.2976554
5. Clemente M., Li F., Valero M., Song W. Smart Seismic Sensing for Indoor Fall Detection, Location, and Notification. IEEE Journal of Biomedical and Health Informatics. 2020. V. 24. P. 524–532. DOI: 10.1109/JBHI.2019.2907498
6. Vacher M., Bouakaz S., Bobillier-Chaumon M.-E., Aman F., Khan R.A., Bekkadja S., Portet F., Guillou E., Rossato S., Lecouteux B. The CIRDO Corpus: Comprehensive Audio/Video Database of Domestic Falls of Elderly People [Elektronny resurs]. Proceedings of 10th International Conference on Language Resources and Evaluation (LREC 2016). Slovenia, 23–28 May 2016. P. 1389–1396. URL: https://hal.archives-ouvertes.fr/hal-01323603 (дата обращения: 01.06.2022).
7. Fatima M., Yousaf M.H., Yasin A., Velastin S.A. Unsupervised fall detection approach using human skeletons. Proceedings of 2021 International Conference on Robotics and Automation in Industry (ICRAI). Pakistan, 26–27 October 2021. P. 1–6. DOI: 10.1109/ICRAI54018.2021.9651467
8. Ramirez H., Velastin S.A., Meza I., Fabregas E., Makris D., Farias G. Fall Detection and Activity Recognition Using Human Skeleton Features. IEEE Access. 2021. V. 9. P. 33532–33542. DOI: 10.1109/ACCESS.2021.3061626
9. Palipana S., Rojas D., Agrawal P., Pesch D. Falldefi: ubiquitous fall detection using commodity wi-fi devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2018. V. 1. P. 1–25. DOI: 10.1145/3161183
10. Wang R.-d., Zhang Y.-l., Dong L.-p., Lu J.-w., Zhang Z.-q., He X. Fall detection algorithm for the elderly based on human characteristic matrix and SVM. Proceedings of 2015 15th International Conference on Control, Automation and Systems (ICCAS). Korea (South), 13-16 October 2015. P. 1190–1195. DOI: 10.1109/ICCAS.2015.7364809.
11. Asif U., Mashford B., Cavallar S.V., Yohanandan S., Roy S., Tang J., Harrer S. Privacy Preserving Human Fall Detection using Video Data. Proceedings of the Machine Learning for Health NeurIPS Workshop (PMLR). 11 December 2020. V. 116. P. 39–51.
12. Liu L., Hou Y., He J., Lungu J., Dong R. An Energy-Efficient Fall Detection Method Based on FD-DNN for Elderly People. Sensors. 2020. V. 20. P. 4192. DOI: https://doi.org/10.3390/s20154192
13. Lin C.-L., Chiu W.-C., Chu T.-C., Ho Y.-H., Chen F.-H., Hsu C.-C., Hsieh P.-H., Chen C.-H., Lin C.-C.K., Sung P.-S., Chen P.-T. Innovative Head-Mounted System Based on Inertial Sensors and Magnetometer for Detecting Falling Movements. Sensors. 2020. V. 20. P. 5774. DOI: https://doi.org/10.3390/s20205774
14. Nahian M.J.A., Ghosh T., Banna H.A., Aseeri M.A., Uddin M.N., Ahmed M.R., Mahmud M., Kaiser M.S. Towards an Accelerometer-Based Elderly Fall Detection System Using Cross-Disciplinary Time Series Features. IEEE Access. 2021. V. 9. P. 39413–39431. DOI: 10.1109/ACCESS.2021.3056441
15. Gjoreski M., Gjoreski H., Lustrek M., Gams M. How accurately can your wrist device recognize daily activities and detect falls? Sensors. 2016. V. 16. P. 600. DOI: 10.3390/s16060800
16. Er P.V., Tan K.K. Non-Intrusive Fall Detection Monitoring for the Elderly Based on Fuzzy Logic. Measurement. 2018. V. 124. P. 91–102. DOI: 10.1016/j.measurement.2018.04.009
17. Nho Y.-H., Lim J.G., Kwon D.-S. Cluster-Analysis-Based User-Adaptive Fall Detection Using Fusion of Heart Rate Sensor and Accelerometer in a Wearable Device. IEEE Access. 2020. V. 8. P. 40389–40401. DOI: 10.1109/ACCESS.2020.2969453
18. Halima I., Laferte J.-M., Cormier G., Fougeres A.-L., Dillenseger J.-L. Depth and Thermal Information Fusion for Head Tracking Using Particle Filter in a Fall Detection Context. Integrated Computputer-Aided Engineering. 2020. V. 27. P. 195–208. DOI: 10.3233/ICA-190615
19. Cai W.Y., Guo J.-H., Zhang M.-Y., Ruan Z.-X., Zheng X.-C., Lv S.-S. GBDT-Based Fall Detection with Comprehensive Data from Posture Sensor and Human Skeleton Extraction. Journal of Healthcare Engineering. 2020. V. 2020. DOI: 10.1155/2020/8887340
20. Bozinovski S. Reminder of the first paper on transfer learning in neural networks, 1976. Informatica. 2020. V. 44. P. 17. DOI: 10.31449/inf.v44i3.2828
21. Charfi I., Miteran J., Dubois J., Atri M., Tourki R. Optimised spatiotemporal descriptors for real-time fall detection: comparison of SVM and Adaboost based classification. Journal of Electronic Imaging. 2013. V. 22. P. 041106. DOI: 10.1117/1.JEI.22.4.041106
22. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Communications of the ACM. 2017. V. 60. P. 84–90. DOI: 10.1145/3065386
23. Gunale K., Mukherji P. Indoor human fall detection system based on automatic vision using computer vision and machine learning algorithms. Journal of Engineering Science and Technology. 2018. V. 13. P. 2587–2605
24. Anishchenko L. Machine learning in video surveillance for fall detection. 2018 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT). 2018. P. 99–102. DOI: 10.1109/USBEREIT.2018.8384560
25. Núňez-Marcos A., Azkune G., Arganda-Carreras I. Vision-Based Fall Detection with Convolutional Neural Networks. Wireless Communications and Mobile Computing. 2017. V. 201. P. 1–16. DOI: 10.1155/2017/9474806
26. Chang W.-J., Hsu C.-H., Chen L.-B. A Pose Estimation-Based Fall Detection Methodology Using Artificial Intelligence Edge Computing. IEEE Access. 2021. V. 9. P. 129965–129976. DOI: 10.1109/ACCESS.2021.3113824
27. Sowmyayani S., Murugan V., Kavitha J. Fall Detection in Elderly Care System Based on Group of Pictures. Vietnam Journal of Computer Science. 2021. V. 8. P. 199–214. DOI: 10.1142/S2196888821500081