Automatic Elderly Fall and Unstable Movement Detection System Using Framewise and LSTM Based Video Analytics on an Embedded Device

Authors

  • Aniqua Nusrat Zereen AI Center, Asian Institute of Technology
  • Anubinda Gurung AI Center, Asian Institute of Technology
  • Amir Rajak AI Center, Asian Institute of Technology
  • Jednipat Moonrinta AI Center, Asian Institute of Technology
  • Matthew Nelson Dailey AI Center, Asian Institute of Technology
  • Mongkol Ekpanyapong AI Center, Asian Institute of Technology
  • Roongtiwa Vachalathiti Faculty of Physical Therapy, Mahidol University
  • Sunee Bovonsunthonchai Faculty of Physical Therapy, Mahidol University

Keywords:

Automatic elder care, Activity profiling, Fall warning, Unstable motion

Abstract

We introduce an edge processing device and cloud computation framework enabling activity profiling, unstable motion alerts, and fall alerts for elderly people living at home under their families’ care. The system analyzes video frames captured by fixed cameras, tracking each visible person and classifying their actions into one of nine ordinary activities, a fall, or unstable movement. Alert notifications are sent to caregivers if a fall or unstable movement is detected. The system comprises an embedded device (NVIDIA TX2) and cloud-based storage and analysis infrastructure. The main modules include video-based human detection, tracking, and recognition; fall detection; activity classification; and detection of painful, unstable, and confused motion likely to lead to falls. The system is designed for accuracy, usability, and cost. Individual module tests and a field test with a volunteer household indicate that the prototype system is ready for the next stage of commercial exploitation, with an accuracy of 91.6% for ordinary actions and falls as well as a recall of 97.02% for unstable motion.

References

Al-Aama, T. (2011). Falls in the elderly: spectrum and prevention. Canadian Family Physician, 57(7), 771-776.

Anderson, D., Keller, J. M., Skubic, M., Chen, X., & He, Z. (2006). Recognizing falls from silhouettes. In 2006 International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 6388-6391). IEEE.

Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1), 172-186.

Chung, J., Ozkaynak, M., & Demiris, G. (2017). Examining daily activity routines of older adults using workflow. Journal of Biomedical Informatics, 71, 82-90.

Chung, P. C., & Liu, C. D. (2008). A daily behavior enabled hidden Markov model for human behavior understanding. Pattern Recognition, 41(5), 1572-1580.

Ghorbani, S., Mahdaviani, K., Thaler, A., Kording, K., Cook, D. J., Blohm, G., & Troje, N. F. (2020). Movi: A large multipurpose motion and video dataset. arXiv preprint. https://arxiv.org/abs/2003.01888

Karim, N. T., Jain, S., Moonrinta, J., Dailey, M. N., & Ekpanyapong, M. (2018). Customer and target individual face analysis for retail analytics. In 2018 International Workshop on Advanced Image Technology (IWAIT) (pp. 1-4). IEEE.

Li, C., Zhong, Q., Xie, D., & Pu, S. (2017). Skeleton-based action recognition with convolutional neural networks. In 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (pp. 597-600). IEEE.

Lin, T. H., Yang, C. Y., & Shih, W. P. (2017). Fall prevention shoes using camera-based line-laser obstacle detection system. Journal of Healthcare Engineering. Article 8264071.

Mubashir, M., Shao, L., & Seed, L. (2013). A survey on fall detection: Principles and approaches. Neurocomputing, 100, 144-152.

Pumpinyo, S., & Koocharoenprasit, S. (2020). A Survey of Leisure Activities that the Elderly Desire. Journal of Advanced Research in Social Sciences, 3(3), 14-19.

Stone, E. E., & Skubic, M. (2011). Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops (pp. 71-77). IEEE.

Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012). Mining actionlet ensemble for action recognition with depth cameras. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1290-1297). IEEE.

Wojke, N., Bewley, A., & Paulus, D. (2017). Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Conference on Image Processing (ICIP) (pp. 3645-3649). IEEE.

Xiao, B., Wu, H., & Wei, Y. (2018). Simple baselines for human pose estimation and tracking. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 466-481).

Yen, H. Y., & Lin, L. J. (2018). Quality of life in older adults: Benefits from the productive engagement in physical activity. Journal of Exercise Science & Fitness, 16(2), 49-54.

Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499-1503.

Zhang, S., Liu, X., & Xiao, J. (2017). On geometric features for skeleton-based action recognition using multilayer lstm networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 148-157). IEEE.

Zhou, Z., Chen, X., Chung, Y. C., He, Z., Han, T. X., & Keller, J. M. (2008). Activity analysis, summarization, and visualization for indoor human activity monitoring. IEEE Transactions on Circuits and Systems for Video Technology, 18(11), 1489-1498.

Zhou, Z., Dai, W., Eggert, J., Giger, J. T., Keller, J., Rantz, M., & He, Z. (2009). A real-time system for in-home activity monitoring of elders. In 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 6115-6118). IEEE.

Downloads

Published

25-11-2021

How to Cite

Zereen, A. N., Gurung, A., Rajak, A., Moonrinta, J., Dailey, M. N., Ekpanyapong, M., Vachalathiti, R., & Bovonsunthonchai, S. (2021). Automatic Elderly Fall and Unstable Movement Detection System Using Framewise and LSTM Based Video Analytics on an Embedded Device. Journal of Digital Communications, 5(5), 117–134. Retrieved from https://so04.tci-thaijo.org/index.php/NBTC_Journal/article/view/253616

Issue

Section

Academic article