Automatic Elderly Fall and Unstable Movement Detection System Using Framewise and LSTM Based Video Analytics on an Embedded Device
Keywords:
Automatic elder care, Activity profiling, Fall warning, Unstable motionAbstract
We introduce an edge processing device and cloud computation framework enabling activity profiling, unstable motion alerts, and fall alerts for elderly people living at home under their families’ care. The system analyzes video frames captured by fixed cameras, tracking each visible person and classifying their actions into one of nine ordinary activities, a fall, or unstable movement. Alert notifications are sent to caregivers if a fall or unstable movement is detected. The system comprises an embedded device (NVIDIA TX2) and cloud-based storage and analysis infrastructure. The main modules include video-based human detection, tracking, and recognition; fall detection; activity classification; and detection of painful, unstable, and confused motion likely to lead to falls. The system is designed for accuracy, usability, and cost. Individual module tests and a field test with a volunteer household indicate that the prototype system is ready for the next stage of commercial exploitation, with an accuracy of 91.6% for ordinary actions and falls as well as a recall of 97.02% for unstable motion.
References
Al-Aama, T. (2011). Falls in the elderly: spectrum and prevention. Canadian Family Physician, 57(7), 771-776.
Anderson, D., Keller, J. M., Skubic, M., Chen, X., & He, Z. (2006). Recognizing falls from silhouettes. In 2006 International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 6388-6391). IEEE.
Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1), 172-186.
Chung, J., Ozkaynak, M., & Demiris, G. (2017). Examining daily activity routines of older adults using workflow. Journal of Biomedical Informatics, 71, 82-90.
Chung, P. C., & Liu, C. D. (2008). A daily behavior enabled hidden Markov model for human behavior understanding. Pattern Recognition, 41(5), 1572-1580.
Ghorbani, S., Mahdaviani, K., Thaler, A., Kording, K., Cook, D. J., Blohm, G., & Troje, N. F. (2020). Movi: A large multipurpose motion and video dataset. arXiv preprint. https://arxiv.org/abs/2003.01888
Karim, N. T., Jain, S., Moonrinta, J., Dailey, M. N., & Ekpanyapong, M. (2018). Customer and target individual face analysis for retail analytics. In 2018 International Workshop on Advanced Image Technology (IWAIT) (pp. 1-4). IEEE.
Li, C., Zhong, Q., Xie, D., & Pu, S. (2017). Skeleton-based action recognition with convolutional neural networks. In 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (pp. 597-600). IEEE.
Lin, T. H., Yang, C. Y., & Shih, W. P. (2017). Fall prevention shoes using camera-based line-laser obstacle detection system. Journal of Healthcare Engineering. Article 8264071.
Mubashir, M., Shao, L., & Seed, L. (2013). A survey on fall detection: Principles and approaches. Neurocomputing, 100, 144-152.
Pumpinyo, S., & Koocharoenprasit, S. (2020). A Survey of Leisure Activities that the Elderly Desire. Journal of Advanced Research in Social Sciences, 3(3), 14-19.
Stone, E. E., & Skubic, M. (2011). Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops (pp. 71-77). IEEE.
Wang, J., Liu, Z., Wu, Y., & Yuan, J. (2012). Mining actionlet ensemble for action recognition with depth cameras. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1290-1297). IEEE.
Wojke, N., Bewley, A., & Paulus, D. (2017). Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Conference on Image Processing (ICIP) (pp. 3645-3649). IEEE.
Xiao, B., Wu, H., & Wei, Y. (2018). Simple baselines for human pose estimation and tracking. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 466-481).
Yen, H. Y., & Lin, L. J. (2018). Quality of life in older adults: Benefits from the productive engagement in physical activity. Journal of Exercise Science & Fitness, 16(2), 49-54.
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499-1503.
Zhang, S., Liu, X., & Xiao, J. (2017). On geometric features for skeleton-based action recognition using multilayer lstm networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 148-157). IEEE.
Zhou, Z., Chen, X., Chung, Y. C., He, Z., Han, T. X., & Keller, J. M. (2008). Activity analysis, summarization, and visualization for indoor human activity monitoring. IEEE Transactions on Circuits and Systems for Video Technology, 18(11), 1489-1498.
Zhou, Z., Dai, W., Eggert, J., Giger, J. T., Keller, J., Rantz, M., & He, Z. (2009). A real-time system for in-home activity monitoring of elders. In 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 6115-6118). IEEE.
Downloads
Published
How to Cite
Issue
Section
License
The Office of the NBTC holds the copyright of articles appearing in the journal. The Office of the NBTC allows the public or individuals to distribute, copy, or republish the work under a Creative Commons license (CC), with attribution (BY), No Derivatives (ND) and NonCommercial (NC); unless written permission is received from the Office of the NBTC.
Text, tables, and figures that appear in articles accepted for publication in this journal are personal opinion and responsibility of the author, and not binding on the NBTC and the Office of the NBTC. In case of errors, each author is solely responsible for their own article, and not concerning the NBTC and the NBTC Office in any way.
