Non-contact multimodal indoor human monitoring systems: A survey

Abstract

Abstract

Indoor human monitoring systems are integral in various applications. They leverage a wide range of sensors, including cameras, radio devices, and inertial measurement units, to collect extensive data from users and the environment. These sensors contribute distinct data modalities, encompassing video feeds from cameras, received signal strength indicators and channel state information from WiFi devices, and three-axis acceleration data from accelerometers. In this context, we present a comprehensive survey of multimodal approaches applied to indoor human monitoring systems, with a specific focus on their relevance in elderly care. Our survey primarily highlights non-contact technologies, particularly cameras and radio devices, as key components in the development of indoor human monitoring systems. Throughout this article, we explore well-established techniques for extracting features from multimodal data sources. Our exploration extends to methodologies for fusing these features and harnessing multiple modalities to improve the accuracy and robustness of machine learning models. Furthermore, we conduct comparative analysis across different data modalities in diverse human monitoring tasks and undertake a comprehensive examination of existing multimodal datasets. This extensive survey not only highlights the significance of indoor human monitoring systems but also emphasizes their versatile applications. In particular, we emphasize their critical role in enhancing the quality of elderly care, offering valuable insights into the development of non-contact monitoring solutions tailored to the needs of aging populations.