Abstract
The widespread Internet of Things (IoT) technologies in day life indoor environments result in an enormous amount of daily generated data, which require reliable data analysis techniques to enable efficient exploitation of this data. The recent developments in deep learning (DL) have facilitated the processing and learning from the massive IoT data and learn essential features swiftly and professionally for a variety of IoT applications on smart indoor environments. This study surveys the recent literature on exploiting DL for different indoor IoT applications. We aim to give insights into how the DL approaches can be employed from various viewpoints to develop improved Indoor IoT applications in two distinct domains: indoor positioning/tracking and activity recognition. A primary target is to effortlessly amalgamate the two disciplines of IoT and DL, resultant in a broad range of innovative strategies in indoor IoT applications, such as health monitoring, smart home control, robotics, etc. Further, we have derived a thematic taxonomy from the comparative analysis of technical studies of the three beforementioned domains. Eventually, we proposed and discussed a set of matters, challenges, and some new directions in incorporating DL to improve the efficiency of indoor IoT applications, encouraging and stimulating additional advances in this auspicious research area.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abuhamad, M., Abusnaina, A., Nyang, D., & Mohaisen, D. (2021). Sensor-based continuous authentication of smartphones’ users using behavioral biometrics: A contemporary survey. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3020076
Alam, F., Faulkner, N., & Parr, B. (2021). Device-free localization: A review of non-RF techniques for unobtrusive indoor positioning. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3030174
Alazrai, R., Awad, A., Alsaify, B., Hababeh, M., & Daoud, M. I. (2020). A dataset for Wi-Fi-based human-to-human interaction recognition. Data in Brief. https://doi.org/10.1016/j.dib.2020.105668
Alemdar, H., Ertan, H., Incel, O. D., & Ersoy, C. (2013). ARAS human activity datasets in multiple homes with multiple residents. https://doi.org/10.4108/icst.pervasivehealth.2013.252120.
Anguita, D., Ghio, A., Oneto, L., Parra, X., & Reyes-Ortiz, J. L. (2013). A public domain dataset for human activity recognition using smartphones.
Bai, J., Lian, S., Liu, Z., Wang, K., & Liu, D. (2017). Smart guiding glasses for visually impaired people in indoor environment. IEEE Transactions on Consumer Electronics. https://doi.org/10.1109/TCE.2017.014980
Banos, O., et al. (2015). Design, implementation and validation of a novel open framework for agile development of mobile health applications. Biomedical Engineering Online. https://doi.org/10.1186/1475-925X-14-S2-S6
Barshan, B., & Yüksek, M. C. (2013). Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. The Computer Journal. https://doi.org/10.1093/comjnl/bxt075
Barut, O., Zhou, L., & Luo, Y. (2020). Multi-task LSTM model for human activity recognition and intensity estimation using wearable sensor data. IEEE Internet of Things Journal, 7(9), 8760–8768. https://doi.org/10.1109/JIOT.2020.2996578
Berthelot, D., Carlini, N., Goodfellow, I., Oliver, A., Papernot, N., & Raffel, C. (2019). MixMatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 32.
Bianchi, V., Bassoli, M., Lombardo, G., Fornacciari, P., Mordonini, M., & De Munari, I. (2019). IoT wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2920283
Blunsden, S., & Fisher, R. B. (2010). The BEHAVE video dataset: Ground truthed video for multi-person behavior classification. Annals of the BMVA, 4, 1–12.
Brinke, J. K., & Meratnia, N. (2019). Dataset: Channel state information for different activities, participants and days. https://doi.org/10.1145/3359427.3361913.
Carreira, J., Noland, E., Hillier, C., & Zisserman, A. (2019). A short note on the kinetics-700 human action dataset. July 2019, [Online]. http://arxiv.org/abs/1907.06987.
CASAS Smart Home Project. (2021). http://casas.wsu.edu/datasets/. Accessed March 25, 2021.
Chavarriaga, R., et al. (2013). The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2012.12.014
Chen, C., Jafari, R., & Kehtarnavaz, N. (2015). UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. https://doi.org/10.1109/ICIP.2015.7350781.
Chen, D., Yongchareon, S., Lai, E. M. K., Yu, J., & Sheng, Q. Z. (2021). Hybrid fuzzy C-means CPD-based segmentation for improving sensor-based multi-resident activity recognition. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3051574
Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., & Nie, F. (2020). A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2019.2927224
Chen, M., et al. (2020). MoLoc: Unsupervised fingerprint roaming for device-free indoor localization in a mobile ship environment. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3004240
Chen, Z., Zhang, L., Jiang, C., Cao, Z., & Cui, W. (2019). WiFi CSI Based passive human activity recognition using attention based BLSTM. IEEE Transactions on Mobile Computing. https://doi.org/10.1109/TMC.2018.2878233
Cheplygina, V., de Bruijne, M., & Pluim, J. P. W. (2019). Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis. https://doi.org/10.1016/j.media.2019.03.009
Chhikara, P., Tekchandani, R., Kumar, N., Chamola, V., & Guizani, M. (2021). DCNN-GA: A deep neural net architecture for navigation of UAV in indoor environment. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3027095
Choi, W., Shahid, K., & Savarese, S. (2009). What are they doing? Collective activity classification using spatio-temporal relationship among people. https://doi.org/10.1109/ICCVW.2009.5457461.
Dang, L. M., Min, K., Wang, H., Piran, M. J., Lee, C. H., & Moon, H. (2020). Sensor-based and vision-based human activity recognition: A comprehensive survey. Pattern Recognition. https://doi.org/10.1016/j.patcog.2020.107561
Deep, S., Zheng, X., Karmakar, C., Yu, D., Hamey, L. G. C., & Jin, J. (2020). A survey on anomalous behavior detection for elderly care using dense-sensing networks. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2019.2948204
Dhall, A., Goecke, R., & Gedeon, T. (2015). Automatic group happiness intensity analysis. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2015.2397456
Dhiman, C., & Vishwakarma, D. K. (2020). View-invariant deep architecture for human action recognition using two-stream motion and shape temporal dynamics. IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2020.2965299
Feng, C., Arshad, S., Zhou, S., Cao, D., & Liu, Y. (2019). Wi-Multi: A three-phase system for multiple human activity recognition with commercial WiFi devices. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2915989
Gao, N. et al. (2020).Generative adversarial networks for spatio-temporal data: A survey. arXiv. 2020.
Gochoo, M., Tan, T. H., Liu, S. H., Jean, F. R., Alnajjar, F. S., & Huang, S. C. (2019). Unobtrusive activity recognition of elderly people living alone using anonymous binary sensors and DCNN. EEE Journal of Biomedical and Health Informatics. https://doi.org/10.1109/JBHI.2018.2833618
Gordon, D., Wirz, M., Roggen, D., Tröster, G., & Beigl, M. (2014). Group affiliation detection using model divergence for wearable devices. https://doi.org/10.1145/2634317.2634319.
Goyal, R. et al. (2017). The ‘something something’ video database for learning and evaluating visual common sense. https://doi.org/10.1109/ICCV.2017.622.
Gu, C., et al. (2018). AVA: A video dataset of spatio-temporally localized atomic visual actions. https://doi.org/10.1109/CVPR.2018.00633.
Gu, F., Khoshelham, K., Yu, C., & Shang, J. (2019). Accurate step length estimation for pedestrian dead reckoning localization using stacked autoencoders. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/TIM.2018.2871808
Guo, L., et al. (2019). Wiar: A public dataset for wifi-based activity recognition. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2947024
Guo, T., Xu, C., He, S., Shi, B., Xu, C., & Tao, D. (2020b). Robust student network learning. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2019.2929114
Guo, X., Ansari, N., Hu, F., Shao, Y., Elikplim, N. R., & Li, L. (2020a). A survey on fusion-based indoor positioning. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2019.2951036
Guo, Z., Xiao, F., Sheng, B., Fei, H., & Yu, S. (2020c). WiReader: Adaptive air handwriting recognition based on commercial WiFi signal. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2997053
Ha, I., Kim, H., Park, S., & Kim, H. (2018). Image retrieval using BIM and features from pretrained VGG network for indoor localization. Building and Environment. https://doi.org/10.1016/j.buildenv.2018.05.026
Haseeb, M. A. A., & Parasuraman, R. (2017). Wisture: RNN-based learning of wireless signals for gesture recognition in unmodified smartphones. arXiv. 2017.
Hayashi, T., Nishida, M., Kitaoka, N., & Takeda, K. (2015). Daily activity recognition based on DNN using environmental sound and acceleration signals. https://doi.org/10.1109/EUSIPCO.2015.7362796.
He, J., & So, H. C. (2020). A hybrid TDOA-fingerprinting-based localization system for LTE network. IEEE Sensors Journal. https://doi.org/10.1109/JSEN.2020.3004179
He, Y., Chen, Y., Hu, Y., & Zeng, B. (2020). WiFi vision: Sensing, recognition, and detection with commodity MIMO-OFDM WiFi. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2989426
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015). ActivityNet: A large-scale video benchmark for human activity understanding. https://doi.org/10.1109/CVPR.2015.7298698.
Hillyard, P., et al. (2018). Experience: Cross-technology radio respiratory monitoring performance study. https://doi.org/10.1145/3241539.3241560.
Huang, J., Lin, S., Wang, N., Dai, G., Xie, Y., & Zhou, J. (2020). TSE-CNN: A two-stage end-to-end CNN for human activity recognition. IEEE Journal of Biomedical and Health Informatics. https://doi.org/10.1109/JBHI.2019.2909688
Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018). Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. https://doi.org/10.1145/3272127.3275108.
Hussain, T., et al. (2020). Multi-view summarization and activity recognition meet edge computing in IoT environments. IEEE Internet of Things Journal. https://doi.org/10.1109/jiot.2020.3027483
Hussain, Z., Sheng, Q. Z., & Zhang, W. E. (2020). A review and categorization of techniques on device-free human activity recognition. Journal of Network and Computer Applications. https://doi.org/10.1016/j.jnca.2020.102738
Huynh-The, T., Hua, C. H., Tu, N. A., & Kim, D. S. (2021). Physical activity recognition with statistical-deep fusion model using multiple sensory data for smart health. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3013272
Jekabsons, G., & Zuravlyovs, V. (2010). Refining Wi-Fi based indoor positioning. In Aict2010—Application of Information and Communication Technologies Proceedings of 4Th International Science Conference, 2010.
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Li, F. F. (2014). Large-scale video classification with convolutional neural networks. https://doi.org/10.1109/CVPR.2014.223.
Khan, P., Reddy, B. S. K., Pandey, A., Kumar, S., & Youssef, M. (2020). Differential channel-state-information-based human activity recognition in IoT networks. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2997237
Khan, A., Wang, S., & Zhu, Z. (2019). Angle-of-arrival estimation using an adaptive machine learning framework. IEEE Communications Letters. https://doi.org/10.1109/LCOMM.2018.2884464
Kim, E. (2020). Interpretable and accurate convolutional neural networks for human activity recognition. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2020.2972628
Kim, M., Han, D., & Rhee, J. K. (2021). Multiview variational deep learning with application to practical indoor localization. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3063512
Ko, W. R., Jang, M., Lee, J., & Kim, J. (2021). AIR-Act2Act: Human–human interaction dataset for teaching non-verbal social behaviors to robots. The International Journal of Robotics Research. https://doi.org/10.1177/0278364921990671
Koppula, H. S., Gupta, R., & Saxena, A. (2013). Learning human activities and object affordances from RGB-D videos. The International Journal of Robotics Research. https://doi.org/10.1177/0278364913478446
Kuehne, H., Jhuang, H., Stiefelhagen, R., & Serre Thomas, T. (2013). Hmdb51: A large video database for human motion recognition. In High performance computing in science and engineering’ 12: Transactions of the high performance computing center, Stuttgart (HLRS) 2012.
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011). Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter. https://doi.org/10.1145/1964897.1964918
Lee, N., Ahn, S., & Han, D. (2018). AMID: Accurate magnetic indoor localization using deep learning. Sensors (switzerland). https://doi.org/10.3390/s18051598
Leutheuser, H., Doelfel, S., Schuldhaus, D., Reinfelder, S., & Eskofier, B. M. (2014). Performance comparison of two step segmentation algorithms using different step activities. https://doi.org/10.1109/BSN.2014.37.
Leutheuser, H., Schuldhaus, D., & Eskofier, B. M. (2013). Hierarchical, multi-sensor based classification of daily life activities: Comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE. https://doi.org/10.1371/journal.pone.0075196
Li, J., Xie, X., Pan, Q., Cao, Y., Zhao, Z., & Shi, G. (2020c). SGM-net: Skeleton-guided multimodal network for action recognition. Pattern Recognition. https://doi.org/10.1016/j.patcog.2020.107356
Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., & Tian, Q. (2021b). Symbiotic graph neural networks for 3D skeleton-based human action recognition and motion prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2021.3053765
Li, Q., Gravina, R., Li, Y., Alsamhi, S. H., Sun, F., & Fortino, G. (2020a). Multi-user activity recognition: Challenges and opportunities. Information Fusion. https://doi.org/10.1016/j.inffus.2020.06.004
Li, X., Wang, Y., Zhang, B., & Ma, J. (2020d). PSDRNN: An efficient and effective HAR scheme based on feature extraction and deep learning. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2020.2968920
Li, X., Yu, L., Chen, H., Fu, C. W., Xing, L., & Heng, P. A. (2021a). Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2020.2995319
Li, Y., Hu, X., Zhuang, Y., Gao, Z., Zhang, P., & El-Sheimy, N. (2020b). Deep reinforcement learning (DRL): Another perspective for unsupervised wireless localization. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2957778
Liu, J., Liu, H., Chen, Y., Wang, Y., & Wang, C. (2020a). Wireless sensing for human activity: A survey. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2019.2934489
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L. Y., & Kot, A. C. (2020b). NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2019.2916873
Lohan, E. S., Torres-Sospedra, J., Leppäkoski, H., Richter, P., Peng, Z., & Huerta, J. (2017). Wi-Fi crowdsourced fingerprinting dataset for indoor positioning. Data. https://doi.org/10.3390/data2040032
Lu, N., Wu, Y., Feng, L., & Song, J. (2019). Deep learning for fall detection: Three-dimensional CNN combined with LSTM on video kinematic data. IEEE Journal of Biomedical and Health Informatics. https://doi.org/10.1109/JBHI.2018.2808281
Luo, F., Poslad, S., & Bodanese, E. (2020). Temporal convolutional networks for multiperson activity recognition using a 2-D LIDAR. IEEE Internet of Things Journal, 7(8), 7432–7442. https://doi.org/10.1109/JIOT.2020.2984544
Ma, Y., Zhou, G., Wang, S., Zhao, H., & Jung, W. (2018). SignFi: Sign language recognition using WiFi. In Proceedings of ACM interactive, mobile, wearable ubiquitous technol, 2018. https://doi.org/10.1145/3191755.
Marszałek, M., Laptev, I., & Schmid, C. (2009). Actions in context. https://doi.org/10.1109/CVPRW.2009.5206557.
Meng, F., Liu, H., Liang, Y., Tu, J., & Liu, M. (2019). Sample fusion network: An end-to-end data augmentation network for skeleton-based human action recognition. IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2019.2913544
Meng, Z., et al. (2020). Gait recognition for co-existing multiple people using millimeter wave sensing (Vol. 34, No. 01, pp. 849–856). https://ojs.aaai.org/index.php/AAAI/article/view/5430.
Micucci, D., Mobilio, M., & Napoletano, P. (2017). UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Applied Sciences. https://doi.org/10.3390/app7101101
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018). Deep learning for IoT big data and streaming analytics: A survey. IEEE Communications Surveys and Tutorials. https://doi.org/10.1109/COMST.2018.2844341
Monfort, M., et al. (2020). Moments in time dataset: One million videos for event understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2019.2901464
Montoliu, R., Sansano, E., Torres-Sospedra, J., & Belmonte, O. (2017). IndoorLoc platform: A public repository for comparing and evaluating indoor positioning systems. https://doi.org/10.1109/IPIN.2017.8115940.
Müller, M., Röder, T., Clausen, M., Eberhardt, B., Krüger, B., & Weber, A. (2007). Documentation mocap database hdm05, 2007.
Nirmal, I., Khamis, A., Hassan, M., Hu, W., & Zhu, X. (2021). Deep learning for radio-based human sensing: Recent advances and future directions. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2021.3058333
Nweke, H. F., Teh, Y. W., Mujtaba, G., & Al-garadi, M. A. (2019). Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions. Information Fusion. https://doi.org/10.1016/j.inffus.2018.06.002
Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013). Berkeley MHAD: A comprehensive multimodal human action database. https://doi.org/10.1109/WACV.2013.6474999.
Oguntala, G., Hu, Y. F., Alabdullah, A. A. S., Abd-Alhameed, R., Ali, M., & Luong, D. (2021). Passive RFID module with LSTM recurrent neural network activity classification algorithm for ambient assisted living. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3051247
Palipana, S., Rojas, D., Agrawal, P., & Pesch, D. (2018). FallDeFi: Ubiquitous fall detection using commodity Wi-Fi devices. In Proceedings of ACM interactive, mobile, wearable ubiquitous technol, 2018. https://doi.org/10.1145/3161183.
Pei, L., et al. (2020). MARS: Mixed virtual and real wearable sensors for human activity recognition with multi-domain deep learning model. arXiv. 2020. https://doi.org/10.1109/jiot.2021.3055859.
Qi, W., Su, H., & Aliverti, A. (2020). A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Transactions on Human-Machine. https://doi.org/10.1109/THMS.2020.2984181
Qian, K., Wu, C., Yang, Z., Liu, Y., & Jamieson, K. (2017). Widar: Decimeter-level passive tracking via velocity monitoring with commodity Wi-Fi. https://doi.org/10.1145/3084041.3084067.
Qian, K., Wu, C., Zhang, Y., Zhang, G., Yang, Z., & Liu, Y. (2018). Widar2.0: Passive human tracking with a single Wi-Fi link. https://doi.org/10.1145/3210240.3210314.
Qin, Z., Zhang, Y., Meng, S., Qin, Z., & Choo, K. K. R. (2020). Imaging and fusing time series for wearable sensor-based human activity recognition. Information Fusion. https://doi.org/10.1016/j.inffus.2019.06.014
Rashid, N., Dautta, M., Tseng, P., & Al Faruque, M. A. (2021). HEAR: Fog-enabled energy-aware online human eating activity recognition. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3008842.
Reiss, A., & Stricker, D. (2012). Introducing a new benchmarked dataset for activity monitoring. https://doi.org/10.1109/ISWC.2012.13.
Rossi, S., Capasso, R., Acampora, G., & Staffa, M. (2018). A multimodal deep learning network for group activity recognition. https://doi.org/10.1109/IJCNN.2018.8489309.
Ryoo, M. S., & Aggarwal, J. K. (2009). Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. https://doi.org/10.1109/ICCV.2009.5459361.
Shahroudy, A., Liu, J., Ng, T. T., & Wang, G. (2016). NTU RGB+D: A large scale dataset for 3D human activity analysis. https://doi.org/10.1109/CVPR.2016.115.
Sheng, B., Fang, Y., Xiao, F., & Sun, L. (2020a). An accurate device-free action recognition system using two-stream network. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2020.2993901
Sheng, B., Xiao, F., Sha, L., & Sun, L. (2020b). Deep spatial-temporal model based cross-scene action recognition using commodity WiFi. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2973272
Shu, X., Tang, J., Qi, G. J., Liu, W., & Yang, J. (2021a). Hierarchical long short-term concurrent memory for human interaction recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2019.2942030
Shu, X., Zhang, L., Sun, Y., & Tang, J. (2021b). Host-parasite: graph LSTM-in-LSTM for group activity recognition. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2020.2978942
Sigurdsson, G. A., Gupta, A., Schmid, C., Farhadi, A., & Alahari, K. (2018). Actor and observer: Joint modeling of first and third-person videos. https://doi.org/10.1109/CVPR.2018.00772.
Sikder, N., & Nahid, A.-A. (2021). KU-HAR: An open dataset for heterogeneous human activity recognition. Pattern Recognition Letters, 146, 46–54. https://doi.org/10.1016/j.patrec.2021.02.024
Singh, A. D., Sandha, S. S., Garcia, L., & Srivastava, M. (2019). Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar.https://doi.org/10.1145/3349624.3356768.
Sobron, I., Del Ser, J., Eizmendi, I., & Velez, M. (2018). Device-free people counting in IoT environments: New insights, results, and open challenges. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2018.2806990
Sohn, I. (2021). Deep belief network based intrusion detection techniques: A survey. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2020.114170
Sohn, K. et al. (2020). FixMatch: Simplifying semi-supervised learning with consistency and confidence. arXiv. 2020.
Soomro, K., Zamir, A. R., & Shah, M. (2012). UCF101: A dataset of 101 human actions classes from videos in the wild. December 2012, [Online]. http://arxiv.org/abs/1212.0402.
Stisen, A., et al. (2015). Smart devices are different: Assessing and mitigating mobile sensing heterogeneities for activity recognition. https://doi.org/10.1145/2809695.2809718.
Sztyler, T.,& Stuckenschmidt, H. (2016)“On-body localization of wearable devices: An investigation of position-aware activity recognition. https://doi.org/10.1109/PERCOM.2016.7456521.
Tang, J., Shu, X., Yan, R., & Zhang, L. (2019a). Coherence constrained graph LSTM for group activity recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2019.2928540
Tang, Y., Lu, J., Wang, Z., Yang, M., & Zhou, J. (2019b). Learning semantics-preserving attention and contextual interaction for group activity recognition. IEEE Transactions on Image Processing. https://doi.org/10.1109/tip.2019.2914577
Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in Neural Information Processing Systems, 30.
Thariq Ahmed, H. F., Ahmad, H., & Cv, A. (2020). Device free human gesture recognition using Wi-Fi CSI: A survey. Engineering Applications of Artificial Intelligence. https://doi.org/10.1016/j.engappai.2019.103281
Torres-Sospedra, J., et al. (2014). UJIIndoorLoc: A new multi-building and multi-floor database for WLAN fingerprint-based indoor localization problems. https://doi.org/10.1109/IPIN.2014.7275492.
Torres-Sospedra, J., et al. (2017). The smartphone-based offline indoor location competition at IPIN 2016: Analysis and future work. Sensors (Switzerland), 10, 100. https://doi.org/10.3390/s17030557
Torres-Sospedra, J., Rambla, D., Montoliu, R., Belmonte, O., Huerta, J. (2015). UJIIndoorLoc-Mag: A new database for magnetic field-based localization problems. https://doi.org/10.1109/IPIN.2015.7346763.
Uddin, M. Z., Hassan, M. M., Alsanad, A., & Savaglio, C. (2020). A body sensor data fusion and deep recurrent neural network-based behavior recognition approach for robust healthcare. Inf. Fusion. https://doi.org/10.1016/j.inffus.2019.08.004
Ugulino, W., Cardador, D., Vega, K., Velloso, E., Milidiú, R., & Fuks, H. (2012). Wearable computing: Accelerometers’ data classification of body postures and movements. https://doi.org/10.1007/978-3-642-34459-6_6.
Virmani, A. & Shahzad, M. (2017). Position and orientation agnostic gesture recognition using WiFi. https://doi.org/10.1145/3081333.3081340.
Wang, F., Feng, J., Zhao, Y., Zhang, X., Zhang, S., & Han, J. (2019a). Joint activity recognition and indoor localization with WiFi fingerprints. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2923743
Wang, F., Gong, W., & Liu, J. (2019c). On spatial diversity in wifi-based human activity recognition: A deep learning-based approach. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2018.2871445
Wang, F., Han, J., Zhang, S., He, X., & Huang, D. (2018)“CSI-Net: Unified human body characterization and pose recognition. arXiv. 2018.
Wang, F., Liu, J., & Gong, W. (2020e). Multi-adversarial in-car activity recognition using RFIDs. IEEE Transactions on Mobile Computing. https://doi.org/10.1109/tmc.2020.2977902
Wang, Q., et al. (2021). Pedestrian dead reckoning based on walking pattern recognition and online magnetic fingerprint trajectory calibration. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3016146
Wang, R. et al. (2014). Studentlife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. https://doi.org/10.1145/2632048.2632054.
Wang, R., Luo, H., Wang, Q., Li, Z., Zhao, F., & Huang, J. (2020d). A spatial-temporal positioning algorithm using residual network and LSTM. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/TIM.2020.2998645
Wang, W., Bai, P., Zhou, Y., Liang, X., & Wang, Y. (2019b). Optimal configuration analysis of AOA localization and optimal heading angles generation method for UAV swarms. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2918299
Wang, X., Wang, X., & Mao, S. (2020b). Deep convolutional neural networks for indoor localization with CSI images. IEEE Transactions on Network Science and Engineering. https://doi.org/10.1109/TNSE.2018.2871165
Wang, X., Wang, X., & Mao, S. (2021b). Indoor fingerprinting with bimodal CSI tensors: A deep residual sharing learning approach. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3026608
Wang, X., Wang, X., Mao, S., Zhang, J., Periaswamy, S. C. G., & Patton, J. (2020c). Indoor radio map construction and localization with deep Gaussian processes. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2996564
Wang, X., Yu, Z., & Mao, S. (2020a). Indoor localization using smartphone magnetic and light sensors: A deep LSTM approach. Mobile Networks and Applications. https://doi.org/10.1007/s11036-019-01302-x
Wang, Z., She, Q., & Ward, T. (2021a). Generative adversarial networks in computer vision: A survey and taxonomy. ACM Computing Surveys. https://doi.org/10.1145/3439723
Weinzaepfel, P., Martin, X., & Schmid, C. (2016) Human action localization with sparse spatial supervision. May 2016, [Online]. http://arxiv.org/abs/1605.05197.
Weiss, G. M., & Lockhart, J. W. (2012). The impact of personalization on smartphone-based activity recognition.
Xiao, C., Han, D., Ma, Y., & Qin, Z. (2019). CsiGAN: Robust channel state information-based activity recognition With GANs. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2936580
Xiao, C., Lei, Y., Ma, Y., Zhou, F., & Qin, Z. (2020). DeepSeg: Deep learning-based activity segmentation framework for activity recognition using WiFi. IEEE Internet of Things Journal. https://doi.org/10.1109/jiot.2020.3033173
Xue, Y., Su, W., Wang, H., Yang, D., & Jiang, Y. (2019). DeepTAL: Deep learning for TDOA-based asynchronous localization security with measurement error and missing data. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2937975
Yan, R., Xie, L., Tang, J., Shu, X., & Tian, Q. (2020). HiGCIN: Hierarchical graph-based cross inference network for group activity recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2020.3034233
Ye, Y., Ye, Y., Qiu, D., Wu, X., Strbac, G., & Ward, J. (2020). Model-free real-time autonomous control for a residential multi-energy system using deep reinforcement learning. IEEE Transactions on Smart Grid. https://doi.org/10.1109/TSG.2020.2976771
Yeung, S., Russakovsky, O., Jin, N., Andriluka, M., Mori, G., & Fei-Fei, L. (2018). Every moment counts: Dense detailed labeling of actions in complex videos. International Journal of Computer Vision. https://doi.org/10.1007/s11263-017-1013-y
Yin, C., Zhang, S., Wang, J., & Xiong, N. N. (2020). Anomaly detection based on convolutional recurrent autoencoder for IoT time series. IEEE Transactions on Systems, Man, and Cybernetics: Systems. https://doi.org/10.1109/tsmc.2020.2968516
Yousefi, S., Narui, H., Dayal, S., Ermon, S., & Valaee, S. (2017). A survey on behavior recognition using WiFi channel state information. IEEE Communications Magazine. https://doi.org/10.1109/MCOM.2017.1700082
Yun, K., Honorio, J., Chattopadhyay, D., Berg, T. L., & Samaras, D. (2012). Two-person interaction detection using body-pose features and multiple instance learning. https://doi.org/10.1109/CVPRW.2012.6239234.
Zhang, C., Tan, K. C., Li, H., & Hong, G. S. (2019). A cost-sensitive deep belief network for imbalanced classification. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2018.2832648
Zhang, H., Hu, Z., Qin, W., Xu, M., & Wang, M. (2021b). Adversarial co-distillation learning for image recognition. Pattern Recognition. https://doi.org/10.1016/j.patcog.2020.107659
Zhang, H., Xiao, Z., Wang, J., Li, F., & Szczerbicki, E. (2020). A novel IoT-perceptive human activity recognition (HAR) approach using multihead convolutional attention. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2949715
Zhang, J., et al. (2021). Data augmentation and dense-LSTM for human activity recognition using WiFi signal. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3026732
Zhang, J., Tang, Z., Li, M., Fang, D., Nurmi, P., Wang, Z. (2018). CrossSense: Towards cross-site and large-scale WiFi sensing. https://doi.org/10.1145/3241539.3241570.
Zhang, J., & Tao, D. (2020). Empowering things with intelligence: A survey of the progress, challenges, and opportunities in artificial intelligence of things. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3039359
Zhang, L., et al. (2020). WiFi-based indoor robot positioning using deep fuzzy forests. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2986685
Zhang, R., Jing, X., Wu, S., Jiang, C., Mu, J., & Yu, F. R. (2021). Device-free wireless sensing for human detection: The deep learning perspective. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.3024234
Zhao, H., Torralba, A., Torresani, L., & Yan, Z. (2019). HACS: Human action clips and segments dataset for recognition and temporal localization. https://doi.org/10.1109/ICCV.2019.00876.
Zhao, Y., Xu, J., Wu, J., Hao, J., & Qian, H. (2020). Enhancing camera-based multimodal indoor localization with device-free movement measurement using WiFi. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2948605
Zheng, L., Hu, B. J., Qiu, J., & Cui, M. (2020). A deep-learning-based self-calibration time-reversal fingerprinting localization approach on Wi-Fi platform. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2981723
Zheng, Y., et al. (2019). Zero-effort cross-domain gesture recognition with Wi-Fi. https://doi.org/10.1145/3307334.3326081.
Zheng, Y., Sheng, M., Liu, J., & Li, J. (2018). Exploiting AoA estimation accuracy for indoor localization: A weighted AoA-based approach. IEEE Wireless Communications Letters. https://doi.org/10.1109/LWC.2018.2853745
Zhou, X., Liang, W., Wang, K. I. K., Wang, H., Yang, L. T., & Jin, Q. (2020). Deep-learning-enhanced human activity recognition for internet of healthcare things. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2020.2985082
Zhu, Q., Chen, Z., & Soh, Y. C. (2019). A novel semisupervised deep learning method for human activity recognition. IEEE Transactions on Industrial Informatics, 15(7), 3821–3830. https://doi.org/10.1109/TII.2018.2889315
Zhu, X., Qu, W., Qiu, T., Zhao, L., Atiquzzaman, M., & Wu, D. O. (2020). Indoor intelligent fingerprint-based localization: Principles, approaches and challenges. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/COMST.2020.3014304
Zhu, Y., Luo, H., Zhao, F., & Chen, R. (2021). Indoor/outdoor switching detection using multisensor densenet and LSTM. IEEE Internet of Things Journal, 8(3), 1544–1556. https://doi.org/10.1109/JIOT.2020.3013853
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Abdel-Basset, M., Chang, V., Hawash, H. et al. Deep learning approaches for human-centered IoT applications in smart indoor environments: a contemporary survey. Ann Oper Res 339, 3–51 (2024). https://doi.org/10.1007/s10479-021-04164-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-021-04164-3