More Web Proxy on the site http://driver.im/

research-article

ATFA: : Adversarial Time–Frequency Attention network for sensor-based multimodal human activity recognition

Authors:

Hao XuAuthors Info & Claims

Volume 236, Issue C

https://doi.org/10.1016/j.eswa.2023.121296

Published: 01 February 2024 Publication History

Abstract

Human activity recognition (HAR) is a key component of mobile and ubiquitous computing. With the increasing number of sensors embedded in devices such as smartphones and smartwatches, models are enhanced to recognize more complex human behaviors. However, this also leads to increased data heterogeneity, caused by diverse user backgrounds, health conditions, and sensing environments, which makes it difficult to align the multimodal data distributions across individuals. This paper proposes a novel unsupervised domain adaptation framework via Adversarial Time-Frequency Attention (ATFA) to efficiently adapt models to new users. In particular, the proposed attention-based modality fusion module captures and fuses important modalities based on their context to reduce redundant information. Additionally, the network explores frequency domain features to improve performance in recognizing human activities. Extensive experiments are conducted on three publicly available HAR datasets to demonstrate the superiority of our proposed method compared to state-of-the-art baselines.

References

[1]

Bai L., Yao L., Wang X., Kanhere S.S., Guo B., Yu Z., Adversarial multi-view networks for activity recognition, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4 (2020) 1–22.

[2]

Buffelli D., Vandin F., Attention-based deep learning framework for human activity recognition with user adaptation, IEEE Sensors Journal 21 (2020) 13474–13483.

[3]

Chavarriaga R., Sagha H., Calatroni A., Digumarti S.T., Tröster G., del R. Millán J., Roggen D., The opportunity challenge: A benchmark database for on-body sensor-based activity recognition, Pattern Recognition Letters 34 (2013) 2033–2042.

[4]

Chen L., Zhang Y., Miao S., Zhu S., Hu R., Peng L., Lv M., SALIENCE: An unsupervised user adaptation model for multiple wearable sensors based human activity recognition, IEEE Transactions on Mobile Computing (2022).

[5]

Chen K., Zhang D., Yao L., Guo B., Yu Z., Liu Y., Deep learning for sensor-based human activity recognition, ACM Computing Surveys 54 (2020) 1–40.

[6]

Chen K., Zhang D., Yao L., Guo B., Yu Z., Liu Y., Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities, ACM Computing Surveys 54 (4) (2021) 1–40.

[7]

Eldele E., Ragab M., Chen Z., Wu M., Kwoh C.-K., Li X., CoTMix: Contrastive domain adaptation for time-series via temporal mixup, 2022, arXiv preprint arXiv:2212.01555.

[8]

Eldele E., Ragab M., Chen Z., Wu M., Kwoh C., Li X., Guan C., Time-series representation learning via temporal and contextual contrasting, 2021, arXiv, abs/2106.14112.

[9]

Ganin Y., Ustinova E., Ajakan H., Germain P., Larochelle H., Laviolette F., Marchand M., Lempitsky V.S., Domain-adversarial training of neural networks, Journal of Machine Learning Research (2015).

[10]

Guan Y., Plötz T., Ensembles of deep LSTM learners for activity recognition using wearables, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (2017) 1–28.

[11]

Hammerla N.Y., Halloran S., Plötz T., Deep, convolutional, and recurrent models for human activity recognition using wearables, 2016, arXiv, abs/1604.08880.

[12]

He Q.-Q., Siu S.W.I., Si Y.-W., Attentive recurrent adversarial domain adaptation with Top-k pseudo-labeling for time series classification, Applied Intelligence (2022) 1–20.

[13]

Li, C., Niu, D., Jiang, B., Zuo, X., & Yang, J. (2021). Meta-HAR: Federated Representation Learning for Human Activity Recognition. In Proceedings of the web conference 2021.

[14]

Li C., Tong C.L., Niu D., Jiang B., Zuo X., Cheng L., Xiong J., Yang J., Similarity embedding networks for robust human activity recognition, ACM Transactions on Knowledge Discovery from Data 15 (6) (2021) 1–17.

[15]

Liu, Q., & Xue, H. (2021). Adversarial Spectral Kernel Matching for Unsupervised Time Series Domain Adaptation. In International joint conference on artificial intelligence.

[16]

Ma, H., Li, W., Zhang, X., Gao, S., & Lu, S. (2019). AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition. In International joint conference on artificial intelligence.

[17]

Ordonez F.J., Roggen D., Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition, Sensors 16 (2016).

[18]

Ozyurt Y., Feuerriegel S., Zhang C., Contrastive learning for unsupervised domain adaptation of time series, 2022, arXiv preprint arXiv:2206.06243.

[19]

Qian, H., Pan, S. J., Da, B., & Miao, C. (2019). A Novel Distribution-Embedded Neural Network for Sensor-Based Activity Recognition. In International joint conference on artificial intelligence.

[20]

Qian, H., Pan, S. J., & Miao, C. (2021). Latent Independent Excitation for Generalizable Sensor-based Cross-Person Activity Recognition. In AAAI conference on artificial intelligence.

[21]

Ragab M., Eldele E., Chen Z., Wu M., Kwoh C., Li X., Self-supervised autoregressive domain adaptation for time series data, IEEE Transactions on Neural Networks and Learning Systems PP (2021).

[22]

Reiss, A., & Stricker, D. (2012). Introducing a New Benchmarked Dataset for Activity Monitoring. In 2012 16th international symposium on wearable computers (pp. 108–109).

[23]

Sanabria A.R., Zambonelli F., Dobson S., Ye J., ContrasGAN: Unsupervised domain adaptation in human activity recognition via adversarial and contrastive learning, Pervasive and Mobile Computing 78 (2021).

[24]

Shen, Q., Feng, H., Song, R., Teso, S., Giunchiglia, F., & Xu, H. (2022). Federated Multi-Task Attention for Cross-Individual Human Activity Recognition. In International joint conference on artificial intelligence.

[25]

Smith, V., Chiang, C. -K., Sanjabi, M., & Talwalkar, A. S. (2017). Federated Multi-Task Learning. In NIPS.

[26]

Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015). Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM conference on embedded networked sensor systems (pp. 127–140).

[27]

Tonekaboni S., Eytan D., Goldenberg A., Unsupervised representation learning for time series with temporal neighborhood coding, 2021, arXiv, arXiv:2106.00750.

[28]

Vaswani A., Shazeer N.M., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I., Attention is all you need, 2017, arXiv, arXiv:1706.03762.

[29]

Wilson, G., Doppa, J. R., & Cook, D. J. (2020). Multi-source deep domain adaptation with weak supervision for time-series sensor data. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1768–1778).

[30]

Wilson G., Doppa J.R., Cook D.J., CALDA: Improving multi-source time series domain adaptation with contrastive adversarial learning, 2021, arXiv, abs/2109.14778.

[31]

Yang, L., & Qiao, L. (2022). Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion. In International conference on machine learning.

[32]

Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. F. (2016). DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. In Proceedings of the 26th international conference on World Wide Web.

[33]

Zhang Y., Yang Q., A survey on multi-task learning, IEEE Transactions on Knowledge and Data Engineering (2021).

Index Terms

ATFA: Adversarial Time–Frequency Attention network for sensor-based multimodal human activity recognition
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Learning paradigms

Index terms have been assigned to the content through auto-classification.

Recommendations

Sensor-based human activity recognition system with a multilayered model using time series shapelets

We exploit time series shapelets for complex human activity recognition.We present a multilayered activity model to represent four types of activities.We implement a prototype system based on smartphone for human activity recognition.Daily living and ...
EPARS: elderly physical activity reminder system using smartphone and wearable sensors
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

The sedentary lifestyle of elderly people may cause chronic diseases, reduce muscle mass and mobility. Moreover, elderly physical inactivity increases the susceptibility to mood disorders, such as anxiety and depression, and can permanently lead to ...
Recognition of coupling-paired activities in daily life
HCCE '12: Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments

Recognition of daily activities is one of major focuses in health care systems. But limited number of studies has been done on recognition of complex human activities in daily life. Therefore in this work an attempt has been done to recognize complex ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 236, Issue C

Feb 2024

1583 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 February 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents