More Web Proxy on the site http://driver.im/

research-article

TFSemantic: A Time–Frequency Semantic GAN Framework for Imbalanced Classification Using Radio Signals

Authors:

Chao YangAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 20, Issue 4

Article No.: 79, Pages 1 - 22

https://doi.org/10.1145/3614096

Published: 11 May 2024 Publication History

Abstract

Recently, wireless sensing techniques have been widely used for Internet of Things (IoT) applications. Unlike traditional device-based sensing, wireless sensing is contactless, pervasive, low cost, and non-invasive, making it highly suitable for relevant IoT applications. However, most existing methods are highly dependent on high-quality datasets, and the minority class will not achieve a satisfactory performance when suffering from a class imbalance problem. In this article, we propose a time–frequency semantic generative adversarial network framework (i.e., TFSemantic) to address the imbalanced classification problem in human activity recognition using radio frequency (RF) signals. Specifically, the TFSemantic framework can learn semantic features from the minority classes and then generate high-quality signals to restore data balance. It includes a data pre-processing module, a semantic extraction module, a semantic distribution module, and a data augmenter module. In the data pre-processing module, we process four different RF datasets (i.e., WiFi, RFID, UWB, and mmWave). We also develop Fourier semantic feature convolution and attention semantic feature embedding methods for the semantic extraction module. A discrete wavelet transform is utilized for reconstructed RF samples in the semantic distribution module. In data augmenter module, we design an associated loss function to achieve effective adversarial training. Finally, we validate the effectiveness of the proposed TFSemantic framework using different RF datasets, which outperforms several state-of-the-art methods.

References

[1]

Harshit Ambalkar, Xuyu Wang, and Shiwen Mao. 2021. Adversarial human activity recognition using Wi-Fi CSI. In Proceedings of the IEEE Canadian Conference on Electrical and Computer Engineering (CCECE’21). 1–5.

[2]

Sergiy Bogomolov, Alexandre Donzé, Goran Frehse, Radu Grosu, Taylor T. Johnson, Hamed Ladan, Andreas Podelski, and Martin Wehrle. 2016. Guided search for hybrid systems based on coarse-grained space abstractions. Int. J. Softw. Tools Technol. Transf. 18, 4 (Aug.2016), 449–467.

Digital Library

[3]

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29.

[4]

Azhar Chara, Tianya Zhao, Xuyu Wang, and Shiwen Mao. 2023. Respiratory biofeedback using acoustic sensing with smartphones. Smart Health 28 (2023), 100387. https://www.sciencedirect.com/science/article/abs/pii/S2352648323000156

[5]

Zhe Chen, Chao Cai, Tianyue Zheng, Jun Luo, Jie Xiong, and Xin Wang. 2021. RF-based human activity recognition using signal adapted convolutional neural network. IEEE Trans. Mobile Comput. 22, 1 (2021), 487–499.

[6]

Fabrizio Costa and Kurt De Grave. 2010. Fast neighborhood subgraph pairwise distance kernel. In Proceedings of the International Conference on Machine Learning (ICML’10). 255–262.

[7]

Lang Deng, Jianfei Yang, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, and Lihua Xie. 2022. Gaitfi: Robust device-free human identification via WiFi and vision multimodal learning. IEEE IoT J. 10, 1 (2022), 625–636.

[8]

Shuya Ding, Zhe Chen, Tianyue Zheng, and Jun Luo. 2020. RF-net: A unified meta-learning framework for RF-enabled one-shot human activity recognition. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys’20). 517–530.

Digital Library

[9]

Baris Erol, Sevgi Z. Gurbuz, and Moeness G. Amin. 2019. GAN-based synthetic radar micro-doppler augmentations for improved human activity recognition. In Proceedings of the IEEE Radar Conference. 1–5.

[10]

Wei Feng, Wenjiang Huang, and Jinchang Ren. 2018. Class imbalance ensemble learning based on the margin theory. MDPI Appl. Sci. 8, 5 (May2018), 815.

[11]

Ruiyang Gao, Mi Zhang, Jie Zhang, Yang Li, Enze Yi, Dan Wu, Leye Wang, and Daqing Zhang. 2021. Towards position-independent sensing for gesture recognition with Wi-Fi. Proc. ACM Interact. Mob. Wear. Ubiq. Technol. 5, 2 (Jun.2021), 1–28.

Digital Library

[12]

Vicente García, Ramón Alberto Mollineda, and José Salvador Sánchez. 2009. Index of balanced accuracy: A performance measure for skewed class distributions. In Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis. 441–448.

Digital Library

[13]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the Conference and Workshop on Neural Information Processing Systems (NIPS’14). 1–9.

[14]

Huaping Guo, Hongbing Liu, Changan Wu, Weimei Zhi, Yan Xiao, and Wei She. 2016. Logistic discrimination based on G-mean and f-measure for imbalanced problem. J. Intell. Fuzzy Syst. 31, 3 (2016), 1155–1166.

Digital Library

[15]

John R Hershey and Peder A Olsen. 2007. Approximating the kullback leibler divergence between gaussian mixture models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’07). IV-317–IV-320.

[16]

Arya Iranmehr, Hamed Masnadi-Shirazi, and Nuno Vasconcelos. 2019. Cost-sensitive support vector machines. Neurocomputing 343 (May2019), 50–64.

Digital Library

[17]

Hao Kong, Li Lu, Jiadi Yu, Yanmin Zhu, Feilong Tang, Yi-Chao Chen, Linghe Kong, and Feng Lyu. 2022. Push the limit of WiFi-based user authentication towards undefined gestures. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’22). 410–419.

Digital Library

[18]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 25. Lake Tahoe, USA.

Digital Library

[19]

Oscar D. Lara and Miguel A. Labrador. 2012. A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 15, 3 (Third Quarter2012), 1192–1209.

[20]

Guillaume Lemaître, Fernando Nogueira, and Christos K Aridas. 2017. Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18, 1 (Jan.2017), 559–563.

Digital Library

[21]

Chenning Li, Manni Liu, and Zhichao Cao. 2020. WiHF: Enable user identified gesture recognition with WiFi. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’20). 586–595.

Digital Library

[22]

Junnan Li, Qingsheng Zhu, Quanwang Wu, and Zhu Fan. 2021. A novel oversampling technique for class-imbalanced learning based on SMOTE and natural neighbors. Inf. Sci. 565 (2021), 438–455.

Digital Library

[23]

Shuheng Li, Ranak Roy Chowdhury, Jingbo Shang, Rajesh K Gupta, and Dezhi Hong. 2021. UniTS: Short-time fourier inspired neural networks for sensory time series classification. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys’21). 234–247.

Digital Library

[24]

Youpeng Li, Xuyu Wang, and Lingling An. 2023. Hierarchical clustering-based personalized federated learning for robust and fair human activity recognition. Proc. ACM Interact. Mob. Wear. Ubiq. Technol. 7, 1 (2023), 1–38.

Digital Library

[25]

Yadong Li, Dongheng Zhang, Jinbo Chen, Jinwei Wan, Dong Zhang, Yang Hu, Qibin Sun, and Yan Chen. 2021. Towards domain-independent and real-time gesture recognition using mmWave signal. arXiv:2111.06195. https://arxiv.org/abs/2111.06195

[26]

Chi Lin, Pengfei Wang, Chuanying Ji, Mohammad S Obaidat, Lei Wang, Guowei Wu, and Qiang Zhang. 2023. A contactless authentication system based on WiFi CSI. ACM Trans. Sens. Netw. 19, 2 (2023), 1–20.

Digital Library

[27]

Jian Liu, Hongbo Liu, Yingying Chen, Yan Wang, and Chen Wang. 2019. Wireless sensing for human activity: A survey. IEEE Commun. Surv. Tutor. 22, 3 (Third Quarter2019), 1629–1645.

[28]

Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. 2019. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR’19). 2537–2546.

[29]

Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, and Woosub Jung. 2018. SignFi: Sign language recognition using WiFi. Proc. ACM Interact. Mob. Wear. Ubiq. Technol. 2, 1 (Mar.2018), 1–21.

Digital Library

[30]

Giovanni Mariani, Florian Scheidegger, Roxana Istrate, Costas Bekas, and Cristiano Malossi. 2018. BAGAN: Data augmentation with balancing GAN. arXiv:1803.09655. Retrieved from https://arxiv.org/abs/1803.09655

[31]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv:1411.1784. Retrieved from https://arxiv.org/abs/1411.1784

[32]

Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional image synthesis with auxiliary classifier GANs. In Proceedings of the International Conference on Machine Learning (ICML’17). 2642–2651.

[33]

Sameera Palipana, Dariush Salami, Luis A Leiva, and Stephan Sigg. 2021. Pantomime: Mid-air gesture recognition with sparse millimeter-wave radar point clouds. Proc. ACM Interact. Mob. Wear. Ubiq. Technol. 5, 1 (Mar.2021), 1–27.

Digital Library

[34]

Maytus Piriyajitakonkij et al. 2020. SleepPoseNet: Multi-view learning for sleep postural transition recognition using UWB. IEEE J. Biomed. Health Inf. 25, 4 (Apr.2020), 1305–1314.

[35]

Akash Deep Singh, Sandeep Singh Sandha, Luis Garcia, and Mani Srivastava. 2019. Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar. In Proc. ACM Workshop on Millimeter-Wave and Terahertz Networks and Sensing Systems (mmNets’19). 51–56.

Digital Library

[36]

Jie Wang, Liang Zhang, Changcheng Wang, Xiaorui Ma, Qinghua Gao, and Bin Lin. 2020. Device-free human gesture recognition with generative adversarial networks. IEEE IoT J. 7, 8 (Apr.2020), 7678–7688.

[37]

Q. Wang, B. Wu, P. Zhu, P. Li, and Q. Hu. 2020. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR’20). 11534–11542.

[38]

Xuyu Wang, Xiangyu Wang, and Shiwen Mao. 2018. RF sensing in the internet of things: A general deep learning framework. IEEE Commun. Mag. 56, 9 (2018), 62–67.

[39]

Xuyu Wang, Chao Yang, and Shiwen Mao. 2020. On CSI-based vital sign monitoring using commodity WiFi. ACM Trans. Comput. Healthc. 1, 3 (2020), 1–27.

Digital Library

[40]

Yanwen Wang and Yuanqing Zheng. 2018. Modeling RFID signal reflection for contact-free activity recognition. Proc. ACM Interact. Mob. Wear. Ubiq. Technol. 2, 4 (Dec.2018), 1–22.

Digital Library

[41]

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (Apr.2004), 600–612.

Digital Library

[42]

Rui Xiao, Jianwei Liu, Jinsong Han, and Kui Ren. 2021. OneFi: One-shot recognition for unseen gesture via COTS WiFi. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys’21). 206–219.

Digital Library

[43]

Chao Yang, Lingxiao Wang, Xuyu Wang, and Shiwen Mao. 2022. Environment adaptive RFID based 3D human pose tracking with a meta-learning approach. IEEE J. Radio Freq. Ident. 6, 1 (Jan.2022), 413–425.

[44]

Chao Yang, Xuyu Wang, and Shiwen Mao. 2020. RFID-Pose: Vision-aided three-dimensional human pose estimation with radio-frequency identification. IEEE Trans. Reliabil. 70, 3 (Sep.2020), 1218–1231.

[45]

Chao Yang, Xuyu Wang, and Shiwen Mao. 2023. TARF: Technology-agnostic RF sensing for human activity recognition. IEEE J. Biomed. Health Inf. 27, 2 (Feb.2023), 636–647.

[46]

Shuochao Yao et al. 2019. STFNets: Learning sensing signals from the time-frequency perspective with short-time fourier neural networks. In Proceedings of the ACM International World Wide Web Conference (WWW’19). 2192–2202.

Digital Library

[47]

Shuangjiao Zhai, Zhanyong Tang, Petteri Nurmi, Dingyi Fang, Xiaojiang Chen, and Zheng Wang. 2022. RISE: Robust wireless sensing using probabilistic and statistical assessments. In Proceedings of the International Conference on Mobile Computing and Networking (MobiCom’21). 309–322.

[48]

Jin Zhang, Fuxiang Wu, Bo Wei, Qieshi Zhang, Hui Huang, Syed W Shah, and Jun Cheng. 2020. Data augmentation and dense-LSTM for human activity recognition using WiFi signal. IEEE IoT J. 8, 6 (Mar.2020), 4628–4641.

[49]

Shujie Zhang, Tianyue Zheng, Zhe Chen, and Jun Luo. 2022. Can we obtain fine-grained heartbeat waveform via contact-free RF-sensing? In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’22). 1759–1768.

Digital Library

[50]

Yi Zhang, Zheng Yang, Guidong Zhang, Chenshu Wu, and Li Zhang. 2021. XGest: Enabling cross-label gesture recognition with RF signals. ACM Trans. Sens. Netw. 17, 4 (2021), 1–23.

Digital Library

[51]

Peijun Zhao, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, and Andrew Markham. 2023. CubeLearn: End-to-end learning for human motion recognition from raw mmWave radar signals. IEEE IoT J. 10, 12 (2023), 10236–10249.

[52]

Tianyue Zheng, Zhe Chen, Shuya Ding, and Jun Luo. 2021. Enhancing RF sensing with deep learning: A layered approach. IEEE Commun. Mag. 59, 2 (Feb.2021), 70–76.

Digital Library

[53]

Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. 2019. Zero-effort cross-domain gesture recognition with Wi-Fi. In Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’19). 313–325.

Digital Library

[54]

Zhipeng Zhou, Feng Wang, Jihong Yu, Ju Ren, Zhi Wang, and Wei Gong. 2022. Target-oriented semi-supervised domain adaptation for WiFi-based HAR. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’22). 420–429.

Digital Library

[55]

Xiaofeng Zhu, Jianye Yang, Chengyuan Zhang, and Shichao Zhang. 2019. Efficient utilization of missing data in cost-sensitive learning. IEEE Trans. Knowl. Data Eng. 33, 6 (2019), 2425–2436.

Cited By

Zhang WLi YAn LWan BWang X(2024)SARS: A Personalized Federated Learning Framework Towards Fairness and Robustness against Backdoor AttacksProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785718:4(1-24)Online publication date: 21-Nov-2024
https://doi.org/10.1145/3678571
Shan YLiao PWang XAn LMao S(2023)Classical to Quantum Transfer Learning Framework for Wireless Sensing Under Domain ShiftGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10437382(3167-3172)Online publication date: 4-Dec-2023
https://doi.org/10.1109/GLOBECOM54140.2023.10437382

Index Terms

TFSemantic: A Time–Frequency Semantic GAN Framework for Imbalanced Classification Using Radio Signals
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

Radio Frequency Antenna Types: Dielectric, Fractal Antenna, Log-Periodic Antenna, Phased Array, Collinear Antenna Array, Feed Horn
A New Under-Sampling Method Using Genetic Algorithm for Imbalanced Data Classification
IMCOM '16: Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication

The class imbalance problem is frequently found in many real-world domains, where many of traditional classifiers often fail to detect minority class objects due to paying less attention to those. In an effort to address this class imbalance problem, a ...
Preprocessing noisy imbalanced datasets using SMOTE enhanced with fuzzy rough prototype selection

The Synthetic Minority Over Sampling TEchnique (SMOTE) is a widely used technique to balance imbalanced data. In this paper we focus on improving SMOTE in the presence of class noise. Many improvements of SMOTE have been proposed, mostly cleaning or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 20, Issue 4

July 2024

603 pages

EISSN:1550-4867

DOI:10.1145/3618082

Editor:
Wen Hu
University of New South Wales, Australia

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 11 May 2024

Online AM: 08 August 2023

Accepted: 25 July 2023

Revised: 06 June 2023

Received: 05 February 2023

Published in TOSN Volume 20, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
669
Total Downloads

Downloads (Last 12 months)476
Downloads (Last 6 weeks)52

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang WLi YAn LWan BWang X(2024)SARS: A Personalized Federated Learning Framework Towards Fairness and Robustness against Backdoor AttacksProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785718:4(1-24)Online publication date: 21-Nov-2024
https://doi.org/10.1145/3678571
Shan YLiao PWang XAn LMao S(2023)Classical to Quantum Transfer Learning Framework for Wireless Sensing Under Domain ShiftGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10437382(3167-3172)Online publication date: 4-Dec-2023
https://doi.org/10.1109/GLOBECOM54140.2023.10437382

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents