More Web Proxy on the site http://driver.im/

research-article

Public Access

Contactless Material Identification with Millimeter Wave Vibrometry

Authors:

Hailan Shanbhag,

Haitham HassaniehAuthors Info & Claims

MobiSys '23: Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services

Pages 475 - 488

https://doi.org/10.1145/3581791.3596850

Published: 18 June 2023 Publication History

Abstract

This paper introduces RFVibe, a system that enables contactless material and object identification through the fusion of millimeter wave wireless signals with acoustic signals. In particular, RFVibe plays an audio sound next to the object that generates micro-vibrations in the object. These micro-vibrations can be captured by shining a millimeter wave radar signal on the object and analyzing the phase of the reflected wireless signal. RFVibe can then extract several features including resonance frequencies and vibration modes, damping time of vibrations, and wireless reflection coefficients. These features are then used to enable more accurate identification, with a step towards generalizing towards different setups and locations. We implement RFVibe using an off-the-shelf millimeter-wave radar and an acoustic speaker. We evaluate it on 23 objects of 7 material types (Metal, Wood, Ceramic, Glass, Plastic, Cardboard, and Foam), obtaining 81.3% accuracy for material classification, a 30% improvement over prior work. RFVibe is able to classify with reasonable accuracy in scenarios that it has not encountered before, including different locations, angles, boundary conditions, and objects.

References

[1]

N. Adair. Radio frequency identification (rfid) power budgets for packaging applications. PGK-491, pages 2--11, 2005.

[2]

F. Adib, C.-Y. Hsu, H. Mao, D. Katabi, and F. Durand. Capturing the human figure through a wall. ACM Transactions on Graphics (TOG), 34(6):1--13, 2015.

[3]

E. M. Amin, R. Bhattacharyya, S. Kumar, S. Sarma, and N. C. Karmakar. Towards low-cost resolution optimized passive uhf rfid light sensing. In WAMICON 2014, pages 1--6. IEEE, 2014.

[4]

V. Aranchuk, A. K. Lal, C. F. Hess, and J. M. Sabatier. Multi-beam laser doppler vibrometer for landmine detection. Optical Engineering, 45(10):104302, 2006.

[5]

O. Buyukozturk, J. G. Chen, N. Wadhwa, A. Davis, F. Durand, and W. T. Freeman. Smaller than the eye can see: Vibration analysis with video cameras. In World Conference on Non-Destructive Testing 2016, 2016.

[6]

O. Büyüköztürk, R. Haupt, C. Tuakta, and J. Chen. Remote detection of debonding in frp-strengthened concrete structures using acoustic-laser technique. In Nondestructive Testing of Materials and Structures, pages 19--24. Springer, 2013.

[7]

P. Castellini, N. Paone, and E. P. Tomasini. The laser doppler vibrometer as an instrument for nonintrusive diagnostic of works of art: application to fresco paintings. Optics and Lasers in Engineering, 25(4--5):227--246, 1996.

[8]

J. G. Chen, A. Davis, N. Wadhwa, F. Durand, W. T. Freeman, and O. Büyüköztürk. Video camera-based vibration measurement for civil infrastructure applications. Journal of Infrastructure Systems, 23(3):B4016013, 2017.

[9]

J. G. Chen, R. W. Haupt, and O. Buyukozturk. The acoustic-laser vibrometry technique for the noncontact detection of discontinuities in fiber reinforced polymer-retrofitted concrete. Materials evaluation, 72(10), 2014.

[10]

L. Collini, R. Garziera, and F. Mangiavacca. Development, experimental validation and tuning of a contact-less technique for the health monitoring of antique frescoes. NDT & E International, 44(2):152--157, 2011.

[11]

A. Davis, M. Rubinstein, N. Wadhwa, G. J. Mysore, F. Durand, and W. T. Freeman. The visual microphone: Passive recovery of sound from video. 2014.

Digital Library

[12]

A. Dhekne, M. Gowda, Y. Zhao, H. Hassanieh, and R. R. Choudhury. Liquid: A wireless liquid identifier. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '18, page 442--454, New York, NY, USA, 2018. Association for Computing Machinery.

Digital Library

[13]

J. Ding and R. Chandra. Towards low cost soil sensing using wi-fi. In The 25th Annual International Conference on Mobile Computing and Networking, pages 1--16, 2019.

[14]

T. Emge and O. Buyukozturk. Remote nondestructive testing of composite-steel interface by acoustic laser vibrometry. Materials evaluation, 70(12), 2012.

[15]

C. Feng, J. Xiong, L. Chang, J. Wang, X. Chen, D. Fang, and Z. Tang. Wimi: Target material identification with commodity wi-fi devices. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pages 700--710, 2019.

[16]

A. Gadre, D. Vasisht, N. Raghuvanshi, B. Priyantha, M. Kotaru, S. Kumar, and R. Chandra. Milton: Sensing product integrity without opening the box using non-invasive acoustic vibrometry. pages 390--402, 2022.

[17]

J. D. Griffin, G. D. Durgin, A. Haldi, and B. Kippelen. Radio link budgets for 915 mhz rfid antennas placed on various objects. In Texas Wireless Symposium, volume 44, 2005.

[18]

J. Guo, T. Wang, Y. He, M. Jin, C. Jiang, and Y. Liu. Twinleak: Rfid-based liquid leakage detection in industrial environments. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pages 883--891. IEEE, 2019.

Digital Library

[19]

U. Ha, J. Leng, A. Khaddaj, and F. Adib. Food and liquid sensing in practical environments using rfids. In Proceedings of the 17th Usenix Conference on Networked Systems Design and Implementation, NSDI'20, page 1083--1100, USA, 2020. USENIX Association.

Digital Library

[20]

R. W. Haupt and K. D. Rolt. Standoff acoustic laser technique to locate buried land mines. Lincoln laboratory journal, 15(1):3--22, 2005.

[21]

S. He, Y. Qian, H. Zhang, G. Zhang, M. Xu, L. Fu, X. Cheng, H. Wang, and P. Hu. Accurate contact-free material recognition with millimeter wave and machine learning. In L. Wang, M. Segal, J. Chen, and T. Qiu, editors, Wireless Algorithms, Systems, and Applications, pages 609--620, Cham, 2022. Springer Nature Switzerland.

Digital Library

[22]

A. Hind. Agilent 101: An introduction to optical spectroscopy, 2013.

[23]

Y. Huang, K. Chen, Y. Huang, L. Wang, and K. Wu. Vi-liquid: unknown liquid identification with your smartphone vibration. In MobiCom, pages 174--187, 2021.

Digital Library

[24]

J. F. James, R. S. Sternberg, and S. A. Rice. The design of optical spectrometers. Physics Today, 23(12):55, 1970.

[25]

C. Jiang, J. Guo, Y. He, M. Jin, S. Li, and Y. Liu. Mmvib: Micrometer-level vibration measurement with mmwave radar. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, MobiCom '20, New York, NY, USA, 2020. Association for Computing Machinery.

Digital Library

[26]

Y. Liang, A. Zhou, H. Zhang, X. Wen, and H. Ma. Fg-liquid: A contact-less fine-grained liquid identifier by pushing the limits of millimeter-wave sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(3):1--27, 2021.

Digital Library

[27]

M. Z. Ozturk, C. Wu, B. Wang, and K. J. R. Liu. Radiomic: Sound sensing via mmwave signals, 2021.

[28]

E. Pan, J. Tang, D. Kosaka, R. Yao, and A. Gupta. Openradar. https://github.com/presenseradar/openradar, 2019.

[29]

H. Saghlatoon, R. Mirzavand, M. M. Honari, and P. Mousavi. Sensor antenna transmitter system for material detection in wireless-sensor-node applications. IEEE Sensors Journal, 18(21):8812--8819, 2018.

[30]

A. A. Shabana. Theory of Vibration, volume 2. Springer, 1999.

[31]

F. Shang, P. Yang, Y. Yan, and X.-Y. Li. Liqray: Non-invasive and fine-grained liquid recognition system. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking, MobiCom '22, page 296--309, New York, NY, USA, 2022. Association for Computing Machinery.

[32]

H. Song, B. Wei, Q. Yu, X. Xiao, and T. Kikkawa. Wieps: Measurement of dielectric property with commodity wifi device---an application to ethanol/water mixture. IEEE Internet of Things Journal, 7(12):11667--11677, 2020.

[33]

S. Su, F. Heide, R. Swanson, J. Klein, C. Callenberg, M. Hullin, and W. Heidrich. Material classification using raw time-of-flight measurements. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3503--3511, 2016.

[34]

X. Sun, W. Deng, X. Wei, D. Fang, B. Li, and X. Chen. Akte-liquid: Acoustic-based liquid identification with smartphones. ACM Trans. Sen. Netw., aug 2022. Just Accepted.

[35]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1--9, 2015.

[36]

K. Tanaka, Y. Mukaigawa, T. Funatomi, H. Kubo, Y. Matsushita, and Y. Yagi. Material classification using frequency-and depth-dependent time-of-flight distortion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 79--88, 2017.

[37]

C. Wang, J. Liu, Y. Chen, H. Liu, and Y. Wang. Towards in-baggage suspicious object detection using commodity wifi. In 2018 IEEE Conference on Communications and Network Security (CNS), pages 1--9. IEEE, 2018.

[38]

C. Wang, X. Zhang, X. Zang, Y. Liu, G. Ding, W. Yin, and J. Zhao. Feature sensing and robotic grasping of objects with uncertain information: A review. Sensors, 20(13):3707, 2020.

[39]

G. Wang, J. Han, C. Qian, W. Xi, H. Ding, Z. Jiang, and J. Zhao. Verifiable smart packaging with passive rfid. IEEE Transactions on Mobile Computing, 18(5):1217--1230, 2018.

[40]

J. Wang, J. Xiong, X. Chen, H. Jiang, R. K. Balan, and D. Fang. Tagscan: Simultaneous target imaging and material identification with commodity rfid devices. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, MobiCom '17, page 288--300, New York, NY, USA, 2017. Association for Computing Machinery.

Digital Library

[41]

L. Wang, C.-Y. Lee, Z. Tu, and S. Lazebnik. Training deeper convolutional networks with deep supervision. arXiv preprint arXiv:1505.02496, 2015.

[42]

Z. Wang. Towards Robust and Secure Audio Sensing Using Wireless Vibrometry and Deep Learning. University of California, Los Angeles, 2020.

[43]

Z. Wang, Z. Chen, A. D. Singh, L. Garcia, J. Luo, and M. B. Srivastava. Uwhear: through-wall extraction and separation of audio vibrations using wireless signals. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pages 1--14, 2020.

Digital Library

[44]

T. Wei, S. Wang, A. Zhou, and X. Zhang. Acoustic eavesdropping through wireless vibrometry. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pages 130--141, 2015.

Digital Library

[45]

J. Weiß and A. Santra. One-shot learning for robust material classification using millimeter-wave radar system. IEEE Sensors Letters, 2(4):1--4, 2018.

[46]

P. Welch. The use of fast fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Transactions on audio and electroacoustics, 15(2):70--73, 1967.

[47]

C. Wu, F. Zhang, B. Wang, and K. R. Liu. msense: Towards mobile material sensing with a single millimeter-wave radio. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(3):1--20, 2020.

Digital Library

[48]

B. Xie, J. Xiong, X. Chen, E. Chai, L. Li, Z. Tang, and D. Fang. Tagtag: Material sensing with commodity rfid. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems, SenSys '19, page 338--350, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

[49]

T. Xue, J. Wu, Z. Zhang, C. Zhang, J. B. Tenenbaum, and W. T. Freeman. Seeing tree structure from vibration. In Proceedings of the European Conference on Computer Vision (ECCV), pages 748--764, 2018.

Digital Library

[50]

Y. Yang, Y. Wang, J. Cao, and J. Chen. Hearliquid: Non-intrusive liquid fraud detection using commodity acoustic devices. IEEE Internet of Things Journal, pages 1--1, 2022.

[51]

H.-S. Yeo, G. Flamich, P. Schrempf, D. Harris-Birtill, and A. Quigley. Radarcat: Radar categorization for input & interaction. pages 833--841, 10 2016.

[52]

D. Zhang, J. Wang, J. Jang, J. Zhang, and S. Kumar. On the feasibility of wi-fi based material sensing. In The 25th Annual International Conference on Mobile Computing and Networking, MobiCom '19, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

[53]

J. Zhang, Y. Zhou, R. Xi, S. Li, J. Guo, and Y. He. Ambiear: Mmwave based voice recognition in nlos scenarios. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 6(3), sep 2022.

Digital Library

[54]

Y. Zhang, G. Laput, and C. Harrison. Vibrosight: Long-range vibrometry for smart environment sensing. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pages 225--236, 2018.

Digital Library

Cited By

Pulikkaseril CRoss DTofini ALize YCollarte F(2024)Instantaneous Material Classification Using a Polarization-Diverse RMCW LIDARSensors10.3390/s2417576124:17(5761)Online publication date: 4-Sep-2024
https://doi.org/10.3390/s24175761
Sheng YLi JMa YZhang J(2024)HomeOSD: Appliance Operating-Status Detection Using mmWave RadarSensors10.3390/s2409291124:9(2911)Online publication date: 2-May-2024
https://doi.org/10.3390/s24092911
Tang MTeckchandani PHe JGuo HSoltanaghai EShu YLiu JTan RHe YChen J(2024)BSENSE: In-vehicle Child Detection and Vital Sign Monitoring with a Single mmWave Radar and Synthetic ReflectorsProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699352(478-492)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699352

Index Terms

Contactless Material Identification with Millimeter Wave Vibrometry

Recommendations

Poster: Contactless Material Identification with Millimeter Wave Vibrometry
MobiSys '23: Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services

This paper introduces RFVibe, a system that enables contactless material and object identification through the fusion of millimeter wave wireless signals with acoustic signals. In particular, RFVibe plays an audio sound next to the object that generates ...
Millimeter-wave planar high gain SIW antenna using half elliptic radiation slots

This article reports a high gain millimeter-wave substrate integrated waveguide SIW antenna using low cost printed circuit board technology. The half elliptic slots which can provide small shunt admittance, low cross polarization level and low mutual ...
Wide-band millimetre-wave antenna based on Si and suspended BCB membrane

This paper presents the design, fabrication and characterization of a millimetre-wave wide-band patch antenna integrated on silicon (Si) and using a BCB membrane suspended over a deep cavity etched in the Si substrate. The antenna structure consists of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '23: Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services

June 2023

651 pages

ISBN:9798400701108

DOI:10.1145/3581791

General Chairs:
Petteri Nurmi,
Pan Hui,
Program Chairs:
Ardalan Amiri Sani,
Yunxin Liu

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

MobiSys '23

Sponsor:

SIGMOBILE

MobiSys '23: 21st Annual International Conference on Mobile Systems, Applications and Services

June 18 - 22, 2023

Helsinki, Finland

Acceptance Rates

MobiSys '23 Paper Acceptance Rate 41 of 198 submissions, 21%;

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
1,026
Total Downloads

Downloads (Last 12 months)473
Downloads (Last 6 weeks)79

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pulikkaseril CRoss DTofini ALize YCollarte F(2024)Instantaneous Material Classification Using a Polarization-Diverse RMCW LIDARSensors10.3390/s2417576124:17(5761)Online publication date: 4-Sep-2024
https://doi.org/10.3390/s24175761
Sheng YLi JMa YZhang J(2024)HomeOSD: Appliance Operating-Status Detection Using mmWave RadarSensors10.3390/s2409291124:9(2911)Online publication date: 2-May-2024
https://doi.org/10.3390/s24092911
Tang MTeckchandani PHe JGuo HSoltanaghai EShu YLiu JTan RHe YChen J(2024)BSENSE: In-vehicle Child Detection and Vital Sign Monitoring with a Single mmWave Radar and Synthetic ReflectorsProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699352(478-492)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699352

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents