Article

A tissue-conductive acoustic sensor applied in speech recognition for privacy

Authors:

Panikos Heracleous,

Yoshitaka Nakajima,

Hiroshi Saruwatari,

Kiyohiro ShikanoAuthors Info & Claims

sOc-EUSAI '05: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies

Pages 93 - 97

https://doi.org/10.1145/1107548.1107577

Published: 12 October 2005 Publication History

Get Access

Abstract

In this paper, we present the Non-Audible Murmur (NAM) microphones focusing on their applications in automatic speech recognition. A NAM microphone is a special acoustic sensor attached behind the talker's ear and able to capture very quietly uttered speech (non-audible murmur) through body tissue. Previously, we reported experimental results for non-audible murmur recognition using a Stethoscope microphone in a clean environment. In this paper, we also present a more advanced NAM microphone, the so-called Silicon NAM microphone. Using a small amount of training data and adaptation approaches, we achieved a 93.9% word accuracy for a 20k vocabulary dictation task. Therefore, in situations when privacy in human-machine communication is preferable, NAM microphone can be very effectively applied for automatic recognition of speech inaudible to other listeners near the talker. Because of the nature of non-audible murmur (e.g., privacy) investigation of the behavior of NAM microphones in noisy environments is of high importance. To do this, we also conducted experiments in real and simulated noisy environments. Although, using simulated noisy data the NAM microphones show high robustness against noise, in real environments the recognition performance decreases markedly due to the effect of the Lombard reflex. In this paper, we also report experimental results showing the negative impact effect of the Lombard reflex on non-audible murmur recognition. In addition to a dictation task, we also report a keyword-spotting system based on non-audible murmur with very promising results.

References

[1]

Y. Nakajima, H. Kashioka, K. Shikano, N. Campbell," Non-Audible Murmur Recognition Input Interface Using Stethoscopic Microphone Attached to the Skin", Proceedings of ICASSP, pp. 708--711, 2003.

Google Scholar

[2]

Y. Zheng, Z. Liu, Z. Shang, M. Sinclair, J. Droppo, L. Deng, A. Acero, Z. Huang," Air- and Bone-Conductive Integrated Microphones for Robust Speech Detection and Enhancement", Proceedings of ASRU, pp. 249--253, 2003.

Google Scholar

[3]

M. Graciarena, H. Franco, K. Sonmez, H. Bratt, "Combining Standard and Throat Microphones for Robust Speech Recognition", IEEE Signal Processing Letters, Vol. 10, No 3, pp. 72--74, 2003.

Crossref

Google Scholar

[4]

S. C. Jou, T. Schultz, Alex Weibel, "Adaptation for Soft Whisper Recognition Using a Throat Microphone", Proceedings of ICSLP, pp. -, 2004.

Google Scholar

[5]

P. Heracleous, Y. Nakajima, A. Lee, H. Saruwatari, K. Shikano, "Non-Audible Murmur (NAM) Recognition Using a Stethoscopic NAM microphone", Proceedings of ICLP, pp. 1469--1472, 2004.

Google Scholar

[6]

P. Heracleous, Y. Nakajima, A. Lee, H. Saruwatari, K. Shikano, "Audible (normal) speech and inaudible murmur recognition using NAM microphone", Proceedings of EUSIPCO, pp. 329--332, 2004.

Google Scholar

[7]

Junqua J-C, "The Lombard Reflex and its Role on Human Listeners and Automatic Speech Recognizers", J. Acoust. Soc. Am. Vol. 1 pp. 510--524, 1993.

Crossref

Google Scholar

[8]

A. Wakao, K. Takeda, F. Itakura, "Variability of Lombard Effects Under Different Noise Conditions", Proceedings of ICSLP, pp. 2009--2012, 1996.

Google Scholar

[9]

T. Kawahara et al., "Free Software Toolkit for Japanese Large Vocabulary Continuous Speech Recognition", Proceedings of ICSLP, pp. IV-476--479, 2000.

Google Scholar

[10]

C. J. Leggetter, C. Woodland, "Maximum Likelihood Linear Regression for Speaker Adaptation of Continuous Density Hidden Markov Models", Computer Speech and Language, Vol. 9, pp. 171--185, 1995.

Crossref

Google Scholar

[11]

C. H. Lee, C. H. Lin, and B. H. Juang, "A study on speaker adaptation of the parameters of continuous density hidden Markov models", IEEE transactions Signal Processing, Vol. 39, pp. 806--814, 1991.

Digital Library

Google Scholar

[12]

P. C. Woodland, D. Pye, M. J. F. Gales, "Iterative Unsupervised Adaptation Using Maximum Likelihood Linear Regression", Proceedings of ICSLP, pp. 1133--1136, 1996.

Google Scholar

[13]

R. C. Rose, D. B. Paul, "A Hidden Markov Model Based Keyword Recognition System," Proc. ICASSP, pages 129--132, 1990.

Google Scholar

Cited By

View all

Rekimoto J(2023)WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech InteractionsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580706(1-12)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580706
Kumar TPadmapriya SBai VBeulah Devamalar PSuresh G(2015)Conversion of non-audible murmur to normal speech through Wi-Fi transceiver for speech recognition based on GMM model2015 2nd International Conference on Electronics and Communication Systems (ICECS)10.1109/ECS.2015.7125023(802-808)Online publication date: Feb-2015
https://doi.org/10.1109/ECS.2015.7125023
Tran VBailly GLvenbruck HToda T(2010)Improvement to a NAM-captured whisper-to-speech systemSpeech Communication10.1016/j.specom.2009.11.00552:4(314-326)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.specom.2009.11.005
Show More Cited By

A tissue-conductive acoustic sensor applied in speech recognition for privacy
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Unvoiced speech recognition using tissue-conductive acoustic sensor

We present the use of stethoscope and silicon NAM (nonaudible murmur) microphones in automatic speech recognition. NAM microphones are special acoustic sensors, which are attached behind the talker's ear and can capture not only normal (audible) speech, ...
Multilingual recognition of non-native speech using acoustic model transformation and pronunciation modeling

This article presents an approach for the automatic recognition of non-native speech. Some non-native speakers tend to pronounce phonemes as they would in their native language. Model adaptation can improve the recognition rate for non-native speakers, ...
Acoustic model adaptation based on pronunciation variability analysis for non-native speech recognition

In this paper, pronunciation variability between native and non-native speakers is investigated, and a novel acoustic model adaptation method is proposed based on pronunciation variability analysis in order to improve the performance of a speech ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

sOc-EUSAI '05: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies

October 2005

316 pages

ISBN:1595933042

DOI:10.1145/1107548

Conference Chairs:
Gerard Bailly
CNRS, France
,
James L. Crowley
INPG - France

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

sOc-EUSAI05

sOc-EUSAI05: Smart Objects & Ambient Intelligence

October 12 - 14, 2005

Grenoble, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
232
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Rekimoto J(2023)WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech InteractionsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580706(1-12)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580706
Kumar TPadmapriya SBai VBeulah Devamalar PSuresh G(2015)Conversion of non-audible murmur to normal speech through Wi-Fi transceiver for speech recognition based on GMM model2015 2nd International Conference on Electronics and Communication Systems (ICECS)10.1109/ECS.2015.7125023(802-808)Online publication date: Feb-2015
https://doi.org/10.1109/ECS.2015.7125023
Tran VBailly GLvenbruck HToda T(2010)Improvement to a NAM-captured whisper-to-speech systemSpeech Communication10.1016/j.specom.2009.11.00552:4(314-326)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.specom.2009.11.005
Hueber TBenaroya EChollet GDenby BDreyfus GStone M(2010)Development of a silent speech interface driven by ultrasound and optical images of the tongue and lipsSpeech Communication10.1016/j.specom.2009.11.00452:4(288-300)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.specom.2009.11.004
Chollet GLandais RHueber TBredin HMokbel CPerrot PZouari L(2007)Some experiments in audio-visual speech processingProceedings of the 2007 international conference on Advances in nonlinear speech processing10.5555/1784829.1784832(28-56)Online publication date: 22-May-2007
https://dl.acm.org/doi/10.5555/1784829.1784832
Chollet GLandais RHueber TBredin HMokbel CPerrot PZouari L(2007)Some Experiments in Audio-Visual Speech ProcessingAdvances in Nonlinear Speech Processing10.1007/978-3-540-77347-4_2(28-56)Online publication date: 2007
https://doi.org/10.1007/978-3-540-77347-4_2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Unvoiced speech recognition using tissue-conductive acoustic sensor

Multilingual recognition of non-native speech using acoustic model transformation and pronunciation modeling

Acoustic model adaptation based on pronunciation variability analysis for non-native speech recognition

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations