poster

Towards Improving Predictive AAC using Crowdsourced Dialogues and Partner Context

Author:

Keith VertanenAuthors Info & Claims

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

Pages 347 - 348

https://doi.org/10.1145/3132525.3134814

Published: 19 October 2017 Publication History

Get Access

Abstract

Augmentative and Alternative Communication (AAC) devices typically rely on a language model to help make predictions or disambiguate user input. We investigate how to improve predictions in two-sided conversational dialogues. We collect and share a new corpus of crowdsourced everyday dialogues. We show how language models based on recurrent neural networks outperform N-gram models on these dialogues. We demonstrate further gains are possible using text obtained from an AAC user's communication partner, even when that text is partial or contains errors.

References

[1]

C. Danescu-Niculescu-Mizil and L. Lee. Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs. In Proc. CMCL, 2011.

Digital Library

Google Scholar

[2]

A. Fiannaca, A. Paradiso, M. Shah, and M. R. Morris. AACrobat: Using mobile devices to lower communication barriers and provide autonomy with gaze-based AAC. In Proc. CSCW, 2017.

Digital Library

Google Scholar

[3]

S. K. Kane, M. R. Morris, A. Paradiso, and J. Campbell. "At times avuncular and cantankerous, with the reflexes of a mongoose": Understanding self-expression through augmentative and alternative communication devices. In Proc. CSCW, 2017.

Digital Library

Google Scholar

[4]

T. Mikolov, M. Karafiát, L. Burget, J. Cernock'y, and S. Khudanpur. Recurrent neural network based language model. In Proc. Interspeech, 2010.

Google Scholar

[5]

R. C. Moore and W. Lewis. Intelligent selection of language model training data. In Proc. ACL, 2010.

Digital Library

Google Scholar

[6]

K. Vertanen and P. O. Kristensson. The imagination of crowds: Conversational AAC language modeling using crowdsourcing and large data sources. In Proc. EMNLP, 2011.

Digital Library

Google Scholar

[7]

B. Wisenburn and D. J. Higginbotham. An AAC application using speaking partner speech recognition to automatically produce contextually relevant utterances: Objective results. Augmentative and Alternative Communication, 24(2).

Google Scholar

Cited By

View all

Fontana De Vargas MYu CShane HMoffatt K(2024)Co-Designing QuickPic: Automated Topic-Specific Communication Boards from Photographs for AAC-Based Language InstructionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642080(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642080
Cai SVenugopalan SSeaver KXiao XTomanek KJalasutram SMorris MKane SNarayanan AMacDonald RKornman EVance DCasey BGleason SNelson PBrenner M(2024)Using large language models to accelerate communication for eye gaze typing users with ALSNature Communications10.1038/s41467-024-53873-315:1Online publication date: 1-Nov-2024
https://doi.org/10.1038/s41467-024-53873-3
Valencia SCave RKallarackal KSeaver KTerry MKane S(2023)“The less I type, the better”: How AI Language Models can Enhance or Impede Communication for AAC UsersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581560(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581560
Show More Cited By

Index Terms

Towards Improving Predictive AAC using Crowdsourced Dialogues and Partner Context
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
2. Social and professional topics
  1. Professional topics
    1. Computing profession
      1. Assistive technologies

Recommendations

A Demonstration of a Tinkerable Augmentative and Alternative Communication Keyboard
IUI '23 Companion: Companion Proceedings of the 28th International Conference on Intelligent User Interfaces

Augmentative and alternative communication (AAC) provides communication assistance for nonspeaking individuals with motor disabilities. Literate AAC users typically rely on a keyboard interface coupled with a speech synthesizer to communicate with ...
The impact of alphabet supplementation and word prediction on sentence intelligiblity of electronically distorted speech

Alphabet supplementation is a low-tech augmentative and alternative communication (AAC) strategy that involves pointing to the first letter of each word spoken. Sentence intelligibility scores increased an average of 25% (Hanson et al., 2004) when ...
Topic modeling in fringe word prediction for AAC
IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces

Word prediction can be used for enhancing the communication ability of persons with speech and language impairments. In this work, we explore two methods of adapting a language model to the topic of conversation, and apply these methods to the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 2017

450 pages

ISBN:9781450349260

DOI:10.1145/3132525

General Chair:
Amy Hurst
University of Maryland, Baltimore County, USA
,
Program Chairs:
Leah Findlater
University of Washington, USA
,
Meredith Ringel Morris
Microsoft Research, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Check for updates

Author Tags

Qualifiers

Poster

Conference

ASSETS '17

Sponsor:

SIGACCESS

ASSETS '17: The 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 20 - November 1, 2017

Maryland, Baltimore, USA

Acceptance Rates

ASSETS '17 Paper Acceptance Rate 28 of 126 submissions, 22%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
166
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)6

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fontana De Vargas MYu CShane HMoffatt K(2024)Co-Designing QuickPic: Automated Topic-Specific Communication Boards from Photographs for AAC-Based Language InstructionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642080(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642080
Cai SVenugopalan SSeaver KXiao XTomanek KJalasutram SMorris MKane SNarayanan AMacDonald RKornman EVance DCasey BGleason SNelson PBrenner M(2024)Using large language models to accelerate communication for eye gaze typing users with ALSNature Communications10.1038/s41467-024-53873-315:1Online publication date: 1-Nov-2024
https://doi.org/10.1038/s41467-024-53873-3
Valencia SCave RKallarackal KSeaver KTerry MKane S(2023)“The less I type, the better”: How AI Language Models can Enhance or Impede Communication for AAC UsersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581560(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581560
Sun XDing JDong YMa XWang RJin KZhang HZhang Y(2022)A Survey of Technologies Facilitating Home and Community-Based Stroke RehabilitationInternational Journal of Human–Computer Interaction10.1080/10447318.2022.205054539:5(1016-1042)Online publication date: 20-Apr-2022
https://doi.org/10.1080/10447318.2022.2050545

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A Demonstration of a Tinkerable Augmentative and Alternative Communication Keyboard

The impact of alphabet supplementation and word prediction on sentence intelligiblity of electronically distorted speech

Topic modeling in fringe word prediction for AAC