Abstract
The ability to communicate with others is of paramount importance for mental well-being. In this paper, we describe an interaction system to reduce communication barriers for people with severe speech and physical impairments (SSPI) such as cerebral palsy. The system consists of two main components: (i) the head-mounted human-computer interaction (HCI) part consisting of smart glasses with gaze trackers and text-to-speech functionality (which implement a communication board and the selection tool), and (ii) a natural language processing pipeline in the backend in order to generate complete sentences from the symbols on the board. We developed the components to provide a smooth interaction between the user and the system thereby including gaze tracking, symbol selection, symbol recognition, and sentence generation. Our results suggest that such systems can dramatically increase communication efficiency of people with SSPI.
Author to whom correspondence should be addressed. This work was carried out as part of the EITKIC 12-1-2012-0001 project, which is supported by the Hungarian Government, managed by the National Development Agency, financed by the Research and Technology Innovation Fund and was performed in cooperation with the EIT ICT Labs Budapest Associate Partner Group.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Blackstone, S.W., Dowden, P., Berg, M.H., Soto, G., Kingsbury, E., Wrenn, M., Liborin, N.: Augmented communicators and their communication partners: a paradigm for successful outcomes. In: CSUN Technology and Persons With Disabilities Conference 2001 (2001)
Pennington, L., Goldbart, J., Marshall, J.: Interaction training for conversational partners of children with cerebral palsy: a systematic review. Int. J. Lang. Comm. Dis. 39(2), 151–170 (2004)
Wiegand, K., Patel, R.: Non-syntactic word prediction for AAC. In: SPLAT 2012, pp. 28–36 (2012)
Arnott, J.L., Alm, N.: Towards the improvement of augmentative and alternative communication through the modelling of conversation. Comput. Speech Lang. 27(6), 1194–1211 (2013)
McCoy, K.F., Pennington, C.A., Badman, A.L.: Compansion: from research prototype to practical integration. Nat. Lang. Eng. 4(1), 73–95 (1998)
Karberis, G., Kouroupetroglou, G.: Transforming spontaneous telegraphic language to well-formed greek sentences for alternative and augmentative communication. In: Vlahavas, I.P., Spyropoulos, C.D. (eds.) SETN 2002. LNCS (LNAI), vol. 2308, pp. 155–166. Springer, Heidelberg (2002)
Patel, R., Pilato, S., Roy, D.: Beyond linear syntax: an image-oriented communication aid. Assistive Technol. Outcomes Benefits 1(1), 57–66 (2004)
ASHA: Augmentative and Alternative Communication Decisions. http://www.asha.org/public/speech/disorders/CommunicationDecisions/
Calvo, A., Chiò, A., Castellina, E., Corno, F., Farinetti, L., Ghiglione, P., Pasian, V., Vignola, A.: Eye tracking impact on quality-of-life of ALS patients. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 70–77. Springer, Heidelberg (2008)
Lőrincz, A., Takács, D.: AGI architecture measures human parameters and optimizes human performance. In: Schmidhuber, J., Thórisson, K.R., Looks, M. (eds.) AGI 2011. LNCS, vol. 6830, pp. 321–326. Springer, Heidelberg (2011)
Gajos, K.Z., Wobbrock, J.O., Weld, D.S.: Improving the performance of motor-impaired users with automatically-generated, ability-based interfaces. In: CHI 2008, pp. 1257–1266 (2008)
Zhang, W.: State-Space Search: Algorithms, Complexity, Extensions, and Applications. Springer, New York (1999)
Goldberg, Y., Orwant, J.: A dataset of syntactic-ngrams over time from a very large corpus of English Books. In: SEM 2013, vol. 1, pp. 241–247
Tiedemann, J.: Parallel data tools and interfaces in OPUS. In: LREC 2012, pp. 23–25 (2012)
Sonntag, D., Zillner, S., Schulz, C., Weber, M., Toyama, T.: Towards medical cyber-physical systems: multimodal augmented reality for doctors and knowledge discovery about patients. In: Marcus, A. (ed.) DUXU 2013, Part III. LNCS, vol. 8014, pp. 401–410. Springer, Heidelberg (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Vörös, G. et al. (2014). Towards a Smart Wearable Tool to Enable People with SSPI to Communicate by Sentence Fragments. In: Cipresso, P., Matic, A., Lopez, G. (eds) Pervasive Computing Paradigms for Mental Health. MindCare 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 100. Springer, Cham. https://doi.org/10.1007/978-3-319-11564-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-11564-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11563-4
Online ISBN: 978-3-319-11564-1
eBook Packages: Computer ScienceComputer Science (R0)