More Web Proxy on the site http://driver.im/

research-article

Speech and Gestures for Smart-Home Control and Interaction for Older Adults

Authors:

Jeannie S.A. LeeAuthors Info & Claims

HealthMedia'18: Proceedings of the 3rd International Workshop on Multimedia for Personal Health and Health Care

Pages 49 - 57

https://doi.org/10.1145/3264996.3265002

Published: 15 October 2018 Publication History

Abstract

Older adults have been encountering difficulties in using modern technological devices to control home appliances as they are lacking in technology literacy and mobility. This led to the usage of remote controllers or requiring assistance from family members, which is not beneficial for older adults since there is less independence. To alleviate this problem, this project aims to develop a prototype system named "Genie" which caters for older adults ranging from 65 to 80 years old, allowing for easy control of smart home appliances through combination of speech and gesture interactions. An experiment was carried out with a total of 20 older adults on the prototype system where the initial results demonstrate a significant increase in usability. Based on the evaluation, such interaction methods show promise to be effective in replacing manual operations of home appliances through the use of simple speech or gesture commands.

References

[1]

Amazon. 2015. Amazon.com Help: Set Up Your Amazon Echo. https://www.amazon.com/gp/help/customer/display.html?nodeId=201601770 . Accessed: 05.11.2017.

[2]

Mohammad Hamdaqa Atli F. Einarsson, Patrekur Patreksson. 2017. SmartHomeML: Towards a Domain-Specific Modeling Language for Creating Smart Home Applications. 2017 IEEE International Congress on Internet of Things (ICIOT) (2017), 82--88.

[3]

Dieter Bohn. 2016. Google Home is smart, loud, and kind of cute. https://www.theverge.com/2016/10/4/13156676/google-home-assistant-speaker-photos-video-device-hands-on . Accessed: 09.11.2017.

[4]

Dieter Bohn. 2017. You can finally say 'Computer' to your Echo to command it. https://www.theverge.com/tldr/2017/1/23/14365338/amazon-echo-alexa-computer-wake-word-star-trek . Accessed: 05.11.2017.

[5]

Richard A. Bolt. 1980. Put-That-There: Voice and Gesture at the Graphics Interface. SIGGRAPH Comput. Graph., Vol. 14, 3 (July 1980), 262--270.

Digital Library

[6]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology, Vol. 3, 2 (2006), 77--101.

[7]

John Brooke. 2013. SUS: A Retrospective., bibinfonumpages29 - 40 pages. http://uxpajournal.org/sus-a-retrospective/

Digital Library

[8]

Chris Burke. {n. d.}. The Decibel Level of Normal Speech. https://classroom.synonym.com/decibel-level-normal-speech-8599569.html . Accessed: 05.11.2017.

[9]

Wendy A. RogersArthur D. FiskSherry E. MeadNeff WalkerElizabeth Fraser Cabrera. 1996. Training older adults to use automatic teller machines. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 38, 3, 425--433.

[10]

Juliana Chan. 2017. Ageing isn't a tsunami. In SMU Office of Research and Tech Transfer . https://research.smu.edu.sg/news/smuresearch/2016/12/20/ageing-isnt-tsunami

[11]

Hartmut WandkeLucienne Christian Stobel and WandkeLucienne Blessing. 2010. Gestural Interfaces for Elderly Users: Help or Hindrance?. In Gesture in Embodied Communication and Human-Computer Interaction, Stefan Kopp and Ipke Wachsmuth (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 269--280.

Digital Library

[12]

G. Clauser and Brent Butterworth. 2010. Is the Google Home the Voice-Controlled Speaker for You? https://thewirecutter.com/reviews/google-home-voice-controlled-speaker/. Accessed: 01.10.2017.

[13]

Lacey Colligan, Henry W.W. Potts, Chelsea T. Finn, and Robert A. Sinkin. 2015. Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. International Journal of Medical Informatics, Vol. 84, 7 (2015), 469 -- 476.

[14]

John Donaldson, Catherine J.Smith, Balambigai Balakrishnan, Mumtaz Md. Kadir, and Sanushka Mudaliar. 2015. Elderly Population in Singapore: Understanding Social, Physical and Financial Needs .

[15]

Matthew Field. 2017. From Amazon Echo to Google Home, the best smart home devices for 2018. https://www.telegraph.co.uk/technology/0/amazon-echo-google-home-best-smart-home-devices-2018/. Accessed on 19.12.2017.

[16]

Kinect for Windows Team. 2014. The Kinect for Windows v2 sensor and free SDK 2.0 public preview are here. https://blogs.msdn.microsoft.com/kinectforwindows/2014/07/15/the-kinect-for-windows-v2-sensor-and-free-sdk-2-0-public-preview-are-here/. Accessed: 12.09.2017.

[17]

Kathrin Gerling, Ian Livingston, Lennart Nacke, and Regan Mandryk. 2012. Full-Body Motion-Based Game Interaction for Older Adults. In ACM, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems .

Digital Library

[18]

Google. 2017. Google Home Mini. https://store.google.com/'srp=/product/google_home_mini . Accessed on 10.11.2017.

[19]

Susan K. Gordon and W. Crawford Clark. 1974. Adult Age Differences in Word and Nonsense Syllable Recognition Memory and Response Criterion. A Journal of Gerontology, Vol. 29 (1974), 659 -- 665.

[20]

Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50, 9 (2006), 904--908.

[21]

Vijayan Sugumaran andMarvin Hubl Joerg Leukel. 2016. The Role of Application Domain Knowledge in Using OWL DL Diagrams: A Study of Inference and Problem-Solving Tasks. (2016).

[22]

J.Wu, Daqing Zhang, Zhaohui Wu, Yingchun Yang, and Shijian Li. 2010. GeeAir: a universal multimodal remote control device for home appliances. Personal and Ubiquitous Computing, Vol. 14, 8 (2010), 723--735.

Digital Library

[23]

Victoria KACHAR. 2003. Terceira idade e informa?tica: aprender revelando potencialidades. Sao Paulo: Cortez.

[24]

Thalmic Labs. 2013. MYO. https://www.thalmic.com . Accessed: 05.09.2017.

[25]

Leapmotion. 2015. Leap Motion. https://store-world.leapmotion.com/products/leap-motion-controller . Accessed on 05.09.2017.

[26]

Chiara Leonardi, Adriano Albertini, Fabio Pianesi, and Massimo Zancanaro. 2010. An Exploratory Study of a Touch-based Gestural Interface for Elderly. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI '10). ACM, New York, NY, USA, 845--850.

Digital Library

[27]

James R. Lewis and Jeff Sauro. 2009. The Factor Structure of the System Usability Scale. In Human Centered Design, Masaaki Kurosu (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 94--103.

Digital Library

[28]

Logitech. 2017. Logitech Harmony Hub. https://www.logitech.com/en-us/product/harmony-hub . (2017). Accessed on 05.09.2017.

[29]

Macedonia and Manuela. 2014. Bringing Back the Body into the Mind: Gestures Enhance Word Learning in Foreign Language. Frontiers in Psychology, Vol. 5 (Dec 2014).

[30]

J. Mark R. P. Thomas, Jens Ahrens and Ivan Tashev. 2012. Optimal 3D beamforming using measured microphone directivity patterns. Acoustic Signal Enhancement; Proceedings of IWAENC 2012; International Workshop, Mountain View, CA, USA.

[31]

W. Y. Mei and Teo Zhiwei. 2016. Technologies for Ageing-in-Place: The Singapore Context .

[32]

NASA. 1986. NASA Task Load Index. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20000021488.pdf

[33]

United Nations. 2017a. Department of Economic and Social Affairs, Population Division (2017). In World Population Ageing 2017 - Highlights (ST/ESA/SER.A/397) .

[34]

United Nations. 2017b. Global Issues: Ageing. In Ageing . http://www.un.org/en/sections/issues-depth/ageing/

[35]

Department of Statistics Singapore. 2017. Population Trends. In Singapore Management Univrsity. Lien Centre for Social Innovation .

[36]

Ollie. 2015. Designing For The Elderly: Ways Older People Use Digital Technology Differently. https://www.smashingmagazine.com/2015/02/designing-digital-technology-for-the-elderly/. Accessed on 10.10.2017.

[37]

KEVIN PARRISH. 2017. ZigBee, Z-Wave, Thread and WeMo: What's the Difference? https://www.tomsguide.com/us/smart-home-wireless-network-primer,news-21085.html . Accessed on 10.09.2017.

[38]

Philips. {n. d.} a. Philips Developers. https://developers.meethue.com . Accessed on 05.09.2017.

[39]

Philips. {n. d.} b. Philips Hue. https://www2.meethue.com/en-sg . Accessed on 05.09.2017.

[40]

Warren JE Rohrer JD, Knight WD. 2008. Word-finding difficulty: a clinical analysis of the progressive aphasias. Brain?: A Journal of Neurology, Vol. 131(Pt 1), 7 (2008), 8 -- 38.

[41]

Margaret Rouse and Matthew Haughn. 2017. Smart speaker. http://whatis.techtarget.com/definition/smart-speaker. Accessed on 05.11.2017.

[42]

Samsung. 2017a. Samsung Developer Portal. (2017). http://docs.smartthings.com/en/latest/getting-started/overview.html

[43]

Samsung. 2017b. Samsung Smarthings developer. https://developers.smartthings.com .

[44]

Samsung. 2017c. Samsung SmartThings. https://www.samsung.com/us/smart-home/smartthings/.

[45]

Patrick Sinclair. 2017. Samsung SmartThings Hub vs Logitech Harmony Hub: Which is Best? (2017). https://www.homerobotics.com/samsung-smartthings-hub-vs-logitech-harmony-hub/

[46]

Kinect's Windows Team. 2014. Set up Kinect for Windows v2 or an Xbox Kinect sensor with Kinect Adapter for Windows. (2014).

[47]

Y.Han, Jonghwan Hyun, and Taeyeol Jeong. 2016. A smart home control system based on context and human speech. In Advanced Communication Technology (ICACT), 2016 18th International Conference on, Pyeongchang, South Korea .

Cited By

Gallo SPaterno FMalizia A(2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627775
Constantin SEyiokur FYaman DBärmann LWaibel A(2023)Interactive Multimodal Robot Dialog Using Pointing Gesture RecognitionComputer Vision – ECCV 2022 Workshops10.1007/978-3-031-25075-0_43(640-657)Online publication date: 19-Feb-2023
https://doi.org/10.1007/978-3-031-25075-0_43
Harrington CMartin-Hammond ABray K(2022)Examining Identity as a Variable of Health Technology Research for Older Adults: A Systematic ReviewProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517621(1-24)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517621
Show More Cited By

Index Terms

Speech and Gestures for Smart-Home Control and Interaction for Older Adults
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces
    2. Interaction techniques
      1. Gestural input

Recommendations

CareHub: Smart Screen VUI and Home Appliances Control for Older Adults
ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

While voice user interfaces (VUIs) are becoming promising tools for users to control home appliances, the VUIs pose great challenges to the older adults. In this paper, I explored how to use multimedia interactions (i.e., voice+visual output) to enhance ...
Recruiting Older Adults in the Wild: Reflections on Challenges and Lessons Learned from Research Experience
PervasiveHealth '18: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare

It is important to understand the older adults prior to the design process. The understanding can better facilitate design conversations between the researchers and the older adults. In this paper, we discussed our experiences of building a relationship ...
Understanding Older Adults' Perceptions of In-Home Sensors Using an Obtrusiveness Framework
Proceedings, Part II, of the 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - Volume 9744

The aim of this study was to determine if dimensions and sub-categories of a previously-tested obtrusiveness framework were represented in interviews conducted with community-dwelling older adults at three- and six-month study visits during an in-home ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HealthMedia'18: Proceedings of the 3rd International Workshop on Multimedia for Personal Health and Health Care

October 2018

77 pages

ISBN:9781450359825

DOI:10.1145/3264996

Program Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Ramesh Jain
University of California, Irvine, USA
,
Noel E. O'Connor
Dublin City University, Ireland
,
Troy McDaniel
Arizona State University, USA
,
Jochen Meyer
OFFIS Institute for Information Technology, Germany

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22, 2018

Seoul, Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
456
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)1

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gallo SPaterno FMalizia A(2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627775
Constantin SEyiokur FYaman DBärmann LWaibel A(2023)Interactive Multimodal Robot Dialog Using Pointing Gesture RecognitionComputer Vision – ECCV 2022 Workshops10.1007/978-3-031-25075-0_43(640-657)Online publication date: 19-Feb-2023
https://doi.org/10.1007/978-3-031-25075-0_43
Harrington CMartin-Hammond ABray K(2022)Examining Identity as a Variable of Health Technology Research for Older Adults: A Systematic ReviewProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517621(1-24)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517621
Kim HKim HIm HIm H(2021)Interacting with Smart Speakers based on Finger Touch GesturesArchives of Design Research10.15187/adr.2021.11.34.4.15734:4(157-171)Online publication date: 30-Nov-2021
https://doi.org/10.15187/adr.2021.11.34.4.157
Abdi Sde Witte LHawley M(2020)Emerging Technologies With Potential Care and Support Applications for Older People: Review of Gray LiteratureJMIR Aging10.2196/172863:2(e17286)Online publication date: 11-Aug-2020
https://doi.org/10.2196/17286

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents