More Web Proxy on the site http://driver.im/

research-article

"What can i help you with?": infrequent users' experiences of intelligent personal assistants

Authors:

Benjamin R. Cowan,

Kellie Morrissey,

Sara Al-Shehri,

Natasha BandeiraAuthors Info & Claims

MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

Article No.: 43, Pages 1 - 12

https://doi.org/10.1145/3098279.3098539

Published: 04 September 2017 Publication History

Abstract

Intelligent Personal Assistants (IPAs) are widely available on devices such as smartphones. However, most people do not use them regularly. Previous research has studied the experiences of frequent IPA users. Using qualitative methods we explore the experience of infrequent users: people who have tried IPAs, but choose not to use them regularly. Unsurprisingly infrequent users share some of the experiences of frequent users, e.g. frustration at limitations on fully hands-free interaction. Significant points of contrast and previously unidentified concerns also emerge. Cultural norms and social embarrassment take on added significance for infrequent users. Humanness of IPAs sparked comparisons with human assistants, juxtaposing their limitations. Most importantly, significant concerns emerged around privacy, monetization, data permanency and transparency. Drawing on these findings we discuss key challenges, including: designing for interruptability; reconsideration of the human metaphor; issues of trust and data ownership. Addressing these challenges may lead to more widespread IPA use.

References

[1]

M. P. Aylett, P. O. Kristensson, S. Whittaker, and Y. Vazquez-Alvarez. 2014. None of a CHInd: Relationship Counselling for HCI and Speech Technology. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14), 749--760.

Digital Library

[2]

M. P. Aylett and S. Lawson. 2016. The Smartphone: A Lacanian Stain, A Tech Killer, and an Embodiment of Radical Individualism. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 501--511.

Digital Library

[3]

C. Baber. Developing interactive speech technology. Interactive speech technology: 1--18.

Digital Library

[4]

L. Bell and J. Gustafson. 1999. Interaction with an animated agent in a spoken dialogue system. In Proceedings of the Sixth European Conference on Speech Communication and Technology, 1143--1146.

[5]

L. Bell, J. Gustafson, and M. Heldner. 2003. Prosodic adaptation in human-computer interaction. In Proceedings of ICPhS 2003, 2453--2456.

[6]

H. P. Branigan, M. J. Pickering, J. M. Pearson, J. F. McLean, and A. Brown. 2011. The role of beliefs in lexical alignment: Evidence from dialogs with humans and computers. Cognition 121, 1: 41--57.

[7]

V. Braun and V. Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2: 77--101.

[8]

J. Cassell. 2007. Body Language: Lessons from the Near-Human. In Genesis Redux: Essays on the History and Philosophy of Artificial Life. University of Chicago Press, Chicago, 346--374.

[9]

W. Chan, N. Jaitly, Q. Le, and O. Vinyals. 2016. Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4960--4964.

[10]

H. H. Clark. 1996. Using Language. Cambridge University Press.

[11]

P. Cohen, A Cheyer, E. Horvitz, R. El Kaliouby, and S. Whittaker. 2016. On the Future of Personal Assistants. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 1032--1037.

Digital Library

[12]

E. Cooper, A. Chang, Y. Levitan, and J. Hirschberg. 2016. Data Selection and Adaptation for Naturalness in HMM-based Speech Synthesis. Proceedings of Interspeech 2016, 357--361.

[13]

B. R. Cowan and H. P. Branigan. 2015. Does voice anthropomorphism affect lexical alignment in speech-based human- computer dialogue? In Proceedings of Interspeech 2015, 155--159.

[14]

B. R. Cowan, H. P. Branigan, H. Begum, L. McKenna, and E. Szekely. 2017. They Know as Much as We Do: Knowledge Estimation and Partner Modelling of Artificial Partners. In Proceedings of the Annual Meeting of the Cognitive Science Society.

[15]

B. R. Cowan, H. P. Branigan, E. Bugis, M. Obregon, and R. Beale. 2015. Voice anthropomorphism, interlocutor modelling and alignment effects on syntactic alignment in human-computer dialogue. International Journal of Human-Computer Studies 83: 27--42.

Digital Library

[16]

B. R. Cowan, D. Gannon, J. Walsh, J. Kinneen, E. O'Keefe, and L. Xie. 2016. Towards Understanding How Speech Output Affects Navigation System Credibility. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 2805--2812.

Digital Library

[17]

J. Edlund, J. Gustafson, M. Heldner, and A. Hjalmarsson. 2008. Towards human-like spoken dialogue systems. Speech Communication 50, 8--9: 630--645.

Digital Library

[18]

Factor1. 2016. Voice Assistant Anyone? Yes please, but not in public! Creative Strategies, Inc. Retrieved August 26, 2016 from http://creativestrategies.com/voice-assistant-anyone-yes-please-but-not-in-public/

[19]

E. Gilmartin, F. Bonin, L. Cerrato, C. Vogel, and N. Campbell. 2015. What's the Game and Who's Got the Ball? Genre in Spoken Interaction. In 2015 AAAI Spring Symposium Series.

[20]

E Gilmartin and N. Campbell. 2014. More Than Just Words: Building a Chatty Robot. In Natural Interaction with Robots, Knowbots and Smartphones, Joseph Mariani, Sophie Rosset, Martine Garnier-Rizet and Laurence Devillers (eds.). Springer New York, 179--185.

[21]

E. Goffman. 1971. Relations in public: microstudies of the public order. Basic Books.

[22]

J. Hauswald, M. A. Laurenzano, Y. Zhang, C. Li, A. Rovinski, A. Khurana, R. G. Dreslinski, T. Mudge, V. Petrucci, L. Tang, and J. Mars. 2015. Sirius: An Open End-to-End Voice and Vision Personal Assistant and Its Implications for Future Warehouse Scale Computers. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '15), 223--238.

Digital Library

[23]

K.S. Hone and C. Baber. 2001. Designing habitable dialogues for speech-based interaction with computers. International Journal of Human-Computer Studies 54, 4: 637--662.

Digital Library

[24]

L. Humphreys. 2005. Cellphones in public: social interactions in a wireless era. New Media & Society 7, 6: 810--833.

[25]

M. Ito. 2005. Mobile Phones, Japanese Youth, and the Re-placement of Social Contact. In Mobile Communications. Springer London, 131--148.

[26]

K. Jokinen and M. McTear. 2009. Spoken Dialogue Systems. Synthesis Lectures on Human Language Technologies 2, 1: 1--151.

[27]

M. Jöst, J. Häußler, M. Merdes, and R. Malaka. 2005. Multimodal Interaction for Pedestrians: An Evaluation Study. In Proceedings of the 10th International Conference on Intelligent User Interfaces (IUI '05), 59--66.

Digital Library

[28]

Kamitis. 2016. Intelligent Personal Assistant-Products, Technologies and Market: 2017-2022.

[29]

James E. Katz and Mark Aakhus. 2002. Perpetual Contact: Mobile Communication, Private Talk, Public Performance. Cambridge University Press.

Digital Library

[30]

A. Kennedy, A. Wilkes, L. Elder, and W. S. Murray. 1988. Dialogue with machines. Cognition 30, 1: 37--72.

[31]

J. Kitzinger. 1995. Qualitative research. Introducing focus groups. BMJ (Clinical research ed.) 311, 7000: 299--302.

[32]

P. Lison and R. Meena. 2014. Spoken Dialogue Systems: The New Frontier in Human-computer Interaction. XRDS 21, 1: 46--51.

Digital Library

[33]

S. Lovato and A. Piper. 2015. "Siri, is This You?": Understanding Young Children's Interactions with Voice Input Systems. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15), 335--338.

Digital Library

[34]

E. Luger and A. Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 5286--5297.

Digital Library

[35]

P. Milhorat, S. Schlögl, G. Chollet, J. Boudy, A. Esposito, and G. Pelosi. 2014. Building the next generation of personal digital Assistants. In 2014 1st International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), 458--463.

[36]

A. E. Moorthy and K. L. Vu. 2014. Voice Activated Personal Assistant: Acceptability of Use in the Public Space. In Human Interface and the Management of Information. Information and Knowledge in Applications and Services, Sakae Yamamoto (ed.). Springer International Publishing, 324--334.

Digital Library

[37]

A. E. Moorthy and K. L. Vu. 2015. Privacy Concerns for Use of Voice Activated Personal Assistant in the Public Space. International Journal of Human-Computer Interaction 31, 4: 307--335.

[38]

J. P. Olive. 1999. The voice user interface. In Global Telecommunications Conference, 1999. GLOBECOM '99, 2051--2055 vol.4.

[39]

S. Oviatt, J. Bernard, and G. A. Levow. 1998. Linguistic adaptations during spoken and multimodal error resolution. Language and speech 41 (Pt 3-4): 419--442.

[40]

S. Oviatt, M. MacEachern, and G. Levow. 1998. Predicting hyperarticulate speech during human-computer error resolution. Speech Communication 24, 2: 87--110.

Digital Library

[41]

T. A. Rodden, J. E. Fischer, N. Pantidi, K. Bachour, and S. Moran. 2013. At Home with Agents: Exploring Attitudes Towards Future Smart Energy Infrastructures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1173--1182.

Digital Library

[42]

M. Russell, S. D'Arcy, and L. Qun. 2007. The Effects of Bandwidth Reduction on Human and Computer Recognition of Children's Speech. IEEE Signal Processing Letters 14, 12: 1044--1046.

[43]

M. J. Russell and S. D'Arcy. 2007. Challenges for computer recognition of children's speech. In SLaTE, 108--111.

[44]

B. Shneiderman. 2000. The limits of speech recognition. Communications of the ACM 43, 9: 63--65.

Digital Library

[45]

L. Wulf, M. Garschall, J. Himmelsbach, and M. Tscheligi. 2014. Hands Free - Care Free: Elderly People Taking Advantage of Speech-only Interaction. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14), 203--206.

Digital Library

[46]

98% of iPhone users have tried Siri, but most don't use it regularly - Business Insider. Retrieved September 7, 2016 from http://uk.businessinsider.com/98-of-iphone-users-have-tried-siri-but-most-dont-use-it-regularly-2016-6?r=US&IR=T

[47]

Here's How Siri Will Work In MacOS Sierra | Digital Trends. Retrieved September 20, 2016 from http://www.digitaltrends.com/computing/siri-macos-sierra-features-interface/

[48]

The Voice UI has Gone Mainstream | Tech.pinions - Perspective, Insight, Analysis. Retrieved February 2, 2017 from https://techpinions.com/the-voice-ui-has-gone-mainstream/46148

Cited By

Shin JKo HBae SKim J(2025)An Exploratory Study on the Impacts of Voice-Based Conversational Agents with Proactive Interactions in the Driving ContextInternational Journal of Human–Computer Interaction10.1080/10447318.2024.244153041:6(3708-3723)Online publication date: 10-Jan-2025
https://doi.org/10.1080/10447318.2024.2441530
Zhong SGatti EHardwick JRibul MCho YObrist M(2025) LLM-mediated domain-specific voice agents: the case of TextileBot Behaviour & Information Technology10.1080/0144929X.2025.2456667(1-33)Online publication date: 3-Feb-2025
https://doi.org/10.1080/0144929X.2025.2456667
Gessinger ISeaborn KSteeds MCowan B(2025)ChatGPT and meInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103400194:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103400
Show More Cited By

Index Terms

"What can i help you with?": infrequent users' experiences of intelligent personal assistants
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Sound-based input / output
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Sound-based input / output

Recommendations

"Accessibility Came by Accident": Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

From an accessibility perspective, voice-controlled, home-based intelligent personal assistants (IPAs) have the potential to greatly expand speech interaction beyond dictation and screen reader output. To examine the accessibility of off-the-shelf IPAs (...
How to Remember What to Remember: Exploring Possibilities for Digital Reminder Systems

Digital reminder systems typically use time and place as triggers to remind people to perform activities. In this paper, we investigate how digital reminder systems could better support the process of remembering in a wider range of situations. We report ...
Finally Johnny Can Encrypt: But Does This Make Him Feel More Secure?
ARES '18: Proceedings of the 13th International Conference on Availability, Reliability and Security

End-to-end (E2E) encryption is an effective measure against privacy infringement. In 2016, it was introduced by WhatsApp for all users (of the latest app version) quasi overnight. However, it is unclear how non-expert users perceived this change, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 2017

874 pages

ISBN:9781450350754

DOI:10.1145/3098279

General Chairs:
Matt Jones
Swansea University
,
Manfred Tscheligi
University of Salzburg and AIT
,
Program Chairs:
Yvonne Rogers
University College London
,
Roderick Murray-Smith
University of Glasgow

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 September 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

H.W. Wilson Foundation Scholarship
UCD Seed Funding Scheme

Conference

MobileHCI '17

Sponsor:

SIGCHI

MobileHCI '17: 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

September 4 - 7, 2017

Vienna, Austria

Acceptance Rates

MobileHCI '17 Paper Acceptance Rate 45 of 224 submissions, 20%;

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

277
Total Citations
View Citations
3,758
Total Downloads

Downloads (Last 12 months)256
Downloads (Last 6 weeks)25

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shin JKo HBae SKim J(2025)An Exploratory Study on the Impacts of Voice-Based Conversational Agents with Proactive Interactions in the Driving ContextInternational Journal of Human–Computer Interaction10.1080/10447318.2024.244153041:6(3708-3723)Online publication date: 10-Jan-2025
https://doi.org/10.1080/10447318.2024.2441530
Zhong SGatti EHardwick JRibul MCho YObrist M(2025) LLM-mediated domain-specific voice agents: the case of TextileBot Behaviour & Information Technology10.1080/0144929X.2025.2456667(1-33)Online publication date: 3-Feb-2025
https://doi.org/10.1080/0144929X.2025.2456667
Gessinger ISeaborn KSteeds MCowan B(2025)ChatGPT and meInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103400194:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103400
Cibrian FChen YAnderson KAbrahamsson CMotti V(2025)Limitations in speech recognition for young adults with down syndromeUniversal Access in the Information Society10.1007/s10209-025-01197-4Online publication date: 15-Feb-2025
https://doi.org/10.1007/s10209-025-01197-4
Lui MBernardes ROliveira MSumlin C(2024)Beyond usability: innovation in the digital age as an antidote to social isolationInternational Journal of Innovation10.5585/2024.2608812:3(e26088)Online publication date: 19-Nov-2024
https://doi.org/10.5585/2024.26088
Subaveerapandiyan AKataria SKarthiga SSundaram S(2024)Examining College Students' Adoption of Chatbots for Assignment AssistanceAI-Assisted Library Reconstruction10.4018/979-8-3693-2782-1.ch010(172-190)Online publication date: 28-Jun-2024
https://doi.org/10.4018/979-8-3693-2782-1.ch010
Doi TNishikawa Y(2024)Usability Analysis of Smart Speakers from a Learnability Perspective for Novel UsersApplied System Innovation10.3390/asi70300367:3(36)Online publication date: 25-Apr-2024
https://doi.org/10.3390/asi7030036
Vaľko DKipikaša JÁdám NKhorshidiyeh H(2024)Design and Implementation of a Cognitive Cloud Assistant: A Text Classification Model ApproachActa Electrotechnica et Informatica10.2478/aei-2024-000924:3(9-14)Online publication date: 19-Sep-2024
https://doi.org/10.2478/aei-2024-0009
Bosco CShojaei FTheisz AOsorio Torres JCureton BHimes AJessup NBarnes PLu YHendrie HHill CShih P(2024)Testing 3 Modalities (Voice Assistant, Chatbot, and Mobile App) to Assist Older African American and Black Adults in Seeking Information on Alzheimer Disease and Related Dementias: Wizard of Oz Usability StudyJMIR Formative Research10.2196/606508(e60650)Online publication date: 9-Dec-2024
https://doi.org/10.2196/60650
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten