More Web Proxy on the site http://driver.im/

research-article

A Virtual Conversational Agent for Teens with Autism Spectrum Disorder: Experimental Results and Design Lessons

Authors:

Mohammad Rafayet Ali,

Seyedeh Zahra Razavi,

Raina Langevin,

Abdullah Al Mamun,

Reza Rawassizadeh,

Lenhart K. Schubert,

Ehsan HoqueAuthors Info & Claims

IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

Article No.: 2, Pages 1 - 8

https://doi.org/10.1145/3383652.3423900

Published: 19 October 2020 Publication History

Abstract

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in realtime, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably:

1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations.

2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility.

References

[1]

J. Baio, L. Wiggins, D. Christensen, M. Maenner, J. Daniels, et al. Prevalence of autism spectrum disorder among children aged 8 years---autism and developmental disabilities monitoring network. MMWR Surveillance Summaries, 2018.

[2]

A. Georgescu, B. Kuzmanovic, D. Roth, G. Bente, and K. Vogeley. The use of virtual characters to assess and train non-verbal communication in high-functioning autism. Frontiers in human neuroscience, 8:807, 2014.

[3]

S. Ozonoff, A. Iosif, F. Baguio, I. Cook, M. Hill, T. Hutman, et al. A prospective study of the emergence of early behavioral signs of autism. Journal of the American Academy of Child & Adolescent Psychiatry, 49(3):256--266, 2010.

[4]

A. Tartaro and J. Cassell. Playing with virtual peers: bootstrapping contingent discourse in children with autism. In ICLS (2). Citeseer, 2008.

[5]

M. Foster, K. Avramides, S. Bernardini, J. Chen, C. Frauenberger, et al. Supporting children's social communication skills through interactive narratives with virtual characters. In ACM International Conference on Multimedia. ACM, 2010.

[6]

S. Bernardini, K. Porayska-Pomsta, and H. Sampath. Designing an intelligent virtual agent for social communication in autism. In AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE'13, 2014.

[7]

M. Milne, D. Powers, and R. Leibbrandt. Development of a software-based social tutor for children with autism spectrum disorders. In Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7, OZCHI '09, 2009.

Digital Library

[8]

E. Mower, M. Black, E. Flores, M. Williams, and S. Narayanan. Rachel: Design of an emotionally targeted interactive agent for children with autism. In IEEE International Conference on Multimedia and Expo, 2011.

Digital Library

[9]

F. Boujarwah, G. Abowd, and R. Arriaga. Socially computed scripts to support social problem solving skills. In SIGCHI Conference on Human Factors in Computing Systems. ACM, 2012.

Digital Library

[10]

D. DeVault, R. Artstein, G. Benn, T. Dey, E. Fast, et al. Simsensei kiosk: A virtual human interviewer for healthcare decision support. In International conference on Autonomous agents and multi-agent systems, 2014.

[11]

I. Hopkins, M. Gower, T. Perez, D. Smith, F. Amthor, et al. Avatar assistant: improving social skills in students with an asd through a computer-based intervention. Journal of autism and developmental disorders, 2011.

[12]

S. Razavi, M. Ali, L. Smith, T.and Schubert, and M. Hoque. The lissa virtual human and asd teens: An overview of initial experiments. In International Conference on Intelligent Virtual Agents. Springer, 2016.

[13]

A. Miner, A. Milstein, S. Schueller, R. Hegde, C. Mangurian, et al. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Internal Medicine, 2016.

[14]

K. Fitzpatrick, A. Darcy, and M. Vierhile. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (woebot): a randomized controlled trial. JMIR Mental Health, 2017.

[15]

S. Razavi, L. Schubert, M. Ali, and M. Hoque. Managing casual spoken dialogue using flexible schemas, pattern transduction trees, and gist clauses. Annual Conference on Advances in Cognitive Systems (ACS), 2017.

[16]

F. Morbini, E. Forbell, D. DeVault, K. Sagae, D. Traum, and A. Rizzo. A mixed-initiative conversational dialogue system for healthcare. In Annual meeting of the special interest group on discourse and dialogue, 2012.

Digital Library

[17]

H. Tanaka, H. Negoro, H. Iwasaka, and S. Nakamura. Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders. PloS one, 2017.

[18]

I. Serban, T. Klinger, G. Tesauro, K. Talamadupula, B. Zhou, et al. Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation. 2016.

[19]

P. Henderson, K. Sinha, N. Angelard-Gontier, N. Ke, G. Fried, et al. Ethical Challenges in Data-Driven Dialogue Systems. 2017.

[20]

R. G. Heimberg. Cognitive-behavioral therapy for social anxiety disorder: Current status and future directions. Biological Psychiatry, 2002.

[21]

S. Siegman, A. W. Feldstein. Nonverbal Behavior and Communication. Psychology Press, 2014.

[22]

M. Ali, D. Crasta, L. Jin, A. Baretto, J. Pachter, et al. LISSA- Live Interactive Social Skill Assistance. In Affective Computing and Intelligent Interaction (ACII), 2015.

Digital Library

[23]

T. Baltru and P. Robinson. OpenFace:an open source facial behavior analysis toolkit.

[24]

P. Boersma and D. Weenink. Praat: doing phonetics by computer.

[25]

M. R. Ali, K. V. Orden, K. Parkhurst, S. Liu, V. Nguyen, et al. Aging and engaging: A social conversational skills training program for older adults. In Intelligent User Interfaces, pages 55--66, 2018.

[26]

P. Chung and C. Liu. A daily behavior enabled hidden Markov model for human behavior understanding. Pattern Recognition, 2008.

[27]

NUANCE: Speech Recognition Solutions.

[28]

H. B. Mann and D. R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Statist., 18(1):50--60, 03 1947.

[29]

R. A. Armstrong. When to use the Bonferroni correction, 2014.

[30]

G. Guest, K. MacQueen, and E. Namey. Introduction to applied thematic analysis. Applied Thematic Analysis, 2012.

[31]

T. McCurdie, S. Taneva, M. Casselman, M. Yeung, C. McDaniel, et al. mHealth consumer apps: the case for user-centered design. Association for the Advancement of Medical Instrumentation, 2012.

[32]

M. Kinzie, W. Cohn, M. Julian, and W. Knaus. A user-centered model for web site design: needs assessment, user interface design, and rapid prototyping. Journal of American Med Info Assoc, 2002.

[33]

J. Mintz. Additional key factors mediating the use of a mobile technology tool designed to develop social and life skills in children with Autism Spectrum Disorders: Evaluation of the 2nd HANDS prototype. Computers and Education, 2013.

[34]

A. Zeeland, M. Dapretto, D. Ghahremani, R. Poldrack, and S. Bookheimer. Reward processing in autism. Autism Research, 2010.

[35]

C. Kasari, M. Sigman, P. Mundy, and N. Yirmiya. Caregiver interactions with autistic children. Journal of Abnormal Child Psychology, 1988.

[36]

D. Chattopadhyay and K. MacDorman. Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision, 2016.

[37]

P. Kenny, A. Hartholt, J. Gratch, W. Swartout, D. Traum, et al. Building Interactive Virtual Humans for Training Environments. (I/ITSEC), 2007.

Cited By

Liu JYu CGao JXie YLiao QWu YWang YDastani MSichman JAlechina NDignum V(2024)LLM-Powered Hierarchical Language Agent for Real-time Human-AI CoordinationProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662979(1219-1228)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662979
Feijóo-García PWrenn CGomes de Siqueira AGhosh RStuart JYao HLok B(2024)Exploring the Effects of User-Agent and User-Designer Similarity in Virtual Human Design to Promote Mental Health Intentions for College StudentsACM Transactions on Applied Perception10.1145/368982222:1(1-41)Online publication date: 28-Nov-2024
https://dl.acm.org/doi/10.1145/3689822
Feijóo-García PWrenn CKalogeras SPayne CLok BOmojokun O(2024)Effects of Gender Synchrony in User-Agent Interactions: Integrating the Designer as a Product Cue in Virtual Human Design for Mental Health SupportProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688326(123-131)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3688326
Show More Cited By

Index Terms

A Virtual Conversational Agent for Teens with Autism Spectrum Disorder: Experimental Results and Design Lessons
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Exploring the Use of a Voice-based Conversational Agent to Empower Adolescents with Autism Spectrum Disorder
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Voice-based Conversational Agents (VCA) have served as personal assistants that support individuals with special needs. Adolescents with Autism Spectrum Disorder (ASD) may also benefit from VCAs to deal with their everyday needs and challenges, ranging ...
Discourse Behavior of Older Adults Interacting with a Dialogue Agent Competent in Multiple Topics
We present a conversational agent designed to provide realistic conversational practice to older adults at risk of isolation or social anxiety, and show the results of a content analysis on a corpus of data collected from experiments with elderly patients ...
Multidemensionality of Caregiver Burden in Parents of Children with Autism Spectrum Disorder

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IVA '20: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents

October 2020

394 pages

ISBN:9781450375863

DOI:10.1145/3383652

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IVA '20

Sponsor:

SIGAI

IVA '20: ACM International Conference on Intelligent Virtual Agents

October 20 - 22, 2020

Scotland, Virtual Event, UK

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
958
Total Downloads

Downloads (Last 12 months)223
Downloads (Last 6 weeks)29

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu JYu CGao JXie YLiao QWu YWang YDastani MSichman JAlechina NDignum V(2024)LLM-Powered Hierarchical Language Agent for Real-time Human-AI CoordinationProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662979(1219-1228)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662979
Feijóo-García PWrenn CGomes de Siqueira AGhosh RStuart JYao HLok B(2024)Exploring the Effects of User-Agent and User-Designer Similarity in Virtual Human Design to Promote Mental Health Intentions for College StudentsACM Transactions on Applied Perception10.1145/368982222:1(1-41)Online publication date: 28-Nov-2024
https://dl.acm.org/doi/10.1145/3689822
Feijóo-García PWrenn CKalogeras SPayne CLok BOmojokun O(2024)Effects of Gender Synchrony in User-Agent Interactions: Integrating the Designer as a Product Cue in Virtual Human Design for Mental Health SupportProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3688326(123-131)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3688326
Rashik MJasim MKucher KSarvghad AMahyar N(2024)Beyond Text and Speech in Conversational Agents: Mapping the Design Space of AvatarsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661563(1875-1894)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661563
Srivastava STheune MCatala AReed CBoratto LGena CMarras MGermanakos PPopescus E(2024)Trust in a Human-Computer Collaborative Task With or Without Lexical AlignmentAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664868(189-194)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3664868
Tang YChen LChen ZChen WCai YDu YYang FSun L(2024)EmoEden: Applying Generative Artificial Intelligence to Emotional Learning for Children with High-Function AutismProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642899(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642899
Dolgikh ASamsonovich ATikhomirova D(2024)Social–Emotional Conversational Agents Based on Cognitive Architectures and Machine LearningPattern Recognition and Image Analysis10.1134/S105466182470064034:3(765-772)Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1134/S1054661824700640
Garzotto FGianotti MPatti APentimalli FVona F(2024)Empowering Persons with Autism Through Cross-Reality and Conversational AgentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337211030:5(2591-2601)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372110
Poglitsch CPirker J(2024)A Qualitative Investigation to Design Empathetic Agents as Conversation Partners for People with Autism Spectrum Disorder2024 IEEE Conference on Games (CoG)10.1109/CoG60054.2024.10645560(1-4)Online publication date: 5-Aug-2024
https://doi.org/10.1109/CoG60054.2024.10645560
Xygkou ASiriaraya PShe WCovaci AAng C(2024)“Can I be More Social with a Chatbot?”: Social Connectedness Through Interactions of Autistic Adults with a Conversational Virtual HumanInternational Journal of Human–Computer Interaction10.1080/10447318.2023.229288040:24(8937-8954)Online publication date: 4-Jan-2024
https://doi.org/10.1080/10447318.2023.2292880
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents