Embodied Conversational Agents in Clinical Psychology: A Scoping Review

Background

Internet-based interventions can be effective in the treatment of various mental disorders compared with care-as-usual (eg, face-to-face treatment) and waiting list control groups [1]. Many interventions, especially those aimed at mood, anxiety, and substance use disorders, are based on cognitive behavioral therapy (CBT). These interventions can be unguided or guided, with guidance typically being provided by licensed health professionals or trained volunteers. Guided interventions are typically more clinically effective than unguided ones [2-4]. Their superiority is most likely explained by the interaction between the participant and the person providing guidance, and although concepts such as treatment adherence have been suggested as a working mechanism [5], that is, human support having a positive effect on adherence, and in turn on effectiveness, it remains unclear how exactly human support accounts for the difference. Thus, there is good reason to explore whether the gap between guided and unguided interventions can be bridged. In this paper we focus on a potential automated solution: Embodied conversational agents (ECAs).

Embodied Conversational Agents

ECAs can be defined as “more or less autonomous and intelligent software entities with an embodiment used to communicate with the user” [6]. Examples of real-world ECAs are interactive characters in video games and virtual characters that assist customers in Web stores. Conceptually, ECAs consist of three components [7]. The first is an application interface that allows users to communicate with the ECA and provide it with information. These interfaces can range from Web-based questionnaires to real-time audio and video input. Second, ECAs are endowed with computer models that give them their “mental” capacities, essentially programmed knowledge used to reason over the factual “observations” derived from the interface. Such models can range from concise decision trees in which different answers on a questionnaire lead to different responses by the ECA, to machine learning–based algorithms that classify real-time video and audio input into a user’s emotional state, allowing the ECA to react empathically. Third, ECAs have an embodiment, or visual representation, which allows them to communicate with users verbally or nonverbally. Embodiments can range from virtual human characters on computer screens to robots, and communication from text messages to human communication modalities such as speech, gestures, and facial expressions. There are advantages and disadvantages to whatever implementation of the design aspects is chosen. Highly advanced ECAs, for example, those using multimodal and real-time user input such as video recordings and natural language, can be more believable than simplistic ones, but their complexity means that they require more development time, greater technological expertise, and that mistakes (eg, in interpreting semantics of natural language) become more likely. Low-tech approaches based, for example, on decision tree mechanisms or relatively simplistic graphics can be utilized to deal with these problems, but they also make for a less realistic experience. These kinds of trade-offs make finding the optimal configuration in a certain setting a nontrivial task.

Are ECAs Ready for Clinical Practice?

Working with an existing Web- and mobile-based cognitive behavioral treatment for depression [8], our interest lies with techniques that can be applied in clinical practice. Given the many design decisions that can be made with respect to an ECA’s configuration, it is not immediately evident what an ECA should look like, and how it should behave in our context, namely as a bridge between guided and unguided interventions in clinical psychology. Paradigms exist that offer concrete design guidelines with respect to some (eg, [9]), or many (eg, [10]), of the aspects of ECA development. However, their empirical foundations generally rest on outcome measures such as “user satisfaction,” “engagement with the ECA,” or “intention to use,” and the context is not necessarily that of clinical psychology. Although such measures might be indicative of clinical effectiveness, they do not necessarily translate to the clinical outcomes we aim to improve. For example, even though users might be fully satisfied with an ECA, this does not necessarily mean that the average treatment outcome (eg, a significant reduction on a clinical measure of depression) will improve. Our chances to successfully bridge the gap between guided and unguided interventions will increase if we can determine how we can apply ECA technology with respect to improving clinical outcomes.

The interdisciplinary nature of ECA research makes almost any intervention that includes an ECA inherently complex. The UK Medical Research Council’s (MRC) framework for complex interventions [11] defines four phases through which such interventions move before being fully embedded in practice: development, piloting, evaluation, and implementation. The difference between the piloting and evaluation phase is crucial. Whereas interventions might still be subject to changes in the piloting phase, the evaluation phase is characterized by a focus on clinical outcomes and a more rigorous study design. Although routine practice sometimes evolves along different lines, as a golden rule, it is only once an intervention has successfully moved through the evaluation phase and can be considered effective and safe to use, that it becomes of practical value to psychologists.

Scoping Review

As a first step toward designing our own ECA, we wanted to review the relevant literature in a systematic manner to find out how ECAs had previously been used in psychotherapy, and to what extent the approaches taken were supported by evidence. Initial exploration of the literature to determine a suitable review method revealed a large variety of ECA applications, study designs, and outcome measures, such that a traditional systematic review (more emphasis on hard evidence) or meta-analytic approach (requires comparable outcomes) appeared inappropriate. We therefore adopted the scoping review method [12]. Scoping reviews aim to map the key concepts underpinning a research area, as well as the main sources and types of evidence that are available [13]. Compared with traditional systematic reviews, scoping reviews address broader topics where many different study designs might be applicable, and do not emphasize quality assessment (eg, the power of the study or nature of control groups) of the included studies, as the research questions are less specific [12].

This scoping review aims to inform health professionals about the technological and clinical possibilities and evidence base for ECA applications in clinical psychology, and to provide an overview of the activity in this field of research.

Study Design

We adopted the Arksey and O’Malley framework for scoping reviews [12], which distinguishes five different stages: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing, and reporting the results. We took into account recommendations about using an iterative team approach throughout stages (1) to (4) [14,15] by having regular discussions with other team members. As theoretical underpinnings to scoping reviews, as well as transparency about the process by which results are obtained, are often lacking [16], we also made an attempt to provide clear concept definitions. Stages (1) to (4) are described in this section and stage (5) in the Results section.

Identifying the Research Question

Given the generic features of human support in psychotherapy, techniques seen in the treatment of disorders other than depression might be applicable in our context as well. Hence, we broadened our scope to include other common mental health disorders known to be a target for e-Mental Health interventions, namely mood disorders, anxiety disorders, post-traumatic stress disorder (PTSD), psychotic disorders, eating disorders, autism spectrum disorders (ASDs), and substance-related disorders.

Study Identification

Our generic search query was as follows:

embodied conversational agent AND mood disorder OR anxiety disorder OR psychotic disorder OR eating disorder OR autism spectrum disorder OR substance-related disorder

Our list of search terms for the ECA concept included those we observed to be most common, for example, “virtual agent,” “virtual character,” “virtual human,” or “avatar.” The terms for the mental disorders were based on those found in PubMed’s MeSH (medical subject headings) index. We searched both psychology and computer science databases, including PubMed (psychology), ScienceDirect (interdisciplinary), WebOfScience (interdisciplinary), ACM (Association for Computing Machinery) Digital Library (computer science), and SpringerLink (artificial intelligence). The detailed search strings and search procedures are described in Multimedia Appendix 1. The final search was conducted in, and included articles published up to, July 2015. References were stored in Microsoft Excel, and duplicates were removed.

Study Selection

Study selection was conducted by two independent reviewers (SP & HL), who screened titles and abstracts on the sequential eligibility criteria, and then assessed the full-text versions of the remaining articles. A third reviewer (JR) was consulted in case of disagreement. We included full articles that:

(1) were written in English, (2) included an ECA in (3) an applied mental health context, (4) conveyed primary research findings, (5) targeted a mood, anxiety, psychotic, eating, autism spectrum, or substance-related disorder, and (6) described an experimental or focus group study.

Regarding criterion (2), for a software entity to be considered an ECA, it required to, first, have a virtual or physical embodiment (eg, Figure 1), second, interact with a user and, third, have a reasonable sense of agency, meaning its behavior had to be autonomous, and the software entity had to exhibit some form of reasoning. As for criterion (3), an applied mental health context implied that ECAs were used in an application that aimed to improve patient outcomes directly related to the targeted disorder, or that a reasonable argument could be made that the proposed application could eventually be used to do so.

Charting the Data

Data extraction was conducted independently by two reviewers (SP & HL). Concepts were mapped in four categories: (1) meta-information, (2) study characteristics, (3) study methodology, and (4) ECA characteristics. Precise definitions of the concepts are listed in Multimedia Appendix 2.

During the data collection process, several concept definitions were refined. In trying to map the studies’ intended interventions and provide a taxonomy, we found that from a low level of abstraction, interventions targeted very specific behaviors or skills. Our listing grew so expansive that we considered a higher-level classification to be useful. Something similar could be said for the ECAs’ social roles, which were difficult to define precisely given the large variety in applications. In our attempt to provide a useful taxonomy, we tagged all studies with their most predominant social role and intended intervention during discussions with all reviewers present. In these discussions, definitions of the different outcome types and development phases (based on the MRC framework for complex interventions [11]) were refined as well, until each of the studies could be tagged unambiguously. The resulting definitions are also listed in Multimedia Appendix 2.

Main Findings

The search identified N=1117 references. After the removal of duplicates, N=958 references remained. Next, a total of N=862 references were excluded by both reviewers after screening titles and abstracts. Of the remaining N=96 references, the reviewers did not agree about the inclusion based on screening in 78 instances, primarily because it was not entirely clear from the limited information provided by title and abstract whether the ECA inclusion criterion was satisfied (N=40). After full assessment of the 96 remaining articles, disagreement on 8 articles remained and was resolved in a discussion with the third reviewer. Finally, N=54 articles were considered eligible for full review. These 54 articles corresponded to N=49 unique studies (Figure 2). Multimedia Appendices 3 and 4, respectively, list the included studies’ intervention and ECA, and experimental design characteristics. Figure 3 depicts the most important results of the overall summative analysis and Figure 1 gives an illustration of some of the ECAs described in this review, more specifically emotional reinforcement with a smiley face [17] in the top-left, a virtual psychiatric nurse [18] in the bottom-left, SPARX (Smart, Positive, Active, Realistic, X-factor thoughts)’s guide character [19] in the middle, the SimSensei Kiosk virtual counselor [20] in the top-right, and humanoid robot KASPAR [21] in the bottom-right.

Autism Spectrum Disorders

Over half the studies (n=26) targeted ASDs (Table 1). They either involved a form of social skills training (n=21), aimed at a variety of target behaviors or skills (Figure 3, graph d), or were presented as an educational aid (n=5) to accommodate children with autism’s special needs. Autism was the only disorder targeted with robotic applications (n=12), and most of the virtual characters appeared in serious games (n=8). Most ECAs assumed the role of a social interaction partner (n=18) or tutor (n=8), and in two studies [22,23] social interaction partners were accompanied by a coach who provided additional feedback on their performance. The relative predictability of ECA behavior compared with that of humans, the possibility to repeatedly practice certain behaviors more often than with human partners, and children with autism’s fascination for technology were important reasons to explore the use of ECA technology in autism treatment.

Depression

A total of n=10 studies targeted depression. These studies revolved around CBT interventions (n=4), counseling (n=3), self-management skills (n=2), and social skills training (n=1). Most (n=6) of the applications were Web based, and the social roles fulfilled by ECAs were a coach (n=6) or health care provider (n=5). The anonymity provided by ECAs, their availability compared with humans, their nonjudgmental nature, and the ability for people to practice social interaction in a safe environment were important reasons to explore the use of ECAs in depression treatment.

Anxiety Disorders

There were N=5 studies that targeted anxiety disorders, either with CBT (n=2) or counseling (n=3) interventions. ECAs assumed the role of a social interaction partner (n=3), a health care provider (n=2), and a tutor (n=1), and most of them were implemented in stand-alone software (n=3). Reasons to use ECAs for anxiety disorders were similar to those mentioned for depression.

Post-Traumatic Stress Disorder

A total of n=4 studies targeted PTSD. Besides two studies on counseling interventions that also targeted depression [20,48], one study proposed a virtual coach in a CBT-based Web-based platform [57], and one concerned a virtual guide in a serious gaming healing environment [58]. The studies involved ECAs in the role of a coach (n=3) and a health care provider (n=1).

Psychotic Disorders

The n=4 studies involving psychotic disorders revolved around social skills training (n=2) and self-management (n=2). Aside from the Web-based job interview training application that also targeted depression [52], two applications were implemented in stand-alone software, and one in a VR environment. Interestingly, this set of studies was the only one to consider ECAs in all four social roles. Important reasons to explore the use of ECAs in the treatment of psychotic disorders were that social skills could be practiced in a safe environment, and that ECAs can always be available to provide support or information.

Substance Use

All ECA applications (n=4) targeting substance use were Web based and included CBT elements. ECAs assumed the role of a coach (n=2) and a health care provider (n=2). The main reason to use ECAs in the context of substance use is that they are more available than supportive humans would be, thereby increasing accessibility.

Principal Findings

This review aimed to inform health professionals about the technological possibilities and evidence base for ECA applications in clinical psychology, and to provide an overview of the activity in this field of research. Research on the use of ECAs in psychotherapy is emerging (Figure 3, graph a), and we reviewed N=49 studies of which the majority targeted ASDs (Figure 3, graph b). A general distinction could be made between applications in which ECAs were used as an adjunct to an intervention that could also have been used independently, and applications in which the interaction between the ECA and user was central. The former were mostly CBT-based programs, educational aids, and self-management interventions, whereas the latter were mostly social interaction skills training interventions and counseling interventions. Social skills training interventions were by far the most popular for ASDs, which also made them the predominant type of intervention overall (Figure 3, graph c). As a result, ECAs in the role of a social interaction partner were the most frequent (Figure 3, graph e). The large variety in ECA applications and types of interventions (eg, Figure 3, graph d) made it a nontrivial task to provide a taxonomy of interventions and ECA social roles. Although clinical behavioral outcomes were studied most often (Figure 3, graph g), they were in many cases restricted to pre-post measurements within experiments that had relatively small sample sizes. Consequently, few studies exceeded the piloting phase (Figure 3, graph h).

Future Work

In Table 1, some notable blank spots can be identified. Most obvious is the scarcity of evaluation and implementation phase studies, which requires more research with larger sample sizes, suitable control groups, and clinical populations. In this respect, the emerging nature of the field is a reassuring consideration. Two other notable blank spots are Web-based ECA applications for ASDs, and the use of ECA technology in VR applications in general. Taking into account the individual studies’ descriptions, there is also still quite some research to be done with regard to effective ECA configurations, by comparing different parameter settings in controlled studies.

Limitations

Our definition of ECAs had three components. With respect to the embodiment and interaction capabilities, we took a liberal stance, but our requirement of agency was rather conservative (autonomous behavior and reasoning). This excluded a fair number of studies (29 during the screening of full articles) from our review that some might consider to be relevant. Regarding the criterion of autonomous behavior, we excluded quite some studies using what is often called a “Wizard of Oz” paradigm, in which the ECA’s behavior is not controlled by a software entity, but by a human operator instead. Examples are a study in which an ECA representing the hallucinated voices of people with schizophrenia spoke the transformed utterances of a therapist [67], and one in which a robot aimed at improving the mood of hospitalized children suffering from cancer was under the control of a researcher [68]. An example of a study that was excluded because the embodied characters lacked reasoning capabilities, that is, the ECA would act the same regardless of user input, was [69].

Although ECA research is almost inherently interdisciplinary, we refrained from going too deep into the technological aspects. This was because our target audience consisted of health professionals with a generally less technical background and we wanted to focus on opening up the ECA domain for them as well as providing them with an overview of the available evidence for application in routine clinical practice. For this reason, we refrained from a highly technical discussion of, for example, verbal and nonverbal ECA capabilities. However, it has to be noted that, depending on how one would like to use ECAs in future work, many more detailed questions could be investigated surrounding ECA design aspects, such as the required capabilities for, and their impact on, specific disorders or types of ECA interventions. With respect to our search strategy, we looked specifically for articles that mention ECAs. As exemplified by the sole included article on SPARX, which is actually supported by more research than reviewed here (eg, [70]), there is a possibility that we missed out on articles describing, for example, serious games or VR environments in which ECAs are used, but not specifically mentioned.

Another limitation relates to the bibliographic databases we considered. Computer science research publications are more dispersed than those of psychology research and computer science databases are less suited to systematic searches. Although our interdisciplinary approach was already broader than what is usual in psychology research, there is a possibility that we might have missed relevant research in, for example, the IEEE (Institute of Electrical and Electronics Engineers) Xplore digital library or Google Scholar. We refrained from using IEEE Xplore digital library due to practical constraints and Google Scholar because its search algorithm is often updated and personalized, which makes it difficult to replicate search results. Additionally, due to practical constraints, we did not conduct searches in the gray literature or manual searches through cross-referencing, nor did we conduct a follow-up search after the original one.

While the idea of applying ECAs in psychotherapy is far from new (eg, [71,72]), to our knowledge this is the first review to specifically consider psychotherapeutic applications of ECAs in a systematic manner. There are several areas of research that are closely related to ours. One of these is social robotics research, which often focuses on providing company to elderly people (eg, [73]) or people suffering from dementia (eg, [74]). While this area of research could be seen as an attempt to prevent the psychological consequences that might ensue from loneliness, we did not consider the focus to be on psychopathology. Robotic applications targeting autism have previously been reviewed (eg, [75]), but without the constraints implied by our ECA concept. Consequently, many of these robotic applications do not adhere to our criterion of agency, that is, they do not act autonomously or intelligently. Besides research on robotics, there is a large corpus of literature on the application of virtual agents in other highly relevant domains from which we can draw inspiration. Although this review focuses solely on psychotherapeutic applications, there seems to be little reason not to consider, for example, motivational [9], pedagogical [76], or lifestyle-support agents [77].

Conclusions

Research into the psychotherapeutic application of ECAs is emerging. We identified 49 studies, with over half of them focusing on autism. The field is characterized by a large variety in all its aspects, for example, type of intervention, target behavior, platform, ECA embodiment, communication modalities, ECA “mental” states, and study design. While there are several studies surpassing the development and piloting phases, as might be expected in a relatively new field, evidence about the clinical effectiveness of ECA applications remains sparse. Technologically advanced ECA applications are very interesting and show promising results, but their complex nature makes it difficult for now to prove that they are effective and safe to use in clinical practice. Therefore, at present, clinical practice seems well served by an additional focus on a more low-tech approach based on the elementary principles that make ECAs effective, that can progress through the development and piloting phases at a faster pace, and that can therefore more easily be proven to be safe and effective for routine clinical practice.