In this Appendix we provide a detailed synthesis through a scoping review of research done in HCI related to the develop of interactive technologically-based interventions that support learning, development, and practice of emotion regulation with adults and/or children. Overall, the main focus was to identify and code most if not all existing HCI work in terms of the psychological mechanisms behind the interventions (where possible), in addition to the more commonly addressed aspects (e.g., type of technology used).
1.1 Scoping Review Methodology
Specifically, we chose a scoping review methodology to develop a detailed synthesis of the type and range of studies available on this emerging topic in digital mental health [
6]: such approach is appropriate to identify the types of evidence available for a topic (which may be split across several fields, or subfields of HCI in our case), to examine how research has been conducted, and to identity knowledge gaps, which is in alignment with our aims. A scoping review is conducted using a systematic procedure, which can be replicated and often aims identify knowledge gaps [
67]. The full coded database of papers as well as the R code used for analysis will be made available through the Open Science Framework.
This approach enabled us to synthesise high level themes related to the kinds of research that has been done related to interactive technology development for learning, developing, or practising ER, with a specific focus on research with an end goal of creating ER interventions. The results of our scoping review provided a synthesis of research to date, serving as a foundation for our analysis of the four delivery mechanisms we focus in our framework (Section 4) and enabling us to highlight gaps or opportunities for HCI, which we discussed in Section 5 of this paper. We present the detailed scoping review methods and results here as context for our paper and for those interested in more information or getting an overview of the current state of the field.
Selection and Filtering:. We conducted a literature review that targeted full papers and notes as well as high quality works-in-progress that were published in top venues in HCI. We targeted the ACM digital library, IEEE databases and used Google Scholar to search for papers published between January 1 2009 and December 8 2021. Our keyword search criteria were ‘interactive technology” (i.e., involved human-computer interaction) and one or more of ‘emotion regulation’ or ‘self-regulation’or ‘stress’or ‘stress-regulation’. The search resulted in 5,574 papers. We then conducted two passes of filtering (see Figure
9) based on title and keywords, excluding papers that were non-English, duplicates, review papers and/or did not address human emotion-regulation, leaving 333 papers.
During the third and fourth passes of filtering, we scanned abstracts and/or full texts and excluded papers that did not involve an interactive technology (e.g. described a study with no digital technology or with a non-interactive technology) and reviewed publication venues, removing non-peer reviewed papers and papers that were published in non-international publications, leaving 130 papers. In the fifth and sixth passes of filtering we reviewed the full text and excluded papers that did not include an evaluation or a user study of any kind or did not include any information that could be used to identify one or more intervention mechanisms, leaving 52 papers. Finally, we conducted a seventh pass in which we read the full text and excluded papers where the end-goal of the research was not an emotion regulation intervention (resulting in 36 papers), which was necessary for the follow-up analysis described in Section 4. This is the dataset that we report on in both Section 4 and in this Appendix.
Inductive Analysis Process. Our analysis process followed an inductive interpretive analysis process, similar to open coding in grounded theory [
20], since we did not have an established analytical framework for coding the articless that were selected in our scoping review [
6]. In this process, we iteratively identified themes or dimensions, which describe the research papers from different perspectives. For each dimension we iteratively and comparatively developed and defined categories, resulting in the development of a codebook, which we then used to code the entire dataset. We used our research goals and focus on ER interventions to structure this inductive and iterative process, beginning with the two researchers (first and last authors) working together to identity the six dimensions of the analysis framework, as described above. The dimensions were derived after reading through a quarter of the articles and skimming many others. We worked individually and then together to derive the following six dimensions:
We included the (1) type of technology dimension because it enabled us to take a snapshot of the kinds of hardware (e.g., platform, input, output) currently being used to create systems used in ER intervention research in HCI. We also examined where each research study fell on a typical research (2) stage in research lifecycle (i.e., early, mid, late stage), which helped us characterize the maturity of each study and better understand current state of the field. Third, we described the underlying (3) HCI research focus by coding clusters of HCI research, each representing different theoretical, methodological and interdisciplinary traditions found in HCI research as applied to ER intervention context. To explicitly capture the psychological components of interventions, we then coded for (4) the intervention mechanism – that is the psychological ‘theory of change’ through which the intervention is assumed to affect emotion regulation (e.g. Reminders, Biofeedback); (5) the level of specificity of that mechanism (i.e., high, medium, low); and the intended use case (6) ongoing vs skill-development support, i.e., whether the intervention was designed to be used indefinitely (and effects would disappear if taken away), or aimed as a temporary scaffolding for skills development. The last dimension is particularly important given the ramifications for the long term effectiveness of interventions, as well as the learning theories underpinning the intervention designs.
Once dimensions were identified, the dataset analysis phase consisted of two of the researchers working inductively to iteratively to develop initial categories and coding rules for each of the six dimensions. This was done during individual and shared analysis sessions, each focusing on a subset of the dimensions (e.g., (3) HCI focus and (4) intervention mechanism), over a small subset of papers. Once categories were defined, the researchers then individually reviewed and coded about 25% of the papers across all six dimensions, together reconciling, revising and finalizing descriptions of categories and coding rules for each of the categories. This codebook was finalised after we had read and analyzed about half the papers, at which we had well-defined dimension categories and stable coding schemes for each dimension.
As such, the resulting categories within each of the dimensions have predominantly emerged from the data and are thus presented together with the results overview below. We also coded (7) the extent to which each system relied on in-the-moment (on-the-spot) vs. out-of-context (offline) training; and cognitive (didactic) vs experiential (experiential) learning; these results are addressed in detail in Section 4.3 rather than here as they are core to our framework.
1.2 Results
Overall the papers are comprised of 36 peer-reviewed papers, split 63.9% ACM, and 19.4% IEEE databases, with the remaining 16.7% available through Google Scholar. In what follows, we first outline the results for each of the dimensions separately, and then highlight some of the arising connections. Each section starts with an overview of how the categories within a dimension are represented in the dataset.
1.2.1 Type of Technology.
We classified papers by the type of hardware technologies used to implement prototypes or systems. The classification was not mutually exclusive, as some systems involved several technologies (i.e., more than one platform, input and/or output device); however it provides a snapshot of the kinds of technologies currently being used in to create ER interventions in HCI research. The most common platforms were mobile devices (16) and desktop or laptop computers (11), with several other devices such as smart watches (4), and bespoke embedded computing prototypes (10). Seven papers described systems that were an assembly involving custom hardware (i.e. system was custom-made assembly of devices and electronic parts), the remainder of papers involved systems created from combining one or more commercially available devices. The sensor(s)—connected to the main platform for systems—included a variety of biosensors (e.g. respiratory, HR, EMD, EEG, GSR sensors) and non-screen based output devices(s) which included VR headsets, tangible fidget, lamp display, haptic seat pad).
1.2.2 Stage of Research.
HCI investigations may take place over a broad ‘lifecycle’ of design research, spanning from early design explorations to large-scale in-the-wild studies. We classified papers based on where the described interventions were on this research trajectory. To simplify matters, we coded the papers as early, mid, or late stage. This qualitative assessment incorporated a range of factors, spanning the research questions addressed by paper authors (e.g., examining viability of a new design concept or form factor, versus efficacy evaluation of robust systems); the purpose and methodology of the evaluation (formative vs summative study designs); as well as the robustness of the examined prototype (e.g., testing a component of a future-envisioned system in-lab, versus in-situ deployments of robust full system implementation).
Specifically, the 13 papers we coded as early stage research focused on the design of prototypes and formative evaluations. The 14 articles coded as mid-stage research focused on iterative design and evaluation of more complex and/or robust prototypes with formative or summative evaluations. Finally, the 9 papers coded late stage research involved hi-fidelity, robust research systems that were ready for wide-spread deployment and summative evaluations in lab or field studies.
1.2.3 HCI Research Focus.
The papers in the data set differed markedly in their theoretical grounding (from post-positivist to critical design) and the contributions they attempt to make (e.g., from verifying intervention efficacy to introducing innovative design ideas). The resulting two main groupings reflect a traditional HCI focus on evaluation, and a design oriented approach to research. Example papers are as follows:
–
Intervention Efficacy Research (n = 26) involves research in which the focus and motivation are on evaluating the effectiveness of a technology-mediated intervention in terms of improving learning, developing or practicing ER in a variety of contexts (e.g., lab, home, everyday activities). Of these most papers were mid to late stage research, with the exceptions tended to focus on early stage feasibility studies (e.g., [
25,
52,
88]. The evaluations were split across all intervention mechanisms described below in Section
1.2.4, including Awareness (5), Physiological Synchronization (3), Reminder and Recommendations (6), Biofeedback (10) and Other (2): In studies of systems based on mechanisms of Awareness, Physiological Synchronization and R&R, research designs tended towards mixed measures observational studies. While the results of many of these studies were largely positive, the measures used typically focused on end-users’ ability to use and understand the systems involved in interventions, rather than evaluating changes in skills development around ER. Most evaluations resulted in findings related to the need to change aspects of the design of the intervention.
Biofeedback-based interventions tended to be evaluated at a higher level of rigor. Those at mid or late-stage were either observational studies (e.g., [
54,
56]) or various forms of comparative (e.g. [
89,
103,
103]) or controlled experiments (e.g., [
5,
51,
108]). There were two RCTs [
81,
104]. Measures often included pre-post test ratings related to stress, anxiety, and depression (e.g., [
5,
51,
54,
81]), performance on follow-up tasks (e.g., [
108]) and some studies also logged and analyzed physiological data during sessions (e.g., [
54,
56,
103,
104]). In many of these studies participants were often children with ER challenges (e.g., ADHD, anxiety, fetal alcohol syndrome). In general, results were often mixed with positive evidence related to showing a direct impact of the intervention on participant’s ability to regulate stress and/or anxiety during the intervention (for an exception see [
81]). However, lack of controls, short duration of interventions and/or lack of direct measures of ER skills development measures outside of the intervention limit validity and generalizability. Only two studies measured transfer of ER skills into everyday life or administered a follow-up test to determine maintenance of effects [
4,
5].
–
Design Oriented Modality Research (n = 10) involved work in which the focus and motivation were on exploring the impact of different modalities of input and output on user perception, engagement and/or other UX-focused measures. These design oriented articles were entirely early to mid-stage research. Out of these, several papers focused on comparing different forms of outputs to support physiological synchronization, including a comparison of placement for haptic actuators representing heart rate [
61], and comparisons of different representations of breath (e.g., haptic versus voice [
72]; tactile versus visual and auditory [
15]; pacing [
65]). Several other papers investigated modalities utilized in biofeedback systems, including the use of visual metaphoric representations of stress over time [
107], the interplay of breath and heart rate with tangible and light outputs [
53], a comparison of different physiological inputs [
73] and exploring the timing and duration of feedback [
74]. These types of explorations are necessary to understand the impact of design decisions on user experience as part of the iterative development of systems that can be used interventions.
While we did not include these as separate categories, some of the papers also mentioned the importance of personalization, for example of design choices relative to placement of haptic actuators [
61]) or triggers for recommendations [
71]. Similarly, other articles mention the importance of social factors as a secondary considerations, for example, the importance of facilitator rapport is cited as a key factor leading to positive results in a biofeedback intervention for children [
5] and teacher created content for interventions [
25]. The role of physical artefacts for scaffolding for parent-child interactions to bridge SEL learning from school to home was explored in [
87]. These and other studies suggest the benefits and importance of considering the personal and social landscape of ER interventions, an area that is currently under-represented in HCI research.
1.2.4 Intervention Mechanisms.
Complementary to the HCI research focus categorisation, the coding of the intervention mechanisms aims to identify the commonly occurring psychological mechanisms that the technology-enabled ER intervention work relied on. The purpose is to highlight the importance of the theoretical grounding used explicitly or assumed as part of the intervention design interventions, as well as identify the commonalities and differences across the varied interventions in our dataset.
–
Biofeedback (n = 14) Biofeedback was the most commonly used intervention mechanism. Systems created with this model all assume that users will try to consciously or unconsciously modify their behavior as a means to alter their physiological or neurological state closer to a target state. Papers vary in terms of input data source, which may be sensed physiologically (e.g., HVR [
107]) or neurologically (e.g., EEG [
4,
5,
56]). Several papers compare input
Modalities related to biofeedback design (e.g., [
73,
74]), and others explore output representations (e.g., light [
107]). Most biofeedback systems are PC-based, however three systems were designed for mobile use [
4,
5,
108]. There is a group of papers in which biofeedback is embedded into video games as a means to practice ER in the context of game-play (e.g., [
54,
56,
81,
103]), when these interventions are evaluated for efficacy, research design choices (e.g., lack of control group, no direct measures of ER) often limited validity and/or generalizability of results. Some interventions explicitly provided support to end-users to learn how to modify regulatory responses (e.g., [
4,
5,
51], but most relied on user’s ability to interpret visual forms of feedback and use that to regulate mental or emotional states. For example, in biofeedback video games, bio-data impacts game mechanics that are communicated to the player through dynamic visual representations (e.g., [
56,
81,
103]). Other biofeedback systems use visual representations based on metaphor theory (e.g., [
4,
5]) that provide metaphor-based cues to how to modify mental and/or emotional states. Over time the goal of biofeedback is to learn and practice emotion-regulation, with and without the support of a system until emotion-regulatory behavior becomes increasingly more implicit or automated. Few authors describe explicitly how this transfer might occur (e.g., [
4,
5,
74]). For example, interventions included biofeedback and psycho-education were proposed as a means to improve learning to modify emotional state and transfer those skills to everyday life (e.g., [
4,
5,
51]). Most authors did not address learning transfer.
–
Physiological synchronization (n = 7) papers were based on a system designed based on the mechanism of automatic and implicit emotion regulation that may occur when the individual perceives an external input that mimics their own biosignals (e.g., the ‘false heart-rate feedback’ studies emerging since the 60s [
99]). Many papers explored the use of haptic representations to prompt a target breath or heart rate (e.g., while driving [
9,
72], through a smartwatch [
15,
21] or wearable custom device [
61]). Overall, this research area is still in early stages in terms of design choices that might effectively duplicate human-human physiological synchronization as a means to support ER. And, as Choi and Ishii point out, the success of this approach may depends on the individual’s level of sensitivity of interoceptive awareness of their own internal body states related to motion and stress [
15]. In addition, although this approach could result in automatic down-regulation of stress through heart or breath rates, the approach does not posit a mechanism for the explicit development of competency around emotion regulation over time, which could be sustained without the system needing to remain active.
–
Awareness (n = 5) papers involved monitoring and tracking individuals’ data as a means to increase user’s awareness of their emotional state(s). Input data was gathered from contextual data (e.g., location [
44]) or from self-report, which was entered manually (e.g., diary [
7]) and/or from sensors (e.g. smartwatch [
103], multi-sensor wristband [
52]). Displays of information take a variety of forms, largely visual. An underlying assumption in this model is that enhanced awareness will motivate users to take a proactive role in their own stress management and/or that users will reflect on the information being provided to them and as a result of that reflection change their future behaviours. However, none of these papers explore the cause and effect relationship between being shown information and actually implementing strategies for enhanced emotion regulation. In addition, none of the articles posit a mechanism for the explicit development of competency around emotion regulation, although it is possible that over time an individual might develop competency through repetitive practice.
–
Reminders and/or Recommendation (n = 6) articles describe interventions that involve reminders or recommendations to follow step-by-step instructions about how to enact emotion regulation strategies. The articles vary in terms of the complexity of emotion regulation strategies suggested. All systems highlight the need for customized reminders and recommendations. These may be triggered based on manually entered self-reports [
31,
89,
91] or sensor-based data indicative of stress [
71,
76] and/or through machine-learning application designed to look for patterns over time [
71]. Some systems involved content that was customized by others (e.g., teachers, coaches) in terms of strategies [
25] or in terms of format (e.g., parents) [
31]. Other content was more generic, for example content was based on strategies from CBT matched to current emotional state [
31]. All articles involved evaluations, two at an early formative stage [
25,
76], one during deployment as part of the design process [
71], and three in more formal deployment [
31,
89,
91]. One key finding was that for sensor-based push reminders and recommendations the timing rarely coincided with the stressful event due to inaccuracies in biodata to represent stress. In addition, Pina et al. suggested that in the moment of stress was not the best timing choice for deliver of reminders and recommendations [
76]. Findings suggest that manually entered data self-reported data combined with customized and just-in-time content had a positive impact over time (e.g. [
31,
89]). No articles offer explicit mechanisms that might reduce the need for reminders and recommendations over time.
–
Other (n = 4) papers included the use of general relaxation exercises [
14] and experience-based interventions designed to provide end-users with the experience of emotion regulation prior to learning how to consciously regulate [
87,
88]. One paper presented a system that was mechanisms agnostic, instead instantiating a range of mechanisms into the intervention [
71].
1.2.5 Specificity (of the Intervention Mechanism).
In contrast to psychology research, where outlining a clear ‘theory of change’ for the intervention is a key part of the peer review process, HCI work often does not explicitly discuss the presumed causal pathways through which intervention effects should emerge. To understand how this (lack of) good practice is present in the emotion regulation intervention space, we coded the level of detail or specificity that was provided about the assumed intervention mechanism and/or theory of change [
34,
70].
We used a tripartite scale of low, moderate and high level of specificity depending on how clearly and explicitly one or more mechanisms leading to improved emotion regulation were described in the paper and how clearly and explicitly those mechanism(s) were linked to intervention elements posited to create effects related to learning, developing or practicing ER skills. For example, a coding of low would result if a theory of behavior change was described, typically in related work, but no explicit mechanisms were described that could be used to design the intervention. A coding of high was used when one or more mechanisms were clearly explained and explicitly linked to design decisions or elements of the intervention, typically technological elements. A paper was coded as medium when the mechanisms were present but not well-described and/or the link to design elements was weak.
Majority of the intervention mechanisms were described by authors in a low (n = 15) or medium (n = 13) level or of detail. For example, in much of the modality-focused research, many authors assume that the right form of input data linked to particular output representations may lead to enhanced awareness of various aspects of emotion regulation, which in turn should improve emotion regulation. In part, this is based on cognitive behavioral therapy in which awareness of emotional state is posited as a key component of learning ER. However, causal mechanisms or direct linkages for these claims are not described, nor are they already established in the HCI—or psychology—literature. Similarly, much of the R&R rationale involves the assumption that giving people the right advice at the right time will improve their ability to ER. While physiological synchronization may lead to ER, there is no mechanism posited that relates this to the development of self-regulation skills. Surprisingly, many of the biofeedback intervention mechanisms are also under-specified in terms of how end-users are supposed to learn/know how to change their physiological or neurological states.
Several successful interventions include multiple mechanisms that together are posited to result in both enacting and developing ER skills that transfer outside of the intervention. For example, [
4,
5,
51] describe one or more intervention mechanisms in a high level of detail, offering direct explanations for causes and effects in emotion regulation skills development, and provide explicit links from these mechanisms to design features and elements of the intervention, which are subsequently evaluated for efficacy.
1.2.6 Use Cases: Skills Development vs Ongoing Support.
A surprisingly strong majority of the papers in the current data set (n = 31) assumes that the developed systems have to provide an on-going support for the intervention to be effective; and thus the effects would disappear if the system was removed. In contrast, only several (n = 5) interventions were designed to include a way to scaffold or reduce end-user dependence on the intervention and/or specify an explicit transfer mechanism that would enable the end-user to apply emotion regulation strategies outside of the intervention. In [
108] and [
81], the authors proposed that using biofeedback games to teach ER through breath regulation in stressful scenarios may promote carryover to other situations. In [
4] and [
5] the intervention enabled children to directly experience ER through metaphor-based biofeedback in the context of coached interventions that used a CBT approach to promote emotion-regulation skill development and transfer. [
87] proposed using a shared parent-child experience around an interactive media, which included narrative hooks to ER strategies taught in school, with the aim to provide parents with language around ER that could be referred to in everyday moments.
