Keywords

1 Introduction

1.1 Background

Chronic stress and its negative impact on health are a growing problem in today’s society [1]. A solution for this problem that is met with increasing approval is the use of health-promoting smartphone applications (mHealth). “Ovia”, “Mevi”, “DeStressify” and “myCompass” are only some of the available stress management apps. First evidence that the use of such apps can have positive impact on a person’s stress level has already been published [2,3,4].

To ensure their effectivity, stress management apps should integrate evidence-based content. This includes well-established coping and relaxation methods, as well as behavior change techniques [5,6,7]. However, current app reviews show that only few of the available stress management apps include a broad range of established methods [8, 9]. Notwithstanding the importance of appropriate content, these methods alone have been suggested as insufficient to promote long-term behavior change through apps [10]. The integration of gamification in mHealth products has been proposed as a possible solution to this problem.

1.2 Gamification in Health Apps

Gamification is defined as the use of game elements in non-game contexts [11]. It can have positive effects on user experience [12] and usability [13], as well as on the user’s motivation and engagement [14]. Moreover, the implementation of gamification can improve usage consistency and thus result in a greater exposure to the content of mHealth products [10]. Nevertheless, it is critically discussed whether gamification should be used in stress management applications. So far, studies report positive as well as negative or neutral effects with respect to behavioral and cognitive aspects. In addition, the effects of gamification are often dependent on the context and aim of the application [12]. For example, a study in the context of smoking cessation suggests that the combination of game elements and behavior change techniques can be well received by users [15]. On the other hand, the developers of the stress management app “Ovia” report that users do not wish for an integration of game elements in this context. This shows that users might not accept game elements in the context of stress management. Users are clearly reluctant towards linking stress management with gamification [2]. This might be one reason why, despite its great potential, gamification is hardly found in current stress management applications [16]. Notwithstanding, the users’ reluctance toward the hypothetical use of gamification in stress management apps, little is known about how game elements are actually perceived in this context.

1.3 Stress-Mentor

To close the identified gaps we developed a first prototype of the stress management app “Stress-Mentor”. This MVP (minimal viable product, [17]) combined the self-monitoring of stress-relevant behaviors and events through a diary [18] with vicarious reinforcement through the appearance of an avatar. The MVP’s general quality was then assessed in a longitudinal study. The MVP was tested by 26 participants over a period of 4 weeks [18]. After the 4-week trial period the participants rated the MVP’s quality with the user version of the mobile application rating scale for health apps (uMARS) [19]. The uMARS was applied as a semi-structured interview in order to receive suggestions for improvement. The results of this first study showed that linking the self-monitoring of stress-relevant behaviors with vicarious reinforcement through an avatar was well received by users. However, the study also revealed that the inclusion of more game elements might be needed to improve the usage behavior [18].

Including the user into the development process in this manner is an important tool to ensure the functionality of health technologies. Thus, as is recommended in iterative app development [20], the user feedback from this first study was used to create an extended app concept.

Based on the feedback additional stress management, behavior change and gamification features were incorporated in Stress-Mentor’s full version. For example, in addition to self-monitoring through a diary and its long-term visualization [21], the full version includes daily stress management exercises (i.e., breathing, progressive muscle relaxation, meditation, visualization, euthymic methods, physical exercises, cognitive aspects, time management, setting priorities, and planning social support and change). Moreover, the full version links these stress management aspects with a number of game elements. Besides the avatar, the full version includes an agent that guides the user through the app by explaining new functions and providing tips on stress and stress management. When using the app for the first time, the agent hands over the responsibility of raising the avatar to the user through a behavioral contract, which provides a narrative context. This means in addition to reflecting the user’s diary entries through its appearance, the user’s progress is visualized through the avatar’s growth in the full version. The user can capture this progress in a photobook. The avatars pictures can also be shared with others. This provides a social component. Additionally, the user’s progress is visualized through progress bars. The full version also supports goal setting by providing the stress management exercises in “tasks of the day” and “tasks of the week”. Here the user can chose one out of three suggested exercises he or she wants to accomplish each day. The app’s consistent usage is rewarded through badges. Another reward system is the points the user receives for every task and diary entry. These points can be exchanged for items for the user’s avatar in a shop. The addition of new gamification aspects over time is aimed at upholding the user’s curiosity. This is further supported through random sales in the app’s shop, randomly provided motivating quotes of famous persons and the randomly provided option to take pictures of the avatar.

The resulting full version of Stress-Mentor thus, combines an extended gamification framework with evidence-based stress management methods and behavior change techniques. For more information regarding Stress-Mentor’s concept see Christmann et al. [22]. Screenshots and content of both the MVP and the full version are displayed in Fig. 1.

Fig. 1.
figure 1

Screenshots and content of the stress management app’s first test version (minimal viable product, MVP, left screenshot) and its full version (right screenshot). The app combines evidence-based stress management techniques with gamification. The content already included in the MVP is highlighted in bold print.

Full size image

The full version’s general quality was again evaluated in a 4-week user study using uMARS. The aim of this study was to assess Stress-Mentor’s quality and to reveal improvements compared to the MVP.

2 Methods

2.1 Participants

Participant recruitment was carried out via an e-mail distribution list of the Technische Universität Kaiserslautern and associated research institutes, as well as an article in a local newspaper. All participants gave written consent to participate in accordance with the declaration of Helsinki.

In 2016 the MVP was tested by 27 participants (mean age = 23.38 years, SD = 3.01, 11 female, 16 male) in a first user study [18]. Of these participants 6 were in employment and 22 were university students.

In 2018 Stress-Mentor’s full version was again tested in a user study with 20 participants. However, one participant did not return for the final study appointment. This resulted in a total of 19 participants (mean age = 33.6 years, SD = 8.8 years, 10 female, 9 male) that were included in the analysis. Of these participants 12 were in employment and 7 were university students.

To ensure that the testing conditions were as realistic as possible the app was installed on the users’ own smartphone (Android version 4.4 or higher) and then used for 4-weeks in both studies. Therefore, the regular use (at least once a day) of a smartphone or tablet was a prerequisite for participation. In addition, participation in both studies required a minimum age of 18 years and fluency in German (all app instructions are only available in German).

2.2 Procedure

At the beginning of the study, each participant was informed about the course and aim of the study, as well as about the collected data. Each participant gave his or her written consent to participate. Subsequently, the participant’s demographic data was collected and the lead investigator installed the full version of the stress management app on the participant’s smartphone or tablet. After the installation, a brief introduction was given by the lead investigator. First the participant had to adjust the settings to his or her gender and age, as well as his or her preferences (tracking of alcohol and coffee consumption, reminder, color schemes and text size). Then he or she made an entry in the app’s health diary (sleep duration and quality, duration and intensity of sport, positive and negative events, general degree of stress, prevailing mood, consumption of water, vegetables and fruit, caffeinated drinks and alcohol). This was followed by a 4-week usage period, during which participants should use the app daily.

The study was completed with a second meeting that took place at least one day and a maximum of 14 days after the end of the 4-week usage period. As was done with the MVP, the participants then again rated the full version’s quality using uMARS [19].

2.3 App Quality

The user version of the mobile application rating scale (uMARS) [19] was applied to assess the quality of both the MVP and the app’s full version. uMARS was specifically designed to assess the quality of health apps from the user perspective [19].

Participants rated each of the questionnaire’s items on a 5-point likert-scale (from 1 inadequate to 5 excellent). uMARS was applied as a semi-structured interview. This means after each rating, participants had the opportunity to explain their answer and provide feedback and suggestions for changes in the app (open response format). Presenting questionnaires as semi-structured interviews provides deeper insight into the reasoning for the ratings, as well as suggestions for possible improvements (e.g., [23]).

A total of three questions were removed from the questionnaire. One question from the category information (16. “Does the app come from a credible source?”) was removed because the participants were informed of the app’s source in detail prior to participation. Furthermore, two questions from the category subjective quality were removed: 18. “How many times do you think you would use this app in the next 12 months if it was relevant to you?” because this question was not included in the MVP questionnaire due to a shorter targeted usage period and 19. “Would you pay for this app?” because it is planned to make the app available for free. The term health behavior in the questions of the app specific section was replaced with stress management.

2.4 Analysis

In order to check whether the addition of established stress management methods and other gamification techniques (see Fig. 1) had a positive effect on the perceived quality of the app, the results from both studies were tested for mean differences. T-tests were calculated to identify differences in mean uMARS ratings of the full version and the MVP. Bonferroni correction was used to adjust the t-test results for an accumulation of alpha errors.

3 Results

The analysis revealed significantly higher ratings for the full version (M = 4.514, SD = 0.327) compared to the MVP (M = 3.992, SD = 0.346) regarding the app’s general quality; t(44) = 5,198, p < .001, d = 1.541. In detail, improvements in the full version (M = 4.221, SD = 0.503) compared to the MVP (M = 2.926, SD = 0.738) could be observed for the category engagement; t(44) = 7.076, p < .001, d = 1.985. Moreover, higher ratings were observed in all questions of this category (i.e., entertainment, interest, customization, interactivity and target group). With regard to aesthetics, the full version (M = 4.667, SD = 0.314) also received better ratings than the MVP (M = 4.235, SD = 0.451); t(44) = 3.600, p = .001, d = 1.078. However, here only the question visual appeal showed a significant improvement, while no difference could be found between the rating regarding Stress-Mentor’s layout and graphics. An improvement of the app’s full version (M = 4.456, SD = 0.590) in comparison with the MVP (M = 4.000, SD = 0.686) could also be detected regarding the category information, t(44) = 2.413, p = .020, d = 0.704. The ratings for both the quality and quantity of the included information increased. No difference between MVP and full version was identified regarding the app’s visual information. Even though the app is now much more complex and has a lot more features, ratings for the app’s functionality did not decrease and its functionality was assessed as excellent for both the MVP (M = 4.806, SD = 0.244) and full version (M = 4.711, SD = 0.315); t(44) = −1.104, p = .278, d = 0.346. This is also reflected in the rating of each of the questions regarding the app’s functionality (i.e., performance, ease of use, navigation and gestural design). There is no difference between MVP and full version with respect to these aspects. In contrast, MVP (M = 2.648, SD = 1.040) and full version (M = 3.763, SD = 0.865) differed with respect to their perceived impact; t(44) = 3.956, p < .001, d = 1.147. The app’s full version received higher ratings for all questions in this category (i.e., awareness, knowledge, attitudes, intention to change, help seeking and behavior change). In addition the full version (M = 4.210, SD = 0.164) received a better overall star rating than the MVP (M = 3.333, SD = 0.207); t(44) = 3.327, p = .002, d = 4.602. Also, participants were more likely to recommend the full version (M = 4.315, SD = 0.217) of Stress-Mentor to others in comparison to the MVP (M = 2.888, SD = 0.235); t(44) = 4.464, p < .001, d = 6.264. The corresponding means and standard errors for the rating of the uMARS questions of both studies are displayed in Fig. 2.

Fig. 2.
figure 2

uMARS (user mobile application rating scale) ratings of the MVP (minimal viable product) and the full version of Stress-Mentor are displayed. Depicted are means and standard errors. Significant differences between the groups are marked with an asterisk (*).

Full size image

4 Discussion

In summary, although not all aspects showed significant improvements between the MVP and the full version, there is a general trend towards enhancing the overall quality of the full version compared to the MVP. The positive ratings with regard to entertainment, interest, customization, interactivity and target group show that the use of gamification can affect perceived user engagement in a positive manner. This goes in line with the results of previous studies showing that gamification can have positive impact on motivation and engagement [14]. Increasing user engagement in this manner is important to improve usage consistency and can thus, increase an intervention’s effectivity [12]. This is supported by the technology acceptance model (TAM3) which identified joy of use as one major predictor for the intention to use technologies [24].

Besides the gamified context, the integration of an extensive number of stress management techniques and information regarding stress and stress relevant behaviors can explain why the participants are more likely to recommend the full version than the MVP. It is also very likely the cause for the improved overall star rating. MVP users commented that they wished for the integration of more stress management methods [18]. The integration of an extensive number of stress management and behavior change techniques in response to this feedback, can also explain the more positive evaluation of the full version’s perceived impact. The inclusion of additional stress management methods in addition to self-monitoring through a diary also explains the increase of the ratings with regard to the app’s perceived impact, as well as the aspects quality and quantity of information. This emphasizes the importance of integrating evidence-based exercises and behavior change techniques into mHealth products [10]. Nonetheless, some users suggested including more graphics, pictures and videos for the stress management exercises.

Moreover, references and links supporting the user in the search for professional help were added to Stress-Mentor’s full version. These links were included in both a menu item and in specific tasks. This likely resulted in a more positive rating regarding the likelihood of the user to seek further help.

Although the full version is much more complex than the MVP, there was no significant deterioration in Stress-Mentor’s navigation and visual information. The functionality of both app versions was rated as excellent. This indicates that the use of gamification could positively affect the system’s usability [13].

The positive evaluations regarding the general quality of the full version suggests that the combination of established stress management methods and gamification [22] that was applied in Stress-Mentor was well received by the users. Although some users of stress management apps explicitly oppose the use of gamification [2] the positive evaluation of Stress-Mentor shows that this is not necessarily the case. The users’ comments emphasized that the embedding of stress-related aspects into a gamified context was perceived in an overall positive way. In fact, the participants found the app interesting to use and enjoyed exploring its contents. This goes in line with previous results that indicate that the combination of gamification with evidence-based content from stress management and behavior change could have positive effects on the user’s engagement and thus, potentially make health apps more effective [15].

However, it is apparent that users are not aware of the effect of gamification [25]. Some of the participants said that the gamification concept would not have been necessary in their opinion. However, they also described that they tried to adjust their behavior so that they could make positive entries in their diary in order to make their avatar look healthier. This observation coincides with previous results on the effectiveness of vicarious reinforcement through avatars (e.g., [26]). It further underpins the potential usefulness of gamification in the context of stress management.

This demonstrates that it is not sufficient to ask potential users whether gamification is desired in a particular context. Rather, studies must examine if the implementation of game elements in a specific context appeals to the user and how their use influences the effectiveness of the product [27, 28]. When integrating gamification, the context and goal of the app should therefore always be taken into account [12]. The positive response to combining gamification with stress management in Stress-Mentor is probably due to the fact that the integrated gamification elements support the actual goal of the app, namely the learning of stress management methods through daily and weekly tasks. Nonetheless, the effect of gamification on actual usage behavior is still unknown. This aspect is the focus of our next study, where we are comparing the usage behavior of a gamified version of Stress-Mentor with a non-gamified version of the app.

5 Conclusion

In summary, an overall trend of improvement could be observed for Stress-Mentor’s full version. The positive ratings confirm that Stress-Mentor is of good general quality, speaking for the app’s successful, iterative development. This study therefore, shows that an iterative development process involving the user can lead to an improvement in the product’s overall quality.

Even though some users of stress management apps were opposed to the use of gamification [2], this study demonstrates that the combination of stress management methods with gamification was, in fact, well received. This highlights the potential usefulness of gamification in the context of stress management.