Keywords

1 Introduction

1.1 Two Years After – Effects of GDPR on Serious Games Studies

The comprehensive EU General Data Protection Regulation (GDPR) [44] that came into effect in May 2018 aims to provide guidance in privacy and data protection and improve scientific integrity of human-related studies. Since GDPR is believed to have a global impact [25], the question arises to what extent the regulations actually changed the scientific conduct and the resulting publications at conferences and in journals.

One area of research concerned with evaluating personal data are applied digital games. Serious Games (SG) have gradually become an instrument for investigation and scientific analysis. Application in research extends from analysis of learning in games [6] over investigating the use of medication [1] and treat phobias [11] to researching decision making and team behaviour [16, 27]. Nearly all SG research is thereby affected by the GDPR as not only experiments with in-game assessment [27] involve data collection but also qualitative interviews that utilise games as a proxy to elicit information [28]. Other, more interactive, research approaches such as participatory SG creation workshops [9] equally require recording participant behaviour.

This paper is thus concerned with investigating the impact of GDPR on scientific reporting of Serious Games (SG) research. With a scoping review [41] of SG studies between 2016 and 2020, split to before and after GDPR came into effect, the reporting of ethics standards and data protection is analysed.

1.2 Research Ethics and Data Protection – Authorities and Guidelines

While for data protection, the legal frame has been laid out more precise with GDPR, guidelines on research ethics are more diverse and loosely defined. Whereas the APA code of conduct, for example, combines guidelines on ethics with data sharing/privacy [5], many European nations regard the two aspects separately. This becomes evident when exemplary, looking at Northern Europe. In Sweden (datainspektionen.se), Norway (datatilsynet.no), Finland (tietosuoja.fi), and Denmark (datatilsynet.dk) data protection is supervised by a single authority. On the other hand, there are generally multiple regional research committees and ethics authorities of different scientific or professional fields. While Sweden just recently moved to a single nationally controlled committee (etikprovningsmyndigheten.se) to administer ethics approvals other Scandinavian countries such as Denmark (nvk.dk) and Finland (tukija.fi) maintain a distributed structure with several regional committees.

Data protection, instead, is supervised much more centrally with single authorities per country that have adopted the GDPR legislative even if not an EU member (e.g. Norway). The far-reaching scope of the regulation may be attributed to the principle of territoriality, which effectively protects every EU citizen with the GDPR even if the data processing party is non-EU related [34]. Thus, researchers from other countries must comply with GDPR if European participants are included in the study. Therefore, it can be assumed that research practices and reporting have adapted widely since coming into force of the regulation.

Much like the diverse structure of ethics boards, numerous ethical guidelines are potentially applicable in the research context. Concerning Serious Games, human-centric research is mostly either medically oriented or related to social science. Ethical considerations regarding vulnerable groups such as children or elderly become specifically relevant when considering Exergames [43]. These SG are developed for improving health or medical conditions and are often researched in clinical trials. Such SG research does not differ from other medical research settings and must follow the same ethical standards. A basic foundation for a medical code of conduct was laid with the Nuremberg Code [37]. However, more actual and elaborated ethical guidelines that are widely quoted in research are the Declaration of Helsinki (DoH) [48] and Good Clinical Practice (GCP) [21] on a worldwide perspective as well as the Clinical Trials Directive in Europe [22]. While the DoH is widely followed in medical research, there has been controversy regarding conflicts with other codes of conduct. Such conflicting guidelines, for example, exist in the UNESCO Universal Declaration on Bioethics and Human Rights [42] and the guidelines of the Nuffield Council [14]. The USA, for example, do not support the regulations any further but instead recommend the orientation on GCP and their own Common Rule [45].

1.3 Research Objectives

As has been outlined, GDPR is a far-reaching directive which is affecting all SG researchers that collect data from European participants. SG researchers are thus compelled to report on their ethical conduct and data protection when publishing articles to respect publication ethics [32]. This paper is therefore concerned with analysing SG research in the last two years (June 2018 to June 2020) and compare research practice with an equal number of SG studies before GDPR came into effect on May 25th, 2018. A scoping review is conducted as the scientific approach to analyse the broad body of research since 2016 for ethical reporting practice [29]. The specific aim of this review is to give insight into the following questions:

  1. 1.

    What are the reporting practices in SG studies from 2016 to 2020 regarding ethics approval, ethics guidelines and data protection?

  2. 2.

    Which ethical principles/guidelines and data protection policies are most reported in SG studies, and are there notable changes after GDPR introduction?

  3. 3.

    How did the coming-into-effect of GDPR affect SG publication ethics concerning reporting the data protection policy or ethical conduct?

2 Method

The scoping review methodology is suitable for gaining insight into applied research concepts and policies as it is concerned with analysing a large body of literature [4]. The following sections outline the mapping methodology followed in this study, as suggested by Peters et al. [31]. According to the guidelines, no quality appraisal was conducted for the studies. Potential bias influences are further reduced by the lexicometric analysis approach [46]. A preliminary search of existing overviews of ethics reporting in SG studies was conducted in all search engines applied for the SG study search but did not find any hits on the specific topic.

2.1 Information Sources and Search

The keyword search was conducted on different online databases with a time limit from 2016 to 2020 while using the university internet connection to have broad access to full-text publications. The search results were downloaded with Zotero reference management software (zotero.com) that automatically retrieves accessible full texts when importing the records. To get a comprehensive overview of reporting practice, the most popular SG terms were defined as keywords, and no further limitations were made in the search strings. Table 1 lists the search engines and search strings for the search that was conducted on May 6th, 2020.

Table 1. Searched databases and applied search settings
Full size table

2.2 Screening and Eligibility

The broad search strategy resulted in duplicates which were excluded at the screening stage. Also records not relevant for examining research ethics reporting such as book chapters or reports were defined and excluded (see Sect. 3.1). Moreover, all entries without full-text access were excluded at screening since ethics approval and data protection are generally not reported in title or abstract. Finally, review studies and studies not reported in English were excluded.

2.3 Data Analysis Process

Selected studies are building a literature corpus that is divided into two subcorpora before and after the coming-into-effect of GDPR. The applied process corresponds to the corpus linguistic approach [40] on a closed, large collection with authentic and representative language. The subsequent lexicometric analysis follows suggestions of Dzudzek et al. [19] and Wiedemann [47] by (1) calculating frequencies of terms regarding research ethics and data protection reporting, (2) key phrase-in-context analysis and (3) comparison of the reporting trends before/after GDPR between the two subcorpora. For the context-observing content analysis, the key terms listed in Table 2 were applied on both subcorpora with the software MAXDictio (maxqda.com). The included records were first imported, and meta-data analyses regarding study characteristics were performed. Successively, frequency examination was run with the outlined word set on each subcorpus separately. The analysis thereby focused on the number of papers reporting the terminology. Next, key phrase in-context analysis was conducted for discovered phrases, and related meaning was evaluated. Studies not reporting the respective phrase in the intended context were excluded from the result tables and figures. For example, when data protection measures were not reported concerning the study conduct but rather as a general requirement in the introduction or when a term was only found in a title in the reference list. Finally, the comparative trends before and after GDPR coming-into-force were analysed to answer the research questions.

Table 2. Key phrases applied in lexicometric analysis of SG studies before and after GDPR coming-into-force
Full size table

3 Results

3.1 Literature Search

The database searches resulted in 20767 citations (Table 3). After exclusion of duplicates, improperly allocated meta-tags were corrected. Proceedings papers listed as book chapters were classified as conference papers during this step. Mostly, the distinction was identifiable from metadata as proceedings or conferences were mentioned in the fields. In rare cases, the full text (if available) was opened for verification.

Table 3. Search results total between January 1st, 2016 and May 6th, 2020
Full size table

The subsequent selection process outlined in Fig. 1 left 2186 studies to divide before and after GDPR coming-into-effect. When splitting according to this date, 1073 studies were eligible for inclusion since May 25th, 2018. To allow for comparative lexicometric examination, the same number of studies before May 25th, 2018 were included in the analysis by going backwards in publication dates. Thus, the resulting cut-off date was February 8th, 2016 and the 40 studies before that date were excluded. The selection procedure resulted in two SG subcorpora of journal and conference papers with 1073 before and 1073 after GDPR coming-into-force.

Fig. 1.
figure 1

Study flow selection process; steps of screening and exclusion of studies with reasons

Full size image

3.2 Ethics Approval and Informed Consent

Lexicometric analysis showed that SG studies report little about their ethical conduct. Only about 29% of the studies before GDPR introduction were reporting one of the investigated ethical aspects. However, overall ethics reporting increased in the more recent study corpus, with about 42% of studies reporting at least one of the elements. This rising tendency in ethics reporting can be observed in Table 4 as all terms were found more frequently in the more current research. Reporting about ethics committees and approval has increased, although both still are stated by fewer than 10%. Context analysis (Fig. 2) then revealed that the vast majority of researchers were reporting to get approval from a committee at their own university or hospital in both periods before GDPR [10] and after [26].

Table 4. Percentages of papers before/after GDPR reporting on ethics approval and consent
Full size table
Fig. 2.
figure 2

Ethics approval reporting before/after GDPR in per cent of total reporting per subcorpus

Full size image

Moreover, two trends became apparent when looking at the approval bodies and comparing before and after subcorpora:

First, researchers are turning more towards not naming the committee but rather stating only that a university has approved [7]. Second, the reporting becomes more diverse with researchers indicating the reference number of the approval document but not the approval body [49] and others reporting just approval without any other information [2] or stating an approval was not necessary [17]. Generally, it can be noticed, that roughly half of the studies (51% before and 44% after GDPR) reporting ethics committee/approval are Exergame studies as compared to only about 20% in each subcorpus total share.

3.3 Ethics Guidelines

By analysing ethics guidelines reporting, even fewer studies can be found that refer to that aspect. About 100 studies of 2146 in total are reporting the application of ethics guidelines which is only 3.6% before and 6.3% of the SG studies after GDPR introduction (Table 5). Again, the proportion of Exergame studies in the rare papers that refer to ethical guidelines is high, with about 60% in the before and 47% in the after subcorpus. SG researchers are only reporting with four of the nine examined phrases. None of the studies has directly stated to follow the Clinical Trials Directive, the Common Rule, any national or international Guidelines for Research Ethics, the Nuffield Council on Bioethics, or the Universal Declaration on Bioethics (UNESCO). Context investigation then disclosed that the Declaration of Helsinki (DoH) was by far the most cited ethical code reported before [8] and after the GDPR effective date [12] (Table 5).

Table 5. Percentages of papers before/after GDPR reporting on ethical guidelines
Full size table

3.4 Data Protection

In the concluding analysis of data protection coverage, the authors of SG studies showed an equally reduced reporting behaviour as for the ethical guidelines (Table 6). Although there is a small increase in data protection reporting between the two study collections, both are at low percentages. Only about 3.7% (before) and 5.6% (after) report on the data protection aspects that were examined. The share of Exergame studies in reporting data protection is, however, lower as within reporting of ethics committees/approval and ethics guidelines. In the period before GDPR introduction, eight studies or roughly 27% and in the period after nine studies or about 19% are concerned with Exergames when reporting on data protection.

Table 6. Percentages of papers before/after GDPR reporting on privacy and data protection
Full size table

Examining the context of data protection phrases revealed that authors emphasised most the anonymisation of study artefacts such as transcripts [30], spreadsheets [38], blog-posts [35] or usage/interaction and activity profiles [24]. In both sub-collections, studies only sporadically reported on anonymisation of participants while also sometimes referring to “anonymising” when actually “pseudonymising” was performed [39]. Only two studies reported concrete pseudonymising after the GDPR introduction [13, 20]. Equally rare was coverage about encryption with only 1% of authors referring to it in each subcorpus by describing, for instance, secured communication [15] and encrypted storage context [23].

As regards compliance with GDPR and reporting thereof, none of the studies in the subcorpus before the GDPR effective date were found mentioning the regulation. However, remarkably there were also only five studies in the two years after GDPR coming-into-force that reported some form of compliance with the directive. By examining the characteristics of GDPR usage in these studies, it became apparent that the regulation was referred to in terms of compliance [18, 36] but also to clarify privacy requirement [13], identifying the two GDPR data-related roles (controller and processor) in connection with the study [15] and demands concerning data storage [20].

4 Discussion

In the light of the outlined findings, it can be noticed that all three investigated areas, research ethics, ethics guidelines and data protection are in general reported in a minimal range. Analysis of the broad collection of studies between 2016 and 2020 revealed little support for changing trends in reporting practice of research ethics or data protection. Researchers kept referring to Serious Games for their game-oriented studies while Exergaming claims a stable share and the term Applied Game did hardly gain a foot in the community. Concerning the first research question, lexicometric analysis showed a minor rising trend reporting ethics approval and informed consent.

However, the slightly higher quota of journal articles in the after GDPR study collection could have contributed to this under the assumption that authors report more details in journals than in conference papers. The most important ethics approval board for SG researchers before and after GDPR introduction remains their home university ethics committee. There is, however, some indication that designating the approving body generally becomes less regarded. Although SG research is predominantly human-centred, only between 10 and 15% of studies are reporting on their ethics approval or informed consent. There is also no evidence from the comparative before/after analysis that would indicate a fast or considerable change of practice. This could present a considerable hindrance for SG applied in research to be trusted as effective and ethically sound. Especially considering that often children, elderly or disabled people are the ones who should benefit from SG. Thus, instead of keeping a low ethics reporting profile, ethical conduct should become an integral part of a SG from the start. In-game informed consent and modular game structures with secure data communication as well as pseudonymised storage of data should become the basis, not the exception of SG research. A transparent process for participants that also involves briefing and debriefing during a research-game session is an attainable objective.

The second question asked on reporting of ethical guidelines and privacy policies can be answered with clear findings from this review. About 80% of studies reporting guidelines declared to follow the DoH. There is no indication of a change of this from the data analysis. Rather, a consistent picture has been revealed over the last four years of SG research. However, the overall reporting of followed ethical principles remains very low (< 6%) in SG studies.

Regarding the third question and data protection policy, there was no preferred use/reporting of guidelines visible in neither the subcorpus of SG studies before nor after GDRP introduction. Data protection and privacy policy are extremely rarely addressed with only one study out of 2146 reporting on privacy policy [33]. Although the GDPR was broadly discussed long before the date of coming-into-effect [3] authors of the examined SG studies chose not to participate in this discussion. Equally remarkable, two years after the binding regulation has come into force, only five studies out of about a thousand are referring to the directive. Indeed, that is the impact of the GDPR on SG publication revealed from comparatively analysing the two study-sets. The findings do not allow for conclusions on a general rising trend in ethics reporting as practices from other time periods would need consideration in this regard. Yet, a considerable contribution from GDPR to such a potential trend could not be found in the data of this study.

Since the results on reporting of ethics and data protection are concerning, it is suggested to repeat the scoping review protocol of this study in a two-year interval to investigate the course of impact of GDPR. At this point, however, Albrecht’s [3] question of how GDPR will change the world can be answered from the perspective of SG research reporting: If at all, then hardly noticeable.

5 Conclusion

The conducted scoping review has provided insight in ethics reporting practices of Serious Games researchers. While researchers showed some increase in reporting on parts of scientific integrity, there was no substantial change found when looking at the writing about ethical guidelines, data protection or the GDPR. Aside from finding only marginal change from GDPR, the scoping review has found that most SG researchers who report on ethics obtain ethics approval from a review board at their own university and follow the Declaration of Helsinki. The outlined problems of conduct and transparency could be addressed in future studies by not only making the situation visible but by supporting SG creation in research with ideation toolsets that include data protection advice and building blocks that comply with GDPR from the very beginning of the development. Ultimately, with GDPR, there are now clear data protection regulations that can be operationalised. Since designing SGs frequently involves matching rules of a domain into a game, data protection rules can be part of the balancing process. Accordingly, this process could then be facilitated with design toolsets oriented on SG research.