Introduction

Supporting people to accept difference, avoid conflict and engage effectively across cultures continues to be essential (UNESCO 2011). Education is a key tool, with many countries incorporating intercultural learning into the curriculum (Kirchschlaeger et al. 2012). Intercultural education aims to bring about an overall improvement of people’s performance in situations taking place in intercultural contexts (McDonald et al. 2008; Gudykunst et al. 1996). Much intercultural education has targeted adults, however, it is increasingly essential that children are also provided with the skills, competences and strategies required to support intercultural engagements.

Intercultural education can be categorized along two dimensions, relating to the learning process and the content focus (see Fig. 1). Whilst there is plentiful didactic and culture-specific material available for children in hard and soft copy, opportunities to engage in experiential learning and particularly to develop culture-general competence and skills are more limited. For many 9–11 year old school children, cultural learning focuses on culture-specific festivals, rituals and highlights.

Fig. 1
figure 1

Intercultural education: learning process & content focus

Full size image

Whilst participation in festivals does offer some degree of experiential learning, this stimulates culture-specific knowledge, educating the child about a culture, rather than providing transferable skills that would enable understanding of any culture. There is a need for approaches that extend learner’s competences by providing them with a series of concrete skills that can then be applied in unfamiliar situations (Fowler and Blohm 2004).

Intercultural education strives to create a harmonious society, where learners are culturally sensitive (Bennett 1986) and empathic, being able to understand another individual’s experience as it is embedded in their culture (Mullavey-O’Byrne 1997). To develop intercultural understanding and sensitivity, children clearly need to learn about challenging intercultural experiences, such as intercultural dissonance and its potential to lead to conflict (Mahler 2012). We define intercultural conflict as conflict of which the cause can be traced back to intercultural dissonance or misunderstandings due to differences in culture. Ting-Toomey defines” cultural conflict competence” as “the mindful management of emotional frustrations and conflict interaction struggles due primarily to cultural or ethnic group membership differences” (Ting-Toomey 2004). Here, we focus not only on group difference but also specifically on differences in culture beyond group membership.

Providing children with experiential learning that supports the development of intercultural conflict competence results in the immediate challenge of which cultures should be in conflict and which aspects of culture should be used to highlight dissonance. Using real cultural conflict is fraught with difficulty and may reinforce stereotypes and antagonism rather than develop the desired intercultural competence learning. In the classroom, intercultural conflict is often addressed through culture-specific scenario-based intercultural exercises, with short videos and/or texts and websites supplemented with discussion topics. Role play is also used in supporting the creation of knowledge and meaning from concrete, culture-specific, though imagined, experiences (Kanfer 1979). Simulation games can address intercultural conflict using a culture-general experiential learning approach, for example, RaFa RaFa the children’s version of BaFa BaFa (Shirts 1995) and BARNGA (Thiagarajan 2006).

Virtual Learning Environments (VLE) populated by autonomous agents that are able to simulate human-like behaviour, including cultural behaviour provide an exciting opportunity for intercultural learning experiences. Although VLEs and virtual agents have been developed for intercultural role play and simulation experiences, these are typically for adult learners. Some use specific real-world cultures for their experiences (Endrass et al. 2011; Johnson and Valente 2009; Wu and Miller 2010), whilst others have used imaginary cultures (Aylett et al. 2009; Mascarenhas et al. 2009). This has resulted in growing interest in creating agents that exhibit cultural behaviour. However, the majority of these are culture-specific to a real culture. For example, the Culturally Affected Behaviour (CAB) model (Solomon et al. 2008) allows the encoding of specific ethnographic data on cultural norms, biases and stereotypes, which are used to influence the behaviour of agents. CUBE-G (Rehm et al. 2007) focuses on the modeling and generation of appropriate nonverbal communication aspects of different national cultures.

This paper presents an alternative culture-general experiential learning approach, MIXER (Moderating Interactions for Cross-Cultural Empathic Relationships), a VLE populated by intelligent, affective and interactive characters targeted at children. MIXER aims to enable 9–11 year olds to experience and learn about intercultural conflict, highlighting strategies and supporting the development of skills and competences. MIXER was developed and evaluated by the eCUTE (Education in Cultural Understanding, Technology Enhanced) project (see www.ecute.eu). In this paper, firstly, the background to our approach for developing intercultural conflict skills and competence is presented. We then briefly describe MIXER’s scenario and design, followed by an overview of our architecture that incorporates mechanisms to generate culturally appropriate affect and behaviour. The approach used to evaluate the learning achieved through using MIXER, considering near and far transfer is outlined along with the results from a 144 children study in the UK. The final section provides a discussion of our results and of the potential of intelligent agents and cultures for learning about intercultural conflict.

Developing Intercultural Skills and Competence

Hofstede et al. (Hofstede et al. 2010) define culture as “the collective programming of the mind that distinguishes the members of one group or category of people from another.” They empirically found a limited set of issues on which societies differ systematically throughout their values and practices. Countries can thus be placed in a six-dimensional space in which their value systems are more or less similar. The Hofstede dimensions are zoomed-out abstractions, derived from aggregated society-wide analysis. They describe the result of this ‘collective programming of the mind’, not the mechanisms. Those mechanisms, the everyday collective programming, involve practices (heroes, symbols, rituals) that are enacted through a myriad of interactions between people. Many of those interactions take place in the context of rituals (large and small) that involve religion or nationality, alongside all possible other visible social categories, norms, roles and rules. The crucial formative interactions take place during infancy and childhood. This programming creates patterns of thinking, feeling and behaving that are shared and learnt by members of the same culture, notwithstanding wide individual variety. Intercultural conflict is frequently the result of a mismatch of these patterns. Through understanding concepts that capture aspects of culture, (such as cultural value patterns) self- and other-awareness is increased, and contributes to competent management of intercultural antagonisms (Ting-Toomey 2009).

Synthetic cultures (Hofstede et al. 2002) are culture scripts based on the manipulation of the value orientations of Hofstede’s Dimensional model (e.g. Power Distance, Uncertainty Avoidance, etc.). Synthetic cultures enable an understanding of culture as being a group with common and collective expectations and behaviours. This negates our understanding of culture being necessarily tied to visible characteristics such as race, religion, nationality, etc. but instead views culture as the collective programming of the mind that distinguishes the members of one group or category of people from another. As indicated above there could, but need not, be a role for race, religion and other factors in shaping that culture. Synthetic cultures provide a useful approach to simplifying the complex notion of culture, providing understandable yet unreal cultures, creating an environment in which to experience culture-general learning through simulating intercultural encounters with virtual characters from different synthetic cultures.

Within this perspective, a basic goal of intercultural learning is to accept people belonging to a given out-group into one’s own “moral circle” (Pizarro et al. 2006; Singer 1981). A “moral circle” is formed by those who adhere to a common purpose and group identity and those who follow a common set of moral rules and who trust each other (Hofstede et al. 2009). Moral circles can be formed in many situations, from the moral circle formed by a class or village to haphazard, ephemeral groupings of people such as those in a theatre queue. Each social setting creates its own ‘moral circle’ consisting of all people who belong to that setting. Consequently, its rules only apply to those who belong, and may only be known to the moral circle’s members. These unwritten rules for behaviour are used to monitor our own and others’ performance. All kinds of covert and overt feedback signals and sanctions are used to try to enforce good behaviour.

In many cases, different groups have different practices, and so at that level a learning process is needed before a visitor can achieve any level of intercultural competence. Language is the most obvious example. But, note also, that language is not enough; without education in cultural practices and values, one cannot really become competent in a different culture even if one learns the language. In MIXER we focus on a simple set of explicit, if unwritten, practices and stay away from the linguistic element by confining ourselves to differences in game rules.

A key focus for intercultural learning in MIXER is for the learner to be willing to include out-group members in their own moral circle and to try to understand their unwritten rules. To achieve this, MIXER has been developed to support learning within the Intercultural Competence Learning Framework (Swiderska et al. 2011), (see Table 1). The framework extends Bennett’s Developmental Model of Intercultural Sensitivity (Bennett 1986), by focusing on the intercultural competencies that facilitate or moderate the development of intercultural sensitivity (Spitzberg and Changnon 2009).

Table 1 eCUTE intercultural competence learning framework
Full size table

Mixer: Learning Approach and Goals

In MIXER, the child user starts out by observing (and judging) the superficial levels of a moral circle (practices). The next step is trying to discover the underlying rules of that moral circle. These rules can (and most likely will) evoke strong emotions in a person, since they determine what is right or wrong in the game. If the child perceives only the practices but not their cause in the underlying rules, this will lead to the child rejecting the group that uses these practices. If the child understands that at the deeper level of fair play the other group makes sense, they may try to learn the different rules as just new ways of achieving fair play. To the extent that their attitudes towards “the others” change, learners should start to react with positive emotions to members of another culture. Thus, a key learning aim for MIXER is for the children to be willing to include the out-group’s members in one’s own moral circle, and try to understand their unwritten rules.

Following the framework in Table 1, starting as a Beginner, it is important to become aware of the specific practices and values of the moral circle. This is defined by the rule set of the game. The Journeyman integrates the knowledge acquired in the beginner’s learning goal, and tries to integrate it within their mental model of the world. By trying to shift perspectives, children begin to understand on a basic level the differences and similarities between their own group and another group. They should now be able to describe the game from the point of view of people from both their own in-group and from the other out-group. The cognitive, emotional, and behavioural learning goals of MIXER should enhance development of awareness, sensitivity, and communication in intercultural competence.

The Mixer Application

MIXER is a computer-based learning experience targeted at 9–11 year olds for use in the classroom context. MIXER engages users in an interactive narrative set in a virtual summer camp where two groups of school children (intelligent agents) play Werewolves, a popular intergenerational game widely known in many cultures.

MIXER depicts a peer conflict scenario, occurring when Tom (protagonist) plays the Werewolves game with two different groups of children at a summer camp. The teams are represented as the Yellow team (in-group) and the Red team (out-group). Each team is composed of six intelligent agents - a game-master, a werewolf and four villagers. These teams provide two cultures or moral circles, each with different values, represented as rule sets for playing Werewolves. Within the Yellow team, the rules are that each player takes turns to say who they think the werewolf is and why, the player with the most votes is then killed off and is out of the game. In the red team, one player states who they believe the werewolf to be, if they do not have majority agreement from the other players then they themselves are killed by the villagers and they are out of the game. As the scenario unfolds, it becomes evident to Tom and the child user that the groups adhere to different rules (reflecting that they belong to two different cultures or moral circles). Tom accuses the red team of cheating, because he does not understand the rule change, and Tom tells the child user that he thinks they just don’t want him to play. MIXER ends by Tom resolving the conflict with the Red team by discussing the differences in the two versions of Werewolves. The final scene in MIXER shows that Tom understands and accepts the rules of the Red team and Tom tells the child user that the red team rules sound “Pretty cool” and that he can’t wait to try out the new rules. This reinforces the message that sometimes what may appear to be unfair or strange behaviour may actually be due to simple differences that can be easily resolved and can result in positive outcomes. This observation and interaction provides the basis for accepting people belonging to a given out-group into one’s own’moral circle’ (Hofstede et al. 2009). It also encourages thought about appropriate coping strategies in conflict situations (Ting-Toomey 2009).

In MIXER, the child user does not directly appear in the virtual world. Instead their role is to interact with Tom, as an invisible friend and to support his play by responding to Tom’s requests for advice on how to react and what to do at different stages of the game. The aim is for the child user to learn, by observing and advising Tom. This positioning of the learner as the invisible friend increases empathic engagement between the child and the synthetic friend character (Paiva et al. 2005). Additionally, detaching the child user from actually playing the game themselves can also produce a distancing effect so as to promote reflection (Aylett et al. 2005).

Mixer Implementation

The graphical visualization of the summer camp and the characters was implemented in the Unity game engine (Helgason 2014). The characters (see Figs. 2 & 3) were designed to be similar to the intended users, with age appropriate appearance and voices, including boys and girls with a mix of ethnicities. As is common in summer camps, the children were dressed in team T-shirts, representing the two teams: the Reds and the Yellows.

Fig. 2
figure 2

Alex and Lisa, characters from MIXER

Full size image
Fig. 3
figure 3

The yellow team during discussion (left) and accusation (right) phase of the Werewolves game

Full size image

The Reds and the Yellows provide the moral circles of MIXER. Each team is formed by 6 intelligent agents (see Fig. 3) that each have intelligent affective behaviour. Such agents with their own goals, motivations and emotions, autonomously select their actions, giving users a sense that they are ‘alive’ and believable. This is achieved using FAtiMA, (Fearnot AffecTIve Mind Architecture) a cognitive appraisal architecture (Dias et al. 2011; Dias and Paiva 2005) (see Fig. 4) in which events are assessed in relation to goals and generate an affective outcome as a result. These affective states are then used both in reactive action-tendency rules for immediate responses and in a deliberative planning system.

Fig. 4
figure 4

FAtiMA architecture (taken from Lim et al. 2012)

Full size image

The FAtiMA architecture has been used in several applications, such as FearNot! (Aylett et al. 2006), ORIENT (Aylett et al. 2010), SEMIRA (Ferreira et al. 2013). FAtiMA’s autonomous agents adopt a cognitive appraisal approach (Dias and Paiva 2005). So, events in the world are perceived by the agents and evaluated in relation to the agent’s internal state (thus, the agent’s attitudes and goals) and from there, an affective outcome is triggered as a result. These affective states are then used in reactive action-tendency rules for immediate responses of the agent. The affective states are also used in a deliberative planning system, generating more complex coping behaviours based on the emotions of hope and fear.

In MIXER we have used FAtiMA to define the agent behaviour, involving establishing their specific goals, actions, inference rules, emotional reactions and action tendencies. Two types of goals were modelled. Active-pursuit goals, representing the goals that the agent actively pursues to achieve a certain state (e.g. to accuse another player), and interest goals are goals that the agent continuously maintains to avoid threatening situations (e.g. to stay in the game – not be ‘killed’).

The advantage of a cognitive appraisal architecture like MIXER is that the basic mechanism of assessing events according to the agent goals and generating an affective state as a result can easily be extended to assess the likely behaviour of other agents. If an agent takes the action it is considering and re-runs its own architecture as if this was an incoming event, it is assessing how another agent would react to this projected action. This is a simulation view of how a Theory of Mind might work and is illustrated in Fig. 5.

Fig. 5
figure 5

Abstraction of FAtiMA to show incorporation of theory of mind

Full size image

FAtiMA’s agent model was extended for MIXER to incorporate Theory of Mind (further described below). The other elements of the Agent Model can be briefly described as:

  • Emotional state: an emotional reaction rule is associated with an external event to appraise variables, desirability of the event, desirability for others and praiseworthiness of the action. The exact value of the variable for each rule is authored beforehand.

  • Memory: FAtiMA’s agents have autobiographical memory, they are aware of their past actions, experiences and emotions. For example, FAtiMA appraises events to generate emotions and this information is stored in the autobiographic memory for future reference, assisting the agent in selecting (re) actions.

  • Deliberative layer: this layer provides deliberation and practical reasoning processes. All behaviour is driven by predefined goals, with the goals of the MIXER agents related to playing Werewolves.

  • Reactive layer: the reactive appraisal process matches events to a set of predefined emotional reaction rules providing a mechanism to appraise and react to certain situations.

In MIXER, the intelligent agent’s appraisal, reactions and action generation are based on knowledge of and relations with others and are consequently culturally dependent. This is achieved through a Theory of Mind mechanism consisting of a collection of Models of Others (see Fig. 6), each representing the beliefs of a particular known agent. A 1-level Theory of Mind is modelled which means that a Model of Other will have its own Theory of Mind including Models of Others, resulting in a recursive hierarchical tree (Aylett et al. 2014). This Theory of Mind enables the agent to play the Werewolves game, as it requires making decisions based on an agent’s interpretation of other players and their motives. With the rational FAtiMA agent minds, the agent will choose options or goals to maximise their expected utility. In this case, to not be ‘killed’ and thus out of the game. This provides the two teams with specific requirements associated with the need to behave in a group, according to the unwritten rules, in this case the voting mechanism to determine who is ‘killed’ in Werewolves. Making the utility of goals, and thus decision-making, depend on cultural values, a link between culture and behaviour can be established. Through the specification of Theory of Mind preconditions to be tested against a particular Model of Other, the deliberative component is thus able to trigger goals according to the beliefs of others within their moral circle.

Fig. 6
figure 6

Theory of mind model

Full size image

The Theory of Mind used in MIXER encodes equivalents to Baron-Cohen’s (Baron-Cohen 1995) Eye-Direction Detection (EDD) and Shared-Attention Mechanism (SAM) capabilities, which work on an agent’s incoming perception so it can run the Theory of Mind mechanism for the perspective of a different agent (Ruth Aylett et al. 2014).

In MIXER, the child interacts with Tom through a tablet using a Pictorial Interaction Language (Endrass et al. 2014), (see Fig. 7). The Pictorial Interaction Language provides children with access to over 70 graphics structured for use in sentences, enabling them to create their advice for Tom. In addition to being fun, intuitive and engaging, this approach also reinforces the child’s role as invisible friend. Whilst everyone can see the virtual environment (on the computer screen), only the child themselves can see their ‘private’ communication with Tom on the tablet.

Fig. 7
figure 7

Pictorial Interaction language

Full size image

Mixer Evaluation

Design

The evaluation for MIXER was designed and developed using Transmedia Evaluation, which seamlessly embeds evaluation into the user experience by creating evaluation materials and activities that are both appropriate and engaging for the target age group, and that collect relevant, useful and high quality data for the research team (Hall and Hume 2011). The aim for the MIXER evaluation was to identify if the following learning goals were achieved:

  • Emotional: MIXER supports children to recognise emotions (for example fear and anxiety) when dealing with the strange behaviours of another group

  • Cognitive: MIXER supports children to start learning the specific practices and values of another group

  • Behavioural: MIXER supports children in being fully present in attending to others’ verbal and non-verbal messages

In addition the following goal related to whether the MIXER technology was an effective approach for technology enhanced learning:

  • Experience: MIXER engages children in the narrative and with the characters, supporting the children’s understanding and learning of strategies for coping with intercultural conflict

To establish if these learning goals were met, MIXER was evaluated in two ways: 1) Directly after the MIXER interaction aiming to identify if MIXER resulted in near transfer or immediate learning; and 2) With a pre- post- test design focusing on whether interacting with MIXER resulted in far transfer or sustained learning. Figure 8 outlines the phases of the evaluation, which included the completion of 3 activity style workbooks and a discussion session (CDF) for children.

Fig. 8
figure 8

MIXER evaluation design

Full size image

Sample

Children aged 9–11 years (N = 144) participated in the MIXER evaluation at Broadway Junior School, Sunderland. Table 2 illustrates the sample demographics.

Table 2 Evaluation sample demographics
Full size table

Materials

Near Transfer Measures – Workbook 2 & CDF

Workbook 2 provides the Experience Evaluation Questionnaire (EEQ). This instrument was developed to evaluate the user learning experience in VLEs populated by embodied characters, based on Hall et al. (2013) and Hall et al. (2006). The EEQ collects data related to children’s immediate learning; their narrative comprehension, empathic engagement; and their perspectives and views of the MIXER characters and experience. The EEQ addresses all four MIXER learning goals: emotional, cognitive, behavioural and experiential. Table 3 outlines the activities incorporated into the EEQ.

The EEQ was provided as Workbook Two in the MIXER evaluation and given to children after their interaction with MIXER. Activities focused on learning, comprehension and responses to MIXER, as can be seen Fig. 9.

Table 3 Experience evaluation questionnaire
Full size table
Fig. 9
figure 9

EEQ Workbook and sample pages

Full size image

Classroom Discussion Forum

After completing the EEQ, children took part in a 20 min Classroom Discussion Forum (CDF) involving a question and answer session led by a trained researcher. CDF questions targeted narrative comprehension, perceptions of red and yellow team behaviour, views of characters and MIXER, etc. In addition to assessing user learning and gaining feedback, the CDF also aimed to further scaffold the children’s learning of intercultural conflict. A key goal of the CDF was to encourage self-reflective learning to take place, consolidating the engaging learning experience that children had with MIXER. Within the CDF, the discussion reinforces that different rule sets are used rather than one team cheating. The CDF also highlights the importance of communication to resolve conflict in the MIXER scenario.

Far Transfer Measures – Workbooks One & Three

The three measures aimed at assessing far transfer were taken from the CQS - Cultural Quotient Scale (Ang et al. 2007); the MESSY Scale - Matson Evaluation of Social Skills (Matson et al. 2010) and Bryant’s Empathy Index (Bryant 1982). The behavioural subscale of the CQS was used as a pre- and post- measure of a child’s capability to adapt verbal and nonverbal behaviour in different situations and cultures. Factor One from the Bryant Empathy Index focuses on understanding feelings and was used as a measure of children’s empathic behaviour. Factor 2 - Social Skills/Assertiveness of the MESSY questionnaire was used to assess the child’s self-perception of their own social skills and competences.

Workbooks One and Three contained the pre and post-test measurements and were given to children a week before interacting with MIXER and a week after. The workbooks contained the same scales, but were designed to look very different from one another, as shown in Fig. 10. This was done to alleviate the repetition that children may feel when completing a post test and it was hoped that this would maintain participant engagement throughout the evaluation procedure and reduce response biases such as satisficing (Krosnick et al. 1996).

Fig. 10
figure 10

Two versions of Bryant’s empathy index transformed using the Transmedia evaluation approach

Full size image

Procedure

Children participated in 3 sessions over 3 weeks, involving pre-test, test, and post-test sessions. The sessions were delivered in a classroom environment by a trained researcher. Children were randomly allocated to either the control or MIXER interaction condition. Children allocated to the control condition watched a video about camping, lasting for a comparable time to the MIXER software interaction. Children in the control condition then completed a related activity book rather than the EEQ.

Pre-test (workbook one) and post-test (workbook three) instruments were completed in weeks one and three. These sessions were conducted as whole class sessions. Children were each given a workbook, a set of stickers and pens/pencils and were instructed to complete the workbook without discussing it with their classmates. Children were left to complete the workbook with as little assistance as possible but were given help if requested.

Near transfer was assessed in week two. Children in the experimental condition completed an interaction with MIXER using a laptop with headphones and the Pictorial Interaction Language on an iPad. The interaction occurs in parallel with on-going class activities. Small groups of children leave the classroom and interact with MIXER in a separate room. After all of the children have interacted with MIXER, workbook two is completed following the protocol described above for workbooks one and three. The session concludes with the children who took part in the MIXER interaction taking part in the CDF. The researcher begins by explaining that they would like to discuss the MIXER application with the class and that children must put their hands up before speaking. The facilitator allows each child to respond and asks children to elaborate or ask questions in response to comments as appropriate in order to gain as much detail from the children as possible before moving onto the next question.

Results

The results section firstly reports on the near transfer of learning outcomes about intercultural conflict providing analysis from the EEQ and CDF data. This is followed by the results of the pre-post evaluation that explored the far-transfer learning outcomes. Chi-square cross-tabulations were calculated on the EEQ data and repeated ANOVA was carried out on the pre-post (T1/T2) data for the CQS, Bryant’s Empathy and Messy questionnaires.

Near Transfer of Learning Outcomes for Emotion Goals Influences on wanting to be Friends with Tom

A key goal for MIXER is for children to engage with and “care” about Tom, taking the role of the invisible friend. Engagement and ‘caring’ towards the central character, Tom is crucial to promote cultural awareness and comprehension. Children were positive about Tom and his werewolf experience, with 83.5 % thinking Tom had fun playing werewolves, 77.1 % feeling Tom was good at making new friends and 79.8 % believed that Tom was good at playing Werewolves. Nearly two thirds of children (65.1 %, n = 71) stated that they would want to be friends with Tom. A significant age association was found, [X 2 (1, 109), = 5.84, p = .05, Cramer’s V = .23], with more 9 year olds (77.5 %) stating that they wanted to be friends with ‘Tom’ compared to 10 (61.0 %) and 11-year (37.5 %) olds.

Helping, Listening and Friendship

The skills of helping, listening and friendship were essential for encouraging conflict resolution, and in turn cultural awareness and comprehension in the MIXER game. Chi-square cross-tabulations revealed a significant association between children wanting to be friends with Tom, and whether they felt that they had helped Tom resolve conflict between the red and yellow teams during the Werewolves game, [X 2 (1, 109) = 11.50, p = .001, Cramer’s V = .33]. Fig. 11 shows the link between children believing they had helped Tom during the game, and wanting to be friends with Tom (yes friends and yes helped = 95.8 %, n = 71, yes friends, didn’t help = 4.2 %, n = 3).

Fig. 11
figure 11

The association between children wanting to be friends with Tom, and whether they felt that they had helped Tom during the Werewolves games

Full size image

A significant relationship between children stating that they wanted to be friends with Tom and whether they felt that Tom had listened to what they had said during the game was uncovered, [X 2 (1, 109) = 6.12, p = .013, Cramer’s V = .24]. The majority of children who stated that they wanted to be friends with Tom, also stated that they felt that Tom had listened to what they had said (friends and Tom had listened to them, n = 66, 93.0 %, friends, but Tom did not listen to them, n = 5, 7.0 %). However, of those who did not want to be friends with Tom a high percentage still felt that Tom had listened to them (not friends, Tom listened to them: n = 29, 76.3 %, not friends, not listened to: n = 9, 23.7 %).

A small significant relationship was found between children’s views of whether they had helped Tom and whether Tom had listened to them during the game, [X 2 (1, 109) = 4.24, p = .04, Cramer’s V = .20]. 89.6 % of children, who believed that they had helped Tom, also felt that Tom had listened to them during the game. Just 10.4 % of children, who said that they had helped Tom, felt that they had not been listened to. However, 69.2 % of children who felt that they had not helped Tom still felt that Tom had listened to them.

Engagement with MIXER

Just over two thirds of children (67 %) would have liked to meet another group of characters, a positive result showing an interest in meeting different groups of children with probably different rule sets. Only 8.3 % of children would have liked MIXER to have lasted ‘for a shorter amount of time,’ with the majority (52.3 %) stated that they would have liked MIXER to have lasted for a ‘longer amount of time.’ This is a positive finding indicating that children were engaged with the MIXER software and enjoyed the interaction.

Only 8.3 % of children identified that they disliked MIXER, whilst just under 90 % of children stated that would want to use MIXER again, (89.9 %, n = 98). This is a positive result demonstrating that children enjoyed MIXER and were engaged with it, which is essential for learning outcomes to be achieved.

Near Transfer of Learning Outcomes for Cognitive Goals Narrative Comprehension

Narrative comprehension of the scenario and conflict was assessed through fact-based questions. The majority of children answered the true/false fact questions correctly: e.g. 94.5 % knew that there wasn’t a clown in the MIXER camp; 93.6 % correctly identified there was a character called Jorden; and 88.1 % selected ‘true’ that Tom met the yellow team first. In the Roving Reporter, 78 % of children correctly reported that Tom had played werewolves twice. 74.3 % correctly identified that Tom hadn’t known how to play when they first met him.

Children were more positive about the yellow team explanation of the rules, with the majority of children believing that the yellow (in-group) team (80.7 %, n = 88,) had explained the rules of the game the best. The majority of children stated that Tom preferred the ‘yellow’ team rules (82.6 %, n = 90), leaving 17.4 % (n = 19) believing that Tom preferred the ‘red’ rules (out-group). This is an expected result as the yellow team provided their explanation in a non-confrontational situation, prior to the game beginning. In contrast, the red team rules were explained after the game was played. Significant time was spent on ensuring that Tom (and the child) understood the yellow team rules, with a clear walkthrough of the rules.

Learning about Intercultural Conflict and Resolution

The aim of MIXER is for children to understand, learn and accept that different cultures have different rules and that this is not “cheating” but difference. From a learning perspective, the desired outcome is that children identify that neither team is cheating. However, the results do not suggest that this learning outcome was fully met. Figure 12 shows only 28.4 % of children felt that neither team had cheated (n = 31). The largest percentage was for the red team (out-group) cheating, and the yellows (in-group) not cheating (46.8 %, n = 51). These results indicate that nearly 50 % of children demonstrated a preference for the in-group and are suggestive of children not fully appreciating the concept of ‘cultural difference’.

Fig. 12
figure 12

The percentage of children who felt that the red, yellow, both or no teams cheated

Full size image

Near Transfer of Learning Outcomes for Behavioural Goals Perceptions of Tom

More positive attitudes and perceptions towards Tom (e.g. ratings of Tom having fun, helping Tom) were taken as evidence to suggest that children were aware of the different cultural scenarios at play in the game between the Red and Yellow teams, and understood the advise they had provided Tom with.

Table 4, below, illustrates children’s perceptions of Tom (had fun, knew what he was doing, good at game) in relation to whether children wanted to be friends with Tom, had helped Tom and had been listened to. Children who wanted to be friends with Tom also felt that Tom had fun (88.7 %), [X 2 (1, 109) = 4.07, p = .04, Cramer’s V = .19]. Just over two thirds of children who wanted to be friends with Tom also felt that Tom knew what he was doing, whilst 58 % of children who did not want to be friends believed that Tom was confused during the game, [X 2 (1, 109) = 6.44, p = .01, Cramer’s V = .25]. Children who felt that they had helped Tom also believed that Tom had fun during the game (86.5 %), [X 2 (1, 109) = 5.15, p = .02, Cramer’s V = .22]. Children who said they had helped Tom were also more likely to believe that Tom knew what he was doing (62.5 %), [X 2 (1, 109) = 4.76, p = .03, Cramer’s V = .21]. Children who believed that they had helped Tom during the game were more likely to think that Tom was good at the game (83.3 %), [X 2 (1, 109) = 6.18, p = .01, Cramer’s V = .24]. Finally, children who felt listened to were more likely to believe that Tom had fun during the game (87.4 %). [X 2 (1, 109) = 8.09, p = .004, Cramer’s V = .27].

Table 4 Associations between wanting to be friends with Tom, feeling they had helped Tom and feelings that Tom listened and children’s perceptions of Tom’s game performance
Full size table

Near Transfer of Learning Outcomes: CDF Results

Children’s narratives from the Classroom Discussion Forums reflected and reinforced the findings from the EEQ data. Children reported their experience of using MIXER as “great”, “good” and “OK” and said that it wasn’t like games that they normally play. During the discussion of the MIXER story the children were able to recall details such as the name of the game played by the characters in MIXER, “Werewolves”, and identify that the two teams of characters did not play the game with the same rules. During a discussion of the rules a lot of children expressed that “The red team cheated”, “The red team played it wrong”, “The red teams rules weren’t fair because if you made a wrong guess and people didn’t think the same as you, you were out. That’s not fair.” When chatting about Tom and the gameplay all children recognised that Tom became upset at the red team, the children gave responses such as “The red team cheated”, “He (Tom) felt like the odd one out because he didn’t know the rules” and “Nobody told him the new rules on purpose” demonstrating that children were aware of the conflict situation. On asking how the game with the red team ended, the children explained that Tom misunderstood the red team rules but that the conflict was resolved following an explanation of the red team rules, “The red team were ok by the end but at first I thought they were mean”.

The CDF results show that the children engaged with and understood the MIXER application and its narrative. Children demonstrated empathy towards Tom and an understanding of the cause and resolution of the conflict. This learning only occurred with the inclusion of the classroom discussion forum, where the lessons of MIXER were discussed and reinforced by teacher led discussions.

Far Transfer of Learning Outcomes for Emotional Goals Bryant’s Empathy Index Results

Higher scores are indicative of higher empathy levels. The mean baseline (T1) empathy score for children was M = 6.39 (SD: 1.81), and M = 6.34 (SD: 1.70) at post-test (T2). A repeated measures ANOVA with time (T1 and T2) as the repeated factor, and condition (MIXER vs. Control) as the between measures factor was calculated to examine the effects of Bryant’s empathy scores after interacting with the MIXER software. The effects of gender were controlled for as a covariate. No significant main effect of time or condition was found, and there was no significant interaction effect. Figure 13 below illustrates that children’s empathy scores in the control condition slightly decreased over time (T1: M = 7.14, SE = .39; T2: M = 6.68, SE = .37), whilst children’s scores in the MIXER condition remained constant (T1: M = 6.24, SE = .19; T2: M = 6.30, SE = .18), suggesting that the MIXER software did not result in increased empathy among children.

Fig. 13
figure 13

Children’s mean Bryant empathy score pre-post- test for the MIXER and control condition

Full size image

Far Transfer of Learning Outcomes for Cognitive and Behavioural Goals CQS Results

Children’s CQS baseline scores ranged from 5 to 25, with a mean score of 13.34 (SD: 4.91), and a mean post-test score of 16.78 (SD: 6.05). This illustrates that children’s CQS scores were higher after interacting with the MIXER software. A Repeated ANOVA was carried out to determine whether there were any significant differences in CQS scores between the MIXER and control condition at baseline (T1) and post-test (T2). A significant interaction effect between time and condition was found, F (1, 120) = 16.02, p < .001, ηp2 = .19. Figure 14 illustrates that children’s CQS scores increased after the MIXER interaction (T1: M = 14.39, SE = .46; T2: M = 18.74, SE = .42) whilst they decreased in the control condition (T1: M = 10.13, SE = .94; T2: M = 8.13, SE = .86).

Fig. 14
figure 14

Children’s mean CQS scores pre-post- test for the MIXER and control condition

Full size image

MESSY Results

Children’s baseline Messy scores ranged from 43–75 showing relatively high social interaction skills, with a mean Messy baseline (T1) score of 59.73 (SD: 7.08) and a mean post-test (T2) score of 59.99 (SD: 8.83). A repeated measures ANOVA with time as the repeated factor and condition as the between measures factor was carried out on baseline and post-test Messy scores. No significant main effects of time or condition emerged, and there was no significant interaction effect between time and condition. Figure 15 illustrates that children’s Messy scores remained constant in the MIXER condition (T1: M = 60.06, SE = .72; T2: M = 60.92, SE = .87), whilst they slightly decreased in the control condition (T1: M = 58.24, SE = 1.55; T2: M = 56.43, SE = 1.87). This suggests that the MIXER software had no impact on children’s social interaction scores.

Fig. 15
figure 15

Children’s mean Messy scores at pre-post- test for the MIXER and control condition

Full size image

Results Summary

The MIXER Evaluation sought to address and meet the emotional, cognitive and behavioural learning goals for both near and far-transfer learning. Table 5 outlines evidence demonstrated by the children during the MIXER evaluation for each of these learning goals.

Table 5 Summary of learning goals results
Full size table

Discussion

MIXER provides experiential learning placing the child in the role of Tom’s invisible friend. Tom was a successful character, with nearly two-thirds of the children wanting to be friends with him. Children liked Tom, found him fun and had positive opinions about his game playing activity and experience. For children to accept a character as their friend within their moral circle and in-group highlights the potential of using virtual characters to experience empathic cultural relationships.

MIXER contributed to near transfer across all of the learning goals. Children cared about Tom and empathized with him, providing advice based on their understanding of his engagement with the Red and Yellow teams. This contributes to the emotional learning goal, with children able to recognise their emotions when dealing with the unknown behaviours of another group, and to provide advice based on their interpretation to Tom. Those children who wanted to be friends with Tom were more likely to feel that he listened to them and that they had helped him.

Children had high levels of narrative comprehension and engagement with the MIXER scenario. Results suggest that children met the cognitive learning outcome, able to start learning the specific practices and values of another group, engaging with Tom in playing Werewolves with different groups of children. The rules of the Werewolf game are presented to the children in two ways in MIXER. Children readily identified and accepted the conflict situation and understood why Tom was upset and angry with the red team. They understood that Tom was a member of the yellow ‘in-group’ and felt that these rules were the best.

The behavioural learning goal of MIXER supporting children in being fully present in attending to others verbal and non-verbal messages was met. Children attended to the characters, understanding their behaviours and engaging with them. The VLE and characters provided an engaging interactive experience, where children engaged with the Werewolves game and believed in how the agents were playing that game. Children also felt that Tom had attended to them, with Tom listening to them and responding to their advice (e.g. they had helped him). The use of the iPad accentuates children’s levels of identification and personal engagement with Tom as a friend.

To some degree MIXER did meet the experience learning outcome. The Theory of Mind, which permitted deception and game mechanics for the agents (e.g. who agents chose as the suspect so as to minimize their own possibility of being ‘killed’), worked well. The agent architecture and synthetic culture provided a good way to experience conflict without needing a ‘real’ culture and offers potential for further experiential learning within the classroom context. Children successfully interacted with and completed MIXER, engaging with the narrative experiencing an intercultural conflict situation and contributing to its resolution. However, the aim to support children in learning that difference is acceptable and that different groups (cultures) have different rules (values) was less successfully met, with over 70 % of the children believing that at least one of the teams were cheating. In an earlier MIXER pilot (Aylett et al. 2014) our results had highlighted this focus on rule sets and cheating. To support the central message from MIXER that difference is acceptable, this was explicitly explained during the classroom discussion sessions (CDF) following the MIXER interaction. With adult support children readily understood that neither team was cheating and the importance of communication in resolving conflict. This underlines how essential debriefing is for bringing home the message that differences in social rules do not necessarily constitute cheating or ‘bad’ behaviour.

In view of the target group’s age and experience, MIXER does not attempt to create believable cultural differences between the groups other than the rules sets. In case a teacher wanted to stretch the learning in that direction, a debriefing could raise such points, e.g. speaking about different rules of behavior in different families, how you learn them from your parents, and so forth.

A potential future direction to reduce the focus on cheating would be to develop multiple rule-sets within MIXER. This would help to reinforce the central learning goals that different rules are acceptable and that each group, rather than cheating, has a different set of rules. However, similar to most other software developed for Personal, Social and Emotional classroom-based learning, our evaluations suggest that the MIXER software needs to be integrated and delivered as a whole learning experience. This would incorporate the experiential learning of MIXER reinforced by teacher-led classroom discussion, providing both didactic and experiential culture-general learning with the message of the experience reinforced by teaching.

Additionally, in terms of lessons learned, one factor that stood out was the need for the MIXER experience to be part of a longer intervention. An optimal solution would involve the MIXER interaction and follow up discussion to be repeated and blended with additional didactic learning experiences. We feel that this would yield better learning outcomes in classroom situation.

With the far transfer results, children’s cultural abilities, as provided through the CQS scores were higher at post-test after the MIXER interaction. This suggests that children had started to learn conceptually about the values and attitudes of the MIXER characters and were aware of cultural differences. Children’s social abilities, as measured through the MESSY were only slightly higher at post-test compared to the control condition after interacting with the MIXER software. This tentatively suggests that children had started to adapt and modify their behaviour/facial expressions/vocalisations. Repeated evaluations with more exposures would be required in order to confirm any significant changes. Although there was no change with Bryant’s Empathy Index, with children’s empathy levels remaining constant after interacting with MIXER, the near transfer results highlight that children were empathically engaging with Tom.

Conclusion

Synthetic cultures and characters provide an alternative approach to providing experiential learning about intercultural conflict and difference. MIXER characters acted and played in believable ways engaging the children with the narrative. Children empathised with the friend character, Tom, with many feeling that they had helped him and that he had listened to them. MIXER contributed to the children’s intercultural learning, with children experiencing emotional, behavioural and cognitive learning, supporting both near and far transfer. To achieve such learning, MIXER must be integrated into a whole learning experience, incorporating the experiential learning of MIXER reinforced by teacher-led classroom discussion to reinforce the key learning goals that difference is acceptable.