6.1 Effects on improving social and collaborative skills and alleviating social loneliness
Overall, based on the results of quantitative and qualitative analysis, our system achieved the intention of improving children’s social and collaborative skills. In the game, most of the children realized the importance of collaboration and developed a sense of teamwork. Recognizing the value of collaboration can help overcome self-centred problems in young children’s development [
21].
"It is a team effort. We should cooperate to support and correct each other."
P1 reported this when being asked about "What’s the most important thing you learned from this game?", and 10 other children mentioned "collaboration" or "teamwork" to this question.
"I learned to be patient if my partner is slow...I was a little upset at first and then I slowly got it under control, because I want everyone to have fun. I’m usually very easy to get angry, but this time I controlled myself."
P12 reported this to the same question, and 3 other children mentioned "be patient" as well. Patience is intimately related to self-control and is essential for children to participate in cooperation [
2]. In addition to the teamwork and patience mentioned above, many children highlighted being considerate of others, completing their own tasks promptly, avoiding causing trouble for their partners, understanding and assisting each other, and discussing calmly when encountering difficulties, etc. Indeed, social and collaborative skills are originally composite structures covering multiple skills, such as communication, team support, and problem-solving [
36]. The specific insights that children learn during the collaborative process serve as the foundation for their eventual growth of generalized social and collaborative skills. According to the interview, we found that they gained a more precise and deeper understanding of collaboration and mastered certain approaches to collaboration.
We also discovered that using Mr. Brick system can help alleviate children’s social loneliness. Slavin et al. [
77] suggested that cooperation not only improves children’s cognition and performance, it also shows great potential to promote social-emotional outcomes. We met a touching case in the experiment, P9, a very introverted six-year-old girl. When she came to our experiment, she held her mother’s hand tightly and hid behind her mother’s back, holding an reversible octopus plushie in her hand. This plushie was originally designed for children with autism who have difficulty expressing their emotions. The plushie’s pink side indicates happy feelings, while the blue side represents negative emotions. P9 was the only one of our participants who asked for the parent to be in the experimental room. Before the experiment, P9 barely spoke and did not dare to make eye contact with the experimenter. Her octopus plushie was switched to the blue side.
As the game progressed, we were delighted to observe some changes. First, we noticed that her facial expression had softened, gradually changing from expressionless to frequently smiling. The frequency of her verbal requests grew, as did her tone and volume. Her verbal interactions with her partner have progressed from simple responses to rich conversations that include descriptions, requests, reminders, instructions, enquiries, etc. Surprisingly, during the break, P9 said that she can complete the game independently without the company of her mother. In the next round, P9 changed the octopus plushie from blue to pink and placed it on her head (Fig.
7). This was a significant positive sign that she was gradually becoming more socially active during the game.
Apart from P9, most children reported feeling more connected to their partners during the game. In addition, when asked whether this game can help them relieve loneliness and maintain a pleasant mood during the pandemic, the majority of the children responded positively. Compared with adults, children and adolescents are at higher risk of mental health problems when they are forced to reduce social activities during the epidemic [
89]. According to early indicators of COVID-19, more than a third of children and adolescents report high levels of loneliness [
53]. Numerous studies [
44,
61] indicated that maintaining social connections with peers can help children through this difficult time, including organizing support groups, keeping regular social interactions, such as video calls; and playing collaborative games. Our games have demonstrated the potential to be a tool for assisting children in alleviating social loneliness.
6.2 Differences between MR game and traditional video game
Many of the differences between the MR game and the traditional video game were also found in interviews.
According to children, the biggest advantage of the MR game is
"a sense of reality"(P21). Many children reported that the MR game gave them a more physical experience than the traditional video game,
"like actually building LEGO bricks,", P2 said. MR game allows direct interaction with physical objects and direct observation of the player’s hand movements, which is more
"simple to learn"(P1, P19) and
"more realistic" (P19, P21). These encouraging remarks from the children on the MR game further validated our design guideline 3, and offered evidence for past studies’ conclusions that augmented manipulatives with conceptual metaphors based on image schemas should facilitate learning[
5,
59].
Fontijn et al. demonstrated that an interface that is physically engaging enhances fun[
26]. In our study, MR game was appraised by children as
"more interesting" (P7) and
"more fun" (P16). Several participants also note the card’s unique tactile feel. For example, P4 mentioned
"The sensation of the board’s attachment to the table reminds me of ice skating, which I find quite enjoyable.". P16 reported that
"It was fun having all the cards lined up in your hand, and I like the beautiful colors of the cards." (see Fig.
8). We can infer that the hybrid setup of MR.Brick opens up their tactile perspective of sensory, which enhances their game experience. This is analogous to zhou et al.’s Magic Story Cube[
90], which simultaneously makes the multi-sensory experience more interesting by enhancing the feeling of physical touch.
According to Fontijn et al. [
26], getting a sense of accomplishment is the first core source of fun, which is determined by the balance between challenge and control. While the challenge offered is generally seen as the most important factor in making a game enjoyable[
57]. In our study, several children mentioned that the MR game gave them a
“sense of control” by using the board to rotate and observe, and using cards to manipulate and select. Here are two examples:
"It(MR game) gives me the feeling that I am the captain of the train, which is very pleasant, and everything is under my command."(P4)
“The traditional video game is all about the electronic element on the screen. But if it’s the MR version, you have a projector that you can look at, you know, you can take full control of the game, and it’s like I’m in control of the world.”(P11)
At the same time, the MR game was reported to be
"a little challenging"(P6, P22) and
"Have a sense of accomplishment when completing hard mode"(P8). Hence, we made a great balance between challenge and control by offering a sense of control to the player, but at the same time difficult enough for the outcome to be uncertain [
26]. Our findings support the conclusions of many previous studies [
18,
58,
86] that demonstrated experimenting with three-dimensional physical objects in mixed-reality situations results in more learning and satisfaction than two-dimensional interaction on a flat screen.
However, compared to the traditional video game, children reported that the MR game caused them to spend more time looking for the correct card than playing the game(P12). Similarly, P20 reported that "finding the right card" is the most difficult thing during the game. Possibly owing to the large quantity and size of the cards, children were unable to lay out and see the patterns on all the cards throughout the experiment.
Some children also cited traditional video game benefits that MR games lack. "You don’t need a lot of props.", P8 commented, thinking it is not convenient if players need to prepare a lot of extra cards to start the game. Also, some children who were more familiar with mouse control said that the traditional video game was more convenient than the MR game. For example:
"I used to take a programming class, so I became fairly proficient with the mouse by holding it like this and dragging it. This (MR game) must be picked up and moved to its proper location. Then this (traditional video game) is pretty easy; simply click the mouse, and that’s all that is required."(P24)
But not all children found the traditional video game easier to play, and some younger children reported that the mouse was harder to use(P1, P2, P8, P9, P10, P16, P19, P23). We did not find significant differences in the rating of operation difficulty and preferences from the interview feedback. According to Yannier et al. [
86], for students’ learning, there were no significant differences between the mouse control and the tangible trigger, and the effect of observing physical phenomena was more powerful than the effect of using a simple tangible trigger. Therefore, we assume that ratings of the difficulty may mainly be related to children’s age and familiarity with mouse control.
Using a mouse to determine the correct position in a 3D environment proved difficult for the vast majority of children. For example,
“I’ve already clicked on that spot, but he doesn’t move, which is quite unpleasant.”, P11 argued. This is mostly owing to the limited spatial awareness skills of our participants [
73]. They are unaware that while viewing the model through the camera in 3D software, the model will have a perspective effect and the actual location of the assembly will be somewhat behind the position of the mouse click. This may explain why their average time spent playing traditional video games was less for simple games than for MR games and longer for sophisticated ones.
Another interesting finding is that during the traditional video game, in order to improve communication efficiency and achieve better collaboration, many children prefer to use tangible props to make up for the limitation in the verbal description, even when we strongly suggest they try not to use the props.“It might take 20 seconds to describe. But only two seconds with this(card).”, P24 told us.
“It’s hard to tell what this part looks like. It’s better to use cards. With props, I don’t have to talk much. It feels more convenient. All you need to do is to pick up the card.” (P21)
Kuzuoka et al. [
48] indicated that gestures significantly increase communication efficiency and substantially reduce the number of verbal expressions required to indicate intention when using video conferencing to convey gesture information for remote collaboration. Because children’s verbal abilities are limited, their subconscious choices provide solid evidence that the mixed-reality with TUI approach is more child-friendly, and it increased the efficiency of our young participants’ remote collaboration by allowing them to employ gestures and props.
6.4 Limitation and future work
For the MR. Brick system, it still has some technical limitations as a proof-of-concept prototype, mainly the equipment required to access the game settings. By removing the restriction in the augmented reality algorithm that necessitates a perpendicular angle for accurate position calculation, it will be possible to reduce the number of cameras to one. The number of screens required can also be reduced to one by integrating the video communication tool into our game. Furthermore, tangible cards can be produced simply by printing, making the game more accessible. In addition, it is important to encourage children’s creativity by providing opportunities to build freely and adapted to their ability during the game. According to Resnick and Silverman [
72], “Low Floors, High Ceilings, and Wide Walls” should be followed when developing tools aimed at facilitating children’s learning and development through making. That is to say, successful tools allow new users to adapt easily, allow more experienced users to develop complex structures, and allow users to create freely. Mr. Brick has the potential to continue to be developed into a more complete educational tool. With the elimination of these limitations, Mr. Brick may be more simply republished on portable devices using the Unity framework, opening up more opportunities for further research or actual marketing.
For the study, although our research has confirmed that Mr. Brick has the effect of developing children’s social and collaborative skills, however, children only played the MR game for 20-30 minutes in our study, which was a short-term experience. It is difficult to say whether this is due to novelty effects. Besides, the game used in our study only contains a small number of relatively simple and fixed game tasks. In the future, we can explore whether this kind of game can improve children’s social and collaborative skills based on long-term research. For example, a semester-long weekly experiment in an elementary school class. Moreover, more measurement such as network satisfaction survey can be involved in the study to help further research in the social communication behaviour of children.
