Keywords

1 Introduction

It is widely considered that creativity is born of individual activity, yet the possibility to interconnect our abilities can generate several options for problem-solving through interdisciplinary mindset or multi-skillset [1]. For this reason, collaboration in creative work provides people numerous possibilities to produce new ideas, objects, or artwork.

In contrast, the collaboration on creative work is implemented in a co-located work environment that lets people have direct interaction and communication with each other quickly. In addition, they can use gestures and facial expressions in such an environment along with different types of visualization tools such as notebooks, pens, and whiteboard as vital contributors to successfully achieve their goals [2].

In this study, we explore more about Virtual Reality (VR) opportunities for collaborative creative work. VR is changing the way that people could use their creative abilities. Furthermore, some studies show that VR has many benefits and possibilities to make creative processes easier [3,4,5].

VR not only gives users immersive experiences to meet with their co-workers but also has a lot of potential to create environments that facilitate social activities and support collaborative creativity. All these show significant progress towards a new interface that can innovate work environments.

Currently, very few applications explicitly cater to the creators who wish to carry out work in a virtual environment; hence, it is essential to continue the exploration of this field. There are currently some applications of VR for artistic purposes, and more artists and designers are beginning to find new ways to explore their potential in creativity and develop their ideas through collaboration with other artists.

2 Related Work

2.1 Japanese Comics (Manga) Creation

Manga are comics or graphic novels created in Japan and have a relation with the early Japanese narrative style. Manga publication is one of the biggest entertainment industries in Japan and generated around 7 billion yens since 1995.

The traditional manga drawing process is carried out by a manga artist who is the writer and illustrator of the original story with assistants who support them. Further, editors review the sketches and manuscripts before being published.

The traditional manga creation process involves five principal steps [6], as shown in Fig. 1.

Fig. 1.
figure 1

Process of manga creation

Full size image

The first step is making a plot. This step involves an introductory meeting or a brainstorming session where the manga artists presents the story, the purpose, characters, development, situation, and other essentials aspects to form the initial idea.

The second step is revising the plot. The purpose of this step is to examine the ideas and determine the details of the storyline. In addition, it includes time schedule and the division of the staff’s roles.

The third step is writing the story script. The story script contains the scenario description, dialogues, and actions of the characters before the actual visualization of each scene.

The fourth step is the storyboard drawing. In this step, all images are placed into frames on the storyboard and movements, facial expressions, and emotions are drafted on the character board.

The fifth step is reviewing. After finishing the storyboard, the editors review it. On some occasions, the editors request changes for frame panel layout or the entire storyboard.

2.2 Support System for Remote Manga Creation

Currently, Japanese comic (manga) is very popular around the world, and many people want to collaborate in the creation of manga content. Nowadays, many manga artists are using a collaborative approach to produce professional publications such as CLAMP group [7], a famous female quartet of Japanese manga artists.

In this collaborative approach, manga creators divide their roles between writers or illustrators to work effectively.

However, this collaboration process is not a simple division of their works roles as writers or illustrators. Such role designations also relate to their essential competencies and best personal skills. A writer must be the person responsible for story planning and writing the story scripts, and an illustrator must be the person in charge of everything related to the storyboard composition and character design.

Manga artists who work in collaboration consider this working style as more efficient. Working together not only helps them to produce impressive work and complex stories but also each of them can use their best skill to improve the final work.

However, sometimes their workplaces are separate, and their activities do not align simultaneously. However, they must maintain communication with each other accurately for a more efficient creation. In such scenarios it was found that working remotely has the following problems; 1) difficulty in communicating ideas, 2) reduction of engagement, 3) absence of active participation, and 4) lack of drawing perspective.

For this reason, numerous online communities exist in Japan that help manga creators collaborate and share their content in online art communities such as PIXIV [8] which connects creators across the country.

There has been research in collaborative creative designs, such as AB-DOKAN [9] which is a system to support the online collaboration of a four-panel manga production by a few people. The results achieved in real time proved that participants like to engage in the collaborative manga creation process and find it enjoyable.

In another study on collaborative storytelling [10] an application was created for the collaborative composition of a comic. This research demonstrated the possibility to utilize collaborative storytelling between two authors of different countries.

Based on the results of these studies, a new collaborative interface for manga creation is explored in this study, that gives the participants a satisfactory way to collaborate in narration material.

2.3 VR System for Creative Remote Work

The concept of creative thinking is the constant searching for innovation, original ideas, and problem-solving. Some studies have demonstrated that creativity could be applied in multiple ways to learn activities and design tasks [11, 12]. People who participate in collaboration activities related to creative thinking can examine and evaluate new ideas or concepts from different perspectives and find new directions to develop solutions.

One of the essential factors for success in collaboration activities is efficient communication, especially in a creative task. However, sometimes, the participants reside in different geographical locations, and they must work remotely.

Distant location collaboration environments can solve problem of distance limitation, but collaboration efforts might decrease because team members feel separated by distance and technology. For example, some people experiment with a loss of interest; they do not develop the engagement with their co-workers and do not participate actively in the project.

For this reason, it is necessary to create new technologies and build interfaces that optimize communication between remote team members to support their success [13].

The immersive virtual environment has the potential to support creative collaboration activities. In several studies, these kinds of environments focus on training, education, or simulation [14,15,16,17]. However, these studies did not show the specific factors that could contribute to artistic tasks in virtual worlds, and most of the VR technologies used were for individual activities.

In recent years, various types of research efforts have been made to achieve the connection of multiple users within a virtual environment [18]. For example, a study related to VR systems for brainstorming activities established a new way of generating ideas and managing problems efficiently during remote work.

Inside immersive work environments, unlike regular video conferences, the environment could be based on the exchange of information by new communication channels via shared spaces in which the users utilize a three-dimensional (3D) avatar and visual elements that maintain their level of commitment in the work [19].

3 Proposed System

SyncMeet is a virtual meeting room that helps users communicate and collaborate better while remotely working. It has an interface for idea discussion which uses an avatar and 3D objects, for example, a whiteboard in a virtual room. The avatar and the 3D objects let the users collaborate quickly, communicate their ideas, and make propositions for modifications in a manga storyboard by highlighting annotations over the storyboard.

3.1 System Architecture

For each user, an HMD (HTC Vive), a base station, a Leap Motion controller for hand gesture, and a computer with an internet connection are necessary. The HTC Vive can track the user’s position precisely inside the physical space and show the corresponding avatar’s position inside the virtual environment.

The Leap Motion lets users make natural hand gestures using hands and fingers. The user can hear their remote collaborator’s voice through the headphones, as shown in Fig. 2.

Fig. 2.
figure 2

SyncMeet architecture.

Full size image

3.2 System Functions

The system has two principal functions as described in the following text.

Avatar

The purpose of the avatar is to enhance the users’ interaction and give the users a sense of co-presence of both the users during the brainstorming collaboration as shown in Fig. 3.

Fig. 3.
figure 3

User avatar and point of view in VR room.

Full size image

The Avatar has two components:

  1. 1)

    Avatar head indicates not only the collaborator’s position and movement inside the VR room, but also their awareness in the remote place.

  2. 2)

    3D Hand Model for hand gesture, which enables users to show emotions like agreement or disagreement and express what they are talking about by pointing as a part of non-verbal communication.

VR Room

It is a virtual space that enables the users to concentrate and lets them have an immersive experience during the brainstorming session. It was designed to resemble an office meeting room with windows that can increase the users’ engagement level on the remote collaboration task, as shown in Fig. 4.

Fig. 4.
figure 4

VR room layout in SyncMeet.

Full size image

It has four tools to facilitate collaboration, detailed as follows:

  1. 1.

    A virtual whiteboard is a visual representation that shows a story script and four-panel manga drawing which can be seen by both the users simultaneously.

  2. 2.

    A drawing marker tool which enables users to make annotations with lines by pinch input gesture. Moreover, it has the option to choose four different colors to highlight the level of importance of each annotation, as shown in Fig. 5.

  3. 3.

    A voice recording tool which records a voice memo of the conversation during the brainstorming session.

  4. 4.

    A screenshot is captured automatically every three minutes, generating a backup of the annotations on the virtual whiteboard.

    Fig. 5.
    figure 5

    VR drawing tool in SyncMeet.

    Full size image

4 Experimental Evaluation

We evaluated the users’ interactions and the application potential of collaborative manga creation in three different environments (face to- face, Skype, and SyncMeet prototype). In this evaluation, all the participants were asked to create a four-panel manga through collaboration between two persons to evaluate the extent and quality of communication and participants’ engagement during a brainstorming session in SyncMeet.

4.1 Participants

The number of participants was six with an age range of 23 to 32 years aged. All the participants were Masters students and Ph.D. students (three women and three men) who have previous experience in the creation of manga (Japanese comics), illustration design, or script development.

The participants were divided into three pairs with two types of roles depending on their skills and background (writer or illustrator). In addition, the participants had to switch their partners per week. Table 1 shows which pairs have been assigned for each environment:

Table 1. Experiment’s participants distribution.
Full size table

4.2 Task

The activity task consists of four steps that are divided among two different categories, namely, the individual task and collaborative task.

Individual Task

This part of the experiment was divided in two steps as shown in Fig. 6.

Fig. 6.
figure 6

Individual creating session.

Full size image

First, the story writers are requested to make each short story with an introduction, development, plot twist, and conclusion known as Kishotenketsu [20]. Once finished, each writer sends the script by email to the team lead, and then, it is further sent to their corresponding illustrator.

Second, the illustrator reads the story script and draws the four-panel manga.

Collaborative Task

The collaborative task has two steps; ice-breaking, and four-panel manga brainstorming.

Third, the following activities were conducted:

  • Self-Introduction: The participants had to introduce themselves and mention one small fact about themselves, for example, their hobby, favorite movie, or favorite food.

  • True or False Game: Each participant writes four statements about themselves and then they must guess their partner’s true statement out of those four.

Fourth, both participants start the brainstorming session. The writer gives feedback and opinions about the four-panel manga. When both participants complete the discussion, they talk about new ideas that can be added on the four-panel manga to improve the results. During this process, both take notes in each environment (Fig. 7).

Fig. 7.
figure 7

Brainstorming session.

Full size image

4.3 Interview and Questionnaire

The NASA TLX was used for measuring the mental workload and additional custom questionnaires with a Likert scale of 5 points range of answer options were used to get further information about the participant’s impressions during the session. Further, videos and pictures of the participants’ activities were recorded through the experiment. In addition, participants were interviewed to explore more options about how they feel about working in each environment.

4.4 Variables

The evaluation variables were both quantitative and qualitative values which are as follows:

Quantitative Variables

  • Mental Workload: Rates the perceived mental workload of the participants’ collaboration in each environment.

Qualitative Variables

  • Work Environment Satisfaction Level: Rates the participant’s satisfaction with the work environment to achieve success on the task and the participants’ commitment.

  • Collaboration Effectiveness Level: Rates the participant’s impression about the team’s commitment and collaboration level to accomplish the goal.

  • Communication Efficiency Level: Rates the participant’s impression about the efficiency of verbal and non-verbal communication between both the participants.

5 Results and Discussion

The result of three different environments for mental workload, as shown in Fig. 9, were M = 30.14, SD = 12.15 (face to face), M = 37.78, SD = 13.17 (SyncMeet) and M = 39.03, SD = 12.06 (Skype). Using the two-way ANOVA for environment and user role, there was no significant difference between the main effect of the environment and participants interaction.

However, during the interview session the participants expressed that they had a better experience using SyncMeet compared to Skype. Participants said that SyncMeet provided them an immersive experience that let them concentrate better and have an improved sense of engagement almost as much as the face to face environment.

Furthermore, they remarked that SyncMeet had the benefit of visual stimulation through 3D objects. The illustrator showed interest in possibly adding new features that could enhance the remote work on drawing activities over the shared space.

The results of work environment satisfaction question 1, “I could utilize all my skills and abilities to solve this task in this environment”, as shown in Fig. 8, were M = 4.17, SD = 0. 753 (face to face), M = 3.83, SD = 0.435 (SyncMeet) and M = 3.67, SD = 0.435 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of the environment with a p-value = 0.043. As the result of multiple compositions, there was a significant difference between face to face and Skype with a p-value = 0.029.

Fig. 8.
figure 8

Work environment satisfaction question 1.

Full size image

The participants described during the interview that face to face environment always has the benefit of using all their skills and abilities easily to solve the problem. In addition, it was easier to talk while they were taking annotations during the session.

The result of work environment satisfaction question 3, “I could concentrate on the task in this environment”, as shown in Fig. 9, were M = 4.33, SD = 0. 404 (face to face), M = 4.17, SD = 0.149 (SyncMeet) and M = 3.50, SD = 0.404 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of environment with a p-value = 0.020. As the result of multiple compositions, there was a significant difference between face to face and Skype with a p-value = 0.0041.

Fig. 9.
figure 9

Work environment satisfaction question 3.

Full size image

The participants mentioned that it was easier to lose concentration during Skype sessions because they were more aware of their own screens for taking annotations, compared to the face to face sessions. However, they described SyncMeet as “out of time-space” making the work more playful that helped them to have better concentration without distractions.

The results of work environment satisfaction question 4, “I was aware of my partner’s intention in this environment”, as shown in Fig. 10 were M = 4.50, SD = 0. 548 for (face to face), M = 4.17, SD = 0.447 (SyncMeet) and M = 3.67, SD = 0.492 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of environment with a p-value = 0.015. As the result of multiple compositions, there was a significant difference between face to face and Skype with a p-value = 0.009.

Fig. 10.
figure 10

Work environment satisfaction question 4.

Full size image

By observation of the video recorded during the experiment, it was determined that the participants quickly get used to the face to face environment and the VR environment. The writer was seen to be more uncomfortable during the Skype session because they could only talk and had to use lengthy verbal explanations to describe their ideas for the drawing and panel composition.

The result of collaboration effectiveness question 3, “I worked with my partner effectively”, as shown in Fig. 11, were M = 5.00, SD = 0 (face to face), M = 4.33, SD = 0.816 (SyncMeet) and M = 4.17, SD = 0.983 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of environment with a p-value = 0.013. As the result of multiple compositions, there was a significant difference between face to face and Skype with a p-value = 0.025.

Fig. 11.
figure 11

Collaboration effectiveness question 3.

Full size image

The participants mentioned that collaboration in the face to face environment was an excellent experience comparing it with Skype. The primary reason was that they felt that they could see and understand what their partner was doing during the experimental session. By comparing with SyncMeet, they stated that it was almost similar to a face to face environment interaction. One of the reasons was that, some of the participants felt more relaxed during the meeting in SyncMeet instead of Skype; because it was easier to act freely inside the VR room and not just paying attention to the screen in typical video conferences.

Both participants expressed that SyncMeet environment lets them share the information directly like face to face environment and is an excellent and novel way to brainstorm ideas or take annotations. However, the writer responded that not having facial expressions made them feel less confident about their work with their partner on a SyncMeet session.

The results of the work environment satisfaction question 6, “I think my partner had a positive attitude to collaborate in this task”, as shown in Fig. 12, were M = 4.55, SD = 0.314 (writer) and M = 4.778, SD = 0.314 (illustrator). Using the two-way ANOVA as the result of multiple compositions, there was a significant difference for the main effect of the user role with a p-value = 0.047.

Fig. 12.
figure 12

Collaboration effectiveness question 6.

Full size image

The illustrators expressed that the writer participation have a more dynamic and innovative way of collaboration on face to face rather than working in Skype. In addition, they mentioned that SyncMeet was almost similar because they could share their visualization of the sketch and story script. All this lets t the writer felt more comfortable as if he/she was working in face to face environment.

The results of communication efficiency question 1, “I communicated efficiently with my partner.”, as shown in Fig. 13 were M = 4.5, SD = 0 (face to face), M = 4.33, SD = 0.816 (SyncMeet) and M = 4.17, SD = 0.983 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of environment with a p-value = 0.015. As the result of multiple compositions, there was a significant difference between face to face and Skype with a p-value = 0.0010.

Fig. 13.
figure 13

Communication efficiency question 1.

Full size image

The results of communication efficiency question 2, “I think my partner communicated with me efficiently”, as shown in Fig. 14, were M = 4.67, SD = 0. 516 (face to face), M = 4.00, SD = 0.632 (SyncMeet) and M = 3.67, SD = 1.033 (Skype). Using the two-way ANOVA, there was a significant difference for the main effect of environment with a p-value = 0.010. As the result of multiple comparison, there was a significant difference between face to face and Skype with a p-value = 0.0011.

Fig. 14.
figure 14

Communication efficiency question 2.

Full size image

Question 1 and 2 about communication efficiency shows that participants perceive both participants more efficiently on face to face environment in comparison with Skype. They mentioned that the lack of eye contact and non-verbal communication does not let them communicate easily and requires a bigger effort to verbally explain their ideas.

Additionally, the writers explained that they are less efficient in their communication through Skype. They were seen to be more uncomfortable because they only could talk and had to give lengthy explanations to describe their ideas for the drawing and panel comparisons.

6 Conclusions and Future Work

A virtual meeting room, SyncMeet was presented for distance collaboration that allowed two people with different narrative skills to create a four-panel manga (Japanese Comic). In SyncMeet, the users could communicate their creative ideas over their work quickly using an avatar and 3D objects inside a virtual work environment.

A fundamental aspect of this research was to create a new work environment for creative collaboration tasks. SyncMeet lets the users comprehend the advantages of VR environments for collaborative creative activities.

The participants expressed that working inside a virtual environment allows them to have an immersive experience and interact actively with their partner. In addition, the participants expressed that even if it was their first-time using VR, they felt very comfortable collaborating and explaining their ideas inside SyncMeet.

In future work, more functions will be added to SyncMeet, for example, tools for directly writing the script and drawing the storyboard in the virtual whiteboard and showing the differences between two version of storyboards. With these new functions, the illustrators and writers will be able to collaborate more efficiently during the creative process.