Assessing a Collaborative Application for Comic Strips Composition

Stories are powerful educational tools for discussing cultural differences and diversity [8]. In the process of narrating a story, authors express their identities and make narrative decisions, characterizing the possible world they create [6]. This aspect becomes particularly important when the narrators are teenagers, as they are in the process of developing their own identity [5].

As ultimate goal of our research, we aim at designing and developing a tool to help teenagers who do not share a common language to write stories collaboratively. The work presented in this paper is an intermediate step toward this goal and describes a tablet application for remote collaborative storytelling, in which ready-made language expressions are selected from a library, rather than freely written by the authors. The constrained-text interaction has been devised in order to offer an alternative to real-time automatic and human translation in multilingual scenarios. In order to explore how the constraints provided by our technology influence users’ satisfaction and creativity, we designed an experimental study with teenagers (15–17 years old) who share the same language and asked them to create a story both in a constrained-text and in a free-text version. The results suggest that composing a story with constrained text is a valuable alternative to other approaches to remote collaborative storytelling, such as free writing. In particular, our study shows that the constrained-text approach supports collaborative storytelling by providing authors with a limited yet satisfying level of expressiveness.

Collaborative storytelling has long been an educational activity to support reflection on identity [4]. Several attempts have been made in designing technological devices to assist the narrative process. Social networking infrastructures such as Facebook [2], or ad-hoc mobile applications such as MAHI [5], have been used to support digital storytelling in remote, showing how teenagers represent themselves and negotiate their identities online. Some studies in HCI and Education have investigated how teenagers express their narrative ideas in the realization of digital contents, such as videogames, describing the educational benefits of digital storytelling through creative activities [6]. Finally, other studies have investigated the benefits of adopting collaborative and co-located storytelling for cross-cultural mediation. These latter studies have shown that collaborative storytelling can be used to facilitate conflict resolution and to support mutual understanding between authors with different cultures [8].

Our study enriches this last research line, investigating remote collaborative storytelling through the use of constrained-text with the eventual goal of fostering multilingual communication and cross-cultural dialogue.

The 3 main design objectives of this study were: (i) to ensure the creation of meaningful stories; (ii) to support remote collaboration between two authors and (iii) to design a system that in the future can allow remote collaboration also between authors who do not share the same language.

Regarding the first objective, we provided content and images for the development of at least 4 different stories on the topic of ecology. To manage remote collaboration, we decided to adopt a turn-based approach, where the authors alternate in composing the story by intentionally giving the turn one to the other. Regarding the multilingual support, although the use of a common language (such as English as lingua franca) has itself an educational value, storytelling in a non-native language can be difficult and it may hinder collaboration if the two authors do not share the same level of knowledge of the language. Only high proficiency in the language on both sides could ensure full expressiveness and richness in the conversation. Hence, two other main approaches might be used to tackle the challenge of multilingual communication: a free-text approach, where the author freely enters text in her own language, to be translated by a person or by an automatic translation system; or a constrained–text approach, where a number of sentences, previously translated and paired, are made available to the authors for selection. The problems with the former option are that the mediation of a human translator might influence the outcome of the storytelling (as in [8]), and that the current tools for automatic translation are still far from being usable on more than a single word or sentence, and their use might impact the collaboration experience [3]. While the problem of the latter option is that the limited content may hinder authors’ expressiveness. However a constrained-text approach has the advantage to minimize the translation process, since the pre-defined sentences can be previously paired in two (or more) languages. The present study is specifically focused on understanding and assessing the impact of a constrained-text approach on remote collaborative composition of comics using a tablet application.

In designing the application, we opted for comic strips as well-known narrative genre among teenagers. The proposed technology is a tablet application that provides a library of images (42 in total) for backgrounds, characters and objects, as well as a repository of predefined language expressions (151 in total) for greetings (e.g. “Hello!” “Bye-bye!”), statements (e.g. “Look at that poor animal, it has been caught by something!”), questions, etc. that the users can arrange to compose the comic strips (Fig. 1). The number of items (both graphical and lexical) allowed a manageable navigation of the interface, requiring a few horizontal scrolls.

The story was collaboratively created by two users remotely located, each one using her tablet. The authors worked in turns in composing the story, while the interface was automatically synchronized between the two tablets.

A screenshot of the application in the constrained-text condition, showing a story partially created (upper part of the figure) and the library of language expressions (lower part).

Full size image

In order to investigate the limitations that this approach may bring to storytelling, we developed a slightly different version of the application, which allows free-text input for dialogues and captions. In all other respects (such as the graphics) the two versions share the same features.

4.1 Objective

A study was designed to compare the constrained-text approach with a traditional free-text condition in a monolingual setting. The objective was to understand whether the constrained approach allowed users to construct meaningful stories (despite the given topic and the limited resources) and whether it ensured a satisfactory experience.

Our hypothesis was that the constrained-text approach could provide a satisfactory support to remote collaborative storytelling with respect to the free-text condition. We consider the constrained-text version as a satisfactory support to storytelling if it guarantees a reasonable level of expressiveness and allows the creation of meaningful stories.

4.2 Measures

During the study both objective and subjective measures were collected and analyzed. Objectives measures consisted of the logs of the activities on the interface, which pertained the interactions with the tablet as well as the story creation. Measures about the interaction regarded the number of turns, their duration, and the number of operations (addition and removal of elements) performed in each turn. Information about the story creation included the number of scenes and of elements used.

Subjective measures included a 3-item questionnaire investigating users’ satisfaction with the story, with the collaboration on the task and the perceived difficulty of using the application (Table 2). Each item was scored on a 5-point Likert Scale. Furthermore, we adopted the Creativity Support Index (CSI) to measure the perceived usefulness of the application in supporting the creation of stories. The CSI is a psychometric survey designed to investigate the ability of a digital tool to support a creative process [1]. CSI is composed of 12 items and 6 different scales for measuring the following dimensions of creativity: Collaboration, Enjoyment, Exploration, Expressiveness, Immersion and Result Worth Effort.

4.3 Procedure

The study was designed as a within-subject experiment with the constrained-text (CT) application as main condition, and the free-text (FT) version, which allows free text in dialogues and captions, as control condition.

The study was conducted in a 3rd year high school class, with 20 participants (11 females and 9 males; mean age = 15.85 years), all speaking the same language (Italian). The activity, although encouraged by the literature teacher, was not considered part of the normal class activity and participants did not receive a grade for their performance.

Before starting the storytelling activity, the participants were told about the goal of the task (i.e. “create a story on the theme of sustainability”) and were trained in using both versions of the application. During the study, each participant received a tablet and was anonymously paired with another student (for a total of 10 pairs). All pairs worked on the creation of a story using the application both in the CT and FT condition, and the starting condition was randomly counterbalanced over the pairs. In each condition, the same incipit of the story was given in advance: a scene with two boys in a schoolyard and a caption saying: “Bill and John meet after school…” For each condition, participants were given 12 min to conclude the story and afterwards they were asked to fill out the questionnaires.

5.1 Quantitative Data Analysis

Structure of the Stories and Interaction Between the Authors.

Each pairs used on average 9 turns for completing the story in the CT condition vs. 7 turns in the FT condition. Each turn lasted on average 2 min, but standard deviation was higher for the CT condition, indicating a more variable time usage. This might be explained by the fact that it is quicker to compose a scene when the graphical and constrained-text elements are identified, while it might take longer when the users spend time navigating through the different categories. On average, the stories were composed of 4 frames and the number of graphical elements used was similar in both conditions. When using the constrained-text application, the participants tended to use a higher number of textual boxes. This might be explained by the fact that in the free-text condition the participants could use a single text box for expressing more concepts. However, as shown in Table 1, metrics from the system logs were not statistically different between conditions (Wilcoxon Signed Ranks Test, p> .05).

Satisfaction and Difficulty.

From the questionnaire investigating the users’ impressions of the system, we could see that participants were generally satisfied with the story created and with the collaboration, and no statistical differences were observed between the two conditions (Table 2). Yet, the constrained-text application was considered more difficult to use, although both scores are below the midpoint of the Likert scale corresponding to “neutral”.

Creativity Support Index Analysis.

Table 3 shows the average CSI scores, including the average factor counts that express which factors are considered more important for the creativity activity regardless to the specific technology used (highest value is 5), and the average factor score, representing how well the specific version of the application supports these factors (highest value is 20). The scores indicate that Collaboration is by far the most important factor considered by the users with respect to collaborative remote comic writing, while Results Worth Effort and Enjoyment were regarded as the less relevant to the task.

The overall CSI score was calculated by multiplying each factor scores by the related factor counts. It represents how well the tool supports the factors that are considered as most important to the storytelling task. The CSI score for the constrained-text application was 65.1 while the free-text application received a score of 75.57. These scores suggest a reasonably good creativity support for both applications [1]. A repeated-measure ANOVA shows that they are statistically different (p < .05), with the higher score for the FT condition (Table 3).

Looking at the individual factors and how they were rated across the two conditions, we can observe that both versions received high values for the Collaboration dimension. A MANOVA indicates a difference between conditions (Pillai’s trace, V = 0.61, F6, 14 = 3.7, p < .05); specifically, all the scores except for Collaboration are higher for the FT condition (Table 3). This suggests that the free-text application provides more support as creative tool in the majority of the factors investigated but Collaboration. This result is relevant considering that collaboration was rated as the most important factor in collaborative comic writing and it was one of the main design objectives.

5.2 Qualitative Analysis of the Stories

The content of the 20 stories was assessed with a qualitative approach. We used the dimensions of (i) relevance to the topic (i.e. ecology), (ii) narrative tension as the presence of a problem and its resolution [7] to mark the meaningfulness of the stories, as well as (iii) level of interactivity between the two characters [6] to assess the level of expressiveness of each story.

Relevance to the Topic.

All the stories in the CT condition are centered on the topic of ecology as required by the task. This is somehow expected since they were built out from a pre-defined library of language expressions on this topic. Nevertheless, it was possible, though not simple, to build off-topic stories with the material provided. On the contrary, several stories in the FT are off-topic (6 out of 10). In 2 of these cases, the FT condition was performed after the CT, so by that time the authors had already created an on-topic story.

Narrative Tension.

Narrative tension is only hinted in some stories and absent in many others. In CT condition, 4 stories have a goal or a solution (such as freeing a squirrel caught in a net) and 5 in FT condition. Still, among the CT stories without a goal or solution, 3 have a clear moral (such as, “it is because everybody behaves like you that we are in trouble”).

Level of Interactivity.

Stories in both conditions contain several dialogues between the two characters and textual elements were widely used in both conditions. However, the general impression is that the plots are simple. That might be a consequence of the limited amount of time available for the tasks.

5.3 Discussion

Satisfaction and Gratification.

Supporting our hypothesis, the subjective data from the questionnaires show that the constrained-text application provided a satisfactory support to storytelling and to collaboration at a level similar to the free-text version. Unsurprisingly, the participants favored the FT application, which they found to provide better creativity support for authors engaged in storytelling. It is worth noting that the FT version was preferred along all dimensions but collaboration, and that collaboration was indeed scored as the most important aspect of the task.

Content of the Stories (relevance to the given theme).

Although the low quality of the stories does not allow a full analysis of the differences in the two conditions, a noticeable difference was that the constrained-text approach seems to encourage on-topic stories and this may be considered as a positive aspect. Moreover, the objective measures (number of scenes and objects used) did not differ significantly, suggesting that the structural level of the stories is similar in the two conditions.

Collaboration.

The analysis of the objective measures suggests that the two applications had similar complexity and overall the turn-exchanging mechanism was easy to use. Yet, in the FT condition, we observed that participants often enacted an appropriation practice: they used the free text to discuss and find an agreement on the story, when they reached an agreement they would delete the text that therefore do not appear in the final version of the stories. Although this practice does not seem to have improved the quality of the stories, the possibility to communicate seems to be needed by the authors. As future work, we plan to add such functionality in the constrained-text application providing a similar constraint-based mechanism.

This paper is an intermediate step toward the design of an application for supporting remote collaborative storytelling between people from different cultures and speaking different languages. In the study presented in this paper, we compared a constrained-text version of an application for composing comic stories with a free-text version of the same application. Although the free-text application provided a higher level of expressiveness and ease of use, the constrained-text approach allowed the creation of simple, yet meaningful, stories. Therefore, the study presented in this paper provides the basis for future work to investigate the use of the constrained-text approach for supporting collaborative storytelling in multilingual setting.