[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3613904.3642819acmconferencesArticle/Chapter ViewFull TextPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Investigating the Design of Augmented Narrative Spaces Through Virtual-Real Connections: A Systematic Literature Review

Published: 11 May 2024 Publication History

Abstract

Augmented Reality (AR) is regarded as an innovative storytelling medium that presents novel experiences by layering a virtual narrative space over a real 3D space. However, understanding of how the virtual narrative space and the real space are connected with one another in the design of augmented narrative spaces has been limited. For this, we conducted a systematic literature review of 64 articles featuring AR storytelling applications and systems in HCI, AR, and MR research. We investigated how virtual narrative spaces have been paired, functionalized, placed, and registered in relation to the real spaces they target. Based on these connections, we identified eight dominant types of augmented narrative spaces that are primarily categorized by whether they virtually narrativize reality or realize the virtual narrative. We discuss our findings to propose design recommendations on how virtual-real connections can be incorporated into a more structured approach to AR storytelling.

1 Introduction

Hailed as an innovative medium for storytelling, Augmented Reality (AR) enables immersive and interactive narrative experiences by superimposing virtual elements on the real 3D space [11, 27]. Through AR, the virtual space in which digital narrative content is contained can take on a concrete form by merging with the real space along the Mixed Reality (MR) continuum [87], offering compelling opportunities to blur the lines between fiction and reality [78, 93]. In achieving this effect, the virtual narrative space and the real space are connected on many levels to constitute an augmented narrative space in the user’s immediate surroundings [72, 117].
Therefore, it is crucial to understand the structure of the augmented narrative space and how the virtual and the real elements shape it. This establishes the foundation for AR storytelling, both in theory and practice [73, 111, 112]. With the advancement of AR technologies, attempts have been made to conceptualize the ways in which these technologies facilitate relationships between the virtual, the real, and ultimately the user in HCI, AR, and MR research [9, 55]. While they provide insight into the design of augmented narrative spaces based on how the virtual and real are combined, efforts to apply and further contemplate the concepts to survey and analyze approaches to AR as a spatial storytelling medium within these fields have been lacking.
To address this issue, we conducted a systematic literature review targeting research articles that presented AR storytelling systems and applications in the HCI, AR, and MR domains from 2003 to 2023. For a total of 64 articles, we investigated virtual–real connections in augmented narrative spaces across Pairing, Registration, Function, and Placement Rule, deriving patterns for each of them. Based on the ways that the four virtual–real connections were linked to one another in all cases, we identified and categorized dominant types of augmented narrative spaces.
The eight dominant types of augmented narrative spaces we found were classified as either those that virtually narrativize reality or those that realize virtual narratives. By analyzing common traits of cases that comprised each major link between virtual–real connections, we conceptualized each type in the former category as Historicized Outdoor Spaces, Enhanced Indoor Spaces, Extended Object Spaces, and Flexible Backdrop Spaces, and those in the latter as Transported Scene-Based Spaces, Transformed Room-Scale Spaces, Transplanted Tabletop Spaces, and Compatible Portal Spaces.
The contributions of our systematic literature review are threefold: First, we provide a framework of virtual–real connections for a deeper understanding of the spatial and narrative structures of the augmented narrative space. Second, we present a landscape overview of augmented narrative space design within HCI, AR, and MR research, spanning from the earlier to most recent cases. Third, we propose design recommendations to integrate our findings into a methodical approach that leverages the common characteristics of different augmented narrative space types as conventions of a newly emerging medium for storytelling.

2 Background

2.1 AR as a Spatial Storytelling Medium

Amidst mounting expectations for the technological advancement and sociocultural acceptance of AR, discussions on its potential application areas have largely focused on the capacity of AR as a paradigm-shifting medium for immersive and interactive storytelling rooted in space [54, 103, 124]. AR succeeds and builds upon conventions of previous media, such as theater, film, video games, and VR, which emphasize the spatial arrangement and representation of narratives [107, 117]. It also transcends their limitations by providing a unique combination of both spatial and narrative characteristics that enable unprecedented possibilities for spatial storytelling [80].
The first of these characteristics is that the virtual narrative space is housed in the real space. A term established in narratology studies, narrative space is where narrative as the presentation or telling of a story—a sequence of events involving characters—occurs [56, 106]. In abstract forms of media such as written text, narrative space is often relegated to the background for storytelling [50]: Its representation is left to mental concoctions as the temporal progression of the narrative takes center stage [16]. In digital storytelling, however, narrative space becomes a concrete, visible entity integrated into the virtual environment, thereby constituting a virtual narrative space [1, 57, 104]. AR takes one step further in establishing the virtual narrative space as an essential element for storytelling, situating it within the real space that shapes the user’s narrative experience [3, 5, 81].
Second, the virtual narrative space and the real space are connected through various dimensions to create an augmented narrative space. Deriving from Manovich’s concept of augmented space, augmented narrative space refers to the real space overlaid with a virtual narrative space in this study [84]. AR is in essence the technological means by which the virtual narrative space and the real space are aligned, connected, and merged with one another [10]. Furthermore, the features of the two spaces comprising augmented narrative space also determine how mutual relationships between them create effective narrative experiences. Regarding the real 3D space, factors such as physical configurations, spatial affordances, and meaning or affect as factors of place identity [110] create a sense of placeness [28] that ultimately impacts the virtual narrative space [33]. In the virtual narrative space, the purpose and type of the story told, along with its spatial design, are associated with the real space it targets [120].
Third, the user can interact with the augmented narrative space within its real space. AR enables the user to simultaneously perceive and spatially interact with both the real and virtual during a narrative experience. MacIntyre et al. [80] state that the personal nature of AR interfaces induces the user to implicitly interact with the augmented narrative space they are situated in, even when the virtual narrative space itself is non-interactive. In this situation, the user’s sense of place is created by the augmented narrative space as a whole. Consequently, how the virtual narrative space and real space are constructed in relation to one another to achieve a “perceived realism [40]” affects various aspects of narrative experience, such as spatial presence, engagement, and agency [118, 121, 122].
Taking these characteristics into account, we recognize the virtual narrative space, the real space, and the user situated within their spheres as fundamental and interdependent elements in the design of augmented narrative space. Therefore, we aim to investigate the various dimensions on which the virtual narrative space and the real space have been connected to construct an augmented narrative space. Furthermore, we discuss how these connections ultimately condition the spatial and narrative aspects of user experience.

2.2 Conceptualizing Virtual-Real Connections in Augmented Narrative Spaces

Many previous works have presented concepts, frameworks, and taxonomies for AR that provide insight into how the virtual space and the real space are connected, some with a specific focus on storytelling. Here, we review these works with special attention to the spatial and narrative aspects that these connections are based on.
The spatial aspects of virtual-real connections in AR are concerned mainly with how the physical properties of the real space affect the visual manifestation of the virtual space. The extent to which the virtual space and the real space can be paired with one another in materializing as an augmented space is one of these aspects [48, 131]. This is all the more important for AR storytelling, as a certain pairing determines not only the spatial form but also the type of narrative configured in a given virtual narrative space, along with the form, scale, and number of real spaces that it can target [44]. Shilkrot et al. [117] categorized AR narratives as situated, location-based, and world-level by where the augmented narrative space takes place. While they did not provide precise definitions for these terms or explain them through virtual-real relationships, these classifications—location-based in particular—are often used to emphasize how a real space is targeted and paired with a virtual narrative space.
Whereas location-based AR narratives refer to virtual narrative spaces tied to specific locations, the terms location-independent [31, 41] and play-anywhere [49, 108, 135] are frequently used in contrast for those that can be experienced everywhere. Additionally, adaptive (or space-adaptive) [43, 67, 121, 123] is another term that describes virtual narrative spaces accommodated in various real spaces. However, these concepts are not entirely orthogonal: location-based and adaptive are sometimes used in parallel to convey the idea of augmented narrative spaces that adapt to a specific location [20, 70]. As a systematic taxonomy of AR narrative types has not yet been proposed, we aim to inductively derive categories for virtual–real connections through our study.
Another spatial aspect is registration, which is the technological process of aligning the virtual space with the real space [141]. Many works have discussed conceptualizations of different AR registration methods. In a taxonomy of virtual information representation in AR, Tonnis et al. [129] defined registration as the spatial relationship between virtual and real objects. However, terms that distinguish types of registration were not given. In most approaches, registration is categorized by the types of computer vision and sensor technologies employed, such as marker and markerless tracking, feature tracking, plane detection, SLAM (Simultaneous Localization and Mapping), GPS, and sensor fusion [4, 13, 17, 144]. On the other hand, Normand et al. [96] provided a taxonomy based on the degree of freedom as opposed to the type of technology used to focus on the device’s position and user. In our study, we also take an approach to registration that is not oriented in technology. We focus on the spatial structures of connections between the virtual and the real that registration methods facilitate to examine their association with the design of augmented narrative spaces.
Regarding narrative aspects of virtual-real connections, the function the virtual narrative space serves with regard to the real space is one issue. For this, Azuma [2, 6] put forth three useful concepts: Remembering, Reinforcing, and Reskinning. Remembering uses historical events and memories associated with a location to enrich AR narratives. Reinforcing draws upon qualities presently inherent in the real space to tell stories that complement and amplify them. In contrast to Remembering and Reinforcing, Reskinning employs augmentations to recharacterize reality and make it suit the objectives of the virtual narrative being told. While Azuma introduced these ideas in the narrower context of location-based AR storytelling, it provides a meaningful point of reference for the various ways in which the virtual narrative space can affect the real space it targets.
Lastly, the placement of virtual narrative elements, such as virtual objects and spatially designed scenes as their collective arrangement, is determined by various features of the real space. Murray [92] stated that the immersive power of MR storytelling lies in conventionalizing virtual boundaries in relation to the spatial presentation of reality. Kalawsky et al. [61] stressed the importance of providing effective links between an AR application and the real space it is applied to. These links were specified as location complexity, content complexity, and semantic relevance by Wither et al. [137]. In more recent works that explored placement in AR for sensemaking and storytelling, the physical and semantic features of the real world, such as specific types of furniture [76] and space syntax [83, 123] were considered as crucial factors. How these concepts can be effectively applied to identify meaningful virtual-real connections in AR storytelling applications and systems will be one of the main issues for this study.

3 Methodology

In this study, we first sought to investigate the various ways in which spatial and narrative aspects of the virtual narrative space have been connected to the real space for AR storytelling within the scope of HCI, AR, and MR research. Second, we wished to uncover how these connections can provide the basis for establishing distinct categories of augmented narrative spaces. To achieve these goals, we formulated the following research questions:
RQ1.
How have virtual narrative spaces been paired, registered, functionalized, and placed with regard to their target real spaces for the design of augmented narrative spaces in HCI, AR, and MR research?
RQ2.
Based on the ways in which virtual narrative spaces have been connected to their target real spaces, what dominant types of augmented narrative spaces can be identified in HCI, AR, and MR research?
To answer these questions, we conducted a systematic literature review of AR storytelling applications and systems that have been presented in HCI, AR, and MR research over the last 20 years.

3.1 Article Selection

Figure 1:
Figure 1: PRISMA flowchart detailing the article selection precedure
In selecting the articles, we adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [101], which is a standardized reporting guideline that has been widely adopted. For the given procedure, we determined three terms: 1) the list of research venues most pertinent to the goals and scope of our study; 2) sets of search keywords to fine-comb through databases for the most relevant results; and 3) inclusion/exclusion principles for accurate deliberation. Details on the selection procedure are given in Figure 1.
Our systematic literature review initially targeted academic journals and conferences included in the top 20 Google Scholar ranking for the HCI category, updated as of July 2023, which were CHI, PACMHCI, IMWUT, TAC, IJHCS, BIT, HRI, IJHCI, Springer VR, IUI, IJIMT, DIS, UIST, IEEE VR, THMS, UAIS, TOCHI, HCII, IJCCI, and MTI (listed in order of rank). Regarding venues not included above and specializing in AR and MR research, which are domains not categorized by any metrics-based academic resource, we added ISMAR, TVCG, and SIGGRAPH on account of their strong relevance to the scope of our study, along with their highly held reputations in academic circles.
Next, we conducted test search trials with various combinations of keywords to cast a wide and dense net that could accurately filter the most fitting collection of articles. We finalized 12 sets with combinations of the following terms: (Augmented Reality OR Mixed Reality) AND (story OR narrative OR experience OR scenario OR game OR tour). These sets were used to search abstracts of peer-reviewed full papers from the chosen venues in the databases of ACM, IEEE, Taylor & Francis, Springer, and Elsevier. This led to the exclusion of MTI from our target venue, which is published by MDPI and ranked 20 on the HCI Google Scholar ranking. We set the search period to include articles from 2003 to 2023, which we deemed suitable to sufficiently cover the body of works relevant to our study. Identical search conditions were applied to all five databases, and the preliminary search results retrieved 1,657 articles from them. After duplicate results from each query and articles that were not full papers were removed, 927 remained.
We then began a manual elimination process conducted in two stages, which were based on three inclusion/exclusion principles given as follows:
(1)
Articles must present their own AR storytelling application or system and contain a sufficient amount of technical details on their actual development and deployment.
(2)
The technological employment of AR in the application or systems the articles present must satisfy all three conditions as defined by Azuma et al. [3]: 1) Combine real and virtual objects in a real environment; 2) run interactively in real-time; and 3) register real and virtual objects with each other.
(3)
The application or system in the articles must be demonstrated by a specific narrative content or experience that satisfies the most widely accepted definition of a story as epitomized by Jahn [56]: A sequence of events involving characters.
Figure 2:
Figure 2: The venues and numbers of the final selected articles
We first applied these principles to the titles and abstracts of the remaining papers, which resulted in 280 articles eligible for the next and final process of perusing the articles’ content to determine whether they satisfied all three principles. As the goal was to identify individual cases of augmented narrative spaces for analysis, articles that proposed AR storytelling systems as authoring tools and applications as single narrative experiences were considered equally so long as the articles in question showcased specific stories that were facilitated by those systems or applications.
To minimize bias and observe objectivity, the first, second, and third authors contributed equally to the manual elimination process: An article was included or excluded only when a unanimous decision was reached by the authors. To achieve a consensus for each article, conflicts were resolved through extensive discussions between the authors. Ultimately, a total of 64 articles were filtered and selected for analysis. The venues and numbers of the final set of articles are presented in Figure 2.

3.2 Article Analysis

We applied grounded theory to our analysis, which is a qualitative research method that scrutinizes data without predefined frameworks and inductively derives codes for specific attributes [32, 91]. Regarding RQ1, the coding of articles was centered on content that provided information on the criteria of connections we selected—Pairing, Registration, Function, and Placement Rule. Following Braun and Clark’s six stages of thematic analysis [14], we first conducted multiple readings of sections detailing the development and implementation of the augmented narrative space presented by the AR storytelling application or system in each article.
Second, we established initial codes through open coding. For this, we listed information concerning our criteria by creating a table for spatial and narrative attributes of the real space (e.g., spatial range, location, characteristic), the virtual narrative space (e.g., spatial range, types of augmentations, scene construction), technology (e.g., device environment, tracking method, system), and user experience (e.g., user role, interaction, movement) that consist the augmented narrative space in each article. In addition, we summarized the narrative content or experience provided, which was based on factors such as genre, character, premise, storyline, narrative setting, and linearity. The first, second, and third authors coded 30, 19, and 18 articles separately and combined the results to form the initial codes.
Third, we corroborated the initial codes to examine common elements, terms, and descriptions concerning the four criteria, which emerged as potential patterns. Fourth, we cross-reviewed these patterns by engaging in discussion to deliberate those that could best categorize the data for each criterion. Fifth, we finalized the patterns and named each of them. Lastly, we organized the data to report the patterns for each criterion. For RQ2, patterns assigned to the four virtual–real connections in each augmented narrative space were charted in list form to identify major branches of links between them. This was to examine whether certain categories for Pairing, Registration, Function, and Placement Rule were associated with one another to constitute distinct types of augmented narrative spaces.

4 Results

In this section, we first present patterns inductively derived for virtual–real connections in augmented narrative space (RQ1). Patterns for each connection, their definition, and the corresponding articles are given in Table 1. Based on the links between the virtual–real connections for every case, we analyze eight dominant types of augmented narrative spaces that were identified (RQ2).

4.1 Virtual–Real Connections in Augmented Narrative Spaces

Table 1:
ConnectionCategory (No.)DefinitionArticles
PairingExclusive (28)VNS is overlaid on single, designated RS[18] [22] [29] [30] [34] [35] [38] [42] [45] [47] [51] [53] [58] [64] [65] [74] [85] [90] [102] [114] [115] [123] [125] [126] [132] [134] [136] [145]
 Universal (19)VNS is overlaid on any RS[21] [23] [26] [36] [46] [68] [69] [77] [89] [94] [95] [97] [99] [105] [116] [133] [138] [143] [146]
 Conditional (17)VNS is overlaid on RS with specified attribute(s)[7] [24] [37] [39] [52] [60] [62] [63] [66] [70] [71] [75] [79] [88] [113] [140] [142]
RegistrationSpatial Mapping (8)Entire spatial range of VNS is layered over RS[21] [23] [39] [89] [97] [99] [123] [133]
 Multiple Anchoring (40)Augmentations in VNS are registered to multiple specific points in RS[7] [18] [22] [24] [26] [29] [30] [34] [35] [36] [37] [45] [46] [52] [53] [58] [60] [62] [63] [65] [66] [68] [69] [70] [75] [79] [85] [88] [90] [94] [95] [102] [113] [115] [132] [136] [140] [142] [143] [146]
 Single Anchoring (16)Augmentations in VNS are registered to a single specific point in RS[38] [42] [47] [51] [64] [71] [74] [77] [105] [114] [116] [125] [126] [134] [138] [145]
FunctionReinforcing (28)VNS complements features currently inherent in RS[7] [18] [24] [30] [35] [38] [42] [45] [46] [47] [51] [53] [64] [66] [69] [71] [74] [75] [85] [88] [90] [115] [123] [125] [126] [134] [140] [145]
 Remembering (9)VNS represents past events and memories related to RS[22] [29] [34] [58] [65] [102] [114][132] [136]
 Reskinning (27)VNS tells stories with no relevance to the past and present features of RS[21] [23] [26] [36] [37] [39] [52] [60] [62] [63] [68] [70] [77] [80] [89] [94] [95] [97] [99] [105] [113] [116] [133] [138] [142] [143] [146]
Placement RulePhysical Feature (17)Augmentations in VNS are placed according to geometric and geographic features of RS[21] [23] [26] [36] [39] [68] [77] [89] [94] [97] [99] [105] [116] [133] [138] [143] [146]
 Semantic Relationship (47)Augmentations in VNS are placed by their relevance to the functions, meanings, and affordances of physical features in RS[7] [18] [22] [24] [30] [29] [34] [35] [37] [38] [42] [45] [46] [47] [51] [52] [53] [58] [60] [62] [64] [63] [65] [66] [69] [70] [71] [74] [75] [79] [85] [88] [90] [95] [102] [113] [114] [115] [123] [125] [126] [132] [134] [136] [140] [142] [145]
Table 1: List of patterns for four virtual–real connections in augmented narrative spaces and the corresponding articles in the review (No.: Total number of articles for each pattern; VNS: Virtual Narrative Space; RS: Real Space)

4.1.1 Pairing.

For Pairing, which refers to how a given virtual narrative space is spatially matched with the real space it targets, three patterns were identified: Exclusive, Universal and Conditional. Exclusive pairings were found for 28 of the 64 articles(43.75%), where the augmented narrative space consisted of a given virtual narrative space that can only be connected to a single, designated real space. A virtual narrative space consisting of an evacuation training game for a specific building would not be operable anywhere else [18]. In 19 Universal pairings (29.69%), a given virtual narrative space could be situated in any real space, so long as its dimensions could physically accommodate the virtual narrative space. Conditional pairings constituted the remaining 17 cases (26.56%). In these cases, a virtual narrative space could be overlaid on an unspecified number of real spaces, which, so long as the minimum standards of augmentation as stated for a universal pairing are met, must satisfy one or more requirements concerning their spatial attributes. For example, any shipping container can be paired with a virtual narrative space that tells stories related to shipping containers [7].

4.1.2 Registration.

Registration concerns how a given virtual narrative space is attached to the target real space. For this connection, Spatial Mapping, Multiple Anchoring, and Single Anchoring were patterns that emerged. The distinction between registration patterns was made by the spatial form of attachment, which resulted from the application of different tracking technologies. Spatial Mapping indicates that the entire spatial range of the virtual narrative space is layered over the target real space. This was the case for eight articles (12.5%) whose systems or applications employed tracking methods that interpret the geometric structure of real spaces, such as SLAM [127]. In Harmonize, the target real space is scanned in its entirety to facilitate a real-scale first-person shooter game [99]. Multiple Anchoring refers to the 40 cases (62.5%) where augmentations in the virtual narrative space were intermittently attached to numerous specified points in the target real space. This was usually achieved through image or marker detection, object tracking, and GPS-based alignment. For example, various landmarks or monuments along the Singapore River connected the virtual narrative space to the whole area [65]. The remaining 16 connections (25%) categorized as Single Anchoring employed the same methods to establish a single attachment point within the spatial range of the target real space. In one application, multiple scenes of a story were augmented on a fixed position over a map of an ancient city [114].

4.1.3 Function.

For Function, which pertains to how the narrative content or experience provided by the virtual narrative space affects the target real space, the patterns ultimately followed Azuma’s classification: Reinforcing, Remembering, and Reskinning [2]. We adopted and applied these terms to the patterns we identified because the codes derived for function were similar to one of these three in their meaning. In 28 cases (43.75%) of Reinforcing, virtual narrative spaces complemented the physical and semantic features presently inherent in their target real spaces. A treasure hunt game augmenting quizzes and tasks on the main attractions of a natural history museum is one example [90]. The codes for these cases included keywords such as enhance, strengthen, bring out, emphasize, and contextualize. There were nine cases (14.06%) of Remembering, in which virtual narrative spaces represented past events and memories related to the target real space1. Recall, evoke, commemorate, treasure, and historicize were some of the descriptions associated with this function. In one case, scenes from a period TV drama were augmented at real-world locations where the events they portrayed took place in the 16th century [102].
The remaining 27 cases (42.19%) for Reskinning involved virtual narrative spaces that are not associated with features of the target real space in any way. Common descriptors included transform, create, turn into, and alter. For Reskinning, three characteristics could be further found. In most cases, the virtual narrative space reskinned the target real space to immerse users in entirely fabricated stories (fictional). Whereas most of them were games, one where users fight one another at an imaginary battlefield being an example [99], some adapted parts of known novels or movies [26, 36]. Furthermore, the target real space became the grounds where practical scenarios for certain situations could be carried out (situational). These cases included training [39, 77, 138], learning [52, 133], and simulation [95] scenarios that are irrelevant to features of their target real spaces.
In other cases, the virtual space brought narrative elements from another real space such as the south pole [68], a museum exhibit [60], or a now-demolished historic site [105] to unrelated target real spaces, placing users in a completely different location from their immediate reality (translocative). However, such traits could not be established as independent categories of function, as more than one of them could be applied to some of the cases at the same time. For instance, a virtual narrative space offering neurofeedback training via a bird-egg collecting game is both situational and fictional [77] in reskinning the real space. Similarly, a virtual narrative space consisting of an exhibition room at a real museum at an unrelated target space to tell the story of a made-up family during the Holocaust is both translocative and fictional  [60]. For this reason, we finalized Reskinning as an indivisible category of function.
Figure 3:
Figure 3: An alluvial diagram depicting links between patterns of four virtual–real connections in augmented narrative space—Pairing, Function, Placement Rule, and Registration—for the 64 articles reviewed

4.1.4 Placement Rule.

Placement Rule specifies the type of information the virtual narrative space retrieves from the real space to determine where and how its elements are placed. For this connection, two patterns emerged—Physical Feature and Semantic Relationship. In the 17 cases (26.56%) for Physical Feature, elements of the virtual narrative space were placed according to the geometric features of the real space components they were connected to, such as those of planar surfaces, images, markers, or geographic features such as GPS coordinates. In one case, a 2D image of a game character is augmented with its 3D version based on the visual features [77]. In the other 47 cases (73.44%), the Semantic Relationship between elements of the virtual narrative space and the physical features of the real space was the basis on which the former was placed. These relationships are based on the functions, meanings, and affordances of the target real space as a whole and also of individual objects and specific locations within their spatial range. For example, specific locations within a university campus identified through GPS coordinates are connected to virtual content that describes their history and meaning [132].

4.2 Types of Augmented Narrative Spaces

Once we identified patterns for Pairing and Registration as connections between spatial aspects of the virtual narrative space and real space, along with Function and Placement Rule as those between narrative aspects of the two spaces, we linked the patterns found for these four connections in all 64 articles and visualized the data in an alluvial diagram, as shown in Figure 3. Often utilized as a graphical summary of multidimensional data for systematic reviews [15, 139], the alluvial diagram depicts prevailing tendencies for virtual-real connections that can collectively define and distinguish different categories of augmented narrative spaces.
The order in which the four connections are linked in the diagram was based on a space-oriented authoring process of AR narratives proposed by Shin and Woo [119, 120]. For a virtual narrative space, 1) selecting the type of real space to target (Pairing) is first considered in 2) determining the function it would perform with regard to that target space (Function). Then, 3) narrative elements would be chosen and placed in relation to features of the target real space (Placement Rule) and 4) registered as a whole to lock the virtual narrative space onto the target real space (Registration). This specific order enabled a more effective visualization of how the four virtual–real connections are considered in the design flow of augmented narrative spaces and what patterns of links between them emerge as a result of that flow.
Consequently, we confirmed eight dominant links between the virtual-real connections. Dominant links were defined as those consisting of more than 5% of the total number of articles. These eight links accounted for 57 cases of the 64 articles reviewed (89.06%). Furthermore, we found that they could be placed in two subsuming categories of augmented narrative spaces: 1) Spaces of Virtually Narrativized Reality and 2) Spaces of Realized Virtual Narratives. The list and number of articles categorized by the types of augmented narrative space are given in Table 2.

4.2.1 Spaces of Virtually Narrativized Reality.

32 articles (50%) comprising four dominant branches of links between virtual-real connections commonly prioritized the target real space in their construction of augmented narrative spaces. In these cases, the virtual narrative space is born out of the target real space, with connections between them all originating from the real: The narrative setting exists within the target real space. Hence, the primary goal of storytelling is to engage the user more deeply with the real space by narrativizing it with the support of augmentations. Within this category, branches are distinguished mainly by the environment and scale of the target real space.
1) Historicized Outdoor Spaces
In the first type of augmented narrative space that virtually narrativizes reality, the virtual narrative space is exclusively paired with the target real space to tell stories from the past that are specifically associated with it. The placement of its narrative elements is related to the real space and its components on semantic levels, and the space as a whole is registered onto the target space through numerous augmentation points (Exclusive-Remembering-Semantic Relationship-Multiple Anchoring). This type comprised eight out of the 64 cases reviewed (12.5%). In these augmented narrative spaces, stories the virtual narrative space overlays are commonly about the history of the target real space, which may not be readily apparent: On the outset, the real space itself does not directly reveal or contain elements of the virtual narrative space. Instead, it provides the setting for them through specific features such as landmarks and locations that were once the actual and physical backgrounds of events that occurred in the past.
These features, or PoIs (Point of Interests) [128], serve as the anchoring points that augmentations of the virtual narrative space are registered onto. They constitute story nodes that are sequentially ordered within an outdoor area of a singular, socially recognized identity, such as cultural and natural heritage sites [22, 29, 34, 102, 132], regional districts (e.g., the Westwood neighborhood in Los Angeles [136] and the Singapore River area [65]), and university campus grounds [58]. Therefore, we termed this link between virtual-real connections as Historicized Outdoor Spaces. In all cases for this type, augmented narrative spaces provided tours of the target real spaces through stories that were essentially linear.
Table 2:
Category (No.)Type (No.)Links Between ConnectionsArticles
Spaces of Virtually Narrativized Reality (32)Historicized Outdoor Spaces (8)Exclusive-Remembering-Semantic Relationship-Multiple Anchoring[22] [29] [34] [58] [65] [102] [132] [136]
 Enhanced Indoor Spaces (8)Exclusive-Reinforcing-Semantic Relationship-Multiple Anchoring[18] [30] [35] [45] [53] [85] [90] [115]
 Extended Object Spaces (10)Exclusive-Reinforcing-Semantic Relationship-Single Anchoring[38] [42] [47] [51] [64] [74] [125] [126] [134] [145]
 Flexible Backdrop Spaces (6)Conditional-Reinforcing-Semantic Relationship-Multiple Anchoring[7] [24] [66] [75] [88] [140]
Spaces of Realized Virtual Narratives (25)Transported Scene-Based Spaces (6)Universal-Reskinning-Physical Feature-Multiple Anchoring [26] [36] [68] [94] [143] [146]
 Transformed  Room-Scale Spaces (6)Universal-Reskinning-Physical Feature-Spatial Mapping[21] [23] [89] [97] [99] [133]
 Transplanted Tabletop Spaces (4)Universal-Reskinning-Physical Feature-Single Anchoring[77] [105] [116] [138]
 Compatible Portal Spaces (9)Conditional-Reskinning-Semantic Relationship-Multiple Anchoring[37] [52] [60] [62] [63] [70] [79][113] [142]
Table 2: Types of augmented narrative spaces defined by eight dominant links between real-virtual connections
2) Enhanced Indoor Spaces
In this type, the virtual narrative space establishes a 1:1 pairing with the target real space to narrativize and emphasize characteristics that presently define it. The placement of narrative elements as augmentations is based on their semantic relevance to the real space and its components, and multiple points within the real space serve as anchors that the virtual narrative space is registered to (Exclusive-Reinforcing-Semantic Relationship-Multiple Anchoring). This was the case for another eight articles (12.5%). The only difference between Historicized Outdoor Spaces and Enhanced Indoor Spaces is their function: the latter reinforces the present state of a real space instead of remembering its past. For the articles we analyzed, however, this difference also led to discrepancies between the categories in the environment of the target real space—outdoors vs. indoors.
Here, the narrative setting is clearly provided by the present configuration of the target real space or by semantic attributes inherent to it. The anchoring points of the real space are objects or specified locations with unique attributes, essentially forming distinct subareas. They are augmented with narrative elements that elaborate their contexts and are pieced together for a continuous narrative experience with varying degrees of narrative linearity. For this reason, six out of the eight cases targeted exhibition rooms in museums to support, complement, and thereby enhance the content of individual artifacts in a story that binds the whole virtual narrative space to the entire spatial range of the target real space [30, 35, 45, 53, 90, 115]. In the other two cases, the virtual narrative space enabled users to experience training scenarios involving the context of specific locations and attributes of the real target space [18, 85].
3) Extended Object Spaces
This type consists of augmented narrative spaces where a virtual narrative space is matched with a single, specific target real space to reinforce its current attributes. Augmentations bear relevance to the target real space on semantic levels and are registered through a single corresponding point (Exclusive-Reinforcing-Semantic Relationship-Single Anchoring). 10 cases (15.63%) were identified for this type. Whereas the only difference between Historicized Outdoor Spaces and Enhanced Indoor Spaces is their function, registration is the sole factor distinguishing Enhanced Indoor Spaces from this type. Here, units of the virtual narrative space are overlaid on a single object, regardless of how they are spatially and temporally arranged.
In five of these cases, the augmented narrative space targeted a 2D image board with unique designs that signified a specific space, such as a fictional urban area [125], a driving route [47], a zoo [145], and predefined reference images [74]. In three other cases, scenes related to the content of certain books were augmented on their pages [38, 42, 51]. The remaining two cases featured larger physical objects, such as a rhinoceros statue at an art exhibit [64] and a functioning hydrogen fuel tank [134] as target areas of the virtual narrative space. In all these cases, the virtual narrative space was downscaled according to the confined spatial range of the target objects. However, it produced the effect of extending the dimensions of the target real object regarding the physical space it occupied, the meaning it conveyed, and the function it performed.
4) Flexible Backdrop Spaces
In the last type of augmented narrative space prioritizing reality, the virtual narrative space was paired with an unspecified number of real spaces that satisfied one or more spatial attributes and emphasized those very attributes through the content or experience it provided. The placement of augmentations depended on the semantic relationship between the virtual and real, registered through multiple objects or locations in the real space (Conditional-Reinforcing-Semantic Relationship-Multiple Anchoring). This was the case for six articles (9.37%). By providing more flexibility in determining the target real space, these space type ultimately leverages the virtual narrative space to interact with elements of the real space, incorporating them as significant backdrops in the story.
Unlike other types that virtually narrativize reality, these spaces are not distinguishable in terms of their spatial environment or scale: Within this type, target real spaces greatly vary in these aspects, ranging from 2D image boards [140] to life-sized objects [66], rooms [7, 88], and outdoor locations [24, 75]. What sets them apart is that the narrative setting, which is rooted in reality, is less concrete. The characteristics of the anchoring points are viewed more as abstract ideas that loosely influence the construction and content of the virtual narrative space. Therefore, engagement with the narrative setting in the target real space occurs on a more symbolic or metaphorical level. This results in stories that are more context-driven than context-specific. For example, standardized simulations of clinical breast examination can be augmented on any real mannequin [66], and an AR learning session delivering general information about a eucalyptus tree can be held at any outdoor location with that tree [24].

4.2.2 Spaces of Realized Virtual Narratives.

In 25 articles (39.06%) that accounted for the other four dominant links between virtual-real connections, the virtual narrative space took precedence over the real space. As opposed to those that narrativize reality, all narrative elements are rooted in the virtual narrative space in these types of augmented narrative spaces. The narrative setting wholly exists in the virtual narrative space or merely borrows props from the real space to enhance its power in reskinning the target real space: Reality is a means through which the virtual narrative space can materialize. The stories are often entirely fictional and are granted greater imaginative freedom, with spatial configurations of the real world playing a secondary role compared to the virtual elements. As a result, users are immersed more deeply in the manifestation of the virtual within the real than the real itself. For this category, types are mainly distinguished by the dimensions in which the real space is transfigured by the virtual narrative space.
1) Transported Scene-Based Spaces
In these spaces, the virtual narrative space can be paired with any target real space, provided that the minimum physical area needed to augment its spatial range is secured. It serves to transport a narrative experience that bears no relevance to the placeness of the target real space. The placement of its elements is based on the geometry or geography of the real, which correspond to specific points in reality (Universal-Reskinning-Physical Features-Multiple Anchoring). Six cases (9.37%) were identified as establishing this particular link.
In four of the cases, all narrative elements, including the setting, were enclosed within the virtual narrative space. The virtual narrative space was then transported to the real space through multiple augmentation points organized as scenes. The augmented narrative spaces facilitated narrative progression through these scenes in varied spatial environments, ranges, and narrative linearity. A miniature Asian garden was augmented onto different markers on an empty tabletop space for sequential interactions between virtual 3D characters [94]. A fully virtual space was designed for an escape room game that can be implemented in any indoor space at a life-size scale, with various anchors augmenting different scenes [143]. For a larger spatial range outdoors, an in-car game featuring a virtual 3D character set to different GPS locations was implemented [146], and select PoIs of Antarctica were transported to fit a smaller open area [68]. In two other cases, scenes out of the novel Wrinkle in Time [26] and the film Princess Bride [36] were augmented to be reenacted anywhere in reality.
2) Transformed Room-Scale Spaces
For this type, there are no restrictions on how the virtual narrative space is paired with the target real space beyond the minimum requirement for augmentation. The augmented narrative space provides a story that is irrelevant to the target real space’s features and ultimately reskins it, registered over its whole region by processing the spatial configuration of the designated area (Universal-Reskinning-Physical Feature-Spatial Mapping). Ultimately, the virtual narrative space transforms the target real space into a distinctly different entity to the point of erasing its original attributes. This type accounted for six articles (9.37%), which all targeted room-scale indoor spaces.
Five cases presented game-based narratives utilizing the entire spatial range of the target real space at room scale. For example, Ninja on a Plane leveraged the planar surfaces detected in the corner of an unspecified office room to turn the whole area into a ninja’s escape route [21]. Any room could become a galaxy of planets to be conquered in Startag [97], a battlefield in Harmonize [99], a language-learning space in Arigato [133], or a coin-collecting game site [89]. SceneAR [23], which is the one other case for this type, detects the surfaces of any target indoor space to spatially arrange a sequence of virtual narrative scenes over the entire spatial range of the target real space, ultimately transforming it into a setting for the virtual narrative space.
3) Transplanted Tabletop Spaces
Transplanted Tabletop Spaces are different from Transported Scene-Based Spaces and Transformed Room-Scale Spaces in their registration only: the augmented narrative spaces for this type register virtual narrative spaces to the real target space through a single marker, image, or location, which is not associated with the story in any meaningful way (Universal-Reskinning-Physical Feature-Single Anchoring). We identified four articles (6.3%) for this type. Compared to Transformed Room-Scaled Spaces, where a real room with specific semantic features is given an entirely different function, the spaces in this category select a confined area within a real space as a purely geometrical entity with no preassigned meaning. Subsequently, a virtual narrative space entirely disconnected from the physical space is directly transplanted onto this semantically vacant area.
Cases that fall under this type all presented augmented narrative spaces that target empty tabletop spaces as their real spatial range. The virtual narrative spaces augmented on them commonly provided content for interactive board games that users engage in through virtual agents at a miniature scale. Eggly [77] is an educational game for children with autism set at a farm, and LegionARius [105] is a strategy game based on ancient Roman military camps and Germanic settlements. These games can be augmented on any space that fits the physical dimensions of the entirely virtual game boards. Woll et al. [138] features a fictional space with an astronaut under alien attack in a completely virtual board game setting. Sekhava et al. [116] augmented a virtual Zigorat (a maze tower) on a board-game-sized surface, which a ladybug character must climb to release her friends.
4) Compatible Portal Spaces
In the last type of realized virtual narratives, virtual narrative spaces are paired with real spaces with one or more specified attributes and serve to reskin them. The placement of virtual elements is based on semantic features of the real target space, and these elements are augmented on multiple points within the real spatial range (Conditional-Reskinning-Semantic Relationship-Multiple Anchoring). Nine cases (14.06%) were found for this particular link. In all these cases, the real objects and locations to which the virtual narrative space was registered were interchangeable with others that shared the specified attributes. Moreover, these elements functioned as portals through which a virtual narrative space bearing no relevance to the target real space could spatially manifest.
Six cases required certain physical objects to be placed in the real space to open up reskinning narrative spaces: BubbleU [63] augmented narrative elements on nine predefined objects, and certain image cards were needed in an AR game for children that could be played on any tabletop space [52]. Be Our Guest [113] specified that the object leading to virtual spaces narrating various tea-drinking cultures should be a cup, and a story of the Holocaust [60] needed the same type of furniture to be set up in the target real space as its original setting at a museum exhibit. Similarly, a room with a table and chairs were the stated conditions of an AR narrative that changes such real spaces into a setting for Three Angry Men, an interactive play [79]. Another case replaced detected objects in a real-world scene with other virtual characters through augmentations to create fictional stories [62]. In the other three cases, portals into the virtual narrative space for outdoor environments were specified, such as any location that resembles the narrative setting of scenes from the movie Nosferatu [37] or any theme park attraction queue for an AR game offering spatial and temporal escape [142]. Li et al.’s location-aware adaptation method for virtual narrative spaces set predefined parameters such as location compatibility and walking distance in a given real space [70].

5 Discussion

In this analysis, we examine the results to understand how the spatial and narrative aspects of the virtual-real connections identified (RQ1) are associated, resulting in the eight dominant types of augmented narrative spaces within two higher-level categories: Spaces of Virtually Narrativized Reality and Realized Virtual Narratives (RQ2). We then investigate how these types facilitate various modes of spatial and narrative experiences. Based on our analysis, we present design recommendations to leverage these virtual-real connections and types of augmented narrative spaces to establish a more systematic approach to AR storytelling. Additionally, we acknowledge the limitations of our approach.

5.1 Analysis of the Results

5.1.1 Spatial and narrative associations between virtual–real connections.

The distribution of patterns for Pairing (Exclusive, Universal, Conditional) and Function (Reinforcing, Remembering, Reskinning), along with the links between them, shows that there are clear associations between how the virtual narrative space and the real space initially find one another to establish the main objective of the augmented narrative space they comprise. For pairing, exclusive connections accounted for the most number of cases (28, 43.75%). This indicates that the target real space has largely been designated and fixed to a specific, meaningful location or area for the design of augmented narrative spaces.
In these cases, the virtual narrative spaces functioned either to reinforce or remember the real space, with considerably more cases for the former (19). For all but 2 cases among the 19 Universal pairings, the virtual reskinned the real. In the remaining 17 cases of Conditional pairings, the virtual narrative space’s function was distributed among Reskinning (10) and Reinforcing (7). This means that depending on the extent to which a target space is specified, the narrative content configuring the virtual is determined: the more exclusive the spatial pairing, the more bounded the story’s setting and premise are to the target real space.
Then, the problem is how elements of stories in the virtual narrative space, as determined by Pairing and Function, are realized through patterns found in Placement Rule (Physical Feature and Semantic Relationship) and Registration (Spatial Mapping, Multiple Anchoring, Single Anchoring) to implement the augmented narrative space as a spatialized narrative experience in reality. For all but two cases, Exclusive or Conditional pairings augmented narrative elements of the virtual based on their Semantic Relationship with the target real space, which was the majority of cases (47, 73.44%). None of these augmented narrative spaces were registered to the real space through Spatial Mapping: For the articles reviewed, semantic associations were mostly recognized through specific points of features in the real space instead of a holistic understanding of its spatial structure.
In the other 17 cases relying on Physical Features for placement, 16 of which were Universal pairings, registration patterns showed an even distribution overall among Multiple Anchoring (6), Single Anchoring (4), and Spatial Mapping (7). This indicates that Placement Rule and Registration are connections that are also impacted by the specificity of the target real space and the narrative content it dictates: Whereas most Exclusive and some Conditional pairings establish meaningful points of connections for stories bound to the target space, Universal pairings generally entail elements of more free-ranging narratives to adapt to the physical features of the real space in various ways. Ultimately, these associations between the four virtual-real connections are the basis on which the eight dominant types of augmented narrative space can be discussed.

5.1.2 Spatial and narrative experiences of different augmented narrative space types.

Space of Virtually Narrativized Reality outnumbered those of Realized Virtual Narratives by seven cases. This owes primarily to the prevalence of Exclusive pairings, which is one of the defining features of augmented narrative spaces prioritizing the real. Furthermore, three out of four types with the most number of cases belonged to Spaces of Virtually Narrativized Reality: Extended Object Spaces (10), Historicized Outdoor Spaces (8), and Enhanced Indoor Spaces (8). Among Spaces of Realized Virtual Narratives, the highest number of cases were Compatible Portal Spaces (9). On the other hand, the remaining four types—Flexible Backdrop Spaces (6), Transported Scene-based Spaces (6), Transformed Room-scaled Spaces (6), and Transplanted Tabletop Spaces (4)—did not differ greatly in number.
These augmented narrative space types illustrate the major ways in which newly developing technologies and ideas for spatial narrative experiences in AR have been implemented in HCI, AR, and MR research. Through Extended Object Spaces, AR storytelling has aimed to narrativize a limited area contained within the user’s immediate surroundings through specific, tangible objects. As the physical dimension of the augmented narrative space is not large enough to situate the user in it, the user takes a fixed position outside its bounds to become a part of an interactive narrative through physical or virtual agents in miniature or to observe a linear story’s progression. An example of the former is ChildAR [47], which presents an augmented narrative space where users drive along different routes by moving an augmented miniature car on a tabletop gameboard space. In one of the latter cases, a book on Korean temple bells is augmented with their history, and the user engages with the narrative outside of it through interactions that do not affect the narrative being told [42].
Conversely, Historicized Outdoor Spaces and Enhanced Indoor Spaces envelope and physically move the user along a spatially arranged narrative in various trajectories [8] and have mostly been developed for the cultural heritage domain. They focused on enlivening narratives inherent in heritage sites and museum exhibits. The Svevo Tour takes the user to various locations in the Italian city of Trieste through a narrative centered on the novelist Italo Svevo [34], and Hammady et al.’s MuseumEye MR application implemented an augmented narrative space for the Tutankhamun exhibit at Cairo’s Egyptian Museum with three storytelling scenes [45]. Compared with the three aforementioned space types, Flexible Backdrop Spaces virtually narrativize reality more indirectly and metaphorically. Consequently, the context provided by the real space, rather than its unique placeness, is embellished when reality is brought to the fore as the narrative setting. For instance, LINA engages the sociocultural context embedded in elementary school classrooms for an educational mystery-solving game that centers on a missing classmate. It can be utilized in classrooms anywhere to reinforce their original function [88].
Turning to Spaces of Realized Virtual Narratives, Compatible Portal Spaces, the type with the second highest number of cases, leverage certain traits that can be found across various real spaces as portals into virtual narrative spaces. This allows users to evade their surroundings while being situated in them. As long as there are some chairs and a table in any real space, a completely fictional story of a jury’s conflict that is disassociated with the target space can be integrated into the reality that the user is inhabiting [79]. Unlike the three other augmented narrative space types in the same category (Transported Scene-Based Spaces, Transformed Room-Scale Spaces, and Tranplanted Tabletop Spaces), Compatible Portal Spaces establish a semantic relationship between narrative elements of the virtual and features of the real: augmented characters sit on the real chairs around the real table in the above example. This indicates that together with Spaces of Virtually Narrativized Reality, this type represents a major format that employs AR narratives for specified real spaces.
For the three other space types of Realized Virtual Narratives, the distinction between them is made by how each type registers the virtual narrative space to the target space, as other connections are the same. This leads to differences in the spatial scale of the narratives being realized to some extent, which is in turn associated with the type of stories being told. Spatial Mapping for Transformed Room-Scale Spaces and Single Anchoring for Transplanted Tabletop Spaces not only give the space types their names but also configure how the different spatial scales are employed within the common narrative genre they represent. Almost all cases in the two space types are games except for SceneAR, which presents a linear story of bees arguing over flowers [23]. The five other cases of Transformed Room-Scale Spaces are first-person role-playing games where users themselves conduct gaming actions at a real scale, experiencing the space through various perspectives and modes of input. In Star Tag, the user is encouraged to move quickly and uses voice and gestures to approach and conquer virtual planets dispersed in space [97].
On the other hand, the spatial dimensions of the four Transplanted Tabletop Spaces dictate that they all take on the form of board games, which limits the user’s spatial interactions to a mobile interface. In terms of their story worlds, both space types tend to represent those that are entirely removed from reality itself and not just their target spaces. For Transported Scene-Based Spaces, which vary in their spatial scale, Multiple Anchoring as the registration method sets the space type apart from Transformed Room-Scale Spaces and Transplanted Tabletop Spaces in that the scenes anchored to various points in the target real space elicit linear narratives as opposed to games. Users experience narrative progression by encountering different scenes remediated within the real space. This means that the augmented narrative space integrates existing multimedia resources within a newly devised narrative sequence [12]. For example, IFiction [26] and ARKaraoke [36] take a select number of scenes from the novel Wrinkle in Time and the film Princess Bride respectively out of their original context and position them in different locations to fictionally reskin the real space. These scenes offer varying levels of immersion according to their spatial scale, which varies within allowing users to occupy the virtual narrative space while moving through the physical environment.

5.2 Design Recommendations

5.2.1 Strategize based on priority—the virtual or the real?

In designing augmented narrative spaces, the foremost step should be to determine whether it is the virtual narrative space or the real space that plays a more central role in its dual composition, i.e., which of the two layers should prevail over the other. Depending on this decision, how the two spatial entities are connected falls into place. If the virtual should be more prominent, stories of creative versatility can be adapted to a diverse range of real spaces via their physical features to showcase the transformative power of the virtual narrative space. If the focus is instead on bringing a certain real space to the fore, the virtual narrative space can recede to the background through a story that illuminates the hidden or apparent features of the real space. This can be achieved by connecting the virtual narrative space to specific and meaningful areas, locations, or objects. In essence, the choice between real and virtual dominance in an augmented narrative space should dictate the main objective of the user experience it provides and build the foundation upon which ensuing design decisions are made.

5.2.2 Consider narrative tropes against augmented narrative space types.

Once the main objective for the design of an augmented narrative space is set, outlining the specifics of the narrative itself through parameters such as genre, character, sequence, and the degree of linearity to constitute certain narrative tropes for that objective is required before it can be spatialized in the virtual and the real. At this stage, features that characterize dominant types of augmented narrative spaces through links between virtual–real connections can be utilized as conventions of AR storytelling. This is to say that certain augmented narrative space types can be referred to and reflected in creating certain narrative tropes. For example, suppose the goal is to realize the virtual narrative space through an immersive and interactive indoor experience set in a fantasy world. In this case, the setting is consistent with Transformed Room-Scale Spaces and can be further developed by following or playing with conventions for this type. The genre of the narrative would typically be a game that is highly interactive and non-linear, with users participating in the narrative as characters with the agency to trigger the sequence of events.

5.2.3 Balance the virtual and the real through spatial user interaction.

When details of the narrative are confirmed, the ensuing design choice mainly involves spatializing narrative elements in the virtual narrative space with regard to the target real space. Our analysis revealed that according to an augmented narrative space’s main objective and link between virtual–real connections, the composition of virtual and real elements differed in their amount and presence: dominantly virtual spaces are more noticeably virtual, and real ones more real. However, it is crucial for users to perceive a balanced integration of the virtual and the real to enhance their immersion during an AR narrative experience [122], either to suspend their disbelief in virtuality [86, 121] or engage themselves better in reality [130]. To mitigate the inevitable imbalance between the virtual and the real in certain types of augmented narrative spaces, we recommend facilitating user interaction with the less dominant elements as part of the design strategy. For instance, enabling users to manipulate real objects to trigger virtual narrative sequences, placed to encourage active movement in the real space of a Transformative Room-Scale Space that prioritizes virtuality, would be beneficial for narrative experience by ensuring that users engage with both the virtual and the real.

5.3 Limitations

There are some limitations that should be addressed in subsequent studies. First, our systematic literature review primarily targeted research articles in the HCI domain, with a few additions of venues representing the most relevant works on AR and MR. This was an intentional decision made to investigate how the design of augmented narrative spaces has been understood and practiced in academic spheres. However, it led to the exclusion of commercial cases that have greatly contributed to the advancement and deployment of AR storytelling. Ingress [25, 82], Pokémon Go [100, 109], and Fragments [59, 121] are some of the most notable examples. Considering that these cases present augmented narrative spaces that defy the scope of those identified in our current review in some aspects, analyses of virtual–real connections in them will provide deeper insight into this issue.
Second, we employed a selective number of virtual–real connections to be identified and utilized in defining types of augmented narrative spaces. This served as a practical and effective framework that encompasses both spatial and narrative aspects of relationships between the virtual narrative space and the real space as two essential components of the augmented narrative space. Nevertheless, they should be considered alongside other traits, such as those that involve the user’s spatial conduct and narrative experience as another meaningful element that ultimately establishes the augmented narrative space as grounds for an embodied experience [19, 98]. This may lead to a more comprehensive understanding of the holistic design principles for AR storytelling.

6 Conclusion AND Future Work

In this work, we conducted a systematic literature review of 64 articles that presented AR storytelling systems or applications in HCI, AR, and MR research over the last 20 years. We inductively derived patterns for Pairing, Registration, Function, and Placement Rule as connections between the virtual and the real in the design of augmented narrative spaces. Furthermore, we examined links between the virtual–real connections to identify eight dominant types of augmented narrative spaces. Through a comparative analysis of each type’s characteristics, we conclude that the design of augmented narrative space is fundamentally shaped by how connections between the virtual and the real define, support, and leverage the degree of dominance of one spatial entity over the other.
In the future, we plan to expand the scope of our review to a wider range of augmented narrative spaces to include commercial applications as well and analyze them through a more rigorous framework. In addition, we intend to integrate our findings on virtual–real connections and augmented narrative space types into the refinement and enhancement of AR storytelling systems. Ultimately, our objective is to glean more detailed and in-depth insights for a more systematic approach to the design of augmented narrative spaces that can result in enhanced user experiences for them.

Acknowledgments

This research was supported by National Research Council of Science and Technology(NST) funded by the Ministry of Science and ICT(MSIT), Republic of Korea(Grant No. CRC 21011). This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the Graduate School of Metaverse Convergence support program(IITP-2024-RS2022-00156435) supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation).

A List of Articles Included in the Systematic Literature Review

(1)
Barba, Evan, et al. "Thinking inside the box: making meaning in a handheld AR experience." 2010 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2010.
(2)
Catal, Cagatay, et al. "Evaluation of augmented reality technology for the design of an evacuation training game." Virtual Reality (Springer VR). 24 (2020): 359-368.
(3)
Chekhlov, Denis, et al. "Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual slam." 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2007.
(4)
Chen, Chun-Wen, and Ya Hsin Chen. "Prototype Development of an Interpretative Game with Location-Based AR for Ecomuseum." 2022 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2022.
(5)
Chen, Mengyu, Andrés Monroy-Hernández, and Misha Sra. "SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality." 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2021.
(6)
Cheng, Alan Y., et al. "Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI). ACM, 2023.
(7)
Chinthammit, Winyu, and Angela Thomas. "iFiction: Mobile technology, new media, Mixed Reality and literary creativity in English teaching." 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2012.
(8)
De Luca, Valerio, et al. "A user experience analysis for a mobile Mixed Reality application for cultural heritage." Virtual Reality (Springer VR). 27 (2023): 2821-2837.
(9)
De Paolis, Lucio Tommaso, et al. "Usability, user experience and mental workload in a mobile Augmented Reality application for digital storytelling in cultural heritage." Virtual Reality (Springer VR). 27:2 (2023): 1117-1143.
(10)
Fenu, Cristina, and Fabio Pittarello. "Svevo tour: The design and the experimentation of an augmented reality application for engaging visitors of a literary museum." International Journal of Human-Computer Studies (IJHCI). 114 (2018): 20-35.
(11)
Franz, Juliano, et al. "A comparative evaluation of techniques for sharing AR experiences in museums." Proceedings of the ACM on Human-Computer Interaction (PACMHCI). 3 (2019): 1-20.
(12)
Gandy, Maribeth, et al. "AR Karaoke: Acting in your favorite scenes." Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2005.
(13)
Ghellal, Sabiha, et al. "The remediation of nosferatu: exploring transmedia experiences." Proceedings of the 2014 conference on Designing Interactive Systems (DIS). 2014.
(14)
Gil, Kyungwon, et al. "AR Petite Theater: Augmented reality storybook for supporting children’s empathy behavior." 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design (ISMAR-MASH’D). IEEE, 2014.
(15)
González, Andrés Vargas, et al. "A comparison of desktop and augmented reality scenario based training authoring tools." 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2019.
(16)
Ha, Taejin, Youngho Lee, and Woontack Woo. "Digilog book for temple bell tolling experience based on interactive augmented reality." Virtual Reality (Springer VR). 15 (2011): 295-309.
(17)
Hammady, Ramy, et al. "A framework for constructing and evaluating the role of MR as a holographic virtual guide in museums." Virtual Reality (Springer VR). 25.4 (2021): 895-918.
(18)
Harazono, Yuki, et al. "Development of an Augmented Reality Training Environment Construction System Using Puzzle-Like Programming in a Real Environment." International Journal of Human–Computer Interaction (IJHCI). 40:2 (2022): 1-14.
(19)
Hassan, Syed Ali, Tariq Rahim, and Soo Young Shin. "ChildAR: an augmented reality-based interactive game for assisting children in their education." Universal Access in the Information Society (UAIS). 21:2 (2022): 545-556.
(20)
Hsu, Hsiu-Ting, and I-Jui Lee. "Using augmented reality technology with serial learning framework to develop a serial social story situation board game system for children with autism to improve social situation understanding and social reciprocity skills." Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference (UAHCI) 2020, Held as Part of the 22nd HCI International Conference, (HCII) 2020. Springer International Publishing, 2020.
(21)
Huang, Yu-Chen, and I-Jui Lee. "Using augmented reality and concept mapping to improve ability to master social relationships and social reciprocity for children with autism Spectrum disorder." Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference (UAHCI) 2020, Held as Part of the 22nd HCI International Conference (HCII) 2020. Springer International Publishing, 2020.
(22)
Huang, Yu-Hsiung, Chih-Wei Liao, and Su-Chu Hsu. "The Elf-AR-QA System Based on IoT Cross-Media and Discussion of Emotional and Pleasure Design." 2022 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2022.
(23)
Javornik, Ana, et al. "An experimental study on the role of augmented reality content type in an outdoor site exploration." Behaviour & Information Technology (BIT). 38:1 (2019): 9-27.
(24)
Jin, Yunshui, Minhua Ma, and Yun Liu. "Interactive Narrative in Augmented Reality: An Extended Reality of the Holocaust." 2020 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2020.
(25)
Susi, Tarja, and Niklas Torstensson. "Who’s texting? Playful game experiences for learning to cope with online risks." HCI in Games: First International Conference (HCI-Games) 2019, Held as Part of the 21st HCI International Conference (HCII) 2019. Springer International Publishing, 2019.
(26)
Kari, Mohamed, et al. "TransforMR: Pose-aware object substitution for composing alternate mixed realities." 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2021.
(27)
Kim, Minji, et al. "Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based Interaction." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI). 2023.
(28)
Kim, Taegyu, et al. "The journey of a White Rhinoceros: Sculpture augmentation for gallery exhibition." 2013 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2013.
(29)
Koh, Raymond Koon Chuan, et al. "Co-creativity fusions in interdisciplinary augmented reality game developments." 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2012.
(30)
Kotranza, Aaron, and Benjamin Lok. "Virtual human+ tangible interface= mixed reality human an initial exploration with a virtual breast exam patient." 2008 IEEE Virtual Reality Conference (VR). IEEE, 2008.
(31)
Lee, Gun A., et al. "AntarcticAR: An outdoor AR experience of a virtual tour to Antarctica." 2013 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2013.
(32)
Li, Pengyuan, and Jiro Tanaka. "A Personal Health-Tracking System Focused on Social Communication for Motivation." 2020 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2020.
(33)
Li, Wanwan, et al. "Location-Aware Adaptation of Augmented Reality Narratives." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI). ACM, 2023.
(34)
Liang, Ci-Jyun, et al. "Enhancing stroke assessment simulation experience in clinical training using augmented reality." Virtual Reality (Springer VR). 25 (2021): 575-584.
(35)
Logothetis, Ilias, et al. "Transforming classic learning games with the use of AR: the case of the word hangman game." 2021 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2021.
(36)
López-Faican, Lissette, and Javier Jaen. "Design and evaluation of an augmented reality cyberphysical game for the development of empathic abilities." International Journal of Human-Computer Studies (IJHCI). 176 (2023): 103041.
(37)
Lyu, Yue, et al. "Eggly: Designing Mobile Augmented Reality Neurofeedback Training Games for Children with Autism Spectrum Disorder." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). 7:2 (2023): 1-29.
(38)
MacIntyre, Blair, and Jay David Bolter. "Single-narrative, multiple point-of-view dramatic experiences in augmented reality." Virtual reality (Springer VR). 7 (2003): 10-16.
(39)
Mao, Chia-Chi, and Chien-Hsu Chen. "Augmented reality of 3D content application in common operational picture training system for army." International Journal of Human–Computer Interaction (IJHCI). 37:20 (2021): 1899-1915.
(40)
Mittmann, Gloria, et al. "LINA-A Social Augmented Reality Game around Mental Health, Supporting Real-world Connection and Sense of Belonging for Early Adolescents." Proceedings of the ACM on Human-Computer Interaction (PACMHCI). 6 (2022): 1-21.
(41)
Montero, Alvaro, et al. "Designing and implementing interactive and realistic augmented reality experiences." Universal Access in the Information Society (UAIS). 18 (2019): 49-61.
(42)
Morrison, Ann, et al. "Like bees around the hive: a comparative study of a mobile augmented reality map." Proceedings of the SIGCHI conference on human factors in computing systems (CHI). 2009.
(43)
Nguyen, Ta Huynh Duy, et al. "Real-time 3D human capture system for mixed-reality art and entertainment." IEEE Transactions on Visualization and Computer Graphics (TVCG). 11:6 (2005): 706-721.
(44)
Nilsson, Susanna, Bjorn Johansson, and Arne Jonsson. "Using AR to support cross-organisational collaboration in dynamic tasks." 2009 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2009.
(45)
Numan, Nels, et al. "Star Tag: a superhuman sport to promote physical activity." 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 2019.
(46)
Oriti, Damiano, et al. "Harmonize: A shared environment for extended immersive entertainment." Virtual Reality (Springer VR). 27 (2021): 1-14.
(47)
Park, Hyerim, et al. "Spatial transition management for improving outdoor cinematic augmented reality experience of the TV show." Virtual Reality (Springer VR). 26:3 (2022): 1059-1077.
(48)
Plecher, David A., et al. "Legionarius-beyond limes." 2022 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2022.
(49)
Sabie, Dina, et al. "Be Our Guest: Intercultural Heritage Exchange through Augmented Reality (AR)." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI). 2023.
(50)
Schiavi, Barbara, et al. "Teach me a story: an augmented reality application for teaching history in middle school." 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 2018.
(51)
Schmalstieg, Dieter, and Daniel Wagner. "Experiences with handheld augmented reality." 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2007.
(52)
Sekhavat, Yoones A., and Hossein Zarei. "Sense of immersion in computer games using single and stereoscopic augmented reality." International Journal of Human–Computer Interaction (IJHCI). 34:2 (2018): 187-194.
(53)
Singh, Abbey, et al. "Story creatar: a toolkit for spatially-adaptive augmented reality storytelling." 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 2021.
(54)
Strada, Francesco, et al. "Leveraging a collaborative augmented reality serious game to promote sustainability awareness, commitment and adaptive problem-management." International Journal of Human-Computer Studies (IJHCI). 172 (2023).
(55)
Wang, Wenjia, et al. "Intertwining History and Places: The Design of TongSEE Location-Based Augmented Reality Application for History Learning." 2022 International Conference on Human-Computer Interaction (HCII). Cham: Springer International Publishing, 2022.
(56)
Weerasinghe, Maheshya, et al. "Arigatō: Effects of Adaptive Guidance on Engagement and Performance in Augmented Reality Learning Environments." IEEE Transactions on Visualization and Computer Graphics (TVCG). 28:11 (2022): 3737-3747.
(57)
Whitlock, Matt, Danielle Albers Szafir, and Kenny Gruchalla. "HydrogenAR: Interactive Data-Driven Presentation of Dispenser Reliability." 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2020.
(58)
Wither, Jason, et al. "The westwood experience: connecting story to locations via mixed reality." 2010 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2010.
(59)
Woll, Robert, et al. "Augmented reality in a serious game for manual assembly processes." 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, 2011.
(60)
Yilmaz, Rabia Meryem, and Yuksel Goktas. "Using augmented reality technology in storytelling activities: examining elementary students’ narrative skill and creativity." Virtual Reality (Springer VR). 21 (2017): 75-89.
(61)
Zambetta, Fabio, et al. "Reducing perceived waiting time in theme park queues via an augmented reality game." ACM Transactions on Computer-Human Interaction (TOCHI). 27:1 (2020): 1-30.
(62)
Zeng, Hong, Xingxi He, and Honghu Pan. "Implementation of escape room system based on augmented reality involving deep convolutional neural network." Virtual Reality (Springer VR). 25 (2021): 585-596.
(63)
Zhou, Siyuan, et al. "The use of augmented reality for solving arithmetic problems for preschool children." Learning and Collaboration Technologies. Human and Technology Ecosystems: 7th International Conference (LCT) 2020, Held as Part of the 22nd HCI International Conference (HCII) 2020. Springer International Publishing, 2020.
(64)
Zuckerman, Oren, Guy Hoffman, and Ayelet Gal-Oz. "In-car game design for children: Promoting interactions inside and outside the car." International Journal of Child-Computer Interaction (IJCCI) 2.4 (2014): 109-119.

Footnote

1
While the distinction Azuma originally made between Reinforcing and Remembering depended chiefly on whether the narrative experience is of a shared or personal nature, we consider the temporal congruence of the narrative experience to the target real space as the decisive factor for this study.

Supplemental Material

MP4 File - Video Preview
Video Preview
MP4 File - Video Presentation
Video Presentation
Transcript for: Video Presentation

References

[1]
Ruth Aylett and Sandy Louchart. 2003. Towards a narrative theory of virtual reality. Virtual Reality 7 (2003), 2–9.
[2]
Ronald Azuma. 2015. Location-based mixed and augmented reality storytelling. Fundamentals of Wearable Computers and Augmented Reality 11, 2 (2015), 259–276.
[3]
Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre. 2001. Recent advances in augmented reality. IEEE computer graphics and applications 21, 6 (2001), 34–47.
[4]
Ronald Azuma, Jong Weon Lee, Bolan Jiang, Jun Park, Suya You, and Ulrich Neumann. 1999. Tracking in unprepared environments for augmented reality systems. Computers & Graphics 23, 6 (1999), 787–793.
[5]
Ronald T Azuma. 1997. A survey of augmented reality. Presence: teleoperators & virtual environments 6, 4 (1997), 355–385.
[6]
Ronald T Azuma. 2016. The most important challenge facing augmented reality. Presence 25, 3 (2016), 234–238.
[7]
Evan Barba, Blair MacIntyre, Rebecca Rouse, and Jay Bolter. 2010. Thinking inside the box: making meaning in a handheld AR experience. In 2010 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities. IEEE, Seoul, Korea, 19–26.
[8]
Steve Benford, Gabriella Giannachi, Boriana Koleva, and Tom Rodden. 2009. From interaction to trajectories: designing coherent journeys through user experiences. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, Boston, USA, 709–718.
[9]
Mark Billinghurst, Adrian Clark, Gun Lee, 2015. A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction 8, 2-3 (2015), 73–272.
[10]
Oliver Bimber and Ramesh Raskar. 2005. Spatial augmented reality: merging real and virtual worlds. CRC press, New York, USA.
[11]
Jay David Bolter, Maria Engberg, and Blair MacIntyre. 2021. Reality media: Augmented and virtual reality. MIT Press, Massachusetts, USA.
[12]
Jay David Bolter and Richard Grusin. 2000. Remediation: Understanding new media. mit Press, Massachusetts, USA.
[13]
Erkan Bostanci, Nadia Kanwal, Shoaib Ehsan, and Adrian F Clark. 2010. Tracking methods for augmented reality. In The 3rd international conference on machine vision. SPIE, Hong Kong, China, 425–429.
[14]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[15]
Jason Cory Brunson. 2020. Ggalluvial: layered grammar for alluvial plots. Journal of Open Source Software 5, 49 (2020), 1–4.
[16]
Sabine Buchholz and Manfred Jahn. 2005. Space in narrative. Routledge encyclopedia of narrative theory. London: Routledge 1, 1 (2005), 551–555.
[17]
Julie Carmigniani, Borko Furht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani, and Misa Ivkovic. 2011. Augmented reality technologies, systems and applications. Multimedia tools and applications 51 (2011), 341–377.
[18]
Cagatay Catal, Akhan Akbulut, Berkay Tunali, Erol Ulug, and Eren Ozturk. 2020. Evaluation of augmented reality technology for the design of an evacuation training game. Virtual Reality 24 (2020), 359–368.
[19]
Marc Cavazza, Olivier Martin, Fred Charles, Xavier Marichal, and Steven J Mead. 2003. User interaction in mixed reality interactive storytelling. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings.IEEE, Tokyo, Japan, 304–305.
[20]
William Chang and Qing Tan. 2010. Augmented reality system design and scenario study for location-based adaptive mobile learning. In 2010 13th IEEE International Conference on Computational Science and Engineering. IEEE, Hong Kong, China, 20–27.
[21]
Denis Chekhlov, Andrew P Gee, Andrew Calway, and Walterio Mayol-Cuevas. 2007. Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual slam. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, Nara, Japan, 153–156.
[22]
Chun-Wen Chen and Ya Hsin Chen. 2022. Prototype Development of an Interpretative Game with Location-Based AR for Ecomuseum. In International Conference on Human-Computer Interaction. Springer, Gothenburg, Sweden, 360–370.
[23]
Mengyu Chen, Andrés Monroy-Hernández, and Misha Sra. 2021. SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Online, 294–303.
[24]
Alan Y Cheng, Jacob Ritchie, Niki Agrawal, Elizabeth Childs, Cyan DeVeaux, Yubin Jee, Trevor Leon, Bethanie Maples, Andrea Cuadra, and James A Landay. 2023. Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg, Germany, 1–22.
[25]
Shira Chess. 2014. Augmented regionalism: Ingress as geomediated gaming narrative. Information, Communication & Society 17, 9 (2014), 1105–1117.
[26]
Winyu Chinthammit and Angela Thomas. 2012. iFiction: Mobile technology, new media, Mixed Reality and literary creativity in English teaching. In 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, Atlanta, USA, 39–46.
[27]
Alan B Craig. 2013. Understanding augmented reality: Concepts and applications. Understanding Augmented Reality 1, 2 (2013), 39–67.
[28]
Jennifer E Cross. 2001. What is sense of place?. In 12th Headwaters Conference. Colorado State University. Libraries, Colorado, USA, 1–14.
[29]
Valerio De Luca, Maria Cristina Barba, Giovanni D’Errico, Benito Luigi Nuzzo, and Lucio Tommaso De Paolis. 2023. A user experience analysis for a mobile Mixed Reality application for cultural heritage. Virtual Reality 27, 1 (2023), 2821–2837.
[30]
Lucio Tommaso De Paolis, Carola Gatto, Laura Corchia, and Valerio De Luca. 2023. Usability, user experience and mental workload in a mobile Augmented Reality application for digital storytelling in cultural heritage. Virtual Reality 27, 2 (2023), 1117–1143.
[31]
Marco De Sá and Elizabeth Churchill. 2012. Mobile augmented reality: exploring design and prototyping techniques. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services. ACM, New York, USA, 221–230.
[32]
GJ Elliott, E Jones, and P Barker. 2002. A grounded theory approach to modelling learnability of hypermedia authoring tools. Interacting with Computers 14, 5 (2002), 547–574.
[33]
Jason Farman. 2015. Stories, spaces, and bodies: The production of embodied space through mobile media storytelling. Communication Research and Practice 1, 2 (2015), 101–116.
[34]
Cristina Fenu and Fabio Pittarello. 2018. Svevo tour: The design and the experimentation of an augmented reality application for engaging visitors of a literary museum. International Journal of Human-Computer Studies 114 (2018), 20–35.
[35]
Juliano Franz, Mohammed Alnusayri, Joseph Malloch, and Derek Reilly. 2019. A comparative evaluation of techniques for sharing AR experiences in museums. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–20.
[36]
Maribeth Gandy, Blair MacIntyre, Peter Presti, Steven Dow, Jay Bolter, Brandon Yarbrough, and Nigel O’Rear. 2005. AR Karaoke: Acting in your favorite scenes. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05). IEEE, Vienna, Austria, 114–117.
[37]
Sabiha Ghellal, Ann Morrison, Marc Hassenzahl, and Benjamin Schaufler. 2014. The remediation of nosferatu: exploring transmedia experiences. In Proceedings of the 2014 conference on Designing interactive systems. ACM, Vancouver, Canada, 617–626.
[38]
Kyungwon Gil, Jimin Rhim, Taejin Ha, Young Yim Doh, and Woontack Woo. 2014. AR Petite Theater: Augmented reality storybook for supporting children’s empathy behavior. In 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design (ISMAR-MASH’D). IEEE, Munich, Germany, 13–20.
[39]
Andrés Vargas González, Senglee Koh, Katelynn Kapalo, Robert Sottilare, Patrick Garrity, Mark Billinghurst, and Joseph LaViola. 2019. A comparison of desktop and augmented reality scenario based training authoring tools. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Beijing, China, 339–350.
[40]
Melanie C Green. 2021. Transportation into narrative worlds. Entertainment-education behind the scenes: Case studies for theory and practice 1, 1 (2021), 87–101.
[41]
Anhong Guo, Ilter Canberk, Hannah Murphy, Andrés Monroy-Hernández, and Rajan Vaish. 2019. Blocks: Collaborative and persistent augmented reality experiences. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–24.
[42]
Taejin Ha, Youngho Lee, and Woontack Woo. 2011. Digilog book for temple bell tolling experience based on interactive augmented reality. Virtual Reality 15 (2011), 295–309.
[43]
Drexel Hallaway, Steven Feiner, and Tobias Höllerer. 2004. Bridging the gaps: Hybrid tracking for adaptive mobile augmented reality. Applied Artificial Intelligence 18, 6 (2004), 477–500.
[44]
Asim Hameed and Andrew Perkis. 2018. Spatial storytelling: Finding interdisciplinary immersion. In Interactive Storytelling: 11th International Conference on Interactive Digital Storytelling, ICIDS 2018. Springer, Dublin, Ireland, 323–332.
[45]
Ramy Hammady, Minhua Ma, Ziad Al-Kalha, and Carl Strathearn. 2021. A framework for constructing and evaluating the role of MR as a holographic virtual guide in museums. Virtual Reality 25, 4 (2021), 895–918.
[46]
Yuki Harazono, Taichi Tamura, Yusuke Omoto, Hirotake Ishii, Hiroshi Shimoda, Yoshiaki Tanaka, and Yoshiyuki Takahashi. 2022. Development of an Augmented Reality Training Environment Construction System Using Puzzle-Like Programming in a Real Environment. International Journal of Human–Computer Interaction 40, 2 (2022), 1–14.
[47]
Syed Ali Hassan, Tariq Rahim, and Soo Young Shin. 2022. ChildAR: an augmented reality-based interactive game for assisting children in their education. Universal Access in the Information Society 21, 2 (2022), 545–556.
[48]
Nicolas Henchoz, Vincent Lepetit, Pascal Fua, and John Miles. 2011. Turning Augmented Reality into a media: Design exploration to build a dedicated visual language. In 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities. IEEE, Basel, Switzerland, 83–89.
[49]
Tobias Höllerer, Jason Wither, and Stephen DiVerdi. 2007. “Anywhere augmentation”: Towards mobile augmented reality in unprepared environments. Location Based Services and TeleCartography 1, 1 (2007), 393–416.
[50]
Sheila Hones. 2011. Literary geography: setting and narrative space. Social & Cultural Geography 12, 7 (2011), 685–699.
[51]
Hsiu-Ting Hsu and I-Jui Lee. 2020. Using augmented reality technology with serial learning framework to develop a serial social story situation board game system for children with autism to improve social situation understanding and social reciprocity skills. In Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference, UAHCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020. Springer, Copenhagen, Denmark, 3–18.
[52]
Yu-Chen Huang and I-Jui Lee. 2020. Using augmented reality and concept mapping to improve ability to master social relationships and social reciprocity for children with autism Spectrum disorder. In Universal Access in Human-Computer Interaction. Applications and Practice: 14th International Conference, UAHCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020. Springer, Copenhagen, Denmark, 19–37.
[53]
Yu-Hsiung Huang, Chih-Wei Liao, and Su-Chu Hsu. 2022. The Elf-AR-QA System Based on IoT Cross-Media and Discussion of Emotional and Pleasure Design. In International Conference on Human-Computer Interaction. Springer, Gothenburg, Sweden, 222–237.
[54]
Charles E Hughes, Christopher B Stapleton, Darin E Hughes, and Eileen M Smith. 2005. Mixed reality in education, entertainment, and training. IEEE computer graphics and applications 25, 6 (2005), 24–30.
[55]
Olivier Hugues, Philippe Fuchs, and Olivier Nannipieri. 2011. New augmented reality taxonomy: Technologies and features of augmented environment. Handbook of augmented reality 1, 1 (2011), 47–63.
[56]
Manfred Jahn. 2005. Narratology: A guide to the theory of narrative. English Department, University of Cologne 28 (2005), 1–93.
[57]
So-Youn Jang, Jisu Park, Maria Engberg, Blair MacIntyre, and Jay D Bolter. 2023. RealityMedia: immersive technology and narrative space. Frontiers in Virtual Reality 4 (2023), 1155700.
[58]
Ana Javornik, Efstathia Kostopoulou, Yvonne Rogers, Ava Fatah gen Schieck, Petros Koutsolampros, Ana Maria Moutinho, and Simon Julier. 2019. An experimental study on the role of augmented reality content type in an outdoor site exploration. Behaviour & Information Technology 38, 1 (2019), 9–27.
[59]
Yunshui Jin, Minhua Ma, and Jiachen Li. 2020. Immersive Storytelling in Augmented Reality: Witnessing the Kindertransport. In Joint International Conference on Serious Games. Springer, Stoke-on-Trent, United Kingdom, 17–33.
[60]
Yunshui Jin, Minhua Ma, and Yun Liu. 2020. Interactive Narrative in Augmented Reality: An Extended Reality of the Holocaust. In International Conference on Human-Computer Interaction. Springer, Copenhagen, Denmark, 249–269.
[61]
RS Kalawsky, AW Stedmon, K Hill, and CA Cook. 2000. A taxonomy of technology: Defining augmented reality. In Proceedings of the human factors and ergonomics society annual meeting. SAGE Publications, Los Angeles, USA, 507–510.
[62]
Mohamed Kari, Tobias Grosse-Puppendahl, Luis Falconeri Coelho, Andreas Rene Fender, David Bethge, Reinhard Schütte, and Christian Holz. 2021. Transformr: Pose-aware object substitution for composing alternate mixed realities. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Online, 69–79.
[63]
Minji Kim, Kyungjin Lee, Rajesh Balan, and Youngki Lee. 2023. Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based Interaction. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg, Germany, 1–18.
[64]
Taegyu Kim, Jay Jang, Joonsuk Park, and Jun Park. 2013. The journey of a White Rhinoceros: Sculpture augmentation for gallery exhibition. In 2013 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, Adelaide, Austria, 39–43.
[65]
Raymond Koon Chuan Koh, Henry Been-Lirn Duh, Cheng-Ho Chen, and Yun-Ting Wong. 2012. Co-creativity fusions in interdisciplinary augmented reality game developments. In 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH). IEEE, Atlanta, USA, 47–56.
[66]
Aaron Kotranza and Benjamin Lok. 2008. Virtual human+ tangible interface= mixed reality human an initial exploration with a virtual breast exam patient. In 2008 IEEE Virtual Reality Conference. IEEE, Reno, USA, 99–106.
[67]
Wallace S Lages and Doug A Bowman. 2019. Walking with adaptive augmented reality workspaces: design and usage patterns. In Proceedings of the 24th International Conference on Intelligent User Interfaces. ACM, Marina Del Ray, USA, 356–366.
[68]
Gun A Lee, Andreas Dünser, Alaeddin Nassani, and Mark Billinghurst. 2013. AntarcticAR: An outdoor AR experience of a virtual tour to Antarctica. In 2013 IEEE international symposium on mixed and augmented reality-arts, media, and humanities (ISMAR-AMH). IEEE, Adelaide, Austria, 29–38.
[69]
Pengyuan Li and Jiro Tanaka. 2020. A Personal Health-Tracking System Focused on Social Communication for Motivation. In International Conference on Human-Computer Interaction. Springer, Copenhagen, Denmark, 415–427.
[70]
Wanwan Li, Changyang Li, Minyoung Kim, Haikun Huang, and Lap-Fai Yu. 2023. Location-Aware Adaptation of Augmented Reality Narratives. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg, Germany, 1–15.
[71]
Ci-Jyun Liang, Charles Start, Hanna Boley, Vineet R Kamat, Carol C Menassa, and Michelle Aebersold. 2021. Enhancing stroke assessment simulation experience in clinical training using augmented reality. Virtual Reality 25 (2021), 575–584.
[72]
Gunnar Liestøl. 2011. Situated simulations between virtual reality and mobile augmented reality: Designing a narrative space. In Handbook of augmented reality. Springer, Pennsylvania, USA, 309–319.
[73]
Gunnar Liestøl. 2018. Story and storage–narrative theory as a tool for creativity in augmented reality storytelling. Virtual Creativity 8, 1 (2018), 75–89.
[74]
Ilias Logothetis, George Papadourakis, Iraklis Katsaris, Konstantinos Katsios, and Nikolas Vidakis. 2021. Transforming classic learning games with the use of AR: the case of the word hangman game. In International Conference on Human-Computer Interaction. Springer, Online, 47–64.
[75]
Lissette López-Faican and Javier Jaen. 2023. Design and evaluation of an augmented reality cyberphysical game for the development of empathic abilities. International Journal of Human-Computer Studies 176 (2023), 103041.
[76]
Weizhou Luo, Anke Lehmann, Hjalmar Widengren, and Raimund Dachselt. 2022. Where should we put it? layout and placement strategies of documents in augmented reality for collaborative sensemaking. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, New Orleans, USA, 1–16.
[77]
Yue Lyu, Pengcheng An, Yage Xiao, Zibo Zhang, Huan Zhang, Keiko Katsuragawa, and Jian Zhao. 2023. Eggly: Designing Mobile Augmented Reality Neurofeedback Training Games for Children with Autism Spectrum Disorder. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1–29.
[78]
Jung Yeon Ma and Jong Soo Choi. 2007. The Virtuality and Reality of Augmented Reality.J. Multim. 2, 1 (2007), 32–37.
[79]
Blair MacIntyre and Jay David Bolter. 2003. Single-narrative, multiple point-of-view dramatic experiences in augmented reality. Virtual reality 7 (2003), 10–16.
[80]
Blair MacIntyre, Jay David Bolter, Emmanuel Moreno, and Brendan Hannigan. 2001. Augmented reality as a new media experience. In Proceedings IEEE and ACM International Symposium on Augmented Reality. IEEE, New York, USA, 197–206.
[81]
Wendy E Mackay. 1998. Augmented reality: linking real and virtual worlds: a new paradigm for interacting with computers. In Proceedings of the working conference on Advanced visual interfaces. ACM, Naples, Italy, 13–21.
[82]
Marta Majorek and Marta Du Vall. 2016. Ingress: an example of a new dimension in entertainment. Games and Culture 11, 7-8 (2016), 667–689.
[83]
Joseph Malloch and Derek Reilly. 2022. Supporting Spatial Thinking in Augmented Reality Narrative: A Field Study. In Interactive Storytelling: 15th International Conference on Interactive Digital Storytelling, ICIDS 2022, Vol. 13762. Springer, Santa Cruz, USA, 270–291.
[84]
Lev Manovich. 2010. The poetics of augmented space. In Mediatecture. Springer, Pennsylvania, USA, 304–318.
[85]
Chia-Chi Mao and Chien-Hsu Chen. 2021. Augmented reality of 3D content application in common operational picture training system for army. International Journal of Human–Computer Interaction 37, 20 (2021), 1899–1915.
[86]
Todd Margolis. 2014. Immersive art in augmented reality. In Augmented reality art: From an emerging technology to a novel creative medium. Springer, Pennsylvania, USA, 149–159.
[87]
Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321–1329.
[88]
Gloria Mittmann, Adam Barnard, Ina Krammer, Diogo Martins, and João Dias. 2022. LINA-A Social Augmented Reality Game around Mental Health, Supporting Real-world Connection and Sense of Belonging for Early Adolescents. Proceedings of the ACM on Human-Computer Interaction 6, CHI PLAY (2022), 1–21.
[89]
Alvaro Montero, Telmo Zarraonandia, Paloma Diaz, and Ignacio Aedo. 2019. Designing and implementing interactive and realistic augmented reality experiences. Universal Access in the Information Society 18 (2019), 49–61.
[90]
Ann Morrison, Antti Oulasvirta, Peter Peltonen, Saija Lemmela, Giulio Jacucci, Gerhard Reitmayr, Jaana Näsänen, and Antti Juustila. 2009. Like bees around the hive: a comparative study of a mobile augmented reality map. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, Boston, USA, 1889–1898.
[91]
Michael J Muller and Sandra Kogan. 2010. Grounded theory method in HCI and CSCW. Cambridge: IBM Center for Social Software 28, 2 (2010), 1–46.
[92]
Janet Murray. 2005. Did it make you cry? Creating dramatic agency in immersive environments. In International Conference on Virtual Storytelling. Springer, Strasbourg, France, 83–94.
[93]
Janet H Murray. 2017. Hamlet on the Holodeck, updated edition: The Future of Narrative in Cyberspace. MIT press, Massachusetts, USA.
[94]
Ta Huynh Duy Nguyen, Tran Cong Thien Qui, Ke Xu, Adrian David Cheok, Sze Lee Teo, ZhiYing Zhou, Asitha Mallawaarachchi, Shang Ping Lee, Wei Liu, Hui Siang Teo, 2005. Real-time 3D human capture system for mixed-reality art and entertainment. IEEE Transactions on Visualization and Computer Graphics 11, 6 (2005), 706–721.
[95]
Susanna Nilsson, Bjorn Johansson, and Arne Jonsson. 2009. Using AR to support cross-organisational collaboration in dynamic tasks. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality. IEEE, Orlando, USA, 3–12.
[96]
Jean-Marie Normand, Myriam Servières, and Guillaume Moreau. 2012. A new typology of augmented reality applications. In Proceedings of the 3rd Augmented Human international conference. ACM, Megeve, France, 1–8.
[97]
Nels Numan, Ayla Kolster, Niels Hoogerwerf, Bernd Kreynen, Jeanique Romeijnders, Tomas Heinsohn Huala, Nestor Z Salamon, J Timothy Balint, Stephan Lukosch, and Rafael Bidarra. 2019. Star Tag: a superhuman sport to promote physical activity. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Osaka, Japan, 1826–1830.
[98]
Jennifer O’Meara. 2023. AR cinema: Visual storytelling and embodied experiences with augmented reality filters and backgrounds. PRESENCE: Virtual and Augmented Reality 30, 1 (2023), 99–123.
[99]
Damiano Oriti, Federico Manuri, Francesco De Pace, and Andrea Sanna. 2021. Harmonize: A shared environment for extended immersive entertainment. Virtual Reality 25, 7 (2021), 3259–3272.
[100]
Janne Paavilainen, Hannu Korhonen, Kati Alha, Jaakko Stenros, Elina Koskinen, and Frans Mayra. 2017. The Pokémon GO experience: A location-based augmented reality mobile game goes mainstream. In Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, Denver, USA, 2493–2498.
[101]
Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews 10, 1 (2021), 89.
[102]
Hyerim Park, Maryam Shakeri, Ikbeom Jeon, Jangyoon Kim, Abolghasem Sadeghi-Niaraki, and Woontack Woo. 2022. Spatial transition management for improving outdoor cinematic augmented reality experience of the TV show. Virtual Reality 26, 3 (2022), 1059–1077.
[103]
John V Pavlik and Frank Bridges. 2013. The emergence of augmented reality (AR) as a storytelling medium in journalism. Journalism & Communication Monographs 15, 1 (2013), 4–59.
[104]
Celia Pearce. 2007. Narrative environments. Birkhäuser Basel, Basel, Switzerland, 200–205.
[105]
David A Plecher, Andreas Wohlschlager, Christian Eichhorn, and Gudrun Klinker. 2022. Legionarius-beyond limes. In International Conference on Human-Computer Interaction. Springer, Gothenburg, Sweden, 618–636.
[106]
Gerald Prince. 2003. A dictionary of narratology. University of Nebraska Press, Lincoln, USA.
[107]
Daniel Punday. 2017. Space across Narrative Media: Towards an Archaeology of Narratology. Narrative 25, 1 (2017), 92–112.
[108]
Gideon Raeburn, Laurissa Tokarchuk, and Martin Welton. 2021. Creating Immersive Play Anywhere Location-Based Storytelling Using Mobile AR. In Augmented Reality, Virtual Reality, and Computer Graphics: 8th International Conference, AVR 2021. Springer, Online, 209–226.
[109]
Philipp A Rauschnabel, Alexander Rossmann, and M Claudia tom Dieck. 2017. An adoption framework for mobile augmented reality games: The case of Pokémon Go. Computers in human behavior 76 (2017), 276–286.
[110]
Edward Relph. 1976. Place and placelessness. Vol. 1. Pion, London, United Kingdom.
[111]
Marie-Laure Ryan. 2001. Narrative as virtual reality. Johns Hopkins University Press, Baltimore, USA.
[112]
Marie-Laure Ryan. 2018. Narrative mapping as cognitive activity and as active participation in storyworlds. Frontiers of Narrative Studies 4, 2 (2018), 232–247.
[113]
Dina Sabie, Hala Sheta, Hasan Shahid Ferdous, Vannie Kopalakrishnan, and Syed Ishtiaque Ahmed. 2023. Be Our Guest: Intercultural Heritage Exchange through Augmented Reality (AR). In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg, Germany, 1–15.
[114]
Barbara Schiavi, Franck Gechter, Céline Gechter, and Albert Rizzo. 2018. Teach me a story: an augmented reality application for teaching history in middle school. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Reutlingen, Germany, 679–680.
[115]
Dieter Schmalstieg and Daniel Wagner. 2007. Experiences with handheld augmented reality. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, Nara, Japan, 3–18.
[116]
Yoones A Sekhavat and Hossein Zarei. 2018. Sense of immersion in computer games using single and stereoscopic augmented reality. International Journal of Human–Computer Interaction 34, 2 (2018), 187–194.
[117]
Roy Shilkrot, Nick Montfort, and Pattie Maes. 2014. nARratives of augmented worlds. In 2014 IEEE International Symposium on Mixed and Augmented Reality-Media, Art, Social Science, Humanities and Design (ISMAR-MASH’D). IEEE, Munich, Germany, 35–42.
[118]
Jae-eun Shin, Hayun Kim, Callum Parker, Hyung-il Kim, Seoyoung Oh, and Woontack Woo. 2019. Is any room really ok? the effect of room size and furniture on presence, narrative engagement, and usability during a space-adaptive augmented reality game. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Beijing, China, 135–144.
[119]
Jae-eun Shin and Woontack Woo. 2018. Design guidelines for a location-based digital heritage storytelling tool to support author intent. In 2018 3rd Digital Heritage International Congress (DigitalHERITAGE) held jointly with 2018 24th International Conference on Virtual Systems & Multimedia (VSMM 2018). IEEE, San Francisco, USA, 1–8.
[120]
Jae-Eun Shin and Woontack Woo. 2023. How Space is Told: Linking Trajectory, Narrative, and Intent in Augmented Reality Storytelling for Cultural Heritage Sites. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg, Germany, 1–14.
[121]
Jae-eun Shin, Boram Yoon, Dooyoung Kim, and Woontack Woo. 2021. A user-oriented approach to space-adaptive augmentation: The effects of spatial affordance on narrative experience in an augmented reality detective game. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, 1–13.
[122]
Jae-eun Shin, Boram Yoon, Dooyoung Kim, and Woontack Woo. 2022. The Effects of Spatial Complexity on Narrative Experience in Space-Adaptive AR Storytelling. IEEE Transactions on Visualization and Computer Graphics 29, 12 (2022), 5137–5148.
[123]
Abbey Singh, Ramanpreet Kaur, Peter Haltner, Matthew Peachey, Mar Gonzalez-Franco, Joseph Malloch, and Derek Reilly. 2021. Story creatar: a toolkit for spatially-adaptive augmented reality storytelling. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, Atlanta, USA, 713–722.
[124]
Christopher Stapleton, Charles Hughes, Michael Moshell, Paulius Micikevicius, and Marty Altman. 2002. Applying mixed reality to entertainment. Computer 35, 12 (2002), 122–124.
[125]
Francesco Strada, Maria Ximena Lopez, Carlo Fabricatore, Alysson Diniz dos Santos, Dimitar Gyaurov, Edoardo Battegazzorre, and Andrea Bottino. 2023. Leveraging a collaborative augmented reality serious game to promote sustainability awareness, commitment and adaptive problem-management. International Journal of Human-Computer Studies 172 (2023), 102984.
[126]
Tarja Susi and Niklas Torstensson. 2019. “Who’s texting?”–playful game experiences for learning to cope with online risks. In HCI in Games: First International Conference, HCI-Games 2019, Held as Part of the 21st HCI International Conference, HCII 2019. Springer, Orlando, USA, 427–441.
[127]
Takafumi Taketomi, Hideaki Uchiyama, and Sei Ikeda. 2017. Visual SLAM algorithms: A survey from 2010 to 2016. IPSJ Transactions on Computer Vision and Applications 9, 1 (2017), 1–11.
[128]
Markus Tatzgern, Raphael Grasset, Eduardo Veas, Denis Kalkofen, Hartmut Seichter, and Dieter Schmalstieg. 2015. Exploring real world points of interest: Design and evaluation of object-centric exploration techniques for augmented reality. Pervasive and mobile computing 18 (2015), 55–70.
[129]
Marcus Tönnis, David A Plecher, and Gudrun Klinker. 2013. Representing information–Classifying the Augmented Reality presentation space. Computers & Graphics 37, 8 (2013), 997–1011.
[130]
Shu-pei Tsai. 2020. Augmented reality enhancing place satisfaction for heritage tourism marketing. Current Issues in Tourism 23, 9 (2020), 1078–1083.
[131]
DWF Van Krevelen and Ronald Poelman. 2010. A survey of augmented reality technologies, applications and limitations. International journal of virtual reality 9, 2 (2010), 1–20.
[132]
Wenjia Wang, Yate Ge, Hang Yu, Xu Lu, XueChen Li, Yao Cheng, and Xiaohua Sun. 2022. Intertwining History and Places: The Design of TongSEE Location-Based Augmented Reality Application for History Learning. In International Conference on Human-Computer Interaction. Springer, Online, 153–169.
[133]
Maheshya Weerasinghe, Aaron Quigley, Klen Čopič Pucihar, Alice Toniolo, Angela Miguel, and Matjaž Kljun. 2022. Arigatō: Effects of Adaptive Guidance on Engagement and Performance in Augmented Reality Learning Environments. IEEE Transactions on Visualization and Computer Graphics 28, 11 (2022), 3737–3747.
[134]
Matt Whitlock, Danielle Albers Szafir, and Kenny Gruchalla. 2020. HydrogenAR: Interactive Data-Driven Presentation of Dispenser Reliability. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Online, 704–712.
[135]
Andrew D Wilson. 2005. PlayAnywhere: a compact interactive tabletop projection-vision system. In Proceedings of the 18th annual ACM symposium on User interface software and technology. ACM, Seattle, USA, 83–92.
[136]
Jason Wither, Rebecca Allen, Vids Samanta, Juha Hemanus, Yun-Ta Tsai, Ronald Azuma, Will Carter, Rachel Hinman, and Thommen Korah. 2010. The westwood experience: connecting story to locations via mixed reality. In 2010 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities. IEEE, Seoul, Korea, 39–46.
[137]
Jason Wither, Stephen DiVerdi, and Tobias Höllerer. 2009. Annotation in outdoor augmented reality. Computers & Graphics 33, 6 (2009), 679–689.
[138]
Robert Woll, Thomas Damerau, Kevin Wrasse, and Rainer Stark. 2011. Augmented reality in a serious game for manual assembly processes. In 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities. IEEE, Basel, Switzerland, 37–39.
[139]
Andy Wai Kan Yeung. 2018. Data visualization by alluvial diagrams for bibliometric reports, systematic reviews and meta-analyses. Current Science 115, 10 (2018), 1942–1947.
[140]
Rabia Meryem Yilmaz and Yuksel Goktas. 2017. Using augmented reality technology in storytelling activities: examining elementary students’ narrative skill and creativity. Virtual Reality 21 (2017), 75–89.
[141]
Suya You and Ulrich Neumann. 2001. Fusion of vision and gyro tracking for robust augmented reality registration. In Proceedings of IEEE Virtual Reality 2001. IEEE, Yokohama, Japan, 71–78.
[142]
Fabio Zambetta, William Raffe, Marco Tamassia, Florian’Floyd ‚ Mueller, Xiaodong Li, Niels Quinten, Rakesh Patibanda, Daniel Dang, and Jon Satterley. 2020. Reducing perceived waiting time in theme park queues via an augmented reality game. ACM Transactions on Computer-Human Interaction (TOCHI) 27, 1 (2020), 1–30.
[143]
Hong Zeng, Xingxi He, and Honghu Pan. 2021. Implementation of escape room system based on augmented reality involving deep convolutional neural network. Virtual Reality 25 (2021), 585–596.
[144]
Feng Zhou, Henry Been-Lirn Duh, and Mark Billinghurst. 2008. Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, Cambridge, United Kingdom, 193–202.
[145]
Siyuan Zhou, Xu Sun, Zhiyu Shi, and Yanyi Lu. 2020. The use of augmented reality for solving arithmetic problems for preschool children. In Learning and Collaboration Technologies. Human and Technology Ecosystems: 7th International Conference, LCT 2020, Held as Part of the 22nd HCI International Conference, HCII 2020. Springer, Copenhagen, Denmark, 574–584.
[146]
Oren Zuckerman, Guy Hoffman, and Ayelet Gal-Oz. 2014. In-car game design for children: Promoting interactions inside and outside the car. International Journal of Child-Computer Interaction 2, 4 (2014), 109–119.

Cited By

View all
  • (2024)Medical Education Escape Room Aligned with Flipped Classroom and Powered by Mobile Augmented RealityElectronics10.3390/electronics1312236713:12(2367)Online publication date: 17-Jun-2024

Index Terms

  1. Investigating the Design of Augmented Narrative Spaces Through Virtual-Real Connections: A Systematic Literature Review
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Please enable JavaScript to view thecomments powered by Disqus.

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
          May 2024
          18961 pages
          ISBN:9798400703300
          DOI:10.1145/3613904
          This work is licensed under a Creative Commons Attribution International 4.0 License.

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 11 May 2024

          Check for updates

          Author Tags

          1. Augmented Reality
          2. Mixed Reality
          3. augmented narrative space
          4. storytelling

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          CHI '24

          Acceptance Rates

          Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

          Upcoming Conference

          CHI 2025
          ACM CHI Conference on Human Factors in Computing Systems
          April 26 - May 1, 2025
          Yokohama , Japan

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)2,664
          • Downloads (Last 6 weeks)673
          Reflects downloads up to 14 Dec 2024

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Medical Education Escape Room Aligned with Flipped Classroom and Powered by Mobile Augmented RealityElectronics10.3390/electronics1312236713:12(2367)Online publication date: 17-Jun-2024

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Login options

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media