What You Experience is What We Collect: User Experience Based Fine-Grained Permissions for Everyday Augmented Reality

Privacy issues caused by data collection within head-worn AR are not the same as those on smartphones [2, 20, 51, 94]. Both devices can use the same physical sensors, such as RGB cameras, depth sensors and microphones; AR devices can also incorporate additional sensors, such as eye tracking and positional tracking sensors. Yet, the data collected when using an AR headset has an increased scope and richness due to the headset’s prolonged use and ‘always on’ capabilities continuously sensing data [2, 3, 95]. For example, AR applications can use behavioural data to disclose a user’s sexual preferences [58], emotions [46], and mental state [54].

Prior research shows that AR can amplify existing privacy issues and known challenges [2]. AR applications have access to a more significant data collection that users may not fully comprehend the breadth of personal information that an application can infer by combining different AR sensors [2, 31, 94]. For example, a user’s eye gaze hotspots and behavioural data can be integrated to build highly targeted advertisements[2, 74, 94], displaying specific adverts based on the user’s emotions that only they can see [2, 74]. The privacy concerns of AR sensing branch further than solely the user [2, 84, 94]. Bystanders around the AR user are equally at risk of being sensed without knowing they are being sensed or providing any consent for the data collection in the first place. [84]. AR devices can have the capability to sense a range of bystander data, such as behaviour, biometric characteristics, and identity [84].

Developers must be cautious when accessing AR sensor data. AR applications requesting full access to raw sensor data provide the application with much more information than is potentially needed [2, 86]. The sensed data could contain sensitive details that the user and bystanders could be uncomfortable with or even unaware they are sharing [3, 84, 94]. Overprivileged access to data can occur when applications request more information than is required for the functionality [27], such as requesting full camera access to track users’ hand movements when access to the hand’s positional data would be sufficient [51]. Due to a lack of understanding by users of why an application requested full access to the sensor, applications force users to make predictions and assumptions about how their data will be processed, stored, and used [2, 56]. Raw data access from a headset, such as video, audio, and infrared data, can uncover potentially sensitive user information and personally identify users [79, 87]. For example, recent research has shown that users can be personally identified through basic positional data within a VR experience to an accuracy of 94.33% [79] using sensors also present on AR headsets.

In summary, users need to be made more knowledgeable of the extent of information that can be inferred from AR sensors, especially during prolonged headset use. Combining different AR sensors that are ‘always-on’ can allow applications to influence and collect private information about a user to which they did not consent to be collected. Hence more work is needed to investigate and control the data flow between the user and the application.

Applications use permissions to facilitate consent from a user to access a resource on a device [44], such as accessing the device’s camera or photos. Users then can allow or deny the application access to the requested resource. There are two forms of permissions, install-time and run-time. Install-time permissions are presented to users at the start when first installing the application [28]. The application presents users with a list of permissions the application wants access to; users can typically grant access to all the requested permissions or deny the application access, subsequently stopping the application’s installation [102]. In contrast, users are presented with run-time permissions only when the application wants access to a particular resource [5, 28, 102]. Asking users for information when needed helps prevent over-privileged applications, where applications can access data without a need [27]. Run-time permissions, in theory, should correspond clearly with an exact functionality [5]. A clear association between the data requested and the application’s functionality should be evident to the user as to why the application wants access to the requested data [5, 13, 113]. Unlike install-time permissions that stop the user from using the application, if the user denies access, the best practice for run-time permissions suggests that the application should still work in general minus the functionality where the denied data is needed [5]. The predominant permission type on Android 6.0 and iOS 6 and above devices is run-time [63, 102], which also includes XR devices like the Meta Quest. Nevertheless, in practice, applications can still present run-time permissions the first time users open the application after installation, thus creating an evolved version of install-time permissions where users struggle to associate permissions with specific functionality.

In summary, permissions come in two forms, install-time and run-time. Install-time permissions, if denied, prevent the application from being installed. Each run-time permission should, in theory, be associated with a specific functionality, and if denied, the application should still be able to run minus the functionality.

Traditionally permissions controls are seen as a binary “Allow” or “Deny” decision. Users can either allow or deny applications full access to the requested resource. Binary permission controls are still in use today; however, from Android 11 onwards, applications requesting permissions to access the device’s location, microphone, or camera can use one-time permissions. One-time permissions provide temporary access to information, allowing users to provide access “While using the app”, “Only at this time”, or “Deny” [5]. iOS 15 and above also uses both binary permissions of “Don’t Allow” and “Allow”; and a similar temporary control used for location “Allow while using App”, “Allow Once”, and “Don’t Allow” [50]. Currently, AR devices do not have such temporary access permission and only use the standard “Allow” and “Deny” binary options. However, such existing temporary access permission systems assume the application is not running indefinitely, failing to account for the ’always on’ capabilities of everyday AR devices and the extended scope of data collection [2, 3, 20, 94].

Even though Android and iOS permissions have similar permissions controls, the method by which they are presented to the user differs. iOS permissions require developers to add an explanation for why the permission is needed, such as “The app records during the night to detect snoring sounds” [50]. Permissions can explain why the application needs access to specific data to increase users’ understanding [59, 113, 114]. However, applications can implement malicious or misleading justifications to trick users into providing access when they might not have otherwise if the justification was accurate [61, 111]. Android permissions can also include explanations, but as an optional feature and not by default [5, 102]. The OS generates the Android prompt by default, hence why the messages are vague. The text is generic enough that developers can use the prompts in multiple situations and applications. On iOS, the application developers write the prompt specifically for the context and application; hence other applications cannot use the same prompt.

Previous work shows that such contextualised text-based explanations similar to the iOS permission prompt performed better than non-contextualised examinations similar to the Android permission prompt for Mobile Augmented Reality applications [44]. Nevertheless, textual and non-textual contextualised justifications are absent within current consumer AR devices. Textual justifications may be a first step to explaining AR permissions better. However, textual justifications do not take advantage of the capabilities of AR devices, such as novel visualisations, displaying non-textual information, and more expansive screen space. Therefore, there is a need for permission systems that leverage AR affordances.

Another method to control permissions is the use of Fine-grained permissions. Fine-grained permissions refer to how precise the data accessed by an application is. iOS and Android can allow users to provide different levels of data availability regarding location data. On Android, a user can select between providing approximate or precise location information to an application [6, 30]. The approximate location permissions allow the app to access the users’ location within 3 square kilometres (1.2 square miles). As the name suggests, the precise location permission allows the application to access the users’ location within 50 meters or less (160 feet) [6]. Apple implemented a similar fine-grained control for location through a toggle switch and provides visual feedback to the user showing the accuracy of both options [50]. Nevertheless, even when the applications can offer fine-grained control for the users’ location, this permission presentation method is not the default prompt. Developers can even evade the option for users to provide approximate location data by requesting only the precise location permission [110] and bypassing interventions explicitly designed to protect user privacy- nudging users into providing more data than necessary.

In summary, permissions controls within AR headsets only give the user a binary decision to allow the application full data access or nothing at all. On both Android and iOS devices, when accessing sensitive sensors, the user can provide the application access based on the session length. However, such a solution would not work for AR headsets as sessions can run indefinitely, once again failing to account for ‘always-on’ sensors. The iOS permission prompts are unique and written by developers, while Android permission prompts are generic and provided by the OS. Neither prompt takes advantage of the new capabilities provided by AR headsets, such as infinite screenspace and immersion.

There is limited literature on controlling data access within an AR context. Gallardo et al. [31] interviewed several AR users to investigate what privacy concerns may exist in an everyday AR context. The AR users stated that AR headsets will need to account for different contexts that users will be in, and data access controls may need to become context-dependent [31], motivating the need for AR data access controls to provide variable levels of permissions based on the current situation. This work shows that a one-size-fits-all permissions approach is inappropriate for everyday AR as it does not account for prolonged headset use [31].

Previous research has looked at potential OS-level abstractions to provide fine-grained data access for AR headsets [51]. The fine-grained permissions were implemented using recognisers. Recognisers allow applications to access finer-grained permissions rather than access to the complete sensor data [51]. For example, if an application wants to render filters onto a person’s face, the recognisers only provide the application with where a person’s face and facial features are [51]. Preventing the application from accessing the complete front camera data, which subsequently leaks information about the users’ surroundings to the application, providing the application with overprivileged access [51]. Currently, recognisers work at varying accuracy levels based on different situations, thus impacting the users’ experience when using the application [51]. Moreover, recognisers do not allow users to customise the level of data access an application has.

Recognisers have also been used as the basis of a world-based data access control [95]. The World-Driven Access Control uses ‘passports’ to communicate to the AR headset that it has entered a sensitive location and must alter its data access settings [95]. This work once again motivates the need for AR permissions to be able to adapt and find new ways of providing data rather than providing full raw data access.

Outside of individual data access, previous work has looked at how information can be shared or protected from others in a multi-user AR context [93, 96]. For example, Rajaramet et al. [93] ran an elicitation study with AR and security and privacy experts to design different interaction techniques and how access control is managed between colocated and remote AR users. Ruth et al. [96] examined how augmentations and applications can be secured and shared between users. Both these works account for one type of context and do not explore when users move between locations or allow users to control the level of data access.

Due to a lack of data access guidance for XR developers [2, 3], over recent years, academia [3, 43], industry [49], and government [68] have developed potential security and privacy standards for general XR. For example, XRSI developed a privacy and safety framework [49] highlighting the need for transparent data handling, user consent mechanisms, and ethical considerations in immersive experiences. XRSI’s framework complements broader standards such as GDPR and NIST guidance [117]. Collectively, all these standards and frameworks represent a focused effort to build both adaptable and robust measures for ensuring future XR systems are secure and private.

Previous research [2, 20, 31, 94] motivates an investigation into how AR devices can present fine-grained permission systems that empower users to manage their privacy rather than becoming another hurdle. However, there is no work looking at how users may engage with such fine-grained permissions when given the chance to customise the level of data access over traditional permissions, if at all.

Past research has shown that when users are presented with run-time permissions, they tend not to read permissions and grant the application access to all the requested data without understanding the consequences of their actions [27, 55]. When users do pay attention to permissions and read the provided information, the permission prompt can lack clarity and be challenging for the user to understand [27, 56]. Permissions are seen to be more complex to comprehend [75]. The lack of clarity is an issue, as previous work has shown that the main factor in when a user allows or denies a permission is based on the user’s expectations, their understanding of the application’s functionality, and their considerations of privacy consequences at that moment [23, 63, 75, 102, 113].

By requesting permissions to access information, it is clear to users what data is accessed by the application [44, 56]. However, a struggle when interacting with permissions is that the reason why the application requests access to specific data is not apparent to users [56]. Permissions that do not explicitly state a reason or are unclear from the context leave users confused about why the application needs access in the first place [59] — ultimately leading users not to allow the application access to the requested data [9].

Users prefer when permission systems clearly explain why the application has requested access to information. When applications present users with the reasoning behind the data access with transparency, the application is also perceived to be more trustworthy [39, 44, 59, 113, 114]. Building users’ trust is vital for applications as users tend to grant permissions for applications they trust [110].

Applications can become overprivileged by requesting access to more resources than required [27]. Previous work has shown that developers tend to get confused over the scope of each permission and thus request more than they require to avoid their application crashing [110]. Users allowing more permissions than is needed can cause a privacy issue later on if the developer starts to collect the information previously granted without the user being aware of the change. Research has shown that users rarely remove permissions after they have been granted for reasons such as a lack of awareness, ‘out of sight out of mind’, or even believing they had no reason to [60, 64, 101, 110]. Overprivileged applications make it more important than ever to configure permissions correctly when provided with the opportunity to, as users tend not to remove previously granted permissions.

Current AR permission systems need more explicit justifications presented to users for why the applications requested access to data. AR users cannot make an informed privacy decision whether to allow or deny the permission due to the lack of information [2].

In summary, users tend not to read permission prompts, and when they do can easily get confused as many prompts lack clarity. Users must understand why the requested data is needed to make an informed privacy decision. Overprivilaged applications constitute a significant privacy concern as many users do not change their permission settings to revoke previously granted access at a later point.

The following permission controls were used as baselines to compare against the permission controls we developed. We chose the Binary control, Android 11 control, and iOS control as they are the state-of-the-art permission systems that are available and openly used today. This study focuses on the evaluation of permission control methods, specifically looking at the prompt designs rather than the interaction methods. As most AR systems are based on Android and use the older Binary control, we included the Android 11 control as it is the updated version of how Android systems currently provide data access. The addition of the iOS control follows the same reasoning, as visionOS used in Apple’s Vision Pro may base data access on iOS. It is important to note that in this study, the design of the baseline permission prompts mirror the actual designs in the real-world, thus only providing the exact same text and information, contributing to our studies ecological validity in comparing to real-world baselines.

All the permission control methods were implemented using C# within Unity with the NRSDK 1.9.1 [83]. We loaded the permissions controls directly onto the XReal Light AR headset (1920x1080 px per eye, 60Hz refresh rate) [82] to use as a standalone headset, not to require a connection to external computing power. While users can interact with the permission control methods using both hand tracking and the Nreal controller, only the controller was used for this study due to the performance of the XReal hand tracking under variable lighting conditions.

Five different contexts were formulated to evaluate our control methods so the participants could have enough “hands-on” time to judge and configure the permission control methods reasonably. Each context is an application archetype upon which we would test our envisaged permission prompts. Subtitle: This app provides situational awareness to AR users about the people in their surroundings. This application archetype is based on the work from Google [33, 85]. Filterr: This app allows AR users to apply face and body filters onto other people. This application archetype is based on Snapchat [107]. Room Designer: This is an interior design app that allows AR users to design the room they are currently in. This application archetype is based on the Ikea Place app [48]. StreetNav: This app provides AR users with directions to walk in order to navigate to a location. This application archetype is based on Google Maps [34]. AR Health: This app tracks AR users’ mental and physical health, and its archetype is based on Apple Health [7]. Each of the contexts was a means of presenting different permission prompts for different application archetypes. Hence, we did not have a functional application for any context and did not collect any of the data the permissions requested.

Five permission control variables were used for this study as presented in section 4. Two fine-grained permission methods Slider w/o Image control and Slider w/ Image control. Three baselines from state-of-the-art permission methods Binary control, Android 11 control, and iOS control. During the study the Android 11 control and iOS control were referred as “Data Access Without Description Control” and “Data Access With Description Control” as we did not want to bias the participants by showing brand names or collect results based on any preconceived notions of the two companies privacy practices.

After each condition, the participants were asked to complete unweighted NASA-TLX questionnaire [45] to measure perceived workload for each permission control method. Next, participants are asked questions on perceived understanding used in [44] and [53]. Once the participants have completed all twenty-five conditions at the end of the study, participants were asked to fill out 5-point Likert questions on perceived usability and trust and rank the permission control methods in order of preference. These questions were asked at the end so that the participants could have sufficient time to use each permission control method. The usability questions are based on the System Usability Scale (SUS) [15] and a question used in [44].

The post-study semi-structured interview was conducted to gain more in-depth insights into the decisions made by the participants about their mindset and thinking when based on the actions in the user study and questionnaire responses. The researcher followed an interview guide (see section A.1) to ensure consistency among all the participants while having the freedom to explore and discuss topics brought up during the interview. Interview questions examined comprehension and attitudes towards the permission control methods experienced, in particular contrasting the Slider w/o Image control and Slider w/ Image control with participant’s prior experience both with mobile device permissions and the baselines presented in this study, before finally discussing enhancements to the fine-grained permission controls.

Our Institution’s Research Ethics Board granted ethics approval before conducting the research. The complete study took 60 minutes of the participants’ time; the participants received £10 Amazon gift cards as reimbursement. We recruited 20 participants by sending invitations over a university mailing list, posting on social media, and word of mouth. All potential participants had to be aged 18+ to take part. Our participants were within the age range of 20-40 (M = 27.45, SD = 5.30). Nine participants self-identified as female and eleven self-identified as male. The option of non-binary and other was available yet was not selected. Zero participants declared they had not heard of AR; two participants declared they had never used AR but knew of it; nine declared they had used mobile AR infrequently (AR games such as Pokemon GO, AR applications); one declared they had used mobile AR frequently (AR games such as Pokemon GO, AR applications); and nine declare they have used an AR headset (Hololens, VR headset with passthrough).

We used the Internet Users’ Information Privacy Concerns questionnaire (IUIPC) [62] to assess participants’ general privacy attitudes. The IUIPC’s score is between 1 and 7, where 7 represents a high privacy attitude. Participants rated their wish for control (M = 5.96, SD = 0.05), awareness (M = 6.43, SD = 0.30), and the perceived ratio between benefit and collection (M = 5.77, SD = 0.10). A limitation of privacy questionnaires is that they only provide the theoretical privacy attitudes of the user and do not provide insight into if the attitudes are present in practice [32]. Nonetheless, the mean scores implying the participants represent a high general privacy attitude.

Once the participant was ready to begin the study, they wore a pair of XReal Light AR glasses [82]. The lead researcher explained to the participant how to use the AR headset and controller. The experiment had five different contexts, each with five permission control methods, allowing the participants to configure the requested permission(s). The order of the contexts and permissions controls presented to the participant was counterbalanced using a 5x5 Latin Square [115]. The participant started the study when presented with a context while wearing the AR headset. The researcher read a description of the context as stated in section 5.1. After confirming they understood the context and needed no further repetition, the participant was shown a permission control method presented in section 5.2. The participant was then asked to explore each option and make an informed choice based on if they were using this application in reality. After the participant configured the permissions, they were asked to complete a questionnaire on a tablet. The participant was asked not to take off the AR glasses while filling out the questionnaire so they could refer back to the permission control method at any time. Once the participant interacted with all five permissions control methods, this process was repeated for all the other contexts.

After completing all conditions, the participant removed the AR headset, completed another questionnaire, and ranked the permission control methods on a tablet. Printouts of the five permission control methods were given to the participant to aid their memory. Once the participant completed the questionnaires, a semi-structured interview was conducted (see section 5.4).

Our sample size is small compared to a global sample size. However, a power analysis was conducted using R studio [91, 109] with the pwrss package [16]. Setting an alpha of 0.05 as the significance criterion and a power of 80% [29], a sample of 20 participants is sufficient to determine a medium effect size (\(\eta _{p}^{2} = 0.06\)) or greater with a repeated measures ANOVA. The power analysis assumes a perfect sample of participants, which in practice is extremely difficult to recruit from a local sample [66]. Nevertheless, our within-subject study has more participants than the average accepted number of local participants for a within-subject study in the human-computer-interaction field [17]. While the pairwise comparisons were carefully conducted with corrections applied to mitigate potential biases, no additional corrections were applied on top of the already adjusted values within potential single-family hypotheses. Thus, care is required when generalising the results to broader contexts [21, 118].

We also cannot completely rule out that participants did not fully understand the context descriptions and may have configured the permissions in a way they might not have if their understanding was correct. However, as detailed in section 6.2, we took all precautions to ensure the participant understood the context, double-checking by asking if they understood the context and repeating the context when needed. Participants may not have understood a question or term before answering the question. However, to minimise confusion, the researcher was present in the room and made sure the participant knew they could ask questions or request clarification if needed.

Quantitative: We performed an Aligned-Rank Transform (ART) [24, 116] using the R version of ARTool [26] to convert our non-parametric data into a format to conduct both a one or two-way repeated-measures ANOVA for statistical significance testing. ART enables parametric tests to be conducted on non-parametric data, in our case, Likert-scale responses or data that was non-normal distributions [100, 116]. The effect sizes are reported when the partial eta squared (\(\eta _{p}^{2}\)) values are as follows: small when \(0.01\ge {}\eta _{p}^{2} < 0.06\), medium when \(0.06\ge {}\eta _{p}^{2} < 0.14\), and large when \(\eta _{p}^{2}\ge {}0.14\). These values were chosen based on previous work [29, 77]. We report significance values where p equals or is less than 0.05. In cases of significant differences, we run ART-C post hoc contrasts [25] tests using Tukey corrections to correct the alpha value for multiple comparisons [25, 116]. We do not explicitly report post-hoc contrasts for the interaction effects between context and the permission control method as 300 pairwise comparisons are available hence, presenting meaningful results would not be feasible. However, we present the interaction graphs within the Appendix for by-eye comparisons and tables indicating significant differences (seesection A.2).

Qualitative: Data from the semi-structured interview were coded using inductive coding [67] to develop a qualitative codebook. A single coder iteratively coded the data to account for emerging codes. Quotes from the interview were grouped using codes. Codes were not defined before the coding process and emerged through patterns and similar answers within the data. The lead and secondary authors reviewed the final codes list and grouped similar codes into main themes.

Understanding of data used. Statistical tests for the participant’s understanding of what data the app will be able to use showed significant differences (“The permission control method gives me a good understanding of what data the app will be able to use: 1=Strongly disagree; 5=Strongly agree”, see Figure 3 for distribution of participant responses). The ANOVA suggests that the main effect of Context was not statistically significant and small (2.35, p = 0.054; \(\eta _{p}^{2} = 0.02\)). The main effect of Permission Control Methods was statistically significant and large (32.07, p < 0.001; \(\eta _{p}^{2} = 0.22\)). The interaction between Context and the Permission Control Methods was statistically significant and medium (2.49, p = 0.001; \(\eta _{p}^{2} = 0.08\)). Post hoc analysis revealed the Binary control performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). The Android 11 control also performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0018), and the Slider w/ Image control (p < 0.0001). The iOS control performed statistically significantly better than the Slider w/o Image control (p = 0.0001) yet was still beaten by the Slider w/ Image control (p < 0.0001). The Slider w/ Image control went onto outperform the Slider w/o Image control (p < 0.0001). This shows that clear descriptions and images allow participants to understand better what data the app would be able to use.

Understanding of application functionality. Statistical tests for the participant’s understanding of the functionalities of the applications showed significant differences (“The permission control method gives me a good understanding of what data the app will be able to use: 1=Strongly disagree; 5=Strongly agree”, see Figure 3 for distribution of participant responses). The ANOVA suggests that the main effect of Context is statistically significant and small (3.23, p = 0.012; \(\eta _{p}^{2} = 0.03\)). The main effect of the Permission Control Methods is statistically significant and large (133.88, p < 0.001; \(\eta _{p}^{2} = 0.54\)). The interaction between Context and the Permission Control Methods is statistically significant and medium (3.11, p < 0.001; \(\eta _{p}^{2} = 0.10\)). Post hoc analysis for Context revealed that the Subtitle context performed statistically significantly worse than both the RoomDesigner (p =0.0248) and StreetNav (p =0.0212) contexts. Post hoc analysis for the Permission Control Methods revealed that the Binary control performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). The Android 11 control also performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). The iOS control performed statistically significantly better than the Slider w/o Image control (p = 0.0001). The Slider w/ Image control performed better than the Slider w/o Image control (p < 0.0001). These results show that the Slider w/ Image control and iOS control were the best at communicating the application’s functionalities.

Understanding of potential effect on privacy. Statistical tests for the participant’s understanding of permission’s potential effect on their privacy (“The permission control method gives me a good understanding of the potential effect to my privacy: 1=Strongly disagree; 5=Strongly agree”, see Figure 3 for distribution of participant responses). The ANOVA suggests that the main effect of Context was not statistically significant and small (0.62, p = 0.646; \(\eta _{p}^{2} = 5.44e-03\)). The main effect of the Permission Control Method was statistically significant and large (24.58, p < 0.001; \(\eta _{p}^{2} = 0.18\)). The interaction between Context and the Permission Control Method was statistically not significant and small (1.33, p = 0.173; \(\eta _{p}^{2} = 0.04\)). Post hoc analysis revealed the Binary control performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0001), and the Slider w/ Image control (p < 0.0001). The Android 11 control also performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). Finally the Slider w/ Image control went onto outperformed both the Slider w/o Image control (p = 0.0044) and the iOS control (p = 0.0018). These results show that the Slider w/ Image control was the clearest method for the participants to understand what would happen to their privacy based on their permission choice.

Understanding of why data was requested. Statistical tests for the participant’s understanding of why the applications needed access to the requested permission (“I understood why the application needed access to the permissions that were being requested: 1=Strongly disagree; 5=Strongly agree”, see Figure 3). The ANOVA suggests that the main effect of Context was statistically significant and small (3.06, p = 0.017;\(\eta _{p}^{2} = 0.03\)). The main effect of the Permission Control Method was statistically significant and large (72.16, p < 0.001; \(\eta _{p}^{2} = 0.39\)). The interaction between Context and Type is statistically significant and medium (1.97, p = 0.014; \(\eta _{p}^{2} = 0.06\)). Post hoc analysis for the Context revealed no significant differences between contexts. Post hoc analysis for the Permission Control Methods revealed the Binary control performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0001), and the Slider w/ Image control (p < 0.0001). The Android 11 control also performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0003), and the Slider w/ Image control (p < 0.0001). Finally, the Slider w/o Image control was outperformed by both the iOS control (p < 0.0001) and the Slider w/ Image control (p = 0.0018). These results show that the Slider w/ Image control and iOS control allowed the participants to see the correlation between the permission requested and why the app needed access to that data easier than the other permission control methods.

The SUS based usability questions were asked after the participants completed all 25 unique conditions, hence the only factor is the Permission Control Method. See Figure 4 for distribution of participant responses).

Easy to Use. Statistical tests showed a significant difference between the Permission Control Methods in their perceived ease of use (“I thought the {control method name} was easy to use: 1=Strongly disagree; 5=Strongly agree”). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (4.85, p = 0.002; \(\eta _{p}^{2} = 0.20\)). Post hoc analysis revealed the Slider w/o Image control performed statistically significantly worse than the iOS control (p = 0.0026), and the Slider w/ Image control (p = 0.0159). These results show that the participants perceived the Slider w/ Image control and iOS control as easier to use than the Slider w/o Image control.

Error Proneness. There were no significant differences between the Permission Control Methods in perceived error proneness (“I thought the {control method name} was error-prone: 1=Strongly disagree; 5=Strongly agree”).

Unnecessary Complexity. Statistical tests showed a significant difference between the Permission Control Methods in their perceived Unnecessary Complexity (“I found the {control method name} was unnecessarily complex: 1=Strongly disagree; 5=Strongly agree”). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (3.35, p = 0.014; \(\eta _{p}^{2} = 0.15\)). Post hoc analysis revealed the Slider w/o Image control performed statistically significantly worse than the Binary control (p = 0.0049). These results show that the Slider w/o Image control without an accompanying image may be considered overly complicated, showing how important the addition of the visual representations are.

The Trust questions were asked after the participants completed all 25 unique conditions, hence the only factor is the Permission Control Method. See Figure 5 for distribution of responses.

Trust of Information Accuracy. Significant differences were found when the participants were asked if they trusted the information provided by the permission methods was accurate (“I trust the information provided by the permission control is accurate”). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (11.96, p < 0.001; \(\eta _{p}^{2} = 0.39\)). Post hoc analysis revealed the Binary control performed statistically significantly worse than the iOS control (p = 0.0016), the Slider w/o Image control (p = 0.0051), and the Slider w/ Image control (p < 0.0001). As well as the Android 11 control also performing statistically significantly worse than the iOS control (p = 0.0042), the Slider w/o Image control (p = 0.0125), and the Slider w/ Image control (p < 0.0001). These results show that the participants trusted the information provided by the permission prompt was accurate more when the permission control method explained why the application requested the data along with the name of the sensor, either through text as seen on the iOS control and Slider w/o Image control or image-based like the Slider w/ Image control.

Trust of Device. After running statistical tests on how much the participants trusted the AR device, significant differences between the Permission Control Methods were found (“I trust the device will respect my privacy decisions”). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (5.15, p = 0.001; \(\eta _{p}^{2} = 0.21\)). Post hoc analysis revealed the Binary control performed statistically significantly worse than the iOS control (p = 0.0071), the Slider w/o Image control (p = 0.0296), and the Slider w/ Image control (p = 0.0015).

Trust of Application. Significant differences were found for how much the participants trusted the application (“I trust the application will respect my privacy decisions”). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (6.20, p < 0.001; \(\eta _{p}^{2} = 0.25\)). Post hoc analysis revealed the Android 11 control performed statistically significantly worse than the iOS control (p = 0.0325), and the Slider w/ Image control (p = 0.0150). The Binary control also performed statistically significantly worse than the iOS control (p = 0.0050), the Slider w/o Image control (p = 0.0478), and the Slider w/ Image control (p = 0.0021). These results show that the participants trusted the application more when the permission control method explained why the application requested the data, either through text as seen on the iOS control and Slider w/o Image control or image-based like the Slider w/ Image control.

The Privacy Decision Opportunities questions were asked after the participants completed all 25 unique conditions, hence the only factor is the Permission Control Method.

When the participants were asked how well each permission control method provided them with the opportunity to set the perceived best privacy decision, significant differences were found after the 5-point Likert questions were analysed (“The permission control method provided the opportunities to set the best privacy decision: 1=Strongly disagree; 5=Strongly agree”, see Figure 6 for distribution of participant responses). The ANOVA suggests that the main effect of the Permission Control Method was statistically significant and large (29.57, p < 0.001; \(\eta _{p}^{2} = 0.61\)). Post hoc analysis revealed the Android 11 control performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0001), and the Slider w/ Image control (p < 0.0001). The Binary control also performed statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). Finally, the Slider w/ Image control performed statistically significantly better than the Slider w/o Image control (p = 0.0261). The results show that the Binary control and the Android 11 control performed poorly compared to the rest of the control methods. The results indicate that the participants need more information than just the name of the sensor being requested.

The overall NASA-TLX scores for the Android 11 control was 8.65 (SD = 10.47), the Binary control was 9.92 (SD = 11.82), the Slider w/ Image control was 10.68 (SD = 9.41), the Slider w/o Image control was 12.32 (SD = 13.19), and finally, the iOS control was 12.89 (SD = 12.76). Statistical analysis showed the main effect of Permission Control Method is statistically significant and large (3.38, p = 0.013; \(\eta _{p}^{2} = 0.15\)). Post hoc analysis showed statistically significant differences between the iOS control (Mdn= 8.66) and the Android 11 control (Mdn= 3.33, p = 0.036). These results show on the unweighted NASA-TLX only the Android 11 control scored low, and the rest scored medium for perceived workload [42, 45, 88]. Note that the medium level in the NASA-TLX is derived from the original weighted scale’s 21/3 division.

Participants ranked the permission control methods in the order of most to least preferred (see Figure 7 for distribution of permission control method rankings). The preferred permission control method was the Slider w/ Image control (Mdn= 1st), then iOS control (Mdn= 2nd), then Slider w/o Image control (Mdn= 3rd), then Android 11 control (Mdn= 4th), and finally the Binary control (Mdn= 5th). Statistical analysis showed the main effect of the Permission Control Method was statistically significant and large (46.21, p < 0.001; \(\eta _{p}^{2} = 0.71\)). Post hoc analysis revealed the Binary control ranked statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p < 0.0001), and the Slider w/ Image control (p < 0.0001). The Android 11 control also ranked statistically significantly worse than the iOS control (p < 0.0001), the Slider w/o Image control (p = 0.0001), and the Slider w/ Image control (p < 0.0001). Finally the Slider w/ Image control ranked statistically significantly better than the Slider w/o Image control (p < 0.0001), and the iOS control (p = 0.0058)

We present qualitative data sectioned into three themes based on the clustering of individual qualitative codes.

Our study showed that Slider w/ Image control performed the best for the user’s perceived understanding of the permission control method. Participants had a good understanding of why data was being requested, what the app would do with the data, what functionalities would be available when the app had data access, and finally, what effects allowing data access has on the participant’s privacy. The addition of the descriptive imagery over Slider w/o Image control also helped participants understand why the permission was requested compared to only providing textual explanations, with no perceived impact on usability, being just as easy to use as the current AR binary controls.

Information privacy in the literature is traditionally seen as personal and based highly on current contexts [18, 65, 80, 81, 105]. Viewed through such a lens, users are in control of their individual privacy when they engage with permission controls and make decisions on what private information to share. However, literature has also challenged the idea of individualism and privacy [11, 92, 99, 104], particularly where others privacy attitudes (or lack of) impact more than the user themselves [11, 92, 99, 104]. For example on social media, no matter how disciplined a users privacy attitudes are, it makes no difference if the users friend of posts, shares or leaks private information about them [11, 99].

In a future where everyday AR headsets that are packed with sensing capabilities are worn by multiple people at any given time, privacy could also be considered a collective problem. Literature has shown applications shift the privacy risk assessment of allowing permissions to their users [110]. Placing privacy decisions in the hands of the user requires the user to have a well-rounded view and access to all the information first. Hence, for users to even have a chance to make a fair and informed decision, users need to have a strong understanding of the implications of their choices in allowing or denying data access. Conversely, both Binary control and Android 11 control performed poorly in how well participants understood the permission control methods. Binary control was consistently last, and Android 11 control, when not tied last, came second last. Both the permission control methods present the least information, thus placing last reflects previous work that permission prompts that show very little information are more challenging for users to understand [27, 56, 75]. These results also show that the participants were cautious about their privacy when considering the new level of privacy invasion that AR can potentially bring [2, 94].

AR (and XR permissions, more generally) have thus far typically copied and pasted from smartphone permission architectures. We posit that this is a short-term solution, as AR has different privacy risks than smartphones [2]. Hence as AR devices develop and grow in consumer adoption, we argue that permissions controls such as the Slider w/ Image control built specifically for AR are needed to enhance the privacy of consumer AR devices and support users in determining an acceptable privacy/functionality trade-off. When users have better privacy controls, everyone benefits. Our results emphasise the utility of, and need for, such an approach.

Whilst our Slider w/ Image control design offers a usable, preferable means of supporting users in decision making around the privacy/functionality trade-off for AR apps, there are a number of challenges that would need to be addressed to put such a design into practice.

Our paper has outlined and evidenced the need for permission control methods that can better address the privacy challenges posed by everyday AR in the near future, proposing that platforms move away from binary permission prompts towards fine-grain permissions that enable users to make informed decisions in balancing their privacy versus the AR application functionality. For such a vision to become a reality, collaboration between developers, academia, and platform creators is needed. All parties must put time and effort into further developing privacy protections that allow applications access to the data they need, such as fine-grained permissions or the Slider w/ Image control. While challenging, such initiatives are still possible with time. For example, fine-grained location permissions are now readily available for both Android and iOS [6, 30, 50] and provide precedent to be extended for other AR sensors. In this work, we placed the Slider w/ Image control within five contexts based on realistic apps that could be seen within everyday AR headsets. A direction that should be evaluated is how well the Slider w/ Image control performs given the nuances of the user’s data access behaviours within more specific high-value or high-impact application archetypes, considering the predominant use cases that eventually emerge around consumer everyday AR such as immersive video streaming [97] or augmented productivity [71].

Moreover, permission systems built specifically for AR should carefully consider the burden on AR users to engage with fine-grained permissions. Whilst our proposed designs did not impact perceived user workload to a problematic degree, there is significant scope for variation here, in terms of the complexity of the prompt information and how it is visually conveyed, that would necessitate carefully assessing the privacy-utility trade-off [106] and the consequent burden placed on users in any eventual real-world implementation. We argue that such an effort must be made to safeguard user privacy before we see consumers’ anticipated mass adoption of everyday AR.