Keywords

1 Introduction

1.1 General VR Interaction in Immersive Environments

In immersive virtual reality (VR), the positional relationship between the virtual body (avatar) in the VR space and the body in real space is perfectly synchronized spatiotemporally such that the sense of self-location perceived through the avatar and that of the self-body completely match with each other. However, in this situation, the movable range of the avatar is restricted by the physical movable range of its body.

The hand of an avatar in the VR space acts as a pointer, and the user interacts with an object in the VR space via the hand of the avatar. However, depending on the type of interaction device employed by the user to control the hand of the avatar, we must adjust the operation amount of the device and the movement range of the hand of the avatar.

For example, many haptic devices are designed such that the range of the workspace is narrower than that of the movement of the actual hand owing to the structure [1,2,3,4]. In such a case, we must set a ratio to ensure that the movement amount of the hand of the avatar becomes larger than the operation amount of the device. This ratio, called the control/display (C/D) ratio, is a commonly used tool for mouse interaction [5,6,7,8].

1.2 Full-Body VR Interaction via the C/D Ratio

To date, most avatars used in immersive VR were displayed with a pointer above their wrists. In recent years, however, the improvements in the motion-sensor technology have enabled to display, in real time, an avatar that is synchronized with the entire body movement of a user.

Even when an avatar is full-bodied, its hand is a pointer in the VR space. That is, even upon using a full-body avatar, when the movement amount of the avatar is adjusted, its movable range can be adjusted by tuning the C/D ratio. However, because the hand of a full-body avatar is connected to its arm, we must limit the C/D ratio to be within the range of movement of the joint angle of the arm.

1.3 Persistence of the Sense of Embodiment

Upon adjusting the C/D ratio for a full-bodied avatar, the sense of proprioception might be uncomfortable, even if the C/D ratio is adjusted within the range of movement of the joint angle of the arm. For example, as depicted in Fig. 1, the actual hand joints are curved, whereas the joints of the avatar in the VR space are completely extended. Conversely, a user visually feels that the joint is completely extended; however, in the sense of proprioception, he or she feels that the joint is bent.

Fig. 1.
figure 1

Task environments in real and VR spaces.

Full size image

The effect of the C/D ratio has been discussed from the viewpoint of operability when an avatar is displayed only in the pointer part [9,10,11,12]. However, when a body, such as an arm, is connected to the pointer in the manner previously described, the sense of embodiment associated with the avatar, as well as its operability, might be affected. The sense of embodiment is the sensation through which an individual misperceives another body (not his or her own body) as his or her own body. It comprises three elements: sense of agency, sense of body ownership, and sense of self-location [13]. The C/D ratio might affect both the sense of agency, which is the sense that one’s body is moving, and sense of self-location, which is the sense of one’s body position.

This study aims to investigate how the avatar appearance and C/D ratio affect the sense of embodiment, especially in terms of the sense of agency and sense of self-location.

2 Methods

2.1 Participants

A total of 17 students were recruited from Suwa University of Science. They were males with the mean age of 21 years (SD = 1.32). They had minimal or no experience with VR, as well as mixed reality, and related applications.

2.2 Apparatus

Visual Display/Rendering.

A head-mounted display (HMD) (manufacturing details: HTC VIVE) was used for the experimental task. The headset covered a nominal field-of-view, approximately 110°, using two 2,160 × 1,200 pixel displays, which were updated at 90 Hz. For visual rendering, Unity3D was employed, rendering the graphics at 60 Hz.

3D Motion Tracker.

A leap-motion sensor was used to track the 3D position of the hand of the user, as well as the orientation at 120 Hz.

Task Environment.

A customized experimental environment was developed for this study. It had three blocks on a table both in real and VR spaces. These blocks were spatially synchronized both in real and VR spaces such that a user could visually recognize them in the VR space, as well as haptically recognize them in the real space (see Fig. 2).

Fig. 2.
figure 2

Task environments in real and VR spaces.

Full size image

2.3 Experimental Task

The protocol for the experimental task was the same for all the conditions. The participants blindfolded themselves before entering the experimental room so that they could not recognize the task conditions. After the experimenter provided them with the instructions of the experiment, each participant was made to wear the HMD and then calibrate his position in a virtual environment that was displayed through the HMD. In the experimental task, a target block was randomly displayed from three directions. Each participant was required to set his hand on the home position when the task began. Upon displaying the target block, each participant was required to touch it using his virtual avatar. After touching the block, he had to return to the home position, following which the next target block was displayed. This trial was conducted 30 times for one experimental condition.

2.4 Experimental Design and Independent Variables

To examine the effect of the C/D ratio and external appearance of the virtual avatar, a “within-subjects factorial experiment” design was used, resulting in six experimental tasks.

External Appearance of Virtual Avatar.

In this experiment, two types of avatars were created, as depicted in Fig. 3. The spherical avatar in (a) was synchronously controlled using the position of the palm of the participant. The human-hand-like avatar depicted in (b) was controlled by synchronizing the joint positions and angles of the fingers and arms of the hand of the participant.

Fig. 3.
figure 3

Two types of avatar for the task.

Full size image

C/D Ratio in the VR Space.

The C/D ratio is the ratio of the amplitude of the hand movement of a real to that of his or her virtual hand. In this study, the C/D ratio was changed thrice (0.9, 1.0, and 1.3), as depicted in Fig. 4. When the C/D ratio was 0.9, the position of the avatar shifted to the front of the position of the hand of the actual user. When it was 1.0, the position of the hand of the actual user exactly matched that of the avatar. When the C/D ratio was 1.3, the position of the avatar shifted to behind the hand of the actual user.

Fig. 4.
figure 4

Concept of the C/D ratio in the experimental task.

Full size image

2.5 Dependent Variables

To systematically investigate the effect of both the external appearance of virtual avatar and the C/D ratio in customized task environments, we utilized several dependent measures, which can be categorized into two types of variables. The first one is task performance, which includes the velocity changes of the hand motion. The second one is subjective performance, which includes the sense-of-embodiment questionnaire (SoEQ), presence questionnaire, and NASA-TLX.

Subjective Performance.

It includes three types of questionnaires through which we asked the participants regarding their feelings associated with the sense of embodiment, presence, and mental workload.

2.6 Procedure

The participants were required to read and sign an informed consent. After completing the paperwork, a short training was conducted to familiarize them to interact in the customized experimental environment. The experimental tasks began after the training session. Each participant completed 30 trials for each of the six tasks in a randomized order. After completing each task, the participants answered the questionnaires.

3 Results and Discussions

3.1 Subjective Performance

Presence Questionnaire.

It is a common measure of presence in immersive VR environments. It comprises seven subscales: 1) realism, 2) possibility to act, 3) quality of interface, 4) possibility to examine, 5) self-evaluation of performance, 6) sounds, and 7) haptics. However, in our experimental task, because there were no sound and haptic effects, ANOVA was applied only to five subscales, excluding the “sounds” and “haptics” subscales.

  1. 1)

    Realism. A two-factor ANOVA, as the within-subject factors, revealed the main effect on C/D ratio factor, \( F\left( {1,16} \right) = 5.554, p < .01, \eta_{{}}^{2} = .067 \). A post-hoc comparison (performed using Bonferroni’s method, where α = .05) indicated a significant difference between the 0.9 condition (\( M = 4.441; SD = 1.156 \)) and 1.0 condition (\( M = 5.1389; SD = 1.034 \)). However, no significant difference was observed between the 0.9 and 1.3 conditions, and between the 1.0 and 1.3 conditions. Table 1 presents the results of ANOVA for “realism.” Figure 5 depicts the result of the average plot.

    Table 1. Result of ANOVA for “realism.”
    Full size table
    Fig. 5.
    figure 5

    Average score plot for “realism.”

    Full size image

Although realism was considered susceptible to visual effects, no visual main effects were observed. However, the main effect of the C/D ratio was different, indicating that the subjective view of realism is influenced more by the restriction of movement than the appearance. In particular, when the C/D ratio was 0.9, the realism was low.

  1. 2)

    Possibility to act. A two-factor ANOVA, as the within-subject factors, revealed the main effect on C/D ratio factor, \( F\left( {2,32} \right) = 7.526, p < .01, \eta_{{}}^{2} = .089 \). A post-hoc comparison (performed using Bonferroni’s method, where α = .01) demonstrated a significant difference between the 0.9 condition (\( M = 4.919; SD = 1.036 \)) and 1.0 condition (\( M = 5.566; SD = 0.835 \)) and between the 0.9 condition (\( M = 4.919; SD = 1.036 \)) and 1.3 condition (\( M = 5.441; SD = 0.839 \)). However, no significant difference was noticed between the 1.0 and 1.3 conditions. Table 2 presents the results of ANOVA for “possibility to act.” Figure 6 depicts the result of the average plot.

    Table 2. Result of ANOVA for “possibility to act.”
    Full size table
    Fig. 6.
    figure 6

    Average score plot for “possibility to act.”

    Full size image

Because possibility to act is a measure related to the easiness of behavior, it might become low when the leaching action is restricted. Consequently, it was observed that possibility to act was low when the C/D ratio was 0.9.

  1. 4)

    Quality of interface, 5) Possibility to examine, and 6) Self-evaluation of performance. The two-factor ANOVA, as the within-subject factors, revealed no main effect on both the factors, namely, the external appearance of the virtual avatar and the C/D ratio.

Sense of Embodiment Questionnaire.

It is a standard measure of the feeling of sense of embodiment in immersive VR environments. It comprises six subscales: 1) location, 2) agency 3) ownership, 4) tactile sensations, 5) appearance, and 6) response. In our experimental task, because there was no tactile effect, ANOVA was applied only to five subscales, excluding the “tactile sensations” subscale.

  1. 1)

    Location. A two-factor ANOVA, as the within-subject factors, revealed the main effect on the C/D ratio factor, \( F\left( {2,32} \right) = 7.743, p < .01, \eta_{{}}^{2} = .039 \). A post hoc comparison (performed using Bonferroni’s method, where α = .01) demonstrated a significant difference between the 0.9 condition (\( M = - 0.588, SD = 3.000 \)) and 1.0 condition (\( M = 0.588, SD = 2.697 \)) and between the 0.9 condition (\( M = - 0.588; SD = 3.000 \)) and 1.3 condition (\( M = 0.618; SD = 2.606 \)). However, no significant difference was noticed between the 1.0 and 1.3 conditions. Table 3 presents the results of ANOVA for “location.” Figure 7 depicts the result of the average plot.

    Table 3. Result of ANOVA for “location.”
    Full size table
    Fig. 7.
    figure 7

    Average score plot for “location.”

    Full size image

Because location represents the sense of self-position, it might be affected if the avatar of a user is not aligned with where he or she feels his or her arm is. Consequently, there was a difference in the main effect of the C/D ratio. Specifically, it is significantly affected when the C/D ratio is 0.9. Additionally, when the avatar appears similar to a human, a visual gap occurs in the joint position; thus, a synergistic effect with the main effect of the appearance was expected; however, no significant difference was observed in the interaction effect.

  1. 2)

    Agency. A two-factor ANOVA, as the within-subject factors, revealed the main effect on the external appearance of the virtual avatar factor,\( F\left( {1,16} \right) = 7.136, p < .01, \eta_{{}}^{2} = .152 \). The result demonstrated a significant difference between the human-like-avatar condition (\( M = 4.3373; SD = 3.068 \)) and spherical-avatar condition (\( M = 0.902, SD = 4.933 \)). Table 4 presents the results of ANOVA for “agency.” Figure 8 depicts the result of the average plot.

    Table 4. Result of ANOVA for “agency.”
    Full size table
    Fig. 8.
    figure 8

    Average score plot for “agency.”

    Full size image

Agency is an exercise subject feeling, and when a user can voluntarily control his or her avatar, it might become high. Therefore, a significant difference was expected in the main effect of the C/D ratio; however, only the visual effect was confirmed. Conversely, the motion subject feeling might be affected by a change in the visual joint position of the avatar.

  1. 3)

    Ownership, 4) Appearance, and 5) Response. A two-factor ANOVA, as the within-subject factors, revealed no main effect on both the factors, namely, the external appearance of the virtual avatar and the C/D ratio.

NASA-TLX.

The NASA-TLX questionnaire is a standard measure of the mental workload in a task environment. It comprises six subscales: 1) mental demand, 2) physical demand 3) temporal demand, 4) performance, 5) effort, 6) and frustration. In this study, we calculated the weight workload (WWL) on the basis of these six subscales to assess the mental work load; thus, ANOVA was applied only to WWL.

  1. 1)

    Weighted Workload. A two-factor ANOVA, as the within-subject factors, revealed no main effect on both the factors, namely, the external appearance of the virtual avatar and the C/D ratio.

4 Conclusion

In this study, we investigated the effects of the differences in the visual joint angles of avatars and the actual joint angles fed back from participants’ sense of proprioception on realistic sensation, somatization sensation, and mental workload. Specifically, we investigated the effects of repeated simple point-to-point movements by using different avatar C/D ratios and appearances. Consequently, the main effect of the C/D ratio was recognized in the subscale of realism in the presence, and it was proved that realism decreased as the constraint of the movement increased. In SoE, the strongest effect of the C/D ratio was observed on the location subscale when the C/D ratio was 0.9. Additionally, when the avatar appeared similar to a human, a visual gap occurred in the joint position; accordingly, a synergistic effect with the main effect of the appearance was expected; however, no significant difference was observed in the interaction effect. The agency subscale might identify visual effects only and is influenced by changes in the visual joint position of the avatar. For the mental workload, the main effects of the C/D ratio and the appearance of the avatar could not be confirmed. In this study, we focused on the subjective evaluations of participants; however, in the future, we will compare them with objective indicators, such as behavioral and physiological data.