FR3129232A1 - Navigation interface in virtual environment - Google Patents

Abstract

A virtual reality device (EQ) comprising at least one display screen (AFF) and an orientation sensor (CAPT-O) configured to output device orientation data (EQ) within a real environment (ENV), characterized in that:the device (EQ) further comprises at least one position sensor (CAPT-L) distinct from the orientation sensor (CAPT-O), said position sensor (CAPT-L ) being configured to communicate with at least one fixed position beacon within the real environment (ENV), so as to output position data of the device (EQ) relative to the real environment (ENV), the device (EQ) being configured to display, in real time, a virtual content associated with the real environment (ENV) via the display screen (AFF), said displayed virtual content depending on said orientation data and on said data of position. Abstract Figure: Figure 2

Description

Navigation interface in virtual environment

This disclosure relates to the field of virtual reality equipment and more specifically integrated position tracking systems for virtual reality equipment.

Virtual reality devices such as virtual reality headsets (or VR headsets) allow their users to visualize a virtual environment (or virtual reality). A virtual reality device is generally worn by the user and broadcasts virtual content in the user's visual and/or auditory field. Such a broadcast can even fully cover the user's visual/auditory field, for example when the user is wearing a VR headset. The user no longer perceives his real environment, which reinforces the feeling of immersion, particularly sought after in the video game field for example.

In some cases, the virtual content broadcast by the device is associated with the real environment surrounding the user, for example by being superimposed on the real environment. The virtual environment can cover all directions in space, so that when the user's point of view changes (for example when the user turns his head), the broadcast virtual content is adapted in real time so that the user has an ever more immersive experience of virtual reality. When the device is attached to the user's head (helmet or goggles typically), any relative movement between the device and the user's head is cancelled, or at least rendered negligible. The capture of movements is facilitated because it becomes possible to capture the movements of the entire head of the user or of the device itself. On the other hand, individual devices such as helmets or glasses are individual and cannot be used simultaneously by several users. Alternate use of such devices in contact with the face of the user also poses difficulties in terms of hygiene and health.

Among the virtual reality devices integrating a system for tracking the movement of the user in space, most allow only orientation tracking of the user (i.e. tracking at three degrees of freedom or 3DoF). In other words, the user can visualize virtual content in all directions of space, i.e. a solid angle of 4π, incorrectly called "360 degrees", for example by remaining in a seated position and by orienting his head according to different axes. radials. But the user cannot move around the virtual environment or interact with it without the help of additional control devices.

There are theoretical principles that allow both orientation tracking and position tracking of the user (i.e. six degrees of freedom or 6DoF tracking). In this case, the user can theoretically move and change orientation in his real environment to control the movements and changes of orientation in his virtual environment. However, in practice, the position/orientation tracking of existing virtual reality devices remains imprecise: the position data is subject to noise and/or bias. Movements in the virtual environment are therefore erratic, distorted and lack "fluidity" compared to movements in the real environment, which obviously affects the user's feeling of immersion. A user of the virtual reality device cannot therefore move freely (for example in a room) while remaining immersed in a precise virtual environment broadcast in real time as the user moves in space real. In addition, most position/orientation tracking systems are subject to spatial constraints: the user's movement space must for example be sufficiently clear and the presence of walls or even bulky objects can prevent operation. existing tracking systems (position in particular).

Summary

There is therefore a need for a high-performance positioning tracking system for a user of a virtual reality device. In addition to precision, virtual reality requires a tracking system capable of operating in an indoor environment by integrating the spatial constraints of such an environment (particularly walls) and providing real-time positioning data.

This disclosure improves the situation.

It is proposed, according to a first aspect, a virtual reality device comprising at least:
- a display screen, and
- an orientation sensor configured to output device orientation data within a real environment,
the device further comprising at least one position sensor separate from the orientation sensor, said position sensor being configured to communicate with at least one fixed position beacon within the real environment, so as to output data position of the device in relation to the real environment,
the device being configured to display, in real time, a virtual content associated with the real environment via the display screen, said displayed virtual content depending on said orientation data and on said position data.

Consequently, such a device allows monitoring of the positioning of the device. Such positioning at a given instant designates both an orientation and a position in the real environment at such a given instant. Equipped with an orientation sensor and a position sensor, the proposed device thus makes it possible to follow movements according to six degrees of freedom. The user can therefore see a virtual content displayed on the display screen when he moves the device in a room (for example by advancing or retreating with respect to a given position) and/or orients the device in different directions (for example by turning the device to show the screen to a second user). Such movements can in particular be random in the sense that it is superfluous to predict behaviors in order to limit the possibilities and simplify the calculations. Indeed, the monitoring of a substantially regular movement (such as the monitoring of the step movement of a user climbing stairs for example) is simplified by the possibility of defining an approximate period of the movement of resetting the data captured at each defined period in order to avoid sensor drift phenomena. However, such a data reset is not relevant in the context of unpredictable movements.

According to another aspect, a method implemented by a virtual reality device is proposed, said device comprising at least:
- a display screen,
- an orientation sensor, and
- a position sensor distinct from the orientation sensor,
the method comprising:
a) generation of device orientation data within a real environment as a function of signals from the orientation sensor,
b) generation of device position data within the real environment as a function of signals from the position sensor, said signals from the position sensor being generated by:
b1) communication of said position sensor with at least one fixed position beacon within the real environment,
c) real-time display of virtual content associated with the real environment via the display screen, said displayed virtual content being a function of at least said orientation data and said position data of the device.

According to another aspect, there is proposed a non-transitory recording medium readable by a computer on which is recorded a program for the implementation of the method when this program is executed by a processor.

The features set out in the following paragraphs can optionally be implemented, independently of each other or in combination with each other:

In one embodiment, the orientation sensor comprises at least:
- a gyroscope and/or a gyrometer, and
- an accelerometer.

Consequently, the virtual reality device is capable of obtaining at least orientation data of the device in the real environment, for example when the device is raised or lowered or even turned by the user with respect to an initial orientation of the device in the real environment.

In one embodiment, the orientation sensor is connected to a first functional unit of the device capable of executing an algorithm for merging the orientation data of the device and captured by the orientation sensor. Such a first functional unit is, moreover, configured to determine, in real time, an estimated value of orientation of the device by merging the orientation data of the device sensed by the orientation sensor.

Consequently, the first functional unit of the device, also referred to as the "orientation module" of the device, takes raw orientation data as input as captured by the gyroscope(s) and/or gyrometers and accelerometers of the orientation sensor . Such orientation data can for example be linked to an angular velocity of the device when it is moved by the user in the real environment.

In one embodiment, the first functional unit is configured to filter the orientation data of the device at the output of the orientation sensor by applying a Madgwick filter.

Consequently, the device is able to process the orientation data so as to improve the precision of the orientation data captured. Advantageously, the application of a Madgwick filter allows analytical processing of the raw orientation data captured by the orientation sensor, unlike the probabilistic approach of other types of filters often applied, such as filters of Kalman. In particular, the Madgwick filter comprises an optimization of the coefficients applied to the output data on the one hand, of the gyroscope(s) and/or gyrometers of the orientation sensor and, on the other hand, of the accelerometer(s) of the orientation sensor , by gradient descent.

In one embodiment, the position sensor being configured to communicate with a plurality of position beacons, at least one position beacon among said plurality corresponding to an Ultra Wide Band tracking beacon, the position sensor and said tracking beacon communicate by Ultra Wide Band pulses.

Therefore, the device is able to communicate with position beacons positioned in the real environment as the device moves. In particular, such communication is not biased by a real indoor environment, unlike communications of the "geolocation and navigation by satellite system" or "GNSS" type, as is the case of "geopositioning by satellite" or " GPS” for example, which require outdoor use. Furthermore, the exchanges between the virtual reality device and the position beacons by ultra-wideband (or UWB) advantageously make it possible to avoid interference with other communication protocols implemented in the real environment, by Bluetooth or Wi-Fi for example. The use of ultra-wideband technology by the virtual reality device also allows communication to be maintained between the device and the real environment (via the position beacons) even when physical obstacles (such as walls, ceilings or other physical objects) exist between the device and the communication beacons.

In one embodiment, the position sensor being configured to communicate with a plurality of position beacons, at least one position beacon among said plurality corresponding to an infrared tracking beacon, the position sensor and said infrared tracking beacon communicate by infrared radiation.

Therefore, the virtual reality device is able to communicate with position beacons positioned in the real environment as the device moves. In particular, such communication has a precision optimized by the emission of infrared rays, for example by a laser transmitter. Such an infrared emission can for example be implemented at a time and in a direction predetermined by the infrared tracking beacons and picked up by the position sensor of the virtual reality device under specific reception conditions, specific to the location of the device in the real environment.

In one embodiment, the virtual reality device further comprising a second functional unit receiving as input a signal from the position sensor, the second functional unit is configured to deduce from said signal at least one measurement relating to an interaction between the sensor position tag and the at least one position beacon.

Consequently, the second functional unit, also referred to as “position module” of the virtual reality device, is capable of processing data relating to at least one interaction between the position sensor and the at least one position beacon. In particular, such an interaction is specific to the location of the position sensor (and a fortiori of the device) in the real environment. Such an interaction can for example correspond to a signal emitted, received and/or exchanged by the position sensor and received, emitted and/or exchanged by a position beacon. Such a measurement relating to such an interaction may correspond to a signal travel time, a signal arrival date, an angle of reception and/or transmission of the signal or even a distance measured between the position sensor and the position beacon.

In one embodiment, the signal comprising a time series of data, the second functional unit is configured to apply a nonlinear Kalman filter to the time series.

Therefore, the interactions between the position sensor and the at least one position beacon can result in a time series of data. The virtual reality device is then able to process and interpret such a time series.

In one embodiment, the second functional unit is configured to deduce in real time at least one estimated position value of the device as a function of the measurements relating to the respective interactions between the position sensor and each of said position beacons.

Consequently, the virtual reality device is able to interpret the measurements relating to the interactions between the position sensor and the position beacons so as to deduce therefrom position data of the device in the real environment. In particular, the plurality of interactions between the position sensor and the position beacons allows the device to obtain several position values, estimated in different ways. For example, the interactions of the position sensor and the position beacons by ultra-wideband can provide a first estimated position value of the device and the interactions of the position sensor and the infrared position beacons can provide a second estimated position value. of the device.

In one embodiment, the position sensor being configured to communicate with at least four Ultra Wide Band tracking beacons, the measurements being relative to respective distances between the position sensor and each of the Ultra Wide Band tracking beacons, the second functional unit is configured to determine a first estimated position value of the device by multilateration as a function of the distances.

Therefore, the interactions between the device's position sensor and the position beacons allow the virtual reality device to obtain a first estimate of its position in the real environment. Such a first position estimate is advantageously obtained in about ten milliseconds due to the ultra-short nature of the UWB pulses. Such a first position estimate of the device can be advantageously obtained regardless of the actual environment (interior or exterior in particular) in which the device is located and regardless of the relative positioning of the position sensor with respect to the respective positioning of each of the position beacons.

In one embodiment, the position sensor being configured to communicate with at least two infrared tracking beacons, the measurements corresponding to respective infrared radiation angles between the position sensor and each of said infrared tracking beacons, the second functional unit is configured to determine a second estimated position value of the device by triangulation as a function of said angles.

Therefore, the interactions between the device's position sensor and the position beacons allow the virtual reality device to obtain a second estimate of its position in the real environment. In particular, such a second position estimate is distinct from the first position estimate obtained.

In one embodiment, the orientation sensor and the position sensor also being connected to a third functional unit of the device, said third functional unit being capable of executing an algorithm for merging said orientation and position data of the device , said third functional unit is configured to estimate a positioning of the device by merging data relating to the orientation of the device and to the position of the device respectively.

In one embodiment, the fusion algorithm is applied on:
- the estimated orientation value of the device,
- the first estimated position value of the device, and
- the second estimated position value of the device,
so as to obtain the estimated positioning of the device.

Consequently, the third functional unit, also referred to as "positioning module" of the virtual reality device is able to deduce a real-time positioning of the device in the real environment. Such positioning may in particular comprise an estimation of an orientation of the device in the real environment and a position (or location) of the device in the real environment. In particular, the method advantageously makes it possible to obtain an estimate of the position taking into account several estimated position values. Such a fusion of data makes it possible to reduce the uncertainty, inaccuracy and noise associated in particular with the measurements of the orientation and position sensors of the virtual reality device.

In one embodiment, the device being also connected to a communication interface capable of receiving at least modeling data from a remote server, the device is configured to display the virtual content according to the estimated positioning and the modeling data.

Consequently, the reality device is capable of broadcasting specific virtual content on the display screen of the device, the specificity of such virtual content being adapted to the positioning of the device as estimated by the positioning module of the device .

In one embodiment, the device includes a graphics engine.

Consequently, the virtual reality device has autonomy in processing and broadcasting the displayed virtual content.

In one embodiment, the virtual content displayed comprises a marking associated with a detection of at least one predefined orientation datum among said orientation data and at least one predefined position datum among said position data of the device .

Consequently, the virtual reality device can autonomously modify and adapt the virtual content displayed, for example by including a specifically positioned marking in the virtual content. Such marking can be related to the positioning as estimated by the device.

Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the appended drawings, in which:

Fig. 1

shows a virtual reality device according to one embodiment.

Fig. 2

shows a system including a virtual reality device according to one embodiment.

Fig. 3

shows an implementation of a display of virtual content by a virtual reality device according to one embodiment.

Fig. 4

shows communication between the virtual reality device and location beacons according to one embodiment.

Claims (18)

Virtual reality device (EQ) comprising at least:
- a display screen (AFF), and
- an orientation sensor (CAPT-O) configured to output device orientation data (EQ) within a real environment (ENV),
characterized in that:
the device (EQ) further comprises at least one position sensor (CAPT-L) distinct from the orientation sensor (CAPT-O), said position sensor (CAPT-L) being configured to communicate with at least one beacon of fixed position (Ax1-Ax4, Bx1, Bx2) within the real environment (ENV), so as to output device position data (EQ) relative to the real environment (ENV),
the device (EQ) being configured to display, in real time, a virtual content associated with the real environment (ENV) via the display screen (AFF), said displayed virtual content depending on said orientation data and on said data of position. Device (EQ) according to claim 1, in which the orientation sensor (CAPT-O) comprises at least:
- a gyroscope and/or a gyrometer (GYR), and
- an accelerometer (ACC). Device according to Claim 3, in which the orientation sensor (CAPT-O) is connected to a first functional unit (MO) of the device (EQ) capable of executing an algorithm for merging the orientation data of the device (EQ) and sensed by the orientation sensor (CAPT-O), and in which said first functional unit (MO) is configured to determine, in real time, an estimated device orientation value (EQ) by merging data from orientation of the device (EQ) captured by the orientation sensor (CAPT-O). Device (EQ) according to Claim 4, in which the first functional unit (MO) is configured to filter the orientation data of the device (EQ) at the output of the orientation sensor (CAPT-O) by applying a filter of Madgwick. Device (EQ) according to one of the preceding claims, the position sensor (CAPT-L) being configured to communicate with a plurality of position beacons, at least one position beacon among said plurality corresponding to an Ultra Large tracking beacon Band (Ax1), and in which the position sensor (CAPT-L) and said UWB tracking beacon (Ax1) communicate by Ultra Wide Band pulses. Device (EQ) according to one of the preceding claims, the position sensor (CAPT-L) being configured to communicate with a plurality of position beacons, at least one position beacon among said plurality corresponding to an infrared tracking beacon ( Bx1), and wherein the position sensor (CAPT-L) and said infrared tracking beacon (Bx1) communicate by infrared radiation. Device (EQ) according to one of the preceding claims, further comprising a second functional unit (ML) receiving as input a signal from the position sensor (CAPT-L), and in which said second functional unit (ML) is configured to deduce from said signal at least one measurement relating to an interaction between the position sensor (CAPT-L) and the at least one position beacon. Apparatus (EQ) according to claim 8 taken in combination with claim 6, said signal comprising a time series of data, and wherein the second functional unit (ML) is configured to apply a non-linear Kalman filter to said time series. Device (EQ) according to Claim 8 taken in combination with one of Claims 6 and 7, in which the second functional unit (ML) is configured to deduce in real time at least one estimated position value of the device (EQ) by based on measurements relating to the respective interactions between the position sensor (CAPT-L) and each of said position beacons (Ax1, Bx1). Device (EQ) according to claim 8 taken in combination with claim 6, the position sensor (CAPT-L) being configured to communicate with at least four Ultra Wide Band tracking beacons (Ax1, Ax2, Ax3, Ax4), said measurements being relative to respective distances between the position sensor (CAPT-L) and each of said Ultra Wide Band tracking beacons (Ax1, Ax2, Ax3, Ax4), and in which the second functional unit (ML) is configured to determine a first estimated device position value (EQ) by multilateration as a function of said distances. Device (EQ) according to claim 8 taken in combination with claim 7, the position sensor (CAPT-L) being configured to communicate with at least two infrared tracking beacons (Bx1, Bx2), said measurements corresponding to angles of respective infrared radiation between the position sensor (CAPT-L) and each of said infrared tracking beacons (Bx1, Bx2), and in which the second functional unit (ML) is configured to determine a second estimated position value of the device (EQ ) by triangulation according to said angles. Device (EQ) according to one of the preceding claims, in which the orientation sensor (CAPT-O) and the position sensor (Tx) are also connected to a third functional unit (MP) of the device (EQ), said third functional unit (MP) being adapted to execute a fusion algorithm of said orientation and position data of the device (EQ), and in which said third functional unit (MP) is configured to estimate a positioning of the device (EQ) by merging data relating to the orientation of the device (EQ) and the position of the device (EQ) respectively. Device (EQ) according to Claim 13 taken in combination with Claims 4, 11 and 12, in which the said fusion algorithm is applied to:
- the estimated orientation value of the device (EQ),
- the first estimated position value of the device (EQ), and
- the second estimated position value of the device (EQ),
so as to obtain the estimated positioning of the device (EQ). Device (EQ) according to one of Claims 13 and 14, the device (EQ) also being connected to a communication interface (COM) capable of receiving at least modeling data from a remote server (SER) , and wherein the device (EQ) is configured to display the virtual content based on the estimated positioning and modeling data. Device (EQ) according to one of the preceding claims, in which the device (EQ) comprises a graphics engine (MG). Method implemented by a virtual reality device (EQ), said device comprising at least:
- a display screen (AFF),
- an orientation sensor (CAPT-O), and
- a position sensor (CAPT-L) separate from the orientation sensor (CAPT-O),
the method comprising:
a) generation of device orientation data (EQ) within a real environment (ENV) based on signals from the orientation sensor (CAPT-O),
b) generation of device position data (EQ) within the real environment (ENV) as a function of signals from the position sensor (CAPT-L), said signals from the position sensor (CAPT-L) being generated by:
b1) communication of said position sensor (CAPT-L) with at least one fixed position beacon within the real environment (ENV),
c) real-time display of virtual content associated with the real environment (ENV) via the display screen (AFF), said displayed virtual content being a function of at least said orientation data and said position data of the device (EQ). Method according to claim 17, in which the virtual content displayed comprises a marking associated with a detection of at least one predefined orientation datum among said orientation data and at least one predefined position datum among said position data of the device (EQ). Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to claim 17 or 18 when this program is executed by a processor (PROC).