DE102014224955A1 - Determining the position of an HMD relative to the head of the wearer - Google Patents

Abstract

Disclosed is a method for determining the position of data glasses on the head of the wearer, comprising: detecting the position of a physical feature of the head of the wearer of the data glasses relative to the data glasses by means of a sensor of the data glasses; Providing a stored position of the physical feature for a reference position of the data glasses; Displaying a representation in a contact-analog manner, taking into account the detected position and the stored position of the physical feature.

Description

The invention relates to a method for determining the position of a data glasses on the head of the wearer and a corresponding data glasses.

Today, data glasses (sometimes called head-mounted displays, called HMD) are known, with the help of the wearer of the data glasses information can be displayed. The data glasses are worn like ordinary glasses, which is used as a visual aid on the head. Compared to ordinary glasses, however, the data glasses include a display that is located near the user's eyes when wearing the data glasses. The display can include two partial displays, one for each eye. On the display, information may be displayed to the user in the form of text, graphics or mixtures thereof. In particular, the display may be semitransparent, that is configured in such a way that the wearer can also recognize the surroundings behind the display. Particularly preferably, the information is displayed to the wearer contact-analog, which is sometimes referred to as augmented reality. In this case, the carrier of the data glasses, the information is displayed at a location that is oriented to the location of an object in the area, so for example, adjacent to the object or this superimposed. In order to realize the contact analogy, typically the position of the object in the environment, the pose of the data glasses in relation to the object as well as the pose of the data glasses to the eyes or the head of the wearer must be known. Pose here describes both the (three-dimensional) position of the data glasses and the orientation (also referred to here as orientation or rotation) in up to three spatial directions of the data glasses.

Data glasses can be used in particular in vehicles to display contact-analog information. This information may concern other road users, objects around the vehicle or objects in the vehicle. In order to be able to display contact-analogous information, in most cases the pose of the data goggles must also be known here.

Inertial sensors are known (acceleration, gyroscope, magnetometer), the relative orientation of an electronic device, i.a. also typically data glasses, can determine.

Data glasses have become known, which likewise each include a camera, which in particular is sensitive to infrared light and makes the recordings in the direction of viewing of the wearer of the data glasses, see, for example WO 2013/012914 ,

Additional influences on the quality of a contact analog display include non-identical and non-static focal lengths, non-Euclidean, planar virtual displays (e.g., radially distorted virtual images), optical distortions of the display.

The contact analog display, in addition to knowing the pose of the data glasses and the position and sometimes orientation of the object, also requires a determination of position and sometimes orientation of the data glasses (both collectively referred to herein as pose) on the wearer's head relative to the head or eye. The orientation of the data glasses refers to a tilted or oblique placement or slipping of the data glasses. The position can be determined for a central point of the glasses. With the help of knowledge about the position and, if necessary, orientation of the data glasses, the contact-analogue display is made possible or substantially improved.

Even after a determination of the position and possibly also orientation of the data glasses on the head of the wearer, a new determination or tracking of the position and possibly orientation of the data glasses relative to the head or eye of the wearer is always necessary for: each new wearer; every new touchdown; every adjustment of the position of the glasses on the head.

The constant remeasurement of the position and possibly also the orientation of the data glasses in relation to the head of the wearer enables an improved, that is more accurate, contact-analogue display.

The invention has for its object to track the displacement of the data glasses with respect to the wearer's head, starting from a reference position of the data glasses, thereby improving contact analog displays.

The object is solved by the subject matters of the independent claims. Advantageous developments are defined in the dependent claims.

A first aspect of the invention relates to a method for determining the position of data glasses on the head of the wearer, comprising: detecting the position of a physical feature of the head of the wearer of the data glasses relative to the data glasses by means of a sensor of the data glasses; Providing a stored position of the physical feature relative to the data glasses for a reference position of the data glasses; Displaying a representation in a contact-analog manner, taking into account the detected position and the stored position of the physical feature. A physical feature of the head is, for example, a birthmark, the structure the skin (especially pigments or pores) or the configuration (points, indentations, other features of the shape) of bone. In some implementations, as a feature of the head, one of all features of the head may be used, except features of the eyeball (including iris, cornea, etc.).

It is therefore proposed here to detect the position of a physical feature of the head of the wearer of the data glasses and to compare it with a position of this physical feature detected for a reference position. This makes it possible to record the extent to which the data glasses have changed their position relative to the reference position and thus with respect to the central eye position. This information will be used hereinafter to adapt the display of a contact analog representation to the new position of the data glasses (and thus the new position of the display of the data glasses in front of the wearer's eyes). In other words, with changing positioning of the data glasses on the head (e.g., readjustment of the glasses, slides, etc.), this is consistently (i.e., permanently background) detected and the calibration of the contact analog display is tracked. This improves the accuracy and quality of the contact analog display of the data glasses. As a result, the displayed representations for the wearer of the data glasses appear more accurately assigned to the location of the environment for which the assignment is to take place, for example superimposed more precisely on the location.

In some implementations, it may be provided that the new pose (new position and new orientation) of the data glasses is determined when displaying the presentation in a contact-analogous manner. In other words, a change in the pose of the data glasses is determined from a pose determined for the reference position, wherein the change in the pose comprises a displacement of the position and a change in the orientation of the data glasses relative to the head of the wearer. For this purpose, in addition to the detected position of the physical feature, characteristics of the feature additionally detected by the sensor, such as rotation relative to the data glasses, can also be taken into account. Alternatively or additionally, the data goggles can comprise a plurality of sensors with the aid of which the position of a plurality of physical features is determined and from this the new pose (new position and new orientation) are derived.

The stored position of the feature of the head can be determined each time the data glasses are put on in a configuration sequence, or permanently stored for a user, so that the configuration sequence only has to be executed once (or possibly at longer intervals). Especially in the latter case results for the user the great advantage that not with each use of the data glasses again an elongated configuration sequence must be performed, but the data glasses can be used directly in principle. The detection of the characteristics of the head is done automatically and requires no further action by the user. Based on the recorded characteristics, the display of contact-analogue representations is adapted. Typically, the detection of the physical features and the tracking of the configuration for the contact analog display are not actively perceived by the user.

The reference position then results from the position in which the data glasses were when the configuration sequence was executed. The stored position of the physical feature is detected by the sensor while the data glasses are in the reference position. For the reference position also a pose can be determined, Referenzpose. For this purpose, if only one sensor is used, besides the position of the physical feature also other properties of the physical feature are stored, such as its structure. In operation, the sensor then also detects this further property of the physical feature, for example a rotation of the structure. If several sensors are used, it may also be sufficient to determine only the position of the respectively detected features, in order to derive a change of the pose for the display.

The initial configuration sequence can be made possible with machine learning, in particular with methods for optical See Through Calibration for contact-analogue Augmented Reality displays are performed, such. in the publication Single-point active alignment method (SPAAM) for optical see-through HMD calibration for augmented reality by Mihran Tuceryan and Nassir Navab.

The sensor of the data glasses, in some implementations, may include one or more of the following: optical sensor, particularly a camera, arranged and attached to the data glasses for detecting skin, mole, ear, or nose irregularities; Pressure sensors in one or both of the temples or the nose pad of the data glasses; Distance sensor for determining the distance to the eyeball, ears or cheeks; Sensor for detecting under-the-skin head features, in particular teeth or bones using electromagnetic waves or ultrasonic waves; System for detecting the displacement of a contact point of the data glasses with the head, in particular the nose pad; Laser sensor for detecting the displacement of the nose pad.

In some implementations, the contact analog representations are used to objects in the vicinity of a vehicle mark while the driver or another occupant of the vehicle is wearing the data glasses. The method further comprises, in such a case: determining the position and orientation of the data glasses in a vehicle; Determining the position of an object to be marked in the environment of the vehicle; wherein the representation to be displayed is designed to mark the object in the environment of the vehicle; The position and orientation of the data glasses and the position of the object to be marked are taken into account when displaying the display.

In order to enable this contact-analogous marking, it is thus necessary to determine the position of the data glasses in the vehicle and also to determine the position of the object to be marked in the surroundings of the vehicle.

The pose of the data glasses can be determined by means of cameras of the vehicle, which record the driver and the data glasses and recognize therein the pose of the data glasses. Alternatively or additionally, the data glasses can also comprise a camera whose images of the interior of the vehicle are then evaluated in order to determine the pose of the data glasses. Further, the smart glasses may include acceleration and yaw rate sensors to determine or update the pose. Also under investigation was the generation of a magnetic field in the vehicle, which is detected by sensors of the data glasses to determine the pose. Examples of such systems are in the applications DE 102014217961 . DE 102014217962 or DE 102014222356 described.

The position of the object in the vicinity of the vehicle is detected by means of vehicle sensors such as radar, lidar or cameras. Methods for this are known in the art.

BRIEF DESCRIPTION OF THE DRAWING

1 schematically shows a data glasses according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT

1 schematically shows a data glasses 1 according to an embodiment. The data glasses include a semi-transparent display with the partial displays 2 and 3 , respectively for the left and right eye. Furthermore, the data glasses include 1 a camera 5 , as well as a processing unit 6 that have a signal line with the two displays 2 and 3 as well as the camera 5 connected is. The camera 5 is arranged so that it continuously takes in operation a part of the right front side of the wearer of the data glasses. In some implementations, the processing unit is 6 separately from the data glasses and the transmission takes place over a longer signal line or wirelessly.

The data glasses also include the camera 20 which provides recordings in the direction of the central viewing direction if the wearer of the data glasses is wearing as intended. Also the camera 20 is with the processing unit 6 connected. The camera 20 is aligned according to the symmetry axis of the data glasses. The processing unit 6 continuously receives the camera recordings during operation 5 and 20 , The camera 20 is used to determine the pose (ie the position and orientation of the data glasses 1 in the vehicle). The pose of the data glasses is needed for the correct display of contact-analogue representations.

The data glasses 1 is designed to the wearer of the data glasses and drivers of a vehicle representations contact analog display, such as navigation instructions or markings of objects in the vicinity of a vehicle. In order to enable a high quality of the contact analogy, an initial configuration sequence is provided and then continuous updates of this configuration sequence using the camera 5 , In the initial configuration sequence, the location of the displays is determined using the SPAAM method 2 and 3 Markers must be displayed so that they appear to the wearer of the data glasses as a contact analog. The position in which the data glasses are located on the wearer's head to complete the configuration sequence is the reference position. At the end of the configuration sequence is using the camera 5 made a recording of the front side of the wearer of the data glasses. In this recording, skin features of the front side are detected. For this it may be necessary to identify mobile head features such as hair and to classify them as an unsuitable feature. Suitable features are, for example, pigments, moles or pores. The location of the appropriate features in the camera's image 5 will be saved.

In the following operation of the data glasses are continuously taking pictures of the camera 5 and determined the position of the previously determined suitable features in the images of the camera. Methods for tracking the features are known in the art and are also used, for example, in the light-based motion detection of computer mice.

The continuously detected positions of the suitable features relative to the stored positions are compared and from this a conclusion of the data glasses is deduced. The displacement of the data glasses can be a change of the relative position to the head in all three spatial directions and also a change in the orientation or the Orientation of the data glasses relative to the head include. Both components of a shift can be considered by the camera and downstream processing. Optionally, a second camera may be provided which takes pictures of the left front side of the carrier and in the same way as the images of the right front side by the camera 5 be processed and taken into account in the determination of the shift of the data glasses.

The initial configuration for displaying a contact-analogue representation on the displays 2 and 3 is adjusted according to the displacements of the data glasses. In this way, the quality and accuracy of the contact-analog representation can be improved.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2013/012914 [0005]
• DE 102014217961 [0021]
• DE 102014217962 [0021]
• DE 102014222356 [0021]

Claims (8)

 A method of determining the position of data glasses on the head of the wearer, comprising: Detecting the position of a physical feature of the head of the wearer of the data glasses relative to the data glasses using a sensor of the data glasses; Providing a stored position of the physical feature relative to the data glasses for a reference position of the data glasses; Displaying a representation in a contact-analog manner, taking into account the detected position and the stored position of the physical feature. Method according to one of the preceding claims, wherein the sensor of the data glasses comprises one or more of the following: - Optical sensor, in particular camera, which is set up and attached to the data glasses for detecting irregularities of the skin, birthmarks, ears or the nose; - Pressure sensors in one or both of the eyeglass temple or the nose pad of the data glasses; - Distance sensor for determining the distance to the eyeball, ears or cheeks; Sensor for detecting under-the-skin head features, in particular teeth or bones by means of electromagnetic waves or ultrasonic waves; - System for detecting the displacement of a contact point of the data glasses with the head, in particular the nose pad; - Laser sensor for detecting the displacement of the nose pad.  Method according to one of the preceding claims, wherein in displaying the representation, the displacement of the position of the data glasses from the reference position is taken into account, wherein the displacement is calculated using the detected and the stored position.  Method according to one of claims 1 to 2, wherein in displaying the display, a change in the pose of the data glasses is determined by a Referenzpose, comprising the reference position, wherein the change in pose a shift in position and change the orientation of the data glasses relative to the head of the Carrier comprises.  Method according to one of the preceding claims, wherein the stored position is determined by means of an adjustment process by interaction with the user of the data glasses.  Method according to one of the preceding claims, wherein when displaying a representation in contact-analogous manner, the detected position and the stored position of the physical feature are used to take into account both a change in position of the data glasses and a changed orientation of the data glasses.  The method of any one of the preceding claims, wherein the method further comprises: Determining the position and orientation of the data glasses in a vehicle; Determining the position of an object to be marked in the environment of the vehicle; wherein the representation to be displayed is designed to mark the object in the environment of the vehicle; The position and orientation of the data glasses and the position of the object to be marked are taken into account when displaying the display.  Data goggles comprising a sensor for detecting the position of a physical feature of the head of the wearer of the data goggles, the data goggles being adapted to carry out a method according to one of the preceding claims 1 to 6.

Images