WO2012019623A1

WO2012019623A1 - Method and device for controlling an apparatus having visual-field detection

Info

Publication number: WO2012019623A1
Application number: PCT/EP2010/004896
Authority: WO
Inventors: David Koch
Original assignee: Siemens Aktiengesellschaft
Priority date: 2010-08-10
Filing date: 2010-08-10
Publication date: 2012-02-16

Abstract

The control of apparatuses or applications should be carried out more accurately using visual-field tracking. For this purpose, the invention relates to a method in which visual-field data (7) of a person (6) are detected and position data and/or orientation data of the vision of the person are determined from said visual-field data. An apparatus or an application (2) is controlled according to the position data and/or orientation data. Because the vision of an observer (6) can be moved only within the limits of the given anatomy, the determined position data and/or orientation data are corrected on the basis of an anatomy model (1) before the apparatus or the application (2) is controlled.

Description

description

Method and device for controlling a device with visual field recognition

The present invention relates to a method for controlling a device by acquiring visual field data of a person, determining position data and / or orientation data of the person's face from the visual field data and controlling the device in dependence on the position data and / or alignment data. Moreover, the present invention relates to a corresponding device for controlling a device.

With the help of a commercially available webcam and low computing power, it is now possible to realize a stable visual field recognition. The visual field detection captures position and rotation data (or orientation data) of the human head. All six degrees of freedom of the viewer can thus be used in different applications (e.g., for navigation and selection in 3D scenes). For this purpose, e.g. in 3D scenes the viewer's real viewing point is coupled to the virtual viewing point in the SD model. This possibility is in principle similar to a navigation with a typical so-called "space mouse", which can control six degrees of freedom compared to a classical PC mouse at the same time.

In contrast to navigation with a "Space Mouse" is the approach using the visual field tracking significantly günsti ^¬ ger and more intuitive for the user. The possibility of a stable, fast and resource saving face recognition (based on a webcam) is technically still relatively new. The company Seeing Machines, for example, has been developing applications for tracking visual fields for several years, and a resource-efficient application based on sophisticated algorithms is only possible with the current generation of user computers currently used for monitoring people and analyzing personal behavior.

The object of the present invention is thus to propose a method and a device for controlling a device, in which with field tracking improved accuracy can be achieved.

According to the invention, this object is achieved by a method for controlling a device by acquiring facial field data of a person, determining position data and / or alignment data of the person's face from the visual field data and controlling the device in dependence on the position data and / or alignment data, as well as by Correcting the determined position data and / or alignment data based on an anatomy model or predetermined correction data before the step of controlling. Such correction data can be determined individually from the movements of a person.

In addition, the invention provides an apparatus for controlling a device having a detection device for acquiring visual field data of a person, a computing device for determining position data and / or orientation data of the person's face from the visual field data and a control device for controlling the device in dependence on the position data and / or alignment data, wherein the calculated position data and / or alignment data can be corrected on the basis of an anatomy model for the control device with the computing device. A "device" is also understood here as an application or application.

Advantageously, the anatomy of a person is thus taken into account in the acquisition of visual field data. Thus, the determined position data and / or alignment data can be cleaned up and achieve increased accuracy. Preferably, a correction matrix is calculated using the anatomy model or the correction data with which the Posi ^¬ tion data and / or orientation data is corrected. Such a correction matrix can be calculated beforehand by a standardized model, so that the correction can be carried out with comparatively low computing power. The term "matrix" can also be understood here to mean a table, a list or the like.

The anatomy model can be created individually for a person. This makes corrections very person-specific. In particular, this can take into account, for example, the size of a person.

Correcting can also be done in response to a person's movement. Moreover, the correction may also be made depending on a posture of the person. In this way, other individual factors may be included in the correction that will improve the overall outcome of Steue ^¬ tion.

In a particular embodiment, the person's face assumes an actual position and / or actual orientation which differs from a respective desired position and / or desired orientation, wherein the actual target difference is taken into account in the correction matrix. Thus, the difference between the actual position and the desired position or the difference between the actual orientation and the desired orientation is calculated and entered into the correction matrix accordingly. In this way, for example, take into account when the controlling person can move their face only in a certain area, because, for example, obstacles in the way.

The correction matrix can be two-dimensional or multi-dimensional. To correct all translatory degrees of freedom alone, the correction matrix should be three-dimensional. In addition, for example, if angles are corrected in two directions, so the correction matrix would have to be five-dimensional. When correcting all translational and rotational degrees of freedom, the correction matrix must be six-dimensional.

The transition from one correction data set to another correction data set of the correction matrix can be done smoothly by means of interpolation. As a result, a continuous control of the device without jumps can be realized.

Furthermore, not only the determined position data and / or orientation data can be corrected with the correction ^matrix , but also further parameters for controlling the device. For example, if only the translational data are determined when the movement of the face is detected, then the rotational variables can also be corrected.

The present invention will now be explained in more detail with reference to the accompanying drawings, in which:

1 shows a schematic diagram for the correction of facial recognition data using an anatomical model ge ^¬ Mäss the present invention;

2 shows a simple three-dimensional correction matrix;

3 shows a five-dimensional correction matrix.

The embodiments described in more detail below represent preferred embodiments of the present invention.

The cause of the inaccuracy of navigation using a conventional visual field tracking is in the correct Inter ^¬ pretation the coordinates of face recognition. In contrast to a "space mouse", the observer's head has no fixed point from which all axes (degrees of freedom) are controlled. tomiesiraulation 1 considered in FIG. In a head-sideways movement of a seated observer, the vertical X-position of the head shifts by a value dx and the associated rotation of the head by a value there, depending on the width. This conditional "falsification" in the sideways movement is based on the rotation of the upper body around the sitting position of the observer.The fixed point in a sideways movement therefore lies with a seated observer in the lower spinal column area.In addition, two falsified position data x and α (um dx and dax falsify, where dax represents an angular deviation from the vertical x-axis about the z-axis perpendicular to the image plane.) These falsified values are due to human anatomy and are misinterpreted in a direct coupling to the application An intuitive and exact navigation in the application is not possible.

Therefore, according to the invention, there is a posture-dependent cleaning of the coordinates from the visual field recognition for intuitive and exact navigation and control of applications (for example 3D scenes 2). These include, on the one hand, the generation of a correction matrix 3 and, on the other hand, the correction 4 of the visual field recognition 5 with the aid of this correction matrix 3.

Depending on the posture of the observer 6 (sitting, standing, etc.), prefabricated correction matrices 3 are initially created. These are multidimensional matrices, which provide correction data for an adjusted position, depending on the detected position of the visual field 7 and optionally also on the current movement of the observer 6.

In the section 8 of FIG.1, the coupling of a real viewpoint (viewpoint) and a virtual Betrach ^¬ tung point is shown symbolically. A webcam 9 detects the movement 10 of the observer's face 6. Then, with the corrected visual field data, a virtual Movement 11 performed for example in the 3D scene 2. However, it can also be controlled in this way, another device, another component or system. 2 shows a three-dimensional correction matrix 3, which the correction data translational (dx, dy, dz) and rotational (dccx, day, daz, ie angle deviations relative to the x, y and z axis) in dependence on the translational position of the viewer 6 defined. The correction matrix 3 contains a plurality of correction data sets 12. In vorlie ^¬ constricting case Korrekturda ^¬ cost rate is 12 for each x, y, z-Triple provided. However, the correction data set 12 has not only correction values for the translatory values x, y and z, but also for the rotational values ax, ay and az.

A more comprehensive and more accurate correction enables the five-dimensional correction matrix 3 'of FIG 3. This is evaluated to ^¬ additionally to the translational position and the rotatory see position of the visual field, to thereby divide the zuwendenden at ^¬ correction data of all six degrees of freedom finer. Each correction element 12 'achievable with the input variables x, y and z thus contains a correction function 13 over a two-dimensional ax-ccy space for each parameter x, y, z, x, ay and az. Corresponding correction values thus result for each five-dimensional coordinate dx, dy, dz, dax, day and daz.

Depending on the application and the accuracy to be used, the correction matrix 3 or 3 'can be one to n-dimensional matrices which define the respective correction data to be applied as a function of the position data taken into consideration (dimension of the matrix and value range). Further conceivable dimensions of the correction matrix can be the attitude or course of movement of the observer. The consideration of the human anatomy model during the creation of the correction matrix takes place, for example, by means of a kinematics simulation. In doing so, an anatomy model of a human being, as it is already used today for the investigation of ergonomics questions (eg "Jack", "RAMSIS" or "Anybody"), is simulated in predetermined postures (sitting, standing)

(http://en.wikipedia.org/wiki/Inverse Kinematics) of the human model, the correction data are determined. For this purpose, possible head positions of the human model are given in a defined grid. The inverse kinematics of the human model then determine to what extent this head position can be reached by the person at all. In addition, a restriction of the degrees of freedom and an ergonomic evaluation provide a typical illustration of human freedom of movement and posture. The ergonomic evaluation of the posture is evaluated in the evaluation of whether a target position is achievable with. An attitude is e.g. as unavailable if it can be reached by the human model kinematically (with inverse kinematics), but falls ergonomically below a medical assessment limit.

If a desired position can only be reached to a limited extent (connected to translational or rotational displacements), then the deviation from the desired position in the correction matrix is recorded at the actual position of the observer in order to be able to determine the desired position later. As already explained in connection with the anatomy Simulation 1 of FIG 1, it comes at a horizontal ^¬ Since downward movement of the person along the y-axis to a distortion of the X-position and the A-rotation of the head. These correction values are recorded in the correction matrix 3 in conjunction with the actual position of the person in order to be able to determine the desired position when the correction matrix is used in practice.

The following are some usage examples where a correction matrix is used to control an application can be used with a visual field recognition. As suggests above, an application on the basis of a face recognition, the calculated position data are adjusted using a correction matrix, ge ^¬ be controlled. In addition, typical (anatomically-related) motion sequences of a user in front of a ^camera for controlling an application can be interpreted more accurately by the use of the correction data. Intuitive gestures to control an application are correctly interpreted by the application despite restrictions on movement in the workplace and human anatomy. In addition, correction matrices are attitude-dependent and differentiate, for example, between standing and sitting users for optimal correction of the position data. The correction matrix may also include correction data for achieving the desired visual field position of the user in dependence on the real actual position of the user visual field. Furthermore, about which he knew ^¬ actual position of the field and the correction data of the correction matrix to calculate the user's desired position. The nominal position takes account of anatomical, ergonomic, posture-related and workplace-related restrictions of the movement of a user compared to the actual position. The transition from one correction data set to the next within the matrix (in the case of a coarse subdivision of the correction matrix) preferably takes place smoothly by means of interpolation.

In the following the creation of a correction matrix is summarized again, whereby the individual points are to be regarded as optional:

The correction matrix is created using an extended ergonomics simulation.

For the creation of a correction matrix are existing, standardized, anatomically and ergonomically valid

Using human models (eg "Jack", "RAMSIS" or "Anybody" make it possible to generate correction data quickly and easily on a well-founded basis). The human model takes into account the anatomy, the free ^¬ degrees of freedom of the joints and an ergonomic use of postures to account for typical and comfortable conversation about the correction matrix.

Motion simulation (advanced ergonomics simulation) takes into account the limited degrees of freedom of movement due to the work environment (e.g., table, chair, chair back, railing, etc.).

The target positions to be examined are to be selected such that typical movements around the zero position (rest position) of the user are covered in order to cover the necessary degrees of freedom for controlling the respective application.

Using inverse kinematics, the simulation firstly investigates the attainment of the desired positions of the visual field and determines the differences in the degrees of freedom to the achievable actual positions (kinematic analysis).

Parallel to the pure kinematic analysis (the anatomy) an ergonomic evaluation of the respective posture takes place.

The ergonomic evaluation of the posture is evaluated when a target position is reached.

An attitude is also considered unavailable if it can be achieved by the human model kinematic (with inverse kinematics), but falls ergonomically below a medical assessment limit.

As soon as a desired position can not be achieved completely kinematically or ergonomically, correction data is recorded in the correction matrix at the location of the actual position of the user actually achieved.

After the definition of the target position data to be examined, the workplace environment and the anatomy adaptation of the human model, the structure of the correction matrix can be carried out automatically by simulation.

The automatic correction matrix creation makes it possible to create an individual correction matrix. This is adapted to the individual anatomy and the ^{degrees of} freedom of movement of a specific work environment.

The user can choose how finely granular the resolution of the correction data should be.

The dimensions of the correction matrix is selectable by the user himself and thus takes into account the individual Freedom ^¬ grade that are evaluated for correction. Thus, applications for different scenarios can be optimized (eg control of esse demonstrators for laymen or applications for experienced CAD users).

Depending on the application, different correction matrices can be used, as not all degrees of freedom of the head position are always evaluated (eg 2D and 3D operation).

LIST OF REFERENCE NUMBERS

Anatomy simulation 3D scenes

Correction matrix correction

Gesichtsfeiderkennun

observer

Facial field

section

webcam

Move

virtual movement Correction data sets Correction element Correction function Angular deviation Angular deviation Falsification

Misleading

Claims

claims

A method of controlling a device (2)

- acquiring facial data (7) of a person,

- Determining position data and / or orientation data of the face of the person (6) from the visual field data (7) and

Controlling the device (2) in dependence on the position data and / or alignment data,

marked by

- Correcting (4) the determined position data and / or alignment data on the basis of an anatomy model (1) or predetermined correction data before the step of the control.

2. The method of claim 1, wherein using the Anatomiemo ^¬ dells (1) or of the correction data, a correction matrix (3) is calculated, with which the position data and / or orientation data is corrected.

3. The method of claim 1 or 2, wherein the anatomical model (1) is created individually for the person.

4. The method according to any one of the preceding claims, wherein the correcting (4) also takes place in dependence on a movement of the person (6).

5. The method according to any one of the preceding claims, wherein the correcting (4) also takes place as a function of a body attitude of the person (6).

6. The method according to any one of claims 2 to 5, wherein the face of the person (6) occupies an actual position and / or actual orientation, which differs from a respective desired position, and the actual target difference in the Correction matrix (3) is taken into account.

7. The method according to any one of claims 2 to 6, wherein the correction matrix (3) is two- or multi-dimensional.

8. The method according to any one of the preceding claims, wherein the transition from one correction data set (12) to another ren correction data set of the correction matrix (3) is carried out by means of interpolation.

9. The method according to any one of the preceding claims, wherein with the correction matrix (3) not only the determined Positi onsdaten and / or alignment data are corrected, but also other parameters for controlling the device (2).

10. Device for controlling a device (2) with

- A detection device (9) for detecting visual field data (7) of a person (6),

- A computing device for determining position data and / or Ausrichtungsdäten the face of the person (6) from the visual field data (7) and

a control device for controlling the device (2) in dependence on the position data and / or alignment devices,

characterized in that

- With the computing device, the determined position data and / or alignment devices on the basis of an anatomy model (1) for the control device can be corrected.