KR101501487B1 - Depth information based Head detection apparatus and method thereof - Google Patents

Abstract

The present invention relates to a head detection technique, and more particularly, to a technique capable of detecting a plurality of heads within a recognition range by improving a head recognition range and a recognition accuracy in a depth image photographed by a depth camera. To this end, a depth image-based head detecting apparatus according to an aspect of the present invention detects a user area based on depth information data in a depth image, searches for a length direction from an upper end of the depth image, An image processing unit for detecting pixels and generating at least one region of interest around the detected uppermost pixel; And
And a head area determination unit for detecting a user head area that satisfies predetermined conditions among the at least one area of interest.

Description

[0001] The present invention relates to a depth-based head detection apparatus and method,

The present invention relates to a head detection technique, and more particularly, to a technique capable of detecting a plurality of heads within a recognition range by improving a head recognition range and a recognition accuracy in a depth image photographed by a depth camera.

In various imaging systems and image processing applications, it is advantageous to automatically recognize the position and / or orientation of the human head of the source image. For example, a user may interact with a program running on a computer system, e.g., a video game program, by moving his head within range of the imaging device. Alternatively, such a head recognition method may also be used in an imaging device to adjust parameters such as effective aperture, exposure time, depth of focus, to optimize for portraiture.

Interaction with computer systems and, in particular, input of data and commands is a generally known issue.

Conventionally, this interaction has occurred through physical input devices such as keyboards, mice, scroll wheels, pens, touch-screens, joysticks, game pads, etc., that generate signals in response to a user's physical actions. However, these physical input devices have a number of disadvantages. For example, they can provide only a limited amount of different input signals that make them feel awkward and scarce in some applications, such as three-dimensional "virtual reality" environments. Moreover, they are easy to carry and their continued use is even more

Negative effects on the user's health, such as RSI (Repetitive Strain Injury).

Alternative input devices and methods are also known. For example, real systems for speech recognition are available. However, speech recognition is not a practical alternative in some applications, such as action games where fast, concise and repetitive inputs are required by the user. In addition, their effectiveness is adversely affected by background noise, and they typically require a learning period to recognize a particular user's voice command.

Another alternative is image recognition. In their simplest form, image recognition systems recognize binary patterns of contrasting colors, such as bar codes, and convert these patterns into binary signals for processing.

More advanced image recognition systems are able to recognize more complex patterns in images and produce a wide variety of signals in response thereto. Such image recognition systems have been proposed, for example, in U.S. Patent No. 6256033 to recognize gestures of a user within the scope of an imaging system. However, conventional imaging systems have no depth recognition and can only generate the 2D projection of the user. As a result, the perception of user gestures is inherently flawed, limited by the range of possible inputs, and riddled with possible recognition errors. In particular, such systems have problems separating the user from his background.

However, the development of 3D imaging systems, for example, provides the possibility to develop shape recognition methods and devices that allow for better user gesture recognition. One such 3D imaging system is disclosed in G. Yahav, G. J. Iddam and D. Mandelboum, "3D Imaging Camera for Gaming Application ". The 3D imaging system disclosed in this paper is a so-called " Time-Of-Flight "in which a depth perception is obtained from the shape of the wavefront of light reflected from objects within the 3D imaging system TOF type. However, other types of imaging systems have also been proposed, such as stereo cameras, LIDAR, radar, sonar, and the like.

For example, in International Patent Application WO 2008/128568 A1, in order to select a subject such as a human body in a 3D image and to segment this object into a plurality of individual regions including the head, It has been proposed to capture images.

In US 7,203,356, among the various alternatives, it has been proposed to use an ellipse or ellipsoidal fitting to determine the position of the human head in the source image captured by the 3D imaging system. However, this prior art does not disclose how parameters of an ellipse or ellipsoid that models the head are obtained.

The object of the present invention is to provide a head detecting method capable of tracking and detecting the head of all users in a camera recognition range even in an environment where there is a change in illumination or in a situation where sufficient illumination is not given do.

According to an aspect of the present invention, there is provided an apparatus for detecting a head of a plurality of users in a depth image including depth information data for foreground and background,

 Detecting a user area on the basis of the depth information data in the depth image, detecting a top pixel in the user area by searching in a longitudinal direction from an upper end of the depth image, An image processor for generating at least one region of interest; And a head area determination unit for detecting a user head area that satisfies a predetermined condition among the at least one area of interest.

Preferably, the image processor detects at least one pixel positioned at the uppermost end of the user area by searching in the longitudinal direction at predetermined intervals from the left or right of the upper end of the depth image.

For example, the size of the ROI may be determined based on depth information data of the uppermost pixel.

For example, the predetermined condition may include a first condition that a ratio of the user area occupying the ROI is equal to or greater than a first set value, a second condition that the user area should be located at a center point position within the ROI, A fourth condition that the user region should be located below the region, and a fourth condition that the user region occupying the region of interest must satisfy the left-right symmetry with respect to the center point position in the region of interest .

Meanwhile, when the head area determining unit detects the first user head area, the head area determining unit deletes the depth information data for the user area corresponding to the first user head area.

At this time, the detection of the second user's head area other than the first user's head area may be performed in a state where the depth information data for the user area corresponding to the first user's head area is deleted.

Preferably, the image processing unit includes a foreground / background separator for separating the foreground and the background from the depth image based on the depth information, a peak detector for detecting a pixel located at the top of the user area, And a region of interest setting unit for generating a candidate region for head region detection around the highest point.

A depth image-based head detection method according to another aspect of the present invention is a method for detecting a head of a plurality of users in a depth image including depth information data on foreground and background, An image processing step of detecting a user area, searching for the topmost pixel in the user area by searching in the longitudinal direction from the top of the depth image, and generating at least one ROI around the detected topmost pixel; And a head region determination step of detecting a user head region that satisfies predetermined conditions among the at least one region of interest.

Preferably, the image processing step includes the steps of: detecting a user area by separating foreground and background based on the depth information data from the depth image; And detecting at least one pixel located at the uppermost end of the user area.

For example, the size of the ROI is determined based on the depth information data of the uppermost pixel.

As described above, according to the present invention, the head of all the users in the camera recognition range can be tracked, and the head of all users can be detected even when a plurality of subjects are simultaneously photographed.

In addition, since the user's head is detected by utilizing the depth information between the subject and the camera in the image photographed by the depth camera, it is possible to provide a strong head detection solution even in an environment where no illumination or illumination changes.

1 is a block diagram of a depth image-based head detection apparatus according to an embodiment of the present invention;
FIG. 2 is a block diagram showing an internal configuration of the image processing unit of FIG. 1. FIG.
FIG. 3 is a diagram illustrating an image obtained by separating only a user region as a result of foreground / background separation in a depth image. FIG.
4A and 4B are views illustrating a process of searching for a user area based on the left upper end of the separated image of FIG.
FIG. 5 is a diagram illustrating a user area search result of FIG. 4, illustrating a process of creating a region of interest for hair detection; FIG.
6A to 6C are diagrams illustrating a process of determining a head region in the ROI of FIG. 5;
FIGS. 7A and 7B are diagrams illustrating a process of searching for a user area for head detection from the beginning after deleting depth information for a user area in which a head area is detected; FIG.
8 is a flowchart illustrating a depth image-based head detection method according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, like reference numerals refer to like elements throughout. In the drawings, like reference numerals are used to denote like elements, and in the description of the present invention, In the following description, a detailed description of the present invention will be omitted.

1 is a block diagram of a depth image-based head detection apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a depth image-based head detection apparatus according to an embodiment of the present invention includes a depth image input unit 110, an image processing unit 120, and a head area determination unit 130.

The depth image input unit 110 receives image data captured by a depth camera mounted on the terminal. Depth cameras generate distance information to objects in a scene. An example is a camera using Time-of-Flight (TOF) technology. The depth camera emits an infrared or optical signal to the scene, measures the distance using the phase difference that the signal returns to the object, and outputs it as a depth image.

The image processing unit 120 detects a user area on the basis of the depth information data in the depth image input to the depth image input unit 110, searches in the longitudinal direction from the top of the depth image, And generates at least one region of interest around the detected uppermost pixel. Here, the user area means an image in which a background area is separated from a depth image in which foreground and background are taken together.

A specific configuration of the image processing unit 120 performing the above functions will be described in detail with reference to FIG. 2 is a block diagram showing an internal configuration of the image processing unit of FIG.

2, the image processing unit 120 may include a foreground / background separating unit 121, a peak detecting unit 122, and a region of interest setting unit 123.

The foreground / background separator 121 separates the object (foreground) and the background using the depth information data of each pixel obtained in the image photographed by the depth camera. This is for extracting the user area part from the photographed depth image. It stores the initial image taken when the depth camera is driven for the first time and removes the background by using the difference image with the current image to be processed do. The depth image in which the background is separated and only the user area is left is as shown in FIG.

The peak detecting unit 122 detects a peak pixel among the pixels corresponding to the user area in the depth image remaining in the user area only. Specifically, the peak detecting unit 122 detects at least one pixel located at the uppermost end of the user region by searching in the longitudinal direction at predetermined predetermined intervals from the left or right of the upper end of the depth image.

For example, as shown in FIG. 4A, the peak detecting unit 122 searches pixels in the longitudinal direction at regular intervals from the upper left corner of the depth image. Here, since the depth image is binarized, the user region and the other regions (separated background regions), that is, pixels corresponding to the respective regions, will have different binarized values.

The peak detecting unit 122 detects a pixel having a maximum height (y-axis direction) among the pixels corresponding to the user region by searching for pixel values in the longitudinal direction. Referring to FIG. 4B, the highest point (indicated by a red circle) in the user area (white area) may not exist in the longitudinal direction, and if it exists, the search result in any one of the longitudinal directions, Only one value will be detected.

The ROI setting unit 123 creates ROIs that are candidates for the head region based on the detected peak. Here, the ROI refers to a polygon generated based on the detected peak. For example, the ROI may be generated as a rectangle as shown in FIG. 5, and a ROI is generated for each detected peak.

At this time, the region of interest may be defined in advance to have a different size depending on the distance between the camera and the user. That is, the size of the ROI can be determined based on the depth information data of the detected peak pixel.

The head area determination unit 130 determines one of the candidate areas for the head area, that is, at least one area of interest, as the head area. At this time, the head area determination unit 130 determines, as a head area, a region of interest that satisfies the above conditions based on a predetermined condition.

Here, the above conditions are as follows, and what meaning the conditions have will be described with reference to Figs. 6A to 6C. FIGS. 6A to 6C are views illustrating a process of determining a head region in the ROI of FIG. 5. FIG.

(a) determining whether a ratio of the user area occupying the ROI is greater than or equal to a first set value,

(b) a second condition that the user area should be located at a center point position in the ROI

(c) a third condition under which the user region should be located below the region of interest,

(d) a fourth condition that the user region occupying the region of interest must satisfy the left-right symmetry with respect to the center point position in the region of interest;

First condition

The ratio of the user area occupying the ROIs must be equal to or greater than the first set value to be determined as the head ROIs. For example, the first set value may be defined as 70%. In the case of the ROI shown in FIG. 6A, the ratio of the user area (white area) occupying the ROI is not more than 70% none.

Second condition

The user area must be positioned at the center point position within the ROI area, so that the ROI area can be determined. Since the center point (indicated by a red dot) of the ROI shown in FIG. 6A is located in the user area (white area), the ROI satisfies the second condition.

Third condition

Since the human head is connected to the body, the user area (white area) must always exist in the lower part of the head. For example, in the case of the region of interest shown in FIG. 6B, since the user region is not positioned below the region of interest (whether the center of the lower segment is located in the user region among the rectangular outlines forming the region of interest) Can not be determined.

Fourth condition

The user region occupying the region of interest must satisfy the symmetry with respect to the center point position in the region of interest so that it can be determined as the head region.

The head area determination unit 130 may determine a region of interest that satisfies all of the first to fourth conditions as a head region or if the first to fourth conditions are selectively selected and the selected condition is satisfied It can also be determined as the head area. However, in order to improve the accuracy of the head area determination, it is desirable to determine a region of interest satisfying all of the first to fourth conditions described above as the candidate region.

If the head area determination unit 130 detects the first user head area, the depth information data for the user area corresponding to the first user head area is deleted.

The header region determination unit 130 determines whether the first user header region is different from the first user header region in a state where the depth information data for the user region corresponding to the first user header region is deleted do.

Referring to FIG. 7A, when the head area determining unit 130 detects the head area (first user head area) for the first user, it deletes the depth information data for the first user, Data is shown in red solid lines.

Based on experimental data, it is assumed that the human body has about three times the size of the head. Therefore, it is assumed that the human body has a width obtained by adding squares of the head region size on both sides of the detected head region (middle rectangle among red shapes), and deletes the corresponding depth information data.

Referring to FIG. 7B, it can be seen that the process of detecting the peak, the region of interest, and the determination of the head region are repeatedly performed for the remaining regions except for the deleted region.

The peak detection unit 122 restarts the search for the peak direction for the detection of the longitudinal direction from the beginning of the depth image (starting from the upper left corner again), even though there is another user in the vicinity of the position where the first user has completed the head region detection And to avoid the case of missing it.

That is, if the first user's head detection is completed, the first user can search from the area after the area where the first user is located. However, in this case, if there are several persons at the same position, I will start searching again from the beginning.

How the depth image-based head detection apparatus described above specifically detects the user's head will be described with reference to FIG. 8 is a flowchart illustrating a depth image-based head detection method according to an embodiment of the present invention.

Referring to FIG. 8, a depth image-based head detection method according to an embodiment of the present invention includes a step of inputting depth image data (S110), a step of separating a user area using foreground / background separation (S120) A step S140 of detecting the highest point in the user area on the basis of the detected maximum point, a step S140 of checking whether the highest point is detected in the search result, (S160) of determining a head region in the ROI, and deleting the depth information data corresponding to the determined head region (S170).

The detailed configuration of each step for providing the depth information-based head detection method has already been described in the head detection apparatus, and thus a detailed method thereof will be omitted.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is to be understood that the invention may be embodied in other specific forms. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

Claims (11)

An apparatus for detecting a head of a plurality of users in a depth image including depth information data for foreground and background,
Detecting a user area on the basis of the depth information data in the depth image, detecting a topmost pixel in the user area by searching in a longitudinal direction from an upper end of the depth image, An image processor for generating at least one region of interest; And a head area determination unit for detecting a user head area that satisfies predetermined conditions among the at least one area of interest,
The head area determination unit determines,
And deleting the depth information data for the user area corresponding to the first user head area when the first user head area is detected
Depth image based head detection device.
The image processing apparatus according to claim 1,
Detecting at least one pixel located at the uppermost end of the user area by searching in the longitudinal direction at predetermined predetermined intervals from the left or right of the upper end of the depth image
Depth image based head detection device.

2. The apparatus of claim 1,
The size of which is determined based on the depth information data of the uppermost pixel
Depth image based head detection device.
2. The method according to claim 1,
(a) determining whether a ratio of the user area occupying the ROI is greater than or equal to a first set value,
(b) a second condition that the user area should be located at a center point position within the ROI,
(c) a third condition under which the user region should be located below the region of interest,
(d) a fourth condition that the user region occupying the ROI must satisfy a left-right symmetry with respect to a center point position in the ROI
Depth image based head detection device.
delete The method according to claim 1,
The detection is performed on the second user head area different from the first user head area in a state where the depth information data for the user area corresponding to the first user head area is deleted
Depth image based head detection device.
The image processing apparatus according to claim 1,
A foreground / background separator for separating foreground and background based on depth information in the depth image,
A peak detecting unit for detecting a pixel located at the uppermost position in the user area from which the background is separated;
And a region of interest setting unit for generating a candidate region for head region detection around the highest point
Depth image based head detection device.
A method for detecting a head of a plurality of users in a depth image including depth information data for foreground and background,
Detecting a user region based on the depth information data in the depth image, detecting a top pixel in the user region by searching in a longitudinal direction from an upper end of the depth image, detecting at least one or more An image processing step of generating a region of interest; And a head area determination step of detecting a user head area satisfying predetermined conditions among the at least one area of interest,
Wherein the head area determination step comprises:
And deleting the depth information data for the user area corresponding to the first user head area when the first user head area is detected
Depth image based head detection method.
9. The image processing method according to claim 8,
Separating foreground and background based on the depth information data in the depth image to detect a user area,
And detecting at least one pixel located at the uppermost end of the user area by searching in the longitudinal direction at predetermined predetermined intervals from the left or right of the upper end of the depth image
Depth image based head detection method.
9. The apparatus of claim 8,
The size of which is determined based on the depth information data of the uppermost pixel
Depth image based head detection method.
9. The method according to claim 8,
(a) determining whether a ratio of the user area occupying the ROI is greater than or equal to a first set value,
(b) a second condition that the user area should be located at a center point position within the ROI,
(c) a third condition under which the user region should be located below the region of interest,
(d) a fourth condition that the user region occupying the ROI must satisfy a left-right symmetry with respect to a center point position in the ROI
Depth image based head detection method.

Images