KR101967858B1 - Apparatus and method for separating objects based on 3D depth image - Google Patents

Abstract

The present invention relates to an object separating apparatus based on a 3D depth image which can be utilized for an image security service such as object detection based on 3D image data, image event detection, etc. by effectively separating an object from a depth image obtained through a 3D camera, and a method thereof. To this end, the apparatus comprises: an image input unit for inputting a 3D depth image; and an object separation unit for performing grouping between pixels having similar pixels among pixel values for detected object after detecting the number of objects and an object position by using the depth information of the input 3D depth image and sequentially removing a background area and a ground area except a grouped object area so as to extract a final object.

Description

[0001] The present invention relates to a 3D depth image-based object separating apparatus and a method thereof,

The present invention relates to a 3D depth depth based image-based object separating apparatus and method thereof, and more particularly to a 3D depth depth-based object separating apparatus and method for efficiently separating an object from a depth image obtained through a 3D camera, And to a 3D depth image-based object separation apparatus and method thereof that can be utilized in an image security service.

In general, various applications for recognizing a user's hand movements, i.e., gestures, using a camera have been proposed. The application accurately recognizes the meaning of the gesture of the user, and executes a command or process corresponding to the recognized meaning.

The recognition of these gestures has been mainly studied in the field of human-computer interaction. Various attempts have been made to recognize gestures such as remote control using a smartphone, communication, and execution of games and applications.

However, although various methods of recognizing a user's gesture using a camera have been proposed, recognition of such a gesture may be limited or inaccurate in some cases. For example, if the user's hand overlaps with the user's face or another object having a color similar to the user's skin color, it is difficult to separate the user's hand from the face or other object in the information of the image acquired by the camera, Therefore, there is a problem that the user can not accurately recognize the actual gesture. In other words, it is necessary to detect an accurate object to recognize the gesture of the object.

In addition, the general 2D-based object separation technology has difficulties in separating objects from the background in various environmental changes such as errors due to the surrounding environment, difficulty in recognizing objects at night, and errors caused by degradation of illumination / image quality. In the image security system, when an event is detected based on a 2D image, a problem of low accuracy occurs.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems and it is an object of the present invention to effectively separate an object from a depth image input from a 3D camera in an environment where various environmental changes exist, The object of the present invention is to provide a 3D depth image-based object separation apparatus and a method thereof, which can improve the accuracy of risk event recognition performance by applying the present invention to an image security system for detecting an object.

According to an aspect of the present invention, there is provided an apparatus for separating objects based on 3D depth images, comprising: an image input unit for inputting 3D depth images; And depth information of the input 3D depth image to detect the position of the object and the number of objects and then perform inter-pixel grouping having similar pixel values among the pixel values of the detected objects, And an object separator for sequentially removing the background area, the ceiling, and the ground area to extract a final object.

The apparatus may further include an image state determination unit for determining a state of the 3D depth image input from the image input unit and providing the 3D depth image to the object separation unit when noise is included in the 3D depth image.

Wherein the object separator comprises: an object detector for detecting the position of the object and the number of objects from the 3D depth image including the noise; A background region removing unit configured to remove a background region (noise) excluding an object using a pixel similarity of the object detected by the object detecting unit; And a ceiling and floor area removing unit for sequentially removing the ceiling area and the ground area from the image from which the background area is removed through the background area removing unit to extract the final object.

The object detection unit can detect the position and the number of the object using the depth learning object detection method through the depth information of the input 3D depth image and acquire and store the position coordinate value of the object.

The background area removing unit may remove a background area (noise) using a CCA (Connected Component Analysis) method.

The ceiling and floor area removing unit may convert the depth image from which the background area is removed in the background area removing unit into three-dimensional coordinates, arbitrarily select three points among the three-dimensional coordinates, determine a plane using the plane equation A ceiling, and a floor from a ground to a specific threshold are determined as a plane, the number of points included in the threshold in the calculated plane is calculated, the above operation is repeated a predetermined number of times, The area can be removed by determining the floor and ceiling.

According to another aspect of the present invention, there is provided a 3D depth image-based object separation method comprising: inputting a 3D depth image; And depth information of the input 3D depth image to detect the position of the object and the number of objects and then perform inter-pixel grouping having similar pixel values among the pixel values of the detected objects, And sequentially removing the background area, the ceiling, and the ground area to extract a final object.

The method may further include determining a state of the input 3D depth image. When the noise is included in the 3D depth image as a result of the determination, the 3D depth image may include noise, .

Wherein the extracting of the object comprises: detecting the location and number of objects of the object from the 3D depth image containing noise; Removing a background region (noise) excluding an object by using a pixel similarity degree of the detected object; And sequentially removing the ceiling area and the ground area from the image from which the background area is removed to extract the final object.

The step of detecting the position and the number of objects of the object may detect the position and the number of the object using the depth learning object detection method through the depth information of the input 3D depth image and acquire and store the position coordinate value of the object .

The step of removing the background region may remove a background region (noise) using a CCA (Connected Component Analysis) method.

The step of extracting the object may include the steps of transforming the background image into a three-dimensional coordinate, selecting arbitrary three points among the three-dimensional coordinates, and determining a plane using the plane equation; Determining a plane from the ceiling and the ground to a specific threshold value, and calculating the number of points included in the threshold in the calculated plane; And repeating the steps a predetermined number of times to determine a plane including a greatest number of points among the number of repetitions as a ground and a ceiling to remove the corresponding region.

According to the present invention, by effectively separating an object from a depth image input from a 3D camera in an environment where various environmental changes exist, it is applied to an image security system that detects an image event based on 3D image data, Can be improved.

In addition, in the environment where various environmental changes according to the present invention exist, the object can be effectively separated from the background of the depth image input from the 3D camera, so that it can be effectively applied in the preprocessing of the 3D image based human body recognition system, And extracts the object of interest from the object, and effectively analyzes the behavior of the object.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic block diagram of a 3D depth image-based object separating apparatus according to the present invention; FIG.
FIG. 2 is a detailed block diagram of the object separator shown in FIG. 1. FIG.
3 is a view illustrating an example of 3D depth image-based object separation according to the present invention.
4 is a flowchart illustrating an operation of a 3D depth image-based object separation method according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. And the present invention is defined by the description of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that " comprises, " or "comprising," as used herein, means the presence or absence of one or more other components, steps, operations, and / Do not exclude the addition.

First, prior to the description of the present invention, the present invention for separating objects from the background in 3D image-based action recognition technology is a key element for accurately performing image security services such as object detection, object tracking, and image event detection.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A 3D depth image-based object separating apparatus and method according to the present invention will now be described in detail with reference to the accompanying drawings.

FIG. 1 is a schematic block diagram of a 3D depth image-based object separating apparatus according to the present invention.

As shown in FIG. 1, the 3D depth image-based object separating apparatus according to the present invention may include an image input unit 10, an image state determination unit 20, and an object separation unit 30.

The image input unit 10 inputs the acquired 3D depth image to the image state determination unit 20. Here, the image input unit 10 may be at least one of at least one 3D camera and a 3D image sensor, and the 3D depth image may include depth information.

The image state determination unit 20 determines the image state of the 3D depth image input from the image input unit 10 and performs the following object separation operation when the image state is not good, that is, do. If the image is in good condition, it performs various actions such as the action of recognizing only the object of the input depth image without performing the separate object separation operation.

If it is determined that the image state is not good as a result of the image state determination unit 20, the object separation unit 30 detects the object position and the number of objects from the depth image input as shown in FIG. And then performs inter-pixel grouping with similar pixel values among the pixel values of the detected objects.

After the inter-pixel grouping is performed, the object separating unit 30 removes the image areas other than the per-group image, and then removes the ceiling and the ground area of the object, thereby separating and extracting the final desired object.

Hereinafter, the detailed configuration and operation of the object separation unit 30 shown in FIG. 1 will be described with reference to FIGS. 2 and 3. FIG.

FIG. 2 is a detailed block diagram of the object separating unit shown in FIG. 1, and FIG. 3 is a view illustrating an example of 3D depth image-based object separation according to the present invention.

2, the object separating unit 30 includes an object detecting unit 31, a background area removing unit 32, a ceiling area removing unit 33, a ground area removing unit 34, and an object extracting unit 35).

The object detecting unit 31 detects the position and the number of objects in the depth image using the depth information from the depth image including noise as shown in FIG. 3 (a), and then detects the coordinates of the detected object . Here, as a method of detecting the positions of objects and the number of objects, a deep learning based object detection method with a high object detection speed can be used. That is, based on the depth information of the inputted depth image, the position and the number of objects are detected according to the modeling value modeled based on various depth information, and then the coordinate values of the detected objects are acquired will be. Here, the deep learning-based object detection method is a general method that is well known in the art, and a detailed description of the method is omitted. It should be understood that an object may be detected using a method other than this method. Here, the screen of the object detection result in the object detection unit 31 is as shown in FIG. 3 (b).

The background region removing unit 32 performs a CCA (Connected Component Analysis) algorithm on the basis of the coordinate value of the object detected by the object detecting unit 31. That is, the CCA algorithm performs pixel grouping with similar pixel values by analyzing similar pixel values connected based on the pixels corresponding to the coordinate values of the object obtained by the object detecting unit 31.

The CCA algorithm has developed thresholding-based binarization techniques for finding and analyzing objects in the input image, and mathematical morphology techniques have been developed to remove noise from the binarization results and refine the results . Morphology Even rough images can distinguish objects and recognize objects. Before that, pixels must be separated into objects. The pixels constituting one object are displayed in a form connected to each other in the image. The CCA algorithm is an analysis method of grouping similar pixels connected to each other and distinguishing objects. Here, the CCA algorithm will be described in more detail as follows.

The CCA algorithm can remove a background area (noise) other than an object by using a median filter that performs a spatial filtering method. Here, the median filter determines the mask size, arranges the neighboring pixel values corresponding to the determined mask size in order of magnitude, and selects a middle value among them. However, if only the median filter is used in the background region removing unit 32, the boundaries of objects included in the depth image will be collapsed.

Accordingly, the background region removing unit 32 may additionally use a morphology erosion algorithm and a morphology dilation algorithm in addition to the median filter. The morphology erosion algorithm is used to strip a layer from an object or a small particle, or to remove an inappropriate pixel or small particle in an image. Conversely, the morphology expansion operation serves to extend a layer from an object or a small size particle, or to expand a particle that has been reduced by the morphology erosion algorithm to its original size.

The background region removing unit 32 may apply the morphology erosion algorithm to the obtained depth image, apply the median filter as the spatial filtering, and then apply the morphology dilation algorithm to effectively minimize the noise of the depth image .

As a result, the background area removing unit 32 removes the surrounding background area based on the detected object, and then provides the removed area image to the ceiling area removing unit 33.

The ceiling area removing unit 33 removes the ceiling area from the image from which the background area is removed through the background area removing unit 32 as shown in FIG. That is, even if the background area excluding the object area is removed after performing the CCA algorithm in the background area removing unit 32, the upper part of the object is determined to be the object area, and the probability of not being removed is high. Therefore, in the ceiling area removing unit 33, the ceiling area is calculated through the plane equation to remove the ceiling area.

The floor area removing unit 34 removes the floor area based on the object from the image in which the ceiling area is removed through the ceiling area removing unit 33 as shown in FIG. 3 (c) . That is, even if the remaining region excluding the object region is removed after the operation of the CCA algorithm in the background region removing unit 32, since the lower portion of the object is determined as the object region and the probability of not being removed is high, Thereby removing the ground area.

The method of removing the ceiling area and the ground area in the ceiling area removing unit 33 and the ground area removing unit 34 will be described in more detail. First, in the background area removing unit 31, The removed depth image is converted into three-dimensional coordinates, arbitrary three points among the three-dimensional coordinates are selected, and the plane is determined using the plane equation.

Then, from the ceiling and the ground to a specific threshold, a plane is defined, and the number of points included in the threshold in the calculated plane is obtained.

Then, such an operation is repeated a predetermined number of times, and the plane including the most points among the results of repetition is determined as the ground and the ceiling, and the corresponding region is removed.

In the above description, the ground area is removed after removing the ceiling area. However, it should be understood that the ground area may be removed first, and then the ceiling area may be removed.

The object extracting unit 35 extracts only a substantial object from the image in which the ceiling area and the ground area are removed as described above.

A 3D depth image-based object separation method according to the present invention, corresponding to the operation of the 3D depth image-based object separation apparatus according to the present invention, will be described step by step with reference to FIG.

4 is a flowchart illustrating a 3D depth image-based object separation method according to an exemplary embodiment of the present invention. Referring to FIG. 4, a 3D depth image obtained through a 3D image sensor or a 3D camera is input (S401). That is, it is determined whether noise is contained in the input 3D depth image (S402).

As a result of the determination, when the image state is not good, that is, when there is a lot of noise or a complex background, the following object separation operation is performed. If the image state is good, an object is immediately extracted from the image without performing a separate object separation operation (S407), and various operations such as an operation using the extracted object are performed.

If the input 3D depth image includes noise, the position and the number of objects in the depth image are detected using the depth information from the input depth image, and then the coordinates of the detected object are detected (S403). Here, as a method of detecting the positions of objects and the number of objects, a deep learning based object detection method with a high object detection speed can be used.

In step S403, the object position and the number of objects are detected, and then the noise (background area) of the depth image is removed by performing the CCA algorithm based on the coordinate values of the detected object (S404). That is, the CCA algorithm analyzes the similar pixel values based on the pixels corresponding to the coordinate values of the object acquired in the step S403, and performs grouping between pixels having similar pixel values, The background area is removed.

If the background area (noise) is removed through the CCA algorithm in step S404, the ceiling area is removed from the image from which the background area is removed (step S405). That is, the ceiling area is removed from the image in which the background area is removed through the CCA algorithm in step S404 as shown in FIG. 3 (c). Here, even if the background area excluding the object area is removed after performing the CCA algorithm, since the upper part of the object is determined as the object area and the probability of not being removed is high, the ceiling area is calculated through the plane equation to remove the ceiling area will be.

In step S405, when the ceiling area is removed, the ground area based on the object is removed from the image in which the ceiling area is removed (S406) as shown in FIG. That is, even if the remaining area excluding the object area is removed after performing the CCA algorithm in step S404, since the lower part of the object is determined to be the object area and the probability of not being removed is high, the ground area is calculated through the plane equation, .

In the method of removing the ceiling area and the ground area, first, the depth image from which the background area is removed is transformed into three-dimensional coordinates through the CCA algorithm, arbitrary three points among the three-dimensional coordinates are selected, Determine the plane.

Then, the number of points included in the threshold is calculated from the ceiling and the ground to a specific threshold, and the number of points included in the threshold is calculated. Then, the above operation is repeated a predetermined number of times, Is determined by the floor and the ceiling, and the area is removed.

In the above embodiment, the ceiling area is removed first and then the ground area is removed. However, it should be understood that the ceiling area may be removed after the ground area is removed first.

If the ceiling area and the ground area are removed through steps S405 and S406, only the substantial objects are extracted (S407).

Although the apparatus and method for separating 3D deep image based objects according to the present invention have been described with reference to the embodiments, the scope of the present invention is not limited to the specific embodiments, Various modifications, alterations, and alterations can be made within the scope of the present invention.

Therefore, the embodiments described in the present invention and the accompanying drawings are intended to illustrate rather than limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and accompanying drawings . The scope of protection of the present invention should be construed according to the claims, and all technical ideas within the scope of equivalents should be interpreted as being included in the scope of the present invention.

10: Image input unit
20: image state determination unit
30: Object separation unit
31: object detection unit
32: Background area removal
33: Ceiling area removal
34: ground area removal
35: object extracting unit

Claims (12)

A 3D depth image-based object separation apparatus,
An image input unit for inputting a 3D depth image; And
After detecting the object position and the number of objects using the depth information of the input 3D depth image, grouping between pixels having similar pixel values among the pixel values of the detected objects is performed, And an object separating unit for sequentially removing the area, the ceiling, and the ground area to extract a final object,
Wherein the object separating unit includes a background area removing unit for removing a background area using a CCA (Connected Component Analysis)
Wherein the background region removing unit applies a morphology erosion algorithm to the object detected in the 3D depth image and then applies a median filter to the 3D depth image to which the morphology erosion algorithm is applied, Values are sorted in order of magnitude, an intermediate value is selected, and a morphology expansion algorithm is applied to remove the background area excluding the object.
The method according to claim 1,
Further comprising an image state determination unit for determining a state of a 3D depth image input from the image input unit and providing the 3D depth image to an object separation unit when noise is included in the 3D depth image, Device.
3. The method of claim 2,
Wherein the object separator comprises:
An object detection unit for detecting the position of the object and the number of objects from the 3D depth image including the noise;
And a ceiling and ground area removing unit for sequentially removing the ceiling area and the ground area from the image from which the background area is removed through the background area removing unit to extract the final object.
The method of claim 3,
Wherein the object detection unit detects the position and the number of the object using the deep running object detection method through the depth information of the input 3D depth image and acquires and stores the position coordinate value of the object, .
delete The method of claim 3,
The ceiling and floor area removing unit may convert the depth image from which the background area is removed in the background area removing unit into three-dimensional coordinates, arbitrarily select three points among the three-dimensional coordinates, determine a plane using the plane equation ,
From the ceiling and the ground to a specific threshold is defined as a plane, the number of points included in the threshold in the calculated plane is calculated,
And repeating the operation a predetermined number of times to determine a plane including a greatest number of points of the number of repetitions as a ground and a ceiling to remove the corresponding region.
A 3D depth image-based object separation method,
Inputting a 3D depth image; And
After detecting the object position and the number of objects using the depth information of the input 3D depth image, grouping between pixels having similar pixel values among the pixel values of the detected objects is performed, Removing the area, the ceiling, and the land area sequentially to extract a final object,
The step of extracting the object includes a step of removing a background area (noise) using a CCA (Connected Component Analysis) method,
Wherein removing the background region comprises:
Applying a morphology erosion algorithm to the detected object in the 3D depth image, applying a median filter to the 3D depth image to which the morphology erosion algorithm is applied, by spatial filtering, Sorting, selecting a median value, and applying a morphology expansion algorithm to remove noise (background area) excluding the object.
8. The method of claim 7,
Further comprising the step of determining the state of the input 3D depth image and extracting the object using the 3D depth image including the noise when noise is included in the 3D depth image as a result of the determination A method for separating 3D depth image based objects.
9. The method of claim 8,
Wherein the step of extracting the object comprises:
Detecting the position of the object and the number of objects from the 3D depth image including the noise; and
And sequentially removing the ceiling area and the ground area from the image from which the background area is removed to extract the final object.
10. The method of claim 9,
The step of detecting the position and the number of objects of the object may include detecting the position and the number of the object using the depth learning object detection method through the depth information of the input 3D depth image to acquire and store the position coordinate value of the object In depth image based object separation method.
delete 10. The method of claim 9,
Wherein the step of extracting the object comprises:
Selecting the arbitrary three points among the three-dimensional coordinates, and determining a plane using the plane equation, after converting the depth-removed depth image into three-dimensional coordinates;
Determining a plane from the ceiling and the ground to a specific threshold value, and calculating the number of points included in the threshold in the calculated plane; And
And repeating the steps a predetermined number of times to determine a plane including a greatest number of points among the number of repetitions as a ground and a ceiling to remove the corresponding region.

Images