KR101915893B1 - Kinect-based Object Detection Method at Wall-Floor Junction Using Region Growing Technique - Google Patents

Abstract

A method and apparatus for object detection at the wall and floor boundaries using the Kinect - based area extension technique are presented. The object detection method at the wall and floor boundaries using the Kinect-based region expansion method proposed in the present invention includes steps of capturing an object in a space through a camera to acquire a depth information image, correcting the obtained depth information image, A step of performing a difference image technique on an image of a space including an object and an image of a space not including an object in the depth information image, a step of performing a difference image using a morphological operation and a connection component analysis on the image subjected to the difference image technique, And detecting the object using the SLIC algorithm and the area extension technique.

Description

FIELD OF THE INVENTION [0001] The present invention relates to an object detection method using a Kinect-based area extension method,

The present invention relates to a method and an apparatus for detecting an object at a bottom portion through a difference image technique and a morphological calculation technique and determining an area through prediction using a region expansion technique and a ratio.

You can use a differential image technique to detect objects at the bottom. The difference technique subtracts the value of each pixel of two images and shows the difference. It can detect the object part of the bottom by subtracting the background image without the object from the image of the object. Morphological operations can also be used to correct the boundaries of the image information. A morphological operation is a technique of analyzing and processing the shape of an image. It uses a predefined shape element to scan each pixel of the image and transform the image. In order to detect an object located on the wall, a region adjacent to the wall and the floor is examined, and a region growing method is used to determine whether the examined pixel is added to the adjacent region.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for accurately detecting objects at wall and floor boundaries using a Kinect-based area extension technique. Detecting the object at the bottom part by the difference image technique and morphological operation technique which is one of the image processing methods and fixing the area of interest by using the area expansion technique and the prediction through the ratio to be able to detect the object attached to the wall Method and apparatus.

In one aspect, a method for detecting an object at a wall and a floor boundary using the Kinect-based region expansion technique proposed in the present invention is a method for acquiring a depth information image by capturing an object in a space through a camera, Performing a difference imaging method on an image of a space including the object and a space including no object in the corrected depth information image, performing a difference imaging method on the image subjected to the difference imaging method, Analyzing the noise, and detecting the object using the SLIC algorithm and the area extension technique.

The step of acquiring a depth information image by capturing an object in a space through a camera and correcting the obtained depth information image may include temporally reducing a plurality of frames into one frame to correct depth information of the depth information image, And performs spatial reduction with respect to the horizontal and vertical sizes of one frame.

The step of removing the noise using the morphological operation and the connection component analysis on the image subjected to the differential image technique removes the noise region around the object and the object using the morphological operation on the image subjected to the differential image technique, The size of each object is measured using the component analysis, and noise below a predetermined size is removed.

The step of detecting the object using the SLIC algorithm and the area extension method includes the steps of designating an object located at the boundary between the wall and the bottom of the detected objects as a region of interest and dividing the region by applying the SLIC algorithm to the region of interest, Setting an object located in the bottom area in the divided area by applying the algorithm to the SEED, performing the area extension method using the set SEED as a starting point, and correcting the shape of the object using the morphological operation.

According to another aspect of the present invention, there is provided an apparatus for detecting an object at a boundary between a wall and a floor using a Kinect-based area expansion technique, which acquires a depth information image by capturing an object in a space through a camera, A difference image correcting unit for correcting the image, a difference image performing unit for performing a difference image method on the image of the space including the object in the corrected depth information image and the image of the space not including the object, A filtering unit for removing noise using morphological operation and connection component analysis for the image, and an object detecting unit for detecting the object using the SLIC algorithm and the area expansion technique.

The depth information correction unit temporally reduces a plurality of frames into one frame in order to correct depth information of the depth information image, and performs spatial reduction on the horizontal and vertical sizes of one frame.

The filtering unit removes the noise region around the object and the object using the morphological operation on the image subjected to the differential image technique, measures the size of each object using the connection component analysis, and then removes noise of a predetermined size or less do.

The object detection unit designates an object located at the boundary between the wall and the bottom of the detected objects as a region of interest, applies a SLIC algorithm to the region of interest to divide the region; Apply the SLIC algorithm to set the object located in the bottom area in the partitioned area to SEED; Performing the area extension technique with the set SEED as a starting point; Correct the shape of the object using morphological operations.

Embodiments of the present invention can accurately detect objects at wall and floor boundaries using the Kinect-based area extension technique. In addition, it can detect object attached to the wall by detecting object at the bottom part by using difference image method and morphological operation method, which is one of image processing methods, and fixing region of interest by using region expansion technique and ratio prediction. .

FIG. 1 is a flowchart illustrating a method for detecting an object at a boundary between a wall and a floor using a Kinect-based area expansion technique according to an embodiment of the present invention.
2 is a view for explaining a process of acquiring a depth information image through a camera according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating an image obtained by reducing a depth information image in a space-time manner according to an embodiment of the present invention.
4 is a diagram illustrating a difference image according to an exemplary embodiment of the present invention.
5 is a diagram illustrating an image in which noise is removed according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of detecting an object using the SLIC algorithm and the area expansion technique according to an embodiment of the present invention. Referring to FIG.
FIG. 7 illustrates a process of designating a region of interest according to an embodiment of the present invention. Referring to FIG.
FIG. 8 is a view for explaining an SLIC algorithm and an area extension technique according to an embodiment of the present invention.
FIG. 9 is a block diagram of an apparatus for detecting an object at a wall and a floor boundary using a Kinect-based area expansion technique according to an embodiment of the present invention. Referring to FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a method of detecting an object at a wall and a floor boundary using a Kinect-based area expansion technique according to an embodiment of the present invention.

The present invention relates to a method and apparatus for detecting pigs in a piggy bank. In the following description, an object can represent a pig. In the method of detecting an object at a wall and a floor boundary using the proposed Kinect-based area expansion method, a step 110 of capturing an object in a space through a camera to obtain a depth information image, correcting the obtained depth information image, A step 120 of performing a difference image technique on an image of a space including an object and an image of a space not including an object in the depth information image 120 and a morphological operation and a connection component analysis on the image subjected to the difference image technique (130), and detecting the object using the SLIC algorithm and the area extension technique (140).

In step 110, an object in a space is photographed through a camera to acquire a depth information image, and the acquired depth information image is corrected. In order to correct the depth information of the depth information image, a plurality of frames are temporally reduced to one frame, and spatial reduction is performed on the horizontal and vertical sizes of one frame. The process of acquiring the depth information image through the camera will be described in more detail with reference to FIG.

2 is a view for explaining a process of acquiring a depth information image through a camera according to an embodiment of the present invention.

FIG. 2A is a view for explaining a process of photographing an object 220 through a Kinect camera installed in a room according to an embodiment of the present invention. FIG. And a depth information image obtained from the camera.

According to one embodiment, a Kinect camera 210 is installed at a height of 4 m, and an object 220 attached to a wall surface is photographed at a maximum angle of view 211 through a Kinect camera 210, Can be obtained. If the object is located at the interface between the wall and the floor in the acquired depth information image, it can have inaccurate depth information.

When photographing an object through a Kinect camera, there is a maximum angle of view, and the wall portion of the room exceeds the maximum angle of view. When the object 220 is positioned at the boundary between the wall and the floor, the object in the photographed image has incorrect depth information. Therefore, in order to correct inaccurate depth information, temporal and spatial reduction of the depth information image can be performed.

FIG. 3 is a diagram illustrating an image obtained by reducing a depth information image in a space-time manner according to an embodiment of the present invention.

FIG. 3A shows an image obtained by spatially reducing a background with an object according to an embodiment of the present invention, and FIG. 3B shows an image obtained by spatially reducing an objectless background according to an embodiment of the present invention.

According to an exemplary embodiment, in order to correct inaccurate depth information in a depth information image acquired by an installed camera, it is possible to perform temporal reduction of three frames into one frame. In addition, it is possible to perform spatial reduction by dividing the horizontal and vertical sizes of each frame by half. In other words, the temporal reduction is to reduce three pixels to one pixel, and the spatial reduction reduces the four pixels to one pixel, so that the temporal reduction and the spatial reduction can be performed by taking the average value of them.

4 is a diagram illustrating a difference image according to an exemplary embodiment of the present invention.

In step 120, a difference image is performed on the image of the space including the object and the image of the space not including the object in the corrected depth information image. At this time, a difference image technique is performed using an image obtained by spatially reducing the background of the object shown in FIG. 3A and an image obtained by spatially reducing the background without the object shown in FIG. 3B.

5 is a diagram illustrating an image in which noise is removed according to an embodiment of the present invention.

In step 130, noise is removed using a morphological operation and connected component analysis on the image subjected to the differential image technique.

5A is an image in which noises around an object and an object are removed through a morphological operation according to an embodiment of the present invention. FIG. 5B is an image in which noise other than an object is removed through connection component analysis according to an embodiment of the present invention .

The noise image is removed from the object and the object by using the morphological operation on the image subjected to the differential image technique, the size of each object is measured using the connection component analysis, and the noise of a predetermined size or less is removed, Only parts can be detected. For example, in the connection component analysis, it is possible to measure sizes by measuring the number of labels after labeling each object by analyzing whether four or eight adjacent pixels are objects.

FIG. 6 is a flowchart illustrating a process of detecting an object using the SLIC algorithm and the area expansion technique according to an embodiment of the present invention. Referring to FIG.

In step 140, the object is detected using the SLIC algorithm and the area extension technique.

In operation 140, an object located at a boundary between a wall and a floor of the detected object is designated as an area of interest, and a region is divided by applying an SLIC algorithm to the region of interest (operation 610). In operation 610, A step 620 of setting an object located in the floor area in the floor area 620, a step 630 of performing an area extension method using the set SEED as a starting point, and a step 640 of correcting the shape of the object using a morphological operation can do.

FIG. 7 illustrates a process of designating a region of interest according to an embodiment of the present invention. Referring to FIG.

In step 610, an object located at the boundary between the wall and the bottom of the detected objects may be designated as a region of interest, and an SLIC algorithm may be applied to the region of interest to divide the region. First, as shown in FIG. 7A, an object located at the boundary 730 between the wall 710 and the floor 720 is designated as a region of interest, and the corresponding portion is separated. In other words, you specify the region of interest around the object that is overlaid on the wall. 7B is a view showing a region of interest in which the surroundings of the object overlapping with the wall surface are separated.

FIG. 8 is a view for explaining an SLIC algorithm and an area extension technique according to an embodiment of the present invention.

FIG. 8A is an image obtained by applying the SLIC algorithm to the ROI. In step 620, an SLIC algorithm may be applied to the ROI to set the object located in the bottom region, i. E., Pig, to SEED in the segmented region. A superpixel is a set of pixels that have similar color information. The SLIC algorithm is an algorithm for dividing into the plurality of superpixels. The established SEED can be used as a starting point for area expansion.

FIG. 8B shows an image in which the pig part in the bottom area is set as SEED, and FIG. 8C shows an image in which the area extension technique is applied. In step 630, an area extension technique may be performed using the set SEED as a starting point. The area extension method is to extend the object part in the wall direction to the boundary of the divided area by applying the SLIC algorithm with the set SEED as a starting point. In other words, the object area can be extended from the SEED to the wall direction until it meets the red line which is the boundary of the area divided by the SLIC algorithm. For example, you can extend the area of the pig to SEED and extend it to the wall, until it reaches the size of a typical pig.

FIG. 8D is an image obtained by correcting the shape of an object through a morphological operation. In step 640, morphological operations are used to correct the shape of the object. The morphological operation is to perform either a closing operation for performing an erosion operation after the expansion operation or an opening operation for performing an expansion operation after the erosion operation. In other words, unnecessary extended regions can be eliminated by using the open and close operations, which are morphological operations.

Such an erosion operation is an operation for removing the surface of an object, and an expansion operation is an operation for expanding the surface of an object. Closed operation can be filled with holes or gaps by performing erosion operation after expansion operation, and open operation can be removed by performing expansion operation after erosion operation.

In the prior art, objects are detected using a simple difference image technique without distinguishing between the floor area and the wall area. However, in the object detection method using the proposed Kinect - based area extension method, it is possible to detect the object more accurately by separating the wall area using the information of the bottom area.

FIG. 9 is a block diagram of an apparatus for detecting an object at a wall and a floor boundary using a Kinect-based area expansion technique according to an embodiment of the present invention. Referring to FIG.

The object detecting apparatus 900 at the wall and floor boundaries using the Kinect-based area expansion technique includes a depth information correcting unit 910, a difference image performing unit 920, a filtering unit 930, and an object detecting unit 940 . The depth information correcting unit 910, the difference image performing unit 920, the filtering unit 930 and the object detecting unit 940 are the same as the steps 110 to 140 of FIG. 1 and the steps 610 to 640 of FIG. As shown in FIG.

The depth information correction unit 910 acquires a depth information image by photographing an object in a space through a camera, and corrects the obtained depth information image. The depth information correction unit 910 temporally shrinks a plurality of frames into one frame to correct the depth information of the depth information image, and performs spatial reduction on the width and height of one frame.

The difference image performing unit 920 performs a difference image method on the image of the space including the object and the image of the space not including the object in the corrected depth information image. The difference image performing unit 920 performs a difference image method using an image obtained by spatially reducing a background having an object and an image obtained by spatially reducing a background without an object.

The filtering unit 930 removes noise using a morphological operation and a connection component analysis on the image subjected to the differential image technique. The filtering unit 930 removes the noise region around the object and the object using the morphological operation on the image subjected to the difference image technique, measures the size of each object using the connection component analysis, So that only the object part can be detected.

The object detection unit 940 detects the object using the SLIC algorithm and the area expansion method.

The object detection unit 940 designates an object located at the boundary between the wall and the bottom of the detected objects as the ROI, and applies the SLIC algorithm to the ROI to divide the ROI. Then, by applying the SLIC algorithm, the object located in the bottom area in the divided area is set as SEED, and the area expansion technique is performed using the set SEED as a starting point. Finally, morphological operations are used to correct the shape of the object.

Embodiments of the present invention can accurately detect objects at wall and floor boundaries using the Kinect-based area extension technique. In addition, it can detect object attached to the wall by detecting object at the bottom part by using difference image method and morphological operation method, which is one of image processing methods, and fixing region of interest by using region expansion technique and ratio prediction. .

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device As shown in FIG. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (12)

Capturing an object in a space through a camera to acquire a depth information image, and correcting the obtained depth information image;
Performing a difference image technique on an image of a space including an object and an image of a space not including an object in the corrected depth information image;
Removing noise by using a morphological operation and a connection component analysis on the image subjected to the differential image technique; And
Detecting an object using the SLIC algorithm and area extension method
Lt; / RTI >
Acquiring a depth information image by photographing an object in a space through a camera, and correcting the obtained depth information image,
In order to correct the depth information of the depth information image, a plurality of frames are temporally reduced in one frame, and spatial reduction is performed on the horizontal and vertical sizes of one frame
Object detection method. delete The method according to claim 1,
The step of removing the noise using the morphological operation and the connection component analysis on the image subjected to the differential image technique may include:
A noise area around an object and an object is removed using a morphological operation on an image subjected to the differential image technique, a size of each object is measured using a connection component analysis, and noise of a predetermined size or less is removed
Object detection method. The method according to claim 1,
The step of detecting the object using the SLIC algorithm and the area extension method includes:
Designating an object located at a boundary between a wall and a bottom of the detected objects as a region of interest and dividing the region by applying an SLIC algorithm to the region of interest;
Setting an object located in a floor area in the divided area to SEED by applying an SLIC algorithm;
Performing an area extension scheme using the set SEED as a starting point; And
Correcting the shape of the object using morphological operations
/ RTI > 5. The method of claim 4,
The region extension technique uses the set SEED as a starting point and applies the SLIC algorithm to extend the object portion to the boundary of the divided region toward the wall
Object detection method. 5. The method of claim 4,
The morphological operation is performed either by a closing operation for performing an erosion operation after the expansion operation or an opening operation for performing an expansion operation after the erosion operation
Object detection method. Acquiring a depth information image by capturing an object in a space through a camera, and correcting the obtained depth information image;
A difference image performing unit for performing a difference image method on an image of a space including an object and an image of a space not including an object in the corrected depth information image;
A filtering unit for removing noise by using a morphological operation and a connection component analysis on an image subjected to the differential image technique; And
An object detection unit for detecting an object using an SLIC algorithm and an area extension method
Lt; / RTI >
The depth-
In order to correct the depth information of the depth information image, a plurality of frames are temporally reduced in one frame, and spatial reduction is performed on the horizontal and vertical sizes of one frame
Object detection device. delete 8. The method of claim 7,
The filtering unit,
A noise area around an object and an object is removed using a morphological operation on an image subjected to the differential image technique, a size of each object is measured using a connection component analysis, and noise of a predetermined size or less is removed
Object detection device. 8. The method of claim 7,
The object detection unit,
Designating an object located at the boundary between the wall and the bottom of the detected objects as a region of interest, applying a SLIC algorithm to the region of interest to divide the region;
Apply the SLIC algorithm to set the object located in the bottom area in the partitioned area to SEED;
Performing the area extension technique with the set SEED as a starting point;
Correct the shape of an object using morphological operations
Object detection device. 11. The method of claim 10,
The object detection unit,
By applying the SLIC algorithm with the set SEED as the starting point, the area extension method is extended to expand the object part in the wall direction to the boundary of the divided area
Object detection device. 11. The method of claim 10,
The object detection unit,
Performing an erosion operation after performing an expansion operation, or performing an erosion operation after performing an erosion operation after the erosion operation,
Object detection device.

Images