KR102186751B1 - Apparatus and method for improving multiple recognition and recognition rate of classification using object detection - Google Patents

Abstract

Provided are a device and a method for multiple recognition and recognition rate improvement of classification using object detection capable of improving a recognition rate (recognition performance) of each object existing in an input image such as a preview image. The provided device comprises: an object area detection unit detecting one or more object areas in an input image; an object area extraction unit extracting the detected one or more object areas; an adjustment unit processing and correcting the extracted one or more object areas to output a plurality of images for each object area; an accuracy calculation unit calculating an accuracy value corresponding to a matching rate with a pre-stored classification model for each of the plurality of images which are adjusted and outputted by the adjustment unit for each extracted object area; and an object name designation unit designating an object name for the corresponding object area based on results calculated by the accuracy calculation unit.

Description

TECHNICAL FIELD [Apparatus and method for improving multiple recognition and recognition rate of classification using object detection}

The present invention relates to an apparatus and method for recognizing multiple objects and improving recognition rate of a class using object detection, and more particularly, to an apparatus and method for recognizing one or more objects present in an input image such as a preview image.

In order to recognize an object existing in an input image such as a preview image, an image classification technique is commonly used.

The image classification technique is to determine whether an object corresponding to a specific class is included in a given input image. For example, it can be determined through how likely an object of class x exists in a given input image. Here, the class may be a cat, dog, or duck.

In the case of the image classification technique, it may be recognized that a cat is present in the input image as shown in FIG. 1.

In such a conventional image classification technique, a benchmark dataset (eg, photo format) is previously collected for each class, but the benchmark dataset includes photos of various angles, brightness, and types. Then, based on various benchmark datasets, machine learning is performed to create a classification model that classifies each class. That is, one representative classification model may be generated for each class. Thereafter, when the test image is input, the object (object) in the test image is classified (classified) based on the classification model.

However, in the case of the conventional image classification technique, a considerably large number of pictures are required for each class in order to generate a classification model for classifying each class.

In addition, the conventional image classification technique does not have a big problem when the number of objects (things) to be classified in an input image is small, but a lot of errors occur when there are a lot of objects (objects) to be classified.

In addition, the recognition rate (recognition performance) by the image classification technique is deteriorated according to the photographing environment, such as an angle and a brightness factor in which an object (object, for example, a cat) is photographed.

Prior Art 1: Korean Patent Registration No. 10-1995294 (Image analysis apparatus and method) Prior Art 2: Korean Patent Registration No. 10-1907883 (Object detection and classification method) Prior Art 3: Korean Patent Registration No. 10-1829386 (target detection device and method)

The present invention has been proposed in order to solve the above-described problems in the related art, and multiple recognition of the classification using object detection capable of improving the recognition rate (recognition performance) for each object existing in an input image such as a preview image. And an apparatus and method for improving recognition rate.

In order to achieve the above object, an apparatus for multiple recognition and recognition rate improvement of a class using object detection according to a preferred embodiment of the present invention includes: an object region detection unit for detecting one or more object regions in an input image; An object region extraction unit extracting the detected one or more object regions; An adjustment unit processing and correcting the extracted one or more object regions to output a plurality of images for each object region; An accuracy calculation unit for calculating an accuracy value corresponding to a matching rate with a pre-stored classification model for each of the plurality of images adjusted and output by the adjustment unit for each of the extracted object regions; And an object name designation unit that designates an object name for a corresponding object area based on the results calculated by the accuracy calculation unit.

The input image may be a preview image.

The processing may be to create an image in which the extracted object area is moved by a predetermined angle in the left/right/upward/downward directions.

The correction may be to generate an image obtained by adjusting at least one of brightness and contrast for the extracted object area.

The object name designation unit calculates an average value after summing the accuracy values for each of the plurality of images, and if the calculated average value is greater than or equal to a preset threshold, the object name corresponding to the classification model is assigned to the object area. Can be specified by object name.

On the other hand, a method for multi-recognition and recognition rate improvement of a class using object detection according to a preferred embodiment of the present invention is a method of classifying a class using object detection in an apparatus for multiple recognition and recognition rate improvement using object detection. A method of multiple recognition and recognition rate improvement, the method comprising: detecting one or more object regions in an input image; Extracting the detected one or more object regions; Processing and correcting the extracted one or more object regions to output a plurality of images for each object region; Calculating an accuracy value corresponding to a matching rate with a pre-stored classification model for each of the plurality of images processed and corrected for each of the extracted object regions and output; And designating an object name for a corresponding object area based on the calculated results.

According to the present invention with such a configuration, each object existing in the input image such as the preview image is processed and corrected according to various environments and objects in the corresponding object area are classified based on these. It is possible to improve the recognition rate (recognition performance) of.

1 is an exemplary diagram employed to describe a conventional image classification technique.
2 is a block diagram of an apparatus for multi-recognition and recognition rate improvement of a class using object detection according to an embodiment of the present invention.
FIG. 3 is an exemplary diagram employed to describe the object area detection unit shown in FIG. 2.
FIG. 4 is an exemplary diagram employed to describe the object extraction unit shown in FIG. 2.
FIG. 5 is an exemplary diagram employed to describe the adjustment unit shown in FIG. 2.
6 is an exemplary diagram employed to describe the accuracy calculation unit shown in FIG. 2.
7 is a flowchart for explaining a method of multi-recognition and recognition rate improvement of a class using object detection according to an embodiment of the present invention.
8 is an exemplary diagram employed in the description of FIG. 7.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate an overall understanding, the same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.

FIG. 2 is a block diagram of an apparatus for multiple recognition and recognition rate improvement of a class using object detection according to an embodiment of the present invention, and FIG. 3 is an exemplary diagram employed in the description of the object region detection unit shown in FIG. , FIG. 4 is an exemplary diagram employed in the description of the object extracting unit shown in FIG. 2, FIG. 5 is an exemplary diagram employed in the description of the adjustment unit shown in FIG. 2, and FIG. 6 is an accuracy calculation shown in FIG. It is an exemplary diagram employed in the description of the part.

The apparatus for multiple recognition and recognition rate improvement of a class using object detection according to an embodiment of the present invention can be employed in electronic devices (eg, smartphones, tablet PCs, mobile phones, etc.) and cameras with built-in camera functions. Do.

In addition, the above-described electronic device may be any electronic device as long as it has a built-in camera function other than those exemplified above.

According to an embodiment of the present invention, an apparatus for multiple recognition and recognition rate improvement of a class using object detection includes an object area detection unit 10, an object area extraction unit 20, an adjustment unit 30, an accuracy calculation unit 40, And an object name designation unit 50.

The object region detection unit 10 detects one or more object regions in the input image.

Here, the input image is an image including one or more objects (objects), and may be a preview image.

Accordingly, the object region detection unit 10 may detect the object region from the input image, as illustrated in FIG. 3 using an artificial intelligence object detection technique. 3 shows a case in which there is only one object (eg, a cat). The object area detection unit 10 may detect the object area by displaying the area where the object is considered to be present in a box-shaped outline 12. .

As a result, the region inside the outline 12 becomes a region of interest (ROI), and this region of interest may be referred to as an object region.

If there are multiple objects in the input image, the object region detection unit 10 may detect each object region.

The object region extraction unit 20 extracts one or more object regions detected by the object region detection unit 10.

That is, the object region extraction unit 20 extracts only the object region from the corresponding input image based on the object region detected by the object region detection unit 10. For example, as illustrated in FIG. 4, the object region extracting unit 20 may crop and extract the object region in the input image.

If there are several detected object regions, the object region extracting unit 20 may extract each object region by cropping.

The adjustment unit 30 processes and corrects each object area extracted by the object area extraction unit 20.

For example, as shown in (a) and (b) of FIG. 5, the adjustment unit 30 rotates the object region at a predetermined angle in the left or right direction based on the vertical center line (not shown) of the extracted object region. For example, within 60 degrees) a moving image is created, and the object area is moved upward or downward based on the horizontal center line (not shown) of the extracted object area as shown in FIGS. 5C and 5D. Each of them creates an image that is moved by a predetermined angle (for example, within 60 degrees). Creating an image such as (a), (b), (c), and (d) of FIG. 5 can be referred to as processing. If necessary, processing may be performed by additionally assuming that snow or rain falls during processing.

In addition, the adjustment unit 30 generates an image obtained by adjusting brightness, contrast, and the like of the extracted object area, as shown in FIG. 5E. Creating an image as shown in (e) of FIG. 5 can be referred to as correction.

The accuracy calculation unit 40 has a built-in classification model for one or more objects (things; also referred to as classes).

The accuracy calculation unit 40 calculates an accuracy value corresponding to a matching rate with the classification model for each of a plurality of images that are output by being adjusted by the adjustment unit 30 with respect to the object region extracted by the object region extracting unit 20.

That is, the accuracy calculation unit 40 calculates an accuracy value corresponding to the match rate with the classification model for each image as shown in (a), (b), (c), (d), and (e) of FIG. 6. Calculate it.

For example, the accuracy calculation unit 40 may calculate that the match rate with the cat (that is, the accuracy value) is 90% for the image of FIG. 5 (a), as shown in FIG. 6 (a). For the image of 5(b), it can be calculated that the match rate with the cat is 80% as shown in FIG. 6(b), and for the image of FIG. 5(c), as shown in FIG. 6(c) It can be calculated that the coincidence rate with is 50%, and it can be calculated that the coincidence rate with the cat is 75% as shown in Figure 6 (d) for the image in Figure 5 (d), and Figure 5 (e) ), it can be calculated that the match rate with the cat is 90%, as shown in (e) of FIG. 6.

The object name designation unit 50 designates an object name for a corresponding object area in the input image based on the results calculated by the accuracy calculation unit 40. Consequently, the object name designated by the object name designation unit 50 refers to an object existing in the object area. In the object name designation unit 50, object names corresponding to each classification model are previously stored.

For example, the match rate with the cat for the image of FIG. 6(a) is 90%, the match rate with the cat for the image of FIG. 6(b) is 80%, and the image of FIG. 6(c) Assuming that the match rate with the cat is 50%, the match rate with the cat for the image of FIG. 6(d) is 75%, and the match rate with the cat for the image of FIG. 6(e) is 90%. , The object name designation unit 50 calculates an average value (eg, 77%) after summing each of these matching rates. That is, the object name designation unit 50 calculates the average value of 77% by doing "(90% + 80% + 50% + 75% + 90%)/5".

Then, the object name designation unit 50 determines whether the average value is greater than or equal to a preset threshold (eg, about 70%), and if the average value is greater than or equal to a preset threshold, the object name for a corresponding object area in the input image (eg, Cat). At this time, the object name designation unit 50 is the classification model used as classification determination data for the current object area (ie, processed and corrected into multiple images) among the various classification models. You can specify the object name as cat.

Here, the threshold value may be set differently for each object or classification model, or the same threshold value may be set for all objects or classification models.

In FIG. 2 described above, the accuracy calculation unit 40 and the object name designation unit 50 are separately configured. However, if necessary, the accuracy calculation unit 40 and the object name designation unit 50 are integrated to provide an object name output unit. You may say.

FIG. 7 is a flowchart illustrating a method of improving recognition rate and multiple recognition of a class using object detection according to an embodiment of the present invention, and FIG. 8 is an exemplary diagram employed in the description of FIG. 7.

First, the object region detection unit 10 detects one or more object regions from an input input image (eg, a preview image) (S10). For example, with respect to the input image as shown in FIG. 8, the object region detection unit 10 may detect four object regions.

Subsequently, the object region extraction unit 20 extracts one or more object regions detected by the object region detection unit 10 (S20). As described above with reference to FIG. 4, the object region extracting unit 20 may extract each object region in the input image by cropping it.

Thereafter, the adjustment unit 30 processes and corrects each object area extracted by the object area extraction unit 20 (S30). That is, as described above with reference to FIG. 5, the adjustment unit 30 generates a plurality of images that are results of processing and correcting for each object area.

In addition, the accuracy calculation unit 40 calculates an accuracy value corresponding to the match rate with the classification model for each of the plurality of images that are adjusted and output by the adjustment unit 30 with respect to the object region extracted by the object region extraction unit 20. Calculate it (S40). That is, the accuracy calculation unit 40 calculates an accuracy value corresponding to the match rate with the classification model for each image as shown in (a), (b), (c), (d), and (e) of FIG. 6. Calculate it.

Subsequently, the object name designation unit 50 designates an object name for a corresponding object area in the input image based on the results calculated by the accuracy calculation unit 40 (S50). Consequently, the object name designated by the object name designation unit 50 refers to an object existing in the object area.

For example, the match rate (i.e., the accuracy value) with the cat for the image of FIG. 6(a) is 90%, the match rate with the cat for the image of FIG. 6(b) is 80%, and FIG. 6 The match rate with the cat for the image of (c) is 50%, the match rate with the cat for the image of (d) of FIG. 6 is 75%, and the match rate with the cat to the image of (e) of FIG. 6 Assuming that this is 90%, the object name designation unit 50 calculates an average value (e.g., 77%) after summing each of these coincidence rates, and the average value is equal to or greater than a preset threshold (e.g., about 70%). Specifies the object name (eg, cat) for the object area in the input image.

In addition, the above-described method for multiple recognition and recognition rate improvement of a class using object detection according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes and code segments for implementing the method can be easily deduced by programmers in the art to which the present invention belongs.

As described above, an optimal embodiment has been disclosed in the drawings and specifications. Although specific terms have been used herein, these are only used for the purpose of describing the present invention, and are not used to limit the meaning or the scope of the present invention described in the claims. Therefore, those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical scope of the present invention should be determined by the technical spirit of the appended claims.

10: object area detection unit 20: object area extraction unit
30: adjustment unit 40: accuracy calculation unit
50: object name designation part

Claims (9)

An object region detector for detecting one or more object regions in the input image;
An object region extraction unit extracting the detected one or more object regions;
An adjustment unit processing and correcting the extracted one or more object regions to output a plurality of images for each object region;
An accuracy calculation unit for calculating an accuracy value corresponding to a matching rate with a pre-stored classification model for each of the plurality of images adjusted and output by the adjustment unit for each of the extracted object regions; And
Includes; an object name designation unit that designates an object name for a corresponding object area based on the results calculated by the accuracy calculation unit,
The object name designation unit,
After summing the accuracy values for each of the plurality of images, an average value is calculated, and if the calculated average value is greater than or equal to a preset threshold value, designating the object name corresponding to the classification model as the object name for the object area. An apparatus for multiple recognition and recognition rate improvement of a class using object detection as a feature. The method according to claim 1,
The apparatus for multiple recognition and recognition rate improvement of a class using object detection, wherein the input image is a preview image. The method according to claim 1,
The above processing,
An apparatus for multi-recognition and recognition rate improvement of a class using object detection, characterized in that generating an image that moves the extracted object area at a predetermined angle in a left/right/up/down direction. The method according to claim 1,
The correction is,
An apparatus for multiple recognition and recognition rate improvement of a class using object detection, characterized in that generating an image obtained by adjusting at least one of brightness and contrast for the extracted object area. delete As a method for multi-recognition and recognition rate improvement of a class using object detection in an apparatus for multi-recognition and recognition rate improvement of a class using object detection,
Detecting one or more object regions in the input image;
Extracting the detected one or more object regions;
Processing and correcting the extracted one or more object regions to output a plurality of images for each object region;
Calculating an accuracy value corresponding to a matching rate with a pre-stored classification model for each of the plurality of images processed and corrected for each of the extracted object regions; And
Including; designating an object name for a corresponding object area based on the calculated results,
The step of designating the object name,
After summing the accuracy values for each of the plurality of images, an average value is calculated, and if the calculated average value is greater than or equal to a preset threshold value, designating the object name corresponding to the classification model as the object name for the object area. A method for improving recognition rate and multiple recognition of a class using object detection as a feature. The method of claim 6,
The above processing,
A method for improving multiple recognition and recognition rate of a class using object detection, characterized in that generating an image that moves the extracted object area at a predetermined angle in a left/right/upward/downward direction. The method of claim 6,
The correction is,
A method for improving multiple recognition and recognition rate of a class using object detection, comprising generating an image obtained by adjusting at least one of brightness and contrast for the extracted object area. delete

Images