KR20100034105A - Object recognition method of robot - Google Patents

Abstract

The present invention relates to a method for recognizing an object of a robot, and more particularly, based on a plurality of patch images selected by dividing an image of a target object into a plurality of areas, the target in the frame of the surrounding image obtained by photographing the surroundings. An object recognition method of a robot for searching for an object, the method comprising: (a) comparing the color histogram of a partial region of the frame with the color histogram of the plurality of patch images to quantify the similarity; (b) comparing the similarity between the color histogram and a predetermined reference value to determine the partial region having a similar color histogram for each patch image as a primary candidate region; (c) comparing the color coincident histogram of the first candidate region with the color coincident histogram of the patch image to quantify the similarity; And (d) determining a primary candidate region having the largest similarity of the color coincident histogram for each patch image as a secondary candidate region.

The present invention greatly improves the speed and accuracy of object recognition in the robot, and at the same time ensures the continuous behavior of the robot. In addition, since the object is searched based on patch images of various sizes, the recognition rate may be further increased, and even when the target object is covered, the object may be quickly recognized.

Therefore, the present invention can ensure the speed, accuracy, continuity of robot operation, etc. of object recognition required for the service robot.

Description

Object recognition method of robot {OBJECT RECOGNITION METHOD OF ROBOT}

The present invention relates to a method for recognizing an object of a robot, and more particularly, to a method for recognizing a object of a robot, which finds a target object from an image of surroundings, based on an image of a target object previously stored by the robot.

The robot's ability to recognize objects is the key to determining the quality of service that a robot can provide.

Like conventional industrial robots, robots that perform repetitive and simple tasks did not require high-level object recognition capabilities, but service robots that are expected to provide direct services to people in the future, for example, require accurate and rapid object recognition capabilities. It is believed to be.

Recently developed robots are also designed to perform various commands using vision sensors. For example, when a person instructs a person to retrieve an object, the robot is designed to perform a command by using the vision sensor to find the object in eagerness and bring it to the person who orders it.

In this example, if the robot didn't find exactly what it was pointing to, or if it took too long to find it, the service it provides would be disappointing. Therefore, the accuracy and speed of object recognition along with the continuous behavior of the robot is considered to be an important factor in determining the service quality of the robot.

Therefore, many researchers have made great efforts to perform robust object recognition.

First, we will briefly examine the research to strengthen the feature points themselves.

The most common feature point using color is the color histogram. Color histograms have strengths that can be easily obtained, but have disadvantages that are not robust.

In order to compensate for the drawbacks of the color histogram, a model using a color cooccurence histrogram including distance information in a conventional color histogram as a feature point has been proposed.

The disadvantages of this model are that it is difficult to recognize when the object is partially obscured, that it is difficult to process data quickly due to the large amount of calculation, and is vulnerable to changes in lighting.

In order to overcome this problem, a patch-based recognition algorithm has been proposed, but this algorithm is difficult to be appropriately used for the size of various objects included in the image.

Another representative feature is SIFT. SIFT is also known to have a long execution time.

On the other hand, instead of suggesting a new model, there is a tendency to study a robust matching method. For example, an object recognition algorithm based on learning has been proposed for better performance. While learning can improve the performance of perception, the fact that it takes a lot of time to learn still remains a disadvantage.

In addition, color-based object recognition is being used for robots that need to complete recognition in a very short time. It is intended for special-purpose robots, such as robots playing soccer games. It is an algorithm suitable for finding edges or finding simple colors, but is not suitable for use in service robots.

In addition, since the result of the recognition may vary greatly depending on the lighting conditions, the robot is also being studied how to adjust the lighting conditions.

As discussed above, in order for the robot to find a particular object well through the image information obtained through the camera, there must be an object recognition algorithm that is robust to environmental changes. In addition, it should be able to recognize the object well even when the object is partially obscured.

When an object is obscured, the person moves to an unobscured position to see if the object they are looking for is correct, and the robot must also define the behavior for the obscured object. The robot must not stop until the algorithm is finished, and must cope with successive incoming images. In other words, the shorter the execution time of the object recognition is, the better. However, if the performance of the recognition is reduced by reducing the execution time, the short time has no meaning.

Therefore, the robot must improve the performance of the recognition while showing continuous behavior through the result of the recognition. For continuous incoming images, it is also important to decide what information of the image should be trusted.

The present invention provides a method capable of robust object recognition against changes in various environments, that is, changes in size of objects, changes in illumination, changes in viewpoint, and partial obstruction.

Although there is a limitation in reducing the performance of the robot while reducing the execution time of the algorithm, the present invention provides a method for deriving a result in the middle of the algorithm and using the same to judge the behavior of the robot. That is, the present invention provides a method of selecting a reliable image from a series of images continuously taken to determine the path of the robot when the obstruction is found.

According to an aspect of the present invention, a robot for searching for a target object in a frame of a surrounding image obtained by photographing the surroundings based on a plurality of patch images selected by dividing an image of the target object into a plurality of areas. An object recognition method comprising the steps of: (a) comparing the color histogram for the partial region of the frame with the color histogram of the plurality of patch images to quantify the similarity; (b) comparing the similarity between the color histogram and a predetermined reference value to determine the partial region having a similar color histogram for each patch image as a primary candidate region; (c) comparing the color coincident histogram of the first candidate region with the color coincident histogram of the patch image to quantify the similarity; And (d) determining the primary candidate region having the greatest similarity of the color coincident histogram for each patch image as the secondary candidate region.

Here, the present invention is (e) the relative displacement of the primary candidate region with respect to the second candidate region and other patch images with respect to the selected reference patch image of the plurality of patch images, the different from the reference patch image Digitizing based on a relative position difference between patch images; (f) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And (g) determining the position of the target object based on the similarity of the color coincident histogram and the relative displacement.

Alternatively, (h) the relative displacement of the secondary candidate regions with respect to the patch images different from the secondary candidate region with respect to the selected reference patch image among the plurality of patch images, the reference patch image and the other patch Quantifying based on the relative position difference between the images; (i) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And (j) determining the position of the target object based on the similarity and the relative displacement of the color coincident histogram. In this case, the determining of the position of the target object based on the similarity and the relative displacement of the color coincident histogram may include: (k) the second candidate corresponding to the other patch images determined for one reference patch image; Adding and multiplying the relative displacement of each of the regions by the similarity of the color coincident histogram; (l) multiplying the result of the addition by the similarity of the color coincident histogram of the secondary candidate region with respect to the reference patch image; (m) repeating steps (k) and (l) with each of the remaining plurality of patch images as a reference patch image; And (n) determining a patch image having a maximum multiplication result of step (l) as a location of the target object.

In addition, the present invention is a step of determining the position of the target object based on the similarity and the relative displacement of the color coincident histogram, (o) for all patch images, the patch of the patch images related to the relative displacement Obtaining a product of the similarity of the color coincident histogram and the relative displacement; And (p) determining the position of the patch images having the largest multiplication result as the position of the target object.

And, the present invention, (q) to quantify the reliability for each of the frame based on the similarity of the color histogram, the similarity of the color coincident histogram, and the relative displacement for each of a plurality of frames for the surrounding image. step; And (r) determining the behavior of the robot according to the position of the target object determined for the high reliability frame. Here, the reliability may be quantified based on the similarity of the color histogram, the similarity of the color coincident histogram, and the relative displacement of the region determined as the position of the target object for each frame, or of the color histogram. The similarity, the similarity of the color coincident histogram, and the weight for each of the relative displacements may be added and then numerically added.

In addition, the present invention may include a plurality of patch images for dividing the image of the target object into a plurality of regions, and may include patch images for dividing the images of the target object having different sizes into partial regions having the same size.

Previously, the present invention also relates to (s) the primary candidate region corresponding to each of the other patch images based on the position in the peripheral image of the secondary candidate region corresponding to one of the plurality of patch images. Calculating a primary position probability that will be located at; (t) multiplying the similarity of the digitized color coincident histogram by the corresponding first position probability; And (u) determining the primary candidate region having the largest multiplication result as the third candidate region for each patch image. Herein, the present invention is based on the position in the peripheral image of the secondary candidate region corresponding to one patch image among the plurality of patch images, the third candidate region corresponding to each of the other patch images. Calculating a third position probability to be located at; And (w) determining the final vector R according to the following equation as the position of the target object.

In addition, the object is a robot object recognition method for searching for a target object in the frame of the peripheral image obtained by photographing the surroundings according to another aspect of the present invention, (a) to the image of the target object of different sizes Pre-storing a plurality of patch images that are divided into partial regions having the same size; (b) comparing the color histogram of the partial region of the frame with the color histogram of the plurality of patch images to quantify the similarity; (c) comparing the similarity between the color histogram and a predetermined reference value to determine the partial region having the similar color histogram for each patch image as a primary candidate region; (d) comparing the color coincident histogram of the primary candidate region with the color coincident histogram of the patch image to quantify the similarity; And (e) determining a primary candidate region having the greatest similarity of the color coincident histogram for each patch image as a secondary candidate region.

Here, the present invention (f) the relative displacement of the primary candidate region with respect to the patch candidate image and the secondary candidate region with respect to the selected reference patch image of the plurality of patch images, different from the reference patch image Digitizing based on a relative position difference between patch images; (g) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And (h) determining the position of the target object based on the similarity of the color coincident histogram and the relative displacement.

Or, the present invention (i) the relative displacement of the secondary candidate region with respect to the second candidate region and the other patch image with respect to the selected reference patch image of the plurality of patch images, the reference patch image and the other patch Quantifying based on the relative position difference between the images; (j) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And (k) determining the position of the target object based on the similarity and the relative displacement of the color coincident histogram.

The present invention greatly improves the speed and accuracy of object recognition in the robot, and at the same time ensures the continuous behavior of the robot. In addition, since the object is searched based on patch images of various sizes, the recognition rate may be further increased, and even when the target object is covered, the object may be quickly recognized.

Therefore, the present invention can ensure the speed, accuracy, continuity of robot operation, etc. of object recognition required for the service robot.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram showing the general configuration of a robot.

As shown in FIG. 1, a general service robot may be roughly classified into a sensor unit 1, a driver 2, a memory 3, and a controller 4.

The sensor unit 1 may be a means for recognizing surrounding conditions, and may include a sensor for detecting temperature, pressure, gas, heat, or the like, or a vision sensor for obtaining an image of the surroundings, an infrared sensor, or the like. Which sensor is actually installed and used is the part of the robot manufacturer's choice according to the purpose of the robot.

The drive unit 2 includes a robot arm, leg, joint, wheel, and the like, and also includes a mechanism including a motor, a gear, and the like for moving them. In addition, it is obvious that the part can be selected according to the use or purpose of the robot.

The memory 3 is essential for the storage of information. As will be described later, in the present invention, the memory 3 stores a partial patch image of at least several items and performs a buffer function for temporarily storing an input image.

The controller 4 analyzes the information collected from the sensor unit 1, determines the action, and then controls the drive unit 2 to perform the action, which corresponds to the human brain.

Since the general configuration of the robot divided into the sensor unit 1, the drive unit 2, the memory 3, and the control unit 4 can be realized with what kind of components, it is well known. Do not

However, since the present invention relates to a method or algorithm for searching for a target object from an image of surroundings, the present invention is based on the premise that an image sensor, a memory 3, and a controller 4 are provided for capturing the surrounding image. Note that other components may be omitted.

2 is a flowchart illustrating a method of recognizing an object of a robot according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the robot first stores a plurality of patch images in the memory 3 that divide images of a target object into a plurality of regions (S1).

Here, the target object may be various items designated or prestored by a user, for example, a cup, a kettle, a bag, an ashtray, a mobile phone, a wallet, or the like. In addition, the area is selected to be of a constant size so that the patch images are uniform in size, and they are a set of complete images of the target object.

Here, the set of patch images, as shown in Figures 3a to 3c is preferably provided with a plurality of sets that are divided into a predetermined size of the image of the target object of different sizes. This is because the size of the target object is changed in the captured image according to the perspective of the robot and the target object, so it is more effective to use patch images of various sizes. This is because making the patch image constant may reduce the complexity of the algorithm.

In addition, the patch image is preferably provided for the image viewed from various directions for one item. Therefore, it is desirable to prepare an image viewed from the front, back, side, etc. for one item, and to predetermine sets of patch images for each of these items.

Next, when an event requiring a search for a specific object occurs (S2), for example, when the user receives a command to bring a specific object, the robot starts to photograph the surroundings to determine the location of the target object (S3). .

The image input from the CCD or CMOS image sensor of the sensor unit 1 is stored in the memory 3 continuously in units of frames (S4).

The controller 4 extracts color histograms of a plurality of patch images of the target object (S5). Here, if there are patch image sets having different sizes, it is necessary to extract a color histogram for each set, and note that the color histogram can be calculated based on several predetermined representative colors other than R, G, and B. Note that, if the memory 3 permits, the color histogram for the patch images can be pre-calculated based on the representative color and stored in the memory 3, where step S5 can be omitted.

Next, partial regions having the same size as the patch image are selected from one frame of the surrounding image, and color histograms of the partial regions are extracted and stored (S6). Here, the partial region may be selected according to the sizes of the patch images when the patch images are provided in sets of different sizes, and the partial region may be selected while sliding a window having a predetermined size at regular intervals.

Next, the color histograms of the patch images and the color histograms between the partial regions of the frame are compared, and the similarity is digitized and stored (S7).

Note that the similarity of the color histogram can be quantified or evaluated in various ways. For example, the sum of differences of values corresponding to the representative colors can be viewed as similarity, or the sum of the minimum values among the values corresponding to the two representative colors can be compared by comparing two color histogram results as shown in Equation 1 below. You can also evaluate.

  <Equation 1>

In Equation 1, r is a partial region in the input image, q is a patch image of the target object, n _c is the number of color set elements or representative colors, η is a normalization factor, and CH (c) is a color of color c. The histogram, Min, represents the minimum value, S _CH (r, q).

By comparing the numerical similarity with a predetermined threshold value, a partial region which is evaluated to be similar to each of the patch images among the partial regions is selected as the primary candidate region (S8). Note that the primary candidate region may be more than one for one patch image, and may not exist if the condition is not satisfied. It is also worth noting that the predetermined threshold depends on the quantified way of similarity. For example, if the similarity is determined as the sum of the differences of the color histograms, the similarity may be evaluated as similar when the threshold is less than or equal to a predetermined threshold. Can be.

<Formula 2>

In Equation 2, Pj is a partial region in which the color histogram similarity with respect to the patch image q _j is larger than the threshold value θ among the partial regions, and represents a primary candidate region for the patch image q _j .

Next, the degree of similarity is digitized by comparing the color coincident histogram of the primary candidate region with the color coincident histogram of the corresponding patch image (S9). As described above, the color coincident histogram is a feature point that adds the distance between colors to the color histogram.

The similarity of the color coincident histogram includes the inter-color distance, which requires a large amount of computation and time when calculating for all regions of the frame. However, since the present invention calculates the similarity of the color co-occurrence histogram for the primary candidate region evaluated as similar through the similarity of the color histogram, it is possible to reduce the amount of calculation as well as to perform a quick calculation.

As mentioned above, the similarity can be evaluated in various ways. As an example, it may be possible to quantify the sum of these values based on the difference between the color histogram and the distance. Alternatively, the degree of similarity may be digitized as in Equation 3 below in connection with Equation 1 and Equation 2.

<Equation 3>

The variable is the same as in <Equation 1>, except that d represents the distance between colors c ₁ and c ₂ .

Next, based on the similarity of the color coincident histogram, a secondary candidate region having a high probability of having a target object among the primary candidate regions is determined (S10). That is, the first candidate region having the largest value among the similarities of the color coherence histograms calculated in step S9 is obtained for each patch image.

As described above, when the similarity of the histogram is calculated as the difference between the two comparison targets, the smaller the difference may be, the greater the similarity may be. However, in the case of <Equation 1> to <Equation 3>, the result of <Equation 3> It can be determined that the largest primary candidate region has the largest similarity. Therefore, the secondary candidate region can be obtained as shown in Equation 4 below:

<Equation 4>

Here, n denotes the number of patch images, and indicates the number of patch images for all sets when a set of patch images of various sizes is provided.

Since the argmax operator means outputting a variable that maximizes the result of the operation, Equation 4 returns a partial region r within the primary candidate region that maximizes S _CCH as a result. Therefore, M has a secondary candidate region for each of the n patch images, and of course, there is no secondary candidate region for the patch image having no primary candidate region.

Next, relative positions between other primary candidate regions or secondary candidate regions around the secondary candidate regions determined for each of the patch images are compared with the original target object, and the relative positions of the candidate regions are determined by the candidate objects. Evaluate whether it provides a validity to be seen as constructing (S11).

That is, the patch images constitute one target object, and these patch images have relative positions. Therefore, it is possible to quantify the evaluation of the relative position between the secondary candidate regions or the primary candidate regions based on the relative positions between the patch images.

Of course, there may be various methods for evaluating relative positions between candidate regions. For example, a difference between pixel coordinates between candidate regions may be measured as a vector, and the size of the vector may be digitized to a value corresponding to a relative position. Alternatively, it may be possible to view a relative position of the reference patch images as an average position and to assign a probability value to other positions.

For example, when the Gaussian distribution is selected as the probability distribution for the relative positions of the patch images, the following Equation 5 may be defined:

<Equation 5>

Here, σ is an experimentally predetermined value inputted by the programmer of the robot, and u is as shown in Equation 6 below.

<Equation 6>

u represents the average position where the partial region corresponding to q _j should exist by adding the position of the secondary candidate region m _i corresponding to the reference q _i to the relative position between the patch images q _i and q _j .

Therefore, the G function of Equation 5 returns a probability value of the candidate region r to quantify the relative position with respect to the average position of qj.

Next, the position of the target object is finally determined based on the similarity between the evaluation value of the relative position quantified in this way and the color coincident histogram (S12).

Here, it should be understood that a method or rule for determining the position of the target object using the evaluation value and the similarity of the relative position may be variously selected.

For example, a secondary candidate region having the highest similarity of color coincident histograms, that is, having the highest color coincident histogram similarity, is defined as q _i and m _i , and is relative to other secondary candidate regions. When there are three or more candidate regions where the numerical evaluation value of the position satisfies the predetermined threshold level or more, it will be possible to determine the corresponding secondary candidate regions as the position of the target object.

Alternatively, one may think of ways to make the most appropriate decision by multiplying the similarity and the evaluation of relative position, as shown in the following equations.

As a first method, consider the following Equation 7.

<Formula 7>

)가 된다.Here, j and i are values selected under conditions of 1 to n, i ≠ j, and the position of m _{i that} produces the largest operation result in this range is the final position vector of the object (

)

The computation of Equation 7 evaluates the similarities and relative displacements of the color coherence histograms for each of the secondary candidate regions making up the relative displacements or corresponding two patch images. The value is multiplied.

Equation (7) shows a series of processes in which this operation is performed repeatedly for all patch images, and the position of the final object is determined as the position of the second candidate region that produces the largest of the results. .

As another second method, consider the following Equation 8.

 <Equation 8>

<Equation 8> is selected based on one secondary candidate region or one patch image corresponding thereto, and the similarity and relative displacement value of the color coincident histogram for each of the secondary candidate regions corresponding to the remaining patch images are selected. Multiply the multiplication results by multiplying the probability that they represent. The final evaluation value is calculated for each patch image by multiplying the similarity of the color coincident histogram of the secondary candidate region based on the final addition value.

This operation is repeatedly performed to select all the patch images as a reference, and Equation 8 returns the secondary candidate region used as the reference representing the maximum value of the repeated results as the position vector of the target object. .

As another third method, consider the following series of equations:

<Equation 9>

The equation multiplies the similarity of the color coincident histogram and the relative displacement evaluation value for each of the primary candidate regions corresponding to each of the other patch images based on one secondary candidate region, and the result is the maximum value. The primary candidate region shown is represented by a V ⁱ vector.

As a result, for one secondary candidate region, a V vector corresponding to all remaining patch images (j = 1 to n, j ≠ i) is generated, so that the following vector having n-1 components is generated. do:

<Equation 10>

This vector is called the tertiary candidate region, and based on this, the final target location can be determined by the following Equation 11:

<Equation 11>

Equation 11 selects one of the secondary candidate regions as a reference and then obtains a sum of the products of the similarity of the color coherence histogram and the evaluation value of the relative displacement for the third candidate regions corresponding to the remaining patch images. Next, the multiplication histogram similarity of the second candidate region selected as the reference result is multiplied.

This series of operations is performed for all secondary candidate regions, and the position of the secondary candidate region representing the maximum value is determined as the position of the final target object.

So far, we have looked at some methods for determining the final position of an object using the similarity of the color coincident histogram and the evaluation of relative position or displacement.

In either case, since it is performed within the primary candidate region, it can be seen that the amount of computation is greatly reduced. In addition, since the similarity of the color coincident histogram is already calculated and stored for each primary candidate region, the computation when making the final decision is only performed by simple multiplication and addition, and thus requires a small amount of computation and computation time. It is only.

In summary, the above-described embodiments of the present invention can be divided into three steps.

One step is to quickly determine the primary candidate region with a small amount of computation using the similarity of the color histogram. The remaining operation is now performed within the range of the primary candidate region.

Second, the similarity of the color coherence histogram with respect to the primary candidate region is obtained to determine a secondary candidate region 1: 1 corresponding to the patch images.

The third is to finally determine the position of the target object among the secondary candidate regions in consideration of the relative displacement. By considering the relative displacement, even if the object is not partially hidden, the possibility of object recognition is increased, while it is advantageous in that it can be found quickly with a small amount of calculation.

It will be appreciated that various methods or rules for finally determining the position of an object can be determined. While four methods or rules have been described above by way of example, the present invention is not limited to these embodiments.

Meanwhile, the robot determines the position of the target object and determines the action according to the flowchart of FIG. 2. If the robot moves, the surrounding images captured by the robot will be input continuously, so different frames will be input.

The frame of the newly input surrounding image must be processed quickly and the next action of the robot must be determined again. However, since this determination process requires a lot of time and operation, it is necessary to determine whether to use the image information of the frame or to trust the image information of the frame more than the image information of the previous frame.

Therefore, as shown in Fig. 2, the next step is to evaluate the reliability of the current frame (S13). The reliability of a frame can be determined by several rules, but it is not easy to evaluate the optimization or appropriateness of the decision rule. Therefore, the rules for frame reliability can be freely determined at the discretion of the engineer who determines the algorithm.

As an example, the reliability for each frame may be evaluated by the following Equation 12:

<Equation 12>

Bel (Frame) = w ₁ CH_MV + w ₂ CCH_MV + w ₃ GEO_MV

Here, w ₁ , w ₂ , w ₃ are weights, CH_MV is a matching value of the color histogram, CCH_MV is a matching value of the color coincident histogram, and GEO_MV is a matching value of the relative displacement.

These matching values may be selected from appropriate values representing the similarity of the color histogram, the similarity of the color coincident histogram, and the degree of relative displacement.

For example, CH_MV may be selected as the similarity of the average color histogram of the secondary candidate regions or partial regions finally determined as the position of the target object in the frame image. Similarly, CCH_MV can be chosen as the similarity of the average color coincident histogram of the partial regions from which CH_MV is calculated, and GEO_MV can represent the accuracy of the quantized relative position for the same regions.

On the other hand, the weights w ₁ , w ₂ , w ₃ are values that can be arbitrarily determined to give a higher weight to the relative reliability or importance of the matching values. For example, w ₂ will generally be chosen larger than w ₁ because the matching value of the color coincident histogram rather than the color histogram is more reliable.

Alternatively, the weights w ₁ , w ₂ , w ₃ can be determined using the concept of entropy, i) calculating w ₁ by entropy based on the number of representative colors that determine the color histogram, ii) it is possible to determine w ₂ by calculating entropy based on the number of cases of color coincident histogram, and iii) similarly, w ₃ can be determined by calculating entropy based on the case of relative displacement of partial regions.

As described above, the reliability as shown in Equation 12 is measured and compared every frame, and the robot's behavior is determined according to the result (S14).

If the reliability is not higher than the previous frame, the robot's behavior will be maintained as determined in the previous frame (S15).

On the other hand, if a new frame with higher reliability is input, the position of the target object is determined based on the new frame, and accordingly, the behavior of the object is newly determined (S16).

The robot's behavior includes movement towards the object, and as the robot moves, continuous shooting is possible, so new frames will be input continuously. The flowchart of FIG. 2 shows how the robot recognizes the target object within the new frames, along with which frame to trust more and determine action. In line with continuous shooting, rapid object recognition and frame reliability must be measured to ensure continuous robot behavior.

According to the present invention, the color histogram or color coincident histogram information on the patch images is fixed, and the candidate area can be reduced from a method having a small amount of calculation in measuring similarity, and a large amount of calculation can be made within the range but accuracy can be determined. Do the method.

Therefore, it is possible to ensure the accuracy and speed of the object recognition of the robot, thereby ensuring the natural behavior of the robot.

Although only some embodiments of the present invention have been described so far, those skilled in the art will understand that various modifications, substitutions, and the like can be made without departing from the spirit and scope of the present invention.

For example, in the embodiments of the present invention, the method of quantifying the similarity may be variously selected, and the method of quantifying the relative displacement may be variously selected in addition to the method using the probability distribution.

In addition, it should be noted that the method of evaluating frame reliability and the order thereof are not limited to the embodiments of the present invention, and may be evaluated in various ways as well as performed in a different order from that in FIG. .

Accordingly, the examples of the present invention should not be construed as limiting the present invention, but are to be understood as exemplary embodiments of the present invention. The spirit and scope of the present invention are defined by the following claims, and the scope should be construed to include the equivalent range.

1 is a schematic diagram showing the general configuration of a robot.

2 is a flowchart illustrating a method of recognizing an object of a robot according to an exemplary embodiment of the present invention.

3A to 3C are diagrams illustrating an example in which various patch images are prepared according to the size of a target object.

Claims (14)

In the object recognition method of the robot for searching for the target object in the frame of the peripheral image obtained by photographing the surroundings, based on a plurality of patch images selected by dividing the image of the target object into a plurality of areas, (a) quantifying the similarity by comparing the color histogram of the partial region of the frame with the color histogram of the plurality of patch images; (b) comparing the similarity between the color histogram and a predetermined reference value to determine the partial region having a similar color histogram for each patch image as a primary candidate region; (c) comparing the color coincident histogram of the first candidate region with the color coincident histogram of the patch image to quantify the similarity; And and (d) determining a primary candidate region having the greatest similarity of the color coincident histogram for each patch image as a secondary candidate region. The method of claim 1, (e) the relative displacement of the primary candidate regions with respect to the second candidate region and other patch images with respect to the selected reference patch image among the plurality of patch images, and the relative displacement between the reference patch image and the other patch images. Quantifying based on the position difference; (f) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And and (g) determining a position of the target object based on the similarity and relative displacement of the color coincident histogram. The method of claim 1, (h) relative displacements of the secondary candidate areas with respect to the selected second patch image and other patch images with respect to the selected reference patch image among the plurality of patch images, relative positions between the reference patch image and the other patch images; Quantifying based on the difference; (i) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And (j) determining the position of the target object based on the similarity and relative displacement of the color coincident histogram. The method of claim 3, wherein determining the position of the target object based on the similarity and relative displacement of the color coincident histogram comprises: (k) multiplying the relative displacement of each of the secondary candidate regions corresponding to the other patch images by the similarity of the color coincident histogram and then adding the determined one patch image; (l) multiplying the result of the addition by the similarity of the color coincident histogram of the secondary candidate region with respect to the reference patch image; (m) repeating steps (k) and (l) with each of the remaining plurality of patch images as a reference patch image; And and (n) determining a patch image having a maximum multiplication result in step (l) as the target object position. The method of claim 2 or 3, wherein determining the position of the target object based on the similarity and the relative displacement of the color coincident histogram, (o) obtaining, for all patch images, a product of the similarity of the color coincident histogram of the patch images related to the relative displacement and the relative displacement; And (p) determining the position of the patch images having the largest multiplication result as the position of the target object. The method according to claim 2 or 3, (q) quantifying the reliability of each of the frames based on the similarity of the color histogram, the similarity of the color coincident histogram, and the relative displacement of each of the plurality of frames of the surrounding image; And (r) determining the behavior of the robot according to the position of the target object determined for the frame having high reliability. The method of claim 6, And the reliability is quantified based on the similarity of the color histogram, the similarity of the color coincident histogram, and the relative displacement of the region determined as the position of the target object for each frame. The method of claim 7, wherein And the reliability is quantified by adding weights to the similarity of the color histogram, the similarity of the color coincident histogram, and the relative displacement, respectively. The method according to claim 2 or 3, The plurality of patch images for dividing the image of the target object into a plurality of areas, the image recognition method of the robot comprising a patch image for dividing the images of the target object of different size into a partial area of the same size. The method of claim 1, (s) a primary in which other patch images will be located in the primary candidate region corresponding to each other based on the position in the peripheral image of the secondary candidate region corresponding to one patch image among the plurality of patch images; Calculating a location probability; (t) multiplying the similarity of the digitized color coincident histogram by the corresponding first position probability; And and determining the primary candidate region having the largest multiplication result as the third candidate region for each patch image. The method of claim 10, (v) a third order in which other patch images will be located in the third candidate region corresponding to each other based on the position in the peripheral image of the secondary candidate region corresponding to one patch image among the plurality of patch images; Calculating a location probability; And (w) determining a final vector (R) according to the following equation as the position of the target object further comprising: detecting the object of the robot: In the object recognition method of the robot for searching for a target object in the frame of the surrounding image obtained by photographing the surroundings, (a) pre-storing a plurality of patch images for dividing the image of the target object of different sizes into partial regions of the same size; (b) comparing the color histogram of the partial region of the frame with the color histogram of the plurality of patch images to quantify the similarity; (c) comparing the similarity between the color histogram and a predetermined reference value to determine the partial region having the similar color histogram for each patch image as a primary candidate region; (d) comparing the color coincident histogram of the primary candidate region with the color coincident histogram of the patch image to quantify the similarity; And and (e) determining a primary candidate region having the largest similarity of the color coincident histogram for each patch image as a secondary candidate region. The method of claim 12, (f) the relative displacement of the primary candidate regions with respect to the second candidate region and other patch images with respect to the selected reference patch image among the plurality of patch images, and the relative position between the reference patch image and the other patch images. Quantifying based on the difference; (g) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And and (h) determining the position of the target object based on the similarity and relative displacement of the color coincident histogram. The method of claim 12, (i) relative displacements of the secondary candidate regions with respect to the second candidate region and other patch images with respect to the selected reference patch image among the plurality of patch images, and the relative displacements between the reference patch image and the other patch images. Quantifying based on the position difference; (j) quantifying the relative displacement by using each of the plurality of patch images as the reference patch image; And and (k) determining the position of the target object based on the similarity and relative displacement of the color coincident histogram.

Images