JP2007310572A - Recognition device and recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a recognition apparatus and a recognition method capable of recognizing an object while further preventing false detection from a static substance. <P>SOLUTION: An object discrimination part 6 discriminates whether or not a substance is an object based on a picked-up image of the substance by an imaging device 2. A static object determination part 7 determines whether or not the substance is static. An object position detection device 3 measures a position of the substance. A map database 9 associates the position with the image of the static substance to store them. A map data reference part 8 extracts the image of the static substance corresponding to the position of the substance from the map database 9. An image correspondence degree comparison part 10 compares correspondence degrees of the picked up image of the substance and the image of the static substance corresponding to the position of the substance. An object discrimination part 3 does not discriminate the substance as an object which has been discriminated as the object since the correspondence degree is a threshold or higher. Accordingly, the picked-up image of the substance and the image of the static substance in the map database are compared to eliminate substances with a high correspondence degree from objects. Thus, false detection from the static substance can be further prevented. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a recognition apparatus and a recognition method for recognizing an object such as a pedestrian on a road, and more particularly to a recognition apparatus and a recognition method capable of recognizing an object while preventing erroneous detection with a stationary object. It is.

As a technique for recognizing a conventional object such as a pedestrian, a technique described in Patent Document 1, for example, is known. This technology uses the position information of stationary objects registered in the map database, and stereo-processes the reference distance, which is the distance from the device to the stationary object obtained from the map database, and the images of multiple cameras provided on the device. The viewing distance obtained in this way is compared. In this technique, only the pixels in front of the stationary object whose viewing distance is shorter than the reference distance are extracted, and the object is detected from the image composed only of the extracted pixels, thereby reducing the burden of image processing. Has been proposed.
JP 2004-171165 A

  However, in the above technique, although an object is detected from an image composed only of pixels in front of a stationary object, the method itself for detecting an object from the image is based on the difference between successive images and the background motion. Because it uses a technique to detect the area of the moving object based on this and extract the area of the moving object, it is easy to misidentify the object as a stationary object when the object such as a pedestrian is stationary There is.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a recognition device and a recognition method capable of recognizing an object while further preventing erroneous detection with a stationary object.

  According to the present invention, an imaging unit that images an object, a target object identifying unit that identifies whether or not the object is a target, and whether or not the object is stationary are determined based on a captured image of the object acquired by the imaging unit. Measured by a stationary object determining means, a target object position detecting means for measuring the position of the object, a map database capable of storing the position and an image of a stationary object at the position in association with each other, and a target object position detecting means A captured image of an object that is identified as a target object by a map data reference unit that extracts an image of a stationary object corresponding to the position of the object from a map database and a target object identification unit and is stationary by a stationary object determination unit And an image matching degree comparing means for comparing the degree of matching with an image of a stationary object corresponding to the position of the object extracted by the map data reference means. When less than the threshold value, a recognition device does not identify a subsequent object an object identified with the object.

  According to this configuration, the imaging unit images an object, the target object identifying unit identifies whether or not the object is a target object, and the stationary object determination unit is configured to stop the object identified as the target object by the target object identifying unit. Therefore, the object to be verified can be narrowed down to an object that is easily mistaken for a stationary object.

  The object position detection means measures the position of the object, the map database stores the position and the image of the stationary object in association with each other, and the map data reference means stores the position of the object measured by the object position detection means. Since the corresponding still object image is extracted from the map database, it is possible to extract the still object image as the basis of verification from the map database.

  Then, the image coincidence degree comparison means compares the degree of coincidence between the captured image of the object and the image of the stationary object corresponding to the position of the object. Since the identified object will not be identified from the target object afterwards, it is possible to directly compare the captured object image with the still object image registered in the map database, and remove the object with a high degree of coincidence from the target object as a stationary object. The object can be recognized while further preventing erroneous detection with a stationary object.

  In this case, the captured image of the object determined to be stationary by the stationary object determination unit is associated with the position of the object measured by the target position detection unit and registered as a still object candidate image in the map database. It is preferable to further include data registration means.

  According to this configuration, since it is determined that the object is stationary by the stationary object determination unit and the captured image of the object that may actually be a stationary object is registered in the map database as a still object candidate image, The content can be updated.

  In this case, the map data reference means extracts an image of a stationary object candidate corresponding to the position of the object measured by the object position detection means from the map database, and the image matching degree comparison means is stationary by the stationary object determination means. The degree of coincidence between the captured image of the object determined to be the image and the still object candidate image corresponding to the position of the object extracted by the map data reference unit is compared. In the above case, it is preferable to register a numerical value added with 1 as the number of matches of the still object candidates in the map database.

  According to this configuration, it is determined whether or not the object is actually a stationary object based on the still object candidate image, and when the degree of coincidence is equal to or greater than the threshold, a numerical value obtained by adding 1 to the number of coincidences is registered in the map database. Therefore, it is possible to update the possibility that the stationary object candidate is actually a stationary object.

  In this case, it is preferable that the map data registration means registers the still object candidate image as a still object image in the map database when the number of matches of the still object candidates is equal to or greater than a threshold value.

  According to this configuration, the map data registration means registers the still object candidate image as a still object image in the map database when the number of matches of the still object candidates is equal to or greater than the threshold value. It is possible to re-register a highly reliable object that is a stationary object as a stationary object.

  According to another aspect of the present invention, an imaging step for imaging an object, a target object identifying step for identifying whether or not the object is a target object based on a captured image of the object acquired in the imaging step, A stationary object determination step for determining whether or not the object is stationary; an object position detection step for measuring the position of the object; and an image of the stationary object corresponding to the position of the object measured in the object position detection step. A map data reference step that is extracted from a map database that is stored in association with an image of a stationary object at the position, and an object that is identified as a target in the target identification step and determined to be stationary in the stationary object determination step An image matching degree comparison step for comparing the degree of coincidence between the captured image and a still object image corresponding to the position of the object extracted in the map data reference step. The object identification step, if the degree of coincidence is not less than the threshold value, the recognition method that does not identify a subsequent object an object identified with the object is provided.

  In this case, the captured image of the object determined to be stationary in the stationary object determination step is associated with the position of the object measured in the object position detection step and registered as a stationary object candidate image in the map database. It is preferable to further include a map data registration step because the contents of the map database can be updated.

  In this case, in the map data reference step, an image of a stationary object candidate corresponding to the position of the object measured in the object position detection step is extracted from the map database, and the image matching degree comparison step is performed in the object identification step. The degree of coincidence between the captured image of the object identified as an object and the image of the stationary object candidate corresponding to the position of the object extracted in the map data reference step is compared. In some cases, it is preferable to register a numerical value obtained by adding 1 to the map database as the number of matches of the stationary object candidate because the possibility that the stationary object candidate is actually a stationary object can be updated. is there.

  In this case, the map data registration step may register the still object candidate image as a still object image in the map database when the number of matches of the still object candidates is equal to or greater than the threshold. An object having high credibility, which is an object, can be registered as a stationary object, which is preferable.

  According to the recognition apparatus and the recognition method of the present invention, it is possible to recognize an object while further preventing erroneous detection with a stationary object.

  Hereinafter, a recognition apparatus and a recognition method according to embodiments of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a functional block diagram showing a configuration of an embodiment of a recognition apparatus according to the present invention. The recognition device 1 of this embodiment is mounted on a vehicle such as an automobile, for example, and recognizes an object such as a pedestrian on a road by distinguishing it from a guardrail, a signboard, a power pole, a tree, and the like on the road.

  As shown in FIG. 1, the recognition device 1 of the present embodiment includes an imaging device (imaging means) 2, an object position detection device (object position detection means) 3, and a data processing device 4. The imaging device 2 captures an image of an object such as a pedestrian and acquires a captured image. In the present embodiment, the imaging device 2 may be a visible light camera or an infrared camera.

  The object position detection device 3 is an object for measuring an absolute position such as a latitude and longitude of an object. Alternatively, the object position detection device 3 may be an object that measures height in addition to latitude and longitude. The object position detection device 3 can be realized by combining a self-position measurement system such as GPS (Global Positioning System) with a distance measurement device such as a laser radar. For simplicity, the object position detection device 3 may be a combination of a distance obtained by performing stereo processing on an image acquired from the imaging device 2 configured by a plurality of cameras and a self-position measurement system. When an absolute position is set for an object or the like designed as a landmark, the value of the absolute position may be applied as it is by the recognition device 1.

  The data processing device 4 is an object that processes the captured image acquired by the imaging device 2 and the position information obtained by the object position detection device 3.

  Hereinafter, the data processing device 4 will be described with reference to FIG. The data processing device 4 includes an object candidate search unit 5, an object identification unit (object identification unit) 6, a stationary object determination unit (stationary object determination unit) 7, and a map data reference unit (map data reference unit). 8. It has a map database 9, an image matching level comparison unit (image matching level comparison unit) 10, a matching number counting unit (matching number counting unit) 11, and a map data registration unit (map data registration unit) 12. ing.

  The object candidate search unit 5 is an object that extracts a part of an image from the captured image input from the imaging device 2 and searches for a candidate for an object for which a learning pattern is to be collected.

  The object identification unit 6 is an object that identifies whether or not the object is an object based on the image input from the object candidate search unit 5. The object identification unit 6 performs this identification by using an identification method acquired by learning before. The learning means can apply SVM or Voila & Jones method. As will be described later, when the object identification unit 6 receives a command from the image matching degree comparison unit 10 or the map data registration unit 12 that the object is a stationary object, the object identification unit 6 interrupts the recognition of the object as the object. Thereafter, the object is not identified as the object.

  The stationary object determination unit 7 is an object that determines whether or not an object is stationary. The determination of whether or not the object is stationary can be made simply by using the captured image from the imaging device 2 for the determination and observing the same object within an absolute position range of at least two errors or less. Can be determined as stationary. The determination of whether or not they are the same object using an image can apply simple template matching if multiple observations are performed within a sufficient time. In addition to the determination of stillness, tracking by image processing, a method of obtaining the motion of an object from an optical flow, and the like can be applied.

  The map data reference unit 8 is an object for extracting an image of a stationary object or a stationary object candidate corresponding to the position of the object measured by the object position detection device 3 from the map database 9. The map data reference unit 8 uses the absolute position of the measured object as a query and searches for whether or not image data such as a stationary object exists in the map database 9. If the data size of the map database 9 can be increased, the map data reference unit 8 may be an object that searches using a set of the absolute position of the object and the absolute position of the recognition device 1 itself as a query. Since how the object is viewed from the recognition device 1 also varies depending on the absolute position of the recognition device 1, more accurate recognition is possible by considering the absolute position of the recognition device 1.

  The map database 9 stores an absolute position and an image of a stationary object or a stationary object candidate at the absolute position in association with each other. FIG. 2 is a table showing an example of information stored in the map database 9 shown in FIG. As shown in FIG. 2, the information stored in the map database 9 does not need to include a specific road map, and the absolute position (latitude, longitude) of an object, an image of a stationary object or a stationary object candidate (It is indicated by image data No. in FIG. 2), and it is sufficient that the correspondence number and the correspondence relationship between the stationary object and the stationary object candidate are included.

  The absolute position of the object may include the altitude of the object. Further, when the map data reference unit 8 is a query using a set of the absolute position of the object and the absolute position of the recognition device 1 itself as a query, the recognition is performed in addition to the absolute position of the object, an image of a stationary object, and the number of matches. The absolute position of the device 1 itself is stored in association with it. Or you may memorize | store a map database including attributes, such as the time when the image of the stationary object candidate was acquired and referred and the size of the object. For example, in the case where the time is also stored, when the map data is regularly maintained (six months or the like), it is possible to delete data that is not referred to after a certain period of time.

  The map database 9 returns the image data to the map data reference unit 8 when there is image data of the corresponding stationary object within the range obtained by adding the measurement error to the absolute position as a query from the map database reference unit 8. A plurality of image data may be returned.

  Returning to FIG. 1, the image matching degree comparison unit 10 is a stationary object corresponding to the captured image of the object determined to be stationary by the stationary object determination unit 7 and the position of the object extracted by the map data reference unit 8. Alternatively, the degree of coincidence with the image of the still object candidate is compared. When there are a plurality of image data corresponding to one absolute position extracted by the map data reference unit 8, the degree of coincidence is determined for each image data. For the determination of the degree of coincidence, template matching or its extension method can be applied simply. In addition, as a method for determining the degree of coincidence, a method of extracting feature points of an image and taking correspondence between the feature points can be applied. The image coincidence degree comparing unit 10 determines that they are coincident when the determined coincidence is equal to or less than a predetermined threshold. If the image matching degree comparison unit 10 determines that the captured image of the object matches the image of the stationary object, the image matching level comparison unit 10 transmits a command to the object identification unit 6 that the object is a stationary object, and the object identification unit 6 does not recognize the object as an object thereafter.

  The coincidence counting unit 11 counts the number of times that the image of the object and the image of the still object candidate corresponding to the position are determined to be coincident by the image coincidence comparing unit 10.

  The map data registration unit 12 counts the captured image of the object determined to be stationary by the stationary object determination unit 7, the position of the object measured by the object position detection device 3, and the coincidence count counting unit 11. This is a thing to be registered in the map database 9 as a still object candidate image in association with the number of matches. Further, the map data registration unit 12 registers the still object candidate image registered in the map database 9 as a still object image when the number of matches counted by the match number counting unit 11 is equal to or greater than a predetermined threshold. To do. The minimum threshold value is 1. This is because, after a captured image of a certain object is newly registered as a still object candidate (number of matches 0), if it is determined that the same captured image matches the image of the still image candidate, the object Is treated as a stationary object. In general, the threshold is preferably 10 times or more so that an object that is not actually a stationary object is not erroneously registered as a stationary object. Furthermore, when the number of matches counted by the match count counting unit 11 is equal to or greater than the threshold, the map data registration unit 12 transmits a command to the object identification unit 6 to the effect that the object is a stationary object. The identification unit 6 interrupts the recognition of the object as the target, and thereafter does not recognize the object as the target.

  FIG. 3 is a flowchart showing a processing procedure by the recognition device 1 described above. The operation of the recognition apparatus 1 will be described using this flowchart.

  First, the target candidate search unit 5 performs a target candidate search on the captured image of the object input from the imaging device 2 (S1 in FIG. 3). The object candidate search is a process in which the entire captured image or a part of the captured image is cut out in an area smaller than the captured image such as a sub window. For example, as shown in FIGS. 4A to 4D, the object candidate search is performed by a processing frame 20 that encloses a card rail 21, a signboard 22, a pedestrian 23, a power pole 24, a tree 25, and the like that can be object candidates. This can be done by cutting out the captured image. Alternatively, this object candidate search can also be performed by performing feature point extraction processing (for example, corner detection) on all or part of the captured image and cutting out an image in the vicinity of the extracted feature points. . In the example of FIG. 4, a stationary object such as the guardrail 21 is erroneously recognized as an object candidate except that the pedestrian 23 is caught in the processing frame 20 in FIG.

  Returning to FIG. 3, the object identifying unit 6 performs object identification on the image cut out by the object candidate search, and determines whether or not the object related to the image is the object (FIG. 3). S2). The object identifying unit 6 sends the image and the identification result to the stationary object determining unit 7 after identifying the object.

  The object position detection device 3 measures the absolute position of the object identified as the object (S3 in FIG. 3). The object position detection device 3 sends the absolute position of the detected object to the map data reference unit 8. The stationary object determination unit 7 determines whether or not the object identified as the target object is stationary (S4 in FIG. 3). In the example of FIG. 3, the processing is performed in the order of S1, S2, S3, and S4. However, the object position measurement processing of S3 can be performed before and after each processing of S1, S2, and S4. The order does not matter.

  The map data reference unit 8 uses the absolute position (latitude, longitude) of the object measured by the object position detection device 3 as a query for the object determined to be stationary by the stationary object determination unit 7, A search is performed to determine whether image data corresponding to the absolute position exists in the database 9 (S5 in FIG. 3). The map database 9 returns image data of a stationary object or a stationary object candidate corresponding to the absolute position.

  When there is still object or still object candidate image data corresponding to the absolute position, the image coincidence degree comparing unit 10 determines the degree of coincidence with the captured image of the object by the imaging device 2 (S6 in FIG. 3). ). If there is no corresponding image data in the process of S5 in FIG. 3, or if it is determined that there is a mismatch in the process of S6 in FIG. 3, the coincidence count counting unit 11 counts the coincidence count to 0, and the map data The registration unit 12 registers the captured image of the object as a still object candidate image in the map database 9 in association with the absolute position and the match count 0 (S7 in FIG. 3).

  If it is determined in step S6 in FIG. 3 that the images match the captured image of the object (S8 in FIG. 3), the image matching level comparison unit 10 determines that the object is stationary. A command to the effect that the object is an object is transmitted to the object identifying unit 6, and the object identifying unit 6 does not recognize the object as an object thereafter (S11 in FIG. 3). When it is determined that they match in the process of S6 in FIG. 3 and the captured image of the object matches the image of the still object candidate, the number of matches of the image data is compared with a set threshold (FIG. 3 S9). In this case, if the number of matches is less than the threshold in the process of S9 in FIG. 3, the match count counter 11 adds 1 to the number of matches (S10 in FIG. 3). On the other hand, if the number of matches is greater than or equal to the threshold value in the process of S8 in FIG. Register with. Further, the map data registration unit 12 transmits a command to the effect that the object is a stationary object to the target object identification unit 6, and the target object identification unit 6 interrupts the recognition of the object as the target object. The object is not recognized as a target (S11 in FIG. 3).

  In the present embodiment, the imaging device 2 images an object, the object identification unit 6 identifies whether or not the object is a target, and the stationary object determination unit 7 is an object that has been identified as a target by the target identification unit 6. In order to determine whether or not the object is stationary, the object to be verified can be narrowed down to an object that is easily misidentified as a stationary object.

  The object position detection device 3 measures the position of the object, the map database 9 stores the position and the image of the stationary object in association with each other, and the map data reference unit 8 is measured by the object position detection device 3. Since an image of a stationary object corresponding to the position of the object is extracted from the map database 9, it is possible to extract an image of a stationary object that is a basis for verification from the map database 9.

  The image coincidence degree comparison unit 10 compares the degree of coincidence between the captured image of the object and the image of the stationary object corresponding to the position of the object, and the object identification unit 6 Since an object identified as an object is not identified as an object thereafter, an image of the imaged object is directly compared with an image of a stationary object registered in the map database 9, and an object with a high degree of coincidence is detected as a stationary object from the object. The object can be recognized while further preventing erroneous detection with a stationary object.

  In the present embodiment, the stationary object determination unit 7 determines that the object is stationary and registers a captured image of an object that may actually be a stationary object in the map database 9 as an image of a stationary object candidate. The contents of the map database 9 can be updated. Further, based on the image of the still object candidate, it is determined whether or not the object is actually a still object. It is also possible to update the possibility that the object candidate is actually a stationary object.

  Further, the map data registration unit 12 registers the still object candidate image as a still object image in the map database 9 when the number of matches of the still object candidates is equal to or greater than the threshold value. An object with high credibility that is an object can be re-registered as a stationary object.

  In the present embodiment, since the stationary objects stored in the map database 9 are limited to those that are erroneously detected as detection objects at a high frequency, a compact database can be realized. In addition, since the method of using the map database 9 is a method in which a captured image of an object is directly compared with an image of a stationary object, the captured image of the object can be applied even if it is acquired with a monocular camera, and is not necessarily limited to stereo images from multiple cameras. No processing is required.

  It should be noted that the recognition apparatus and the recognition method of the present invention are not limited to the above-described embodiments, and it is needless to say that various changes can be made without departing from the gist of the present invention.

It is a functional block diagram which shows the structure of one Embodiment of the recognition apparatus concerning this invention. It is a table | surface which shows an example of the information memorize | stored in the map database shown in FIG. It is a flowchart which shows the process sequence of the recognition apparatus shown in FIG. It is a figure which shows the example which performed object candidate search with respect to the image input from the imaging device shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Recognition apparatus, 2 ... Imaging device, 3 ... Object position detection apparatus, 4 ... Data processing apparatus, 5 ... Object candidate search part, 6 ... Object identification part, 7 ... Stationary object determination part, 8 ... Map data Reference unit, 9 ... map database, 10 ... image matching degree comparison unit, 11 ... coincidence count counting unit, 12 ... map data registration unit, 20 ... processing frame, 21 ... guard rail, 22 ... signboard, 23 ... pedestrian, 24 ... Telephone pole, 25 ... trees.

Claims (8)

Imaging means for imaging an object;
Object identifying means for identifying whether or not the object is a target object based on a captured image of the object acquired by the imaging means;
Stationary object determination means for determining whether or not the object is stationary;
Object position detecting means for measuring the position of the object;
A map database capable of storing a position and an image of a stationary object at the position in association with each other;
Map data reference means for extracting an image of a stationary object corresponding to the position of the object measured by the object position detection means from the map database;
A captured image of the object that has been identified as an object by the object identifying means and determined to be stationary by the stationary object determining means, and a stationary object corresponding to the position of the object extracted by the map data reference means Image matching degree comparing means for comparing the degree of matching with the image of
With
The object identification unit is a recognition device that does not identify an object identified as the object from the object thereafter when the degree of coincidence is equal to or greater than a threshold value.   The captured image of the object determined to be stationary by the stationary object determination unit is associated with the position of the object measured by the target position detection unit and registered as a still object candidate image in the map database. The recognizing device according to claim 1, further comprising map data registration means. The map data reference means extracts an image of the stationary object candidate corresponding to the position of the object measured by the object position detection means from the map database;
The image coincidence degree comparison unit is a captured image of the object determined to be stationary by the stationary object determination unit, and an image of a stationary object candidate corresponding to the position of the object extracted by the map data reference unit. And the degree of coincidence with
The map data registration means, when the degree of coincidence is equal to or greater than a threshold, registers a numerical value obtained by adding 1 as the number of coincidence of the stationary object candidates in the map database;
The recognition apparatus according to claim 2. The map data registration means, when the number of matches of the stationary object candidate is equal to or greater than a threshold, registers the image of the stationary object candidate as an image of a stationary object in the map database;
The recognition apparatus according to claim 3. An imaging step for imaging an object;
A target object identifying step for identifying whether the object is a target object based on a captured image of the object acquired in the imaging step;
A stationary object determination step for determining whether or not the object is stationary;
An object position detecting step for measuring the position of the object;
A map data reference step for extracting an image of a stationary object corresponding to the position of the object measured in the object position detection step from a map database in which the position and the image of the stationary object at the position are stored in association with each other;
The captured image of the object identified as the object in the object identification step and determined to be stationary in the stationary object determination step, and the stationary object corresponding to the position of the object extracted in the map data reference step An image matching level comparison step for comparing the matching level with the image;
Including
In the object identification step, when the degree of coincidence is equal to or greater than a threshold value, a recognition method in which an object identified as the object is not identified from the object thereafter. The captured image of the object determined to be stationary at the stationary object determination step is associated with the position of the object measured at the target object position detection step, and the image of the stationary object candidate is stored in the map database. A map data registration step to be registered;
The recognition method according to claim 5, further comprising: The map data reference step extracts an image of the stationary object candidate corresponding to the position of the object measured in the target object position detection step from the map database,
The image matching degree comparison step includes: a captured image of the object identified as a target in the target identification step; and a still object candidate image corresponding to the position of the object extracted in the map data reference step. Compare the match,
In the map data registration step, when the degree of coincidence is equal to or greater than a threshold value, a numerical value obtained by adding 1 is registered in the map database as the number of matches of the stationary object candidates.
The recognition method according to claim 6. In the map data registration step, when the number of matches of the stationary object candidate is equal to or greater than a threshold, the image of the stationary object candidate is registered as an image of a stationary object in the map database.
The recognition method according to claim 7.

Images