JP4752319B2 - Image recognition apparatus and image recognition method - Google Patents

Description

  The present invention relates to an image recognition apparatus and an image recognition method used for image recognition of a recognition object.

Conventionally, as a device for recognizing a recognition object based on a captured image, a sign region image is extracted from an image of a cityscape continuously photographed by an in-vehicle camera, as described in Japanese Patent Application Laid-Open No. 2000-155839. There is known a device that creates a panoramic image from still images taken by a camera at each time and searches for marker candidates from the panoramic image. This apparatus intends to shorten the processing time by reducing the data amount of the panoramic image compared to the data amount of a large number of still images continuous in time series.
JP 2000-155839 A

  In such an apparatus, since it is necessary to search for the recognition target object with respect to the entire image of the search target image, the processing time becomes long. Moreover, when searching for a recognition target object for the first time with respect to a captured image, there exists a possibility of misrecognizing objects other than the target object to search, and there exists a possibility that recognition accuracy may fall.

  Therefore, an object of the present invention is to provide an image recognition apparatus and an image recognition method that can perform image recognition processing quickly and improve recognition accuracy by using map data indicating the position of a recognition object.

  That is, the image recognition apparatus according to the present invention includes an imaging unit for imaging a recognition target, a position detection unit for detecting a position of the imaging unit, map information recording the position of the recognition target, and the imaging Search means for searching for the recognition object imaged by the imaging means based on position information of the means, and an area in which the recognition object searched by the search means is displayed in an image imaged by the imaging means Area setting means for setting the image and recognition means for performing image recognition of the recognition object for the area.

  According to the present invention, the position of a recognition object appearing in an image is predicted using map information, and image processing is performed in a limited range, thereby reducing erroneous detection of the recognition object and improving processing speed. Can be planned.

In the image recognition apparatus according to the present invention, the region setting means sets the size of the region according to the position detection error of the imaging means in the position detection means. In addition, the image recognition apparatus according to the present invention, the region setting means, it is preferable to set the size of the area in accordance with the error definitive the distance detection from the imaging means to said object to be recognized.

In the image recognition apparatus according to the present invention, it is preferable that the area setting unit sets the size of the area according to a position error of the recognition object in the map information. In the image recognition apparatus according to the present invention, the region setting means, it is preferable to set the size of the area in accordance with the position error that by the communication delay the recognition object.

  According to these inventions, even if a position detection error or the like occurs, the size of the area is set so that the recognition object is displayed in the area where image recognition is performed. It is possible to set an area for reliably displaying an object. Thereby, the image recognition of a recognition target object can be performed reliably.

  In the image recognition apparatus according to the present invention, it is preferable that the recognition object is a road sign. In the image recognition device according to the present invention, it is preferable that the recognition object is a traffic light.

  The image recognition method according to the present invention is based on a position detection step for detecting the position of an imaging means for imaging a recognition object, map information recording the position of the recognition object, and position information of the imaging means. A search step for searching for the recognition object imaged by the imaging means, and an area setting for setting an area in which the recognition object searched in the search step is displayed in the image picked up by the imaging means A step and a recognition step of performing image recognition of the recognition object for the region.

  According to the present invention, the position of a recognition object appearing in an image is predicted using map information, and image processing is performed in a limited range, thereby reducing erroneous detection of the recognition object and improving processing speed. Can be planned.

In the image recognition method according to the present invention, the region setting step sets the size of the region in accordance with the position detection error of the imaging means in the position detection step . In the image recognition method according to the present invention, the region setting step, it is preferable to set the size of the area in accordance with the error definitive the distance detection from the imaging means to said object to be recognized.

In the image recognition method according to the present invention, it is preferable that the region setting step sets the size of the region according to a position error of the recognition object in the map information. In the image recognition method according to the present invention, it is preferable that the region setting step sets the size of the region according to a position error of the recognition target object due to communication delay.

  According to these inventions, even if a position detection error or the like occurs, the size of the area is set so that the recognition object is displayed in the area where image recognition is performed. It is possible to set an area for reliably displaying an object. Thereby, the image recognition of a recognition target object can be performed reliably.

  In the image recognition method according to the present invention, it is preferable that the recognition object is a road sign. In the image recognition method according to the present invention, it is preferable that the recognition object is a traffic light.

  According to the present invention, by using the map information indicating the arrangement position of the recognition object, the image recognition process can be performed quickly and the recognition accuracy can be improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a schematic configuration diagram of an image recognition apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, an image recognition device 1 according to the present embodiment is a device that recognizes a recognition object with respect to a captured image, and is mounted on a vehicle, for example. The image recognition apparatus 1 includes a camera 2, an image processing unit 3, and a database 4. The camera 2 is an image pickup means for picking up an object to be recognized, and for example, a camera provided with an image sensor such as a CCD or C-MOS is used. The camera 2 is preferably a compound eye camera capable of stereo viewing. In this case, the three-dimensional position of the object in the image can be detected based on the images captured by the two cameras.

  Note that one camera may be used as the camera 2. In this case, it is preferable to provide a measuring instrument that measures the distance from the camera 2 to the recognition object, such as a radar device.

  The image processing unit 3 is connected to the camera 2 and is an image processing unit that inputs an image captured by the camera 2, performs image processing on the captured image, and recognizes a recognition target in the captured image. The image processing unit 3 includes, for example, an image capture unit 31, a recognition object information search unit 32, a search distance setting unit 33, a recognition object image position calculation unit 34, a sensor error influence calculation unit 35, and a vehicle position / orientation / orientation detection. Unit 36, search frame setting unit 37, recognition object type setting unit 38, object recognition unit 40, and result output unit 41.

  The image capturing unit 31 captures an image captured by the camera 2. The recognition object information search unit 32 searches for a recognition object that may be photographed by the camera 2, and includes map information and vehicle position information (position information of the camera 2) that records the position of the recognition object. The recognition object to be imaged by the camera 2 is searched based on the above. The search distance setting unit 33 sets a distance range when searching for a recognition object. The recognition object image position calculation unit 34 calculates the display position of the recognition object in the captured image of the camera 2.

  The sensor error influence calculation unit 35 calculates the influence of the display position of the recognition target object due to the sensor error. The vehicle position / orientation / orientation detector 36 detects the position, orientation and attitude of the vehicle. As a device for detecting the position of the vehicle, for example, a GPS (Global Positioning System) is used. The search frame setting unit 37 sets a search frame for performing recognition processing of a recognition object on the captured image. The recognition target object type setting unit 38 sets the type of the recognition target object to be searched based on the map information. The object recognition unit 40 performs recognition processing in the search frame by a technique such as pattern matching. The result output unit 41 outputs the recognition result to the outside of the apparatus. For example, a display device or a speaker is used.

  The database 4 functions as a recording unit that records map information. In this map information, the position of the recognition object on the map is recorded. The map information also includes the types of recognition objects such as road signs and traffic lights, the center position of the recognition object, the elevation of the recognition object, the relative height of the recognition object from the ground or road, and the installation of the recognition object. It is preferable to record in advance the azimuth angle, the size of the recognition object, and the maximum measurement position estimation error of the recognition object.

  Next, the operation of the image recognition apparatus and the image recognition method according to this embodiment will be described.

  FIG. 2 is a flowchart showing the basic operation and the image recognition method of the image recognition apparatus according to this embodiment. The control process of FIG. 2 is executed by the image processing unit 3, for example.

  First, as shown in S10 of FIG. 2, camera image acquisition processing is performed. This process is a process in which an image signal output from the camera 2 is taken into the image processing unit 3 and a captured image is read. The captured image is an image obtained by capturing the front of the vehicle and is stored as a digital image. When the image signal output from the camera 2 is not digitized, the image processing unit 3 may perform analog-digital conversion (A / D conversion).

  And it transfers to S12 and the acquisition process of the position, direction, and attitude | position of a vehicle is performed. This process is a process for acquiring data on the position, orientation, and attitude of the vehicle on which the camera 2 is mounted. For example, the position and direction of the vehicle are detected by GPS mounted on the vehicle. In addition, the attitude of the vehicle is detected by a sensor installed in the vehicle. Further, by detecting the position, orientation and orientation of the vehicle, it is possible to calculate the position, imaging direction and orientation of the camera 2 attached to the vehicle.

  Then, the process proceeds to S14, and search target search processing is performed. The search target means a recognition target. For example, when a road sign is a recognition target, the sign becomes a search target. The search process of the search target is a process of searching for a search target that may be imaged by the camera 2, and the camera is based on the map information and the vehicle position information (position information of the camera 2) in which the position of the search target is recorded. The recognition target object imaged by 2 is searched. For example, it exists in the range of a predetermined distance (for example, 100 m) from the vehicle position, exists in the range of a predetermined angle (for example, 45 degrees) from the traveling direction of the vehicle, and the installation direction angle is in the traveling direction of the vehicle. A search target that is within a predetermined angle range (for example, 30 degrees) is selected. The number of search targets selected here is recorded as N.

  Then, the process proceeds to S16, and zero is set to the recognition counter value i. Then, the process proceeds to S18, where it is determined whether or not the recognition counter value i is equal to or greater than the search target number N. If the recognition counter value i is not greater than or equal to the number of search targets N, the intra-image position calculation process for the search target Ti is performed (S20).

  The intra-image position calculation process of the search target Ti is a process of calculating the position of the search target Ti in the captured image of the camera 2. As this processing, first, based on the latitude and longitude coordinate values (ψp, λp), altitude and height information of the position of the target Ti recorded in the map data, the coordinate value (Xpg) of the world plane rectangular coordinate system of the target Ti. , Ypg, Zpg). Then, the longitude and latitude coordinate values (ψom, λom) of the current vehicle position are converted into coordinate values (Xcg, Ycg, Zcg) of the world plane rectangular coordinate system.

  Then, the coordinate values (Xpg, Ypg, Zpg) of the target Ti are converted into the coordinate values (Xps, Yps, Zps) of the sensor coordinate system. Here, the sensor coordinate system is a coordinate system set with the installation position of the camera 2 as the origin. The conversion to the sensor coordinate system is performed using the following equation (1).

In this equation (1), ρ is the pitch angle of the vehicle, and φ is the roll angle of the vehicle. Further, ax, ay, and az are position coordinate values in the vehicle coordinate system at the origin of the sensor coordinate system. The coordinate values (Xps, Yps, Zps) of the target Ti in the sensor coordinate system are converted into coordinate values (X _L , Y _L ), (X _R , Y _R ) in the left and right camera coordinate systems. The conversion to the left and right camera coordinate system is performed using the following equations (2) and (3).

The equation (2), in _{(3), H1, H2,} p 10 ~p 211 is a camera parameter which is derived in the calibration of the camera 2 of the stereo vision. Based on the equations (2) and (3), it is calculated at which position in the camera image the target Ti is projected.

  Then, the process proceeds to S22, and search frame Fi setting processing is performed. The search frame Fi is an area to be searched for image recognition of the target Ti, and is set on the image. The search frame Fi setting process is a process of setting a search frame Fi having a predetermined horizontal width and vertical width around the coordinate value of the target Ti.

As the coordinate values of the target Ti, the coordinate values (X _L , Y _L ) and (X _R , Y _R ) of the left and right camera coordinate system calculated in S20 are used. The horizontal width and vertical width of the search frame Fi are set according to the distance from the position of the vehicle (camera 2) to the target Ti.

For example, as shown in FIGS. 3 and 4, tables for determining the horizontal width W _L and the vertical width H _L of the left image according to the distance Ds from the vehicle position to the target Ti are set in advance. Similarly, the width W _R of the right _image, a table to determine the vertical width H _R is set in advance in accordance with the distance Ds from the vehicle position to the target Ti. By using these tables, the width W _L of the left _image, the horizontal width of the vertical width H _L and the right image W _R, the vertical width H _R is calculated. At that time, when a measurement error of the vehicle position is output for each measurement, it is preferable to use a calculation table that takes into account the measurement error of the vehicle, that is, the GPS measurement error.

Then, the search frame of the left image is set by coordinate values (X _L , Y _L ), the horizontal width W _L of the left image, and the vertical width H _L. Also, the search frame coordinate value of the right image _{(X R, Y R),} the width _{W R} of the left image, is set by the vertical width _{H R.}

  At this time, the search frame is preferably set in consideration of the position detection error of the target Ti by the stereo camera and the position detection error of the camera 2 by GPS. The search frame is preferably set in consideration of a target position error when creating map data. The search frame is preferably set in consideration of a target position error due to communication delay. Further, the search frame is preferably set in consideration of a target position error due to a detection error of the azimuth angle and pitch angle of the vehicle.

  For example, the position detection error in the left-right direction of the target Ti by the stereo camera is Sx, the position detection error in the front-rear direction is Sy, the position detection error in the height direction is Sz, and the position detection error in the left-right direction of the camera 2 by GPS is Px. The position detection error in the front-rear direction is Py, the position detection error in the height direction is Pz, the position error in the left-right direction of the target when creating map data, the position error in the front-rear direction, and the position error in the height direction are Mp, The position error in the horizontal direction of the target due to communication delay, the position error in the front-rear direction, and the position error in the height direction are set as Cm, the target position error due to the detection error of the azimuth angle of the vehicle is set as Hd, and the detection error of the pitch angle of the vehicle Assuming that the target position error is Pt, the integrated value ΣerrX of the position error in the left-right direction of the target is Sx + Px + Hd + Cm + Mp. Further, the integrated value ΣerrY of the position error in the front-rear direction of the target is Sy + Py + Cm + Mp. Further, the integrated value ΣerrZ of the position error in the height direction of the target is Sz + Pz + Pt + Cm + Mp. The size (horizontal width, vertical width) of the search frame Fi is set so that the target Ti does not deviate from the search frame Fi even if the positional displacement of the target Ti occurs maximally according to these error integrated values ΣerrX, ΣerrY, ΣerrZ. You only have to set it.

  Then, the process proceeds to S24, and the type setting process of the search target Ti is performed. This process is a process for setting the type of the search target Ti. For example, the type of the search target Ti is set based on the map data, such as a traffic light or a stop sign.

  Then, the process proceeds to S26, and the image recognition process of the search target Ti is performed in the image area of the search frame Fi. This recognition process is performed using, for example, a pattern matching technique. As the pattern matching, a sum of differences, a normalized correlation, a neural network, or the like may be used. For example, as shown in FIG. 5, a search frame Fi is set in an image 21 captured by the camera 2, and image recognition of the search target is performed in an image area in the search frame Fi.

  And it transfers to S28 of FIG. 2 and a recognition result output process is performed. This process is a process of outputting the recognition result by display or voice. For example, when image recognition is performed on a recognition target object, a message indicating that the recognition target object has been recognized is displayed by a display device (not shown). In addition, a voice or a buzzer notifies that the recognition target has been recognized by a speaker (not shown).

  Then, the process proceeds to S30, i + 1 is set to the recognition counter value i, and the process returns to S18. If it is determined in S18 that the recognition counter value i is equal to or greater than the number of search targets N, it is determined whether or not a termination condition for the control process is satisfied (S32). If it is determined that the termination condition for the control process is not satisfied, the process returns to S10. On the other hand, when it is determined that the termination condition for the control process is satisfied, the control process is terminated.

  As described above, according to the image recognition apparatus and the image recognition method according to the present embodiment, the recognition object captured by the camera 2 based on the map information in which the position of the recognition object is recorded and the position information of the camera 2 is detected. A search frame in which a recognition target is displayed in the image captured by the camera 2 is set, and image recognition of the recognition target is performed in the search frame. Thus, recognition processing can be performed by performing image processing within a limited range in the captured image. Therefore, the erroneous detection of the recognition target object can be reduced and the processing speed can be improved.

  In addition, by setting the size of the search frame so that the recognition target is displayed in the search frame for performing image recognition even if an error occurs in the position detection of the recognition target, the range is limited. An area for reliably displaying the recognition target can be set. Thereby, the image recognition of a recognition target object can be performed reliably.

  In the present embodiment, the case of recognizing road signs and traffic lights on the road has been described. However, the image recognition apparatus and the image recognition method according to the present invention are not limited to the above, and other than signs and traffic lights. It may be applied to the case of recognizing an image.

1 is a schematic configuration diagram of an image recognition apparatus according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the image recognition apparatus and image recognition method which concern on embodiment of this invention. It is explanatory drawing of the setting process of the search frame in the image recognition apparatus of FIG. It is explanatory drawing of the setting process of the search frame in the image recognition apparatus of FIG. It is explanatory drawing of the image recognition process in the image recognition apparatus of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image recognition apparatus, 2 ... Camera (imaging means), 3 ... Image processing part, 4 ... Database (recording means).

Claims (12)

An imaging means for imaging a recognition object;
Position detecting means for detecting the position of the imaging means;
Search means for searching for the recognition object imaged by the imaging means based on the map information recording the position of the recognition object and the position information of the imaging means;
Area setting means for setting an area in which the recognition object searched by the search means is displayed in an image picked up by the image pickup means;
Recognition means for performing image recognition of the recognition object with respect to the region;
Equipped with a,
The area setting means sets the size of the area according to a position detection error of the imaging means in the position detection means . The region setting means, the image recognition apparatus according to claim 1, characterized in that setting the size of the area in accordance with the error definitive the distance detection from the imaging means to said object to be recognized. The region setting means, the image recognition apparatus according to claim 1 or 2, characterized in that setting the size of the area in accordance with the position error of the recognition object in the map information. The region setting means, the image recognition apparatus according to any one of claims 1 to 3, characterized in that setting the size of the area in accordance with the position error that by the communication delay the recognition object. The image recognition apparatus according to any one of claims 1 to 4, wherein the recognition object is a road sign. The image recognition apparatus according to any one of claims 1 to 4, wherein the recognition object is a traffic signal. A position detection step of detecting the position of an imaging means for imaging the recognition object;
A search step of searching for the recognition object imaged by the imaging means based on the map information recording the position of the recognition object and the position information of the imaging means;
A region setting step for setting a region in which the recognition object searched in the search step is displayed in the image picked up by the image pickup means;
A recognition step of performing image recognition of the recognition object for the region ,
The region setting step sets the size of the region in accordance with a position detection error of the imaging means in the position detection step . The region setting step, the image recognition method according to claim 7, characterized in that setting the size of the area in accordance with the error definitive the distance detection from the imaging means to said object to be recognized. The image recognition method according to claim 7 , wherein the region setting step sets the size of the region in accordance with a position error of the recognition object in the map information. The region setting step, the image recognition method according to any one of claims 7-9, characterized in that setting the size of the area in accordance with the position error that by the communication delay the recognition object. The image recognition method according to claim 7 , wherein the recognition object is a road sign. The image recognition method according to claim 7 , wherein the recognition object is a traffic light.

Images