JP2008134877A - Perimeter monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a "perimeter monitoring device" efficiently detecting a parallel running vehicle. <P>SOLUTION: When one's own vehicle 100 travels at a speed v, an image pattern of a road sign on a road surface included in an image e1 photographed by a right camera 3 includes large vibration in the back-and-forth direction of the own vehicle 100 generated by a speed difference between the own vehicle 100 and the road surface as compared to an image pattern of a two-wheeled vehicle 300 included in an image c1 obtained by photographing the parallel running vehicle by the right camera 3. The vibration appears as bluntness of a luminance change waveform e2 in the back-and-forth direction of the own vehicle 100. By extracting an edge having a larger absolute value of inclination in the back-and-forth direction of the own vehicle than a threshold value Th(v) set such that the threshold value Th(v) becomes smaller as the own vehicle speed v becomes larger, included in the image photographed by the right camera 3 (c3, e3), the edge of the image pattern obtained by photographing the parallel running vehicle can be properly distinguished from the image pattern edge obtained by photographing an in-road fixed object such as road marking and can be detected. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for detecting a situation around an automobile and notifying a user.

As a technology for detecting the situation around the vehicle and notifying the user, the driver's blind spot area generated at the left and right side positions of the vehicle is photographed with a camera, and other vehicles existing in the blind spot area are detected based on the photographed image. A technique for detecting and notifying the driver of the presence of the other vehicle is known (for example, Patent Document 1).
Special Table 2006-519427

Now, the detection of other vehicles based on images taken by the camera is normally performed by performing image recognition processing for recognizing the feature pattern of the vehicle on the taken images, but the amount of image recognition processing is as follows. It is relatively large and cannot detect other vehicles efficiently.
On the other hand, it is also conceivable to detect other vehicles located on the side of the vehicle by using a sensor whose direction is the side of the vehicle, such as a radar device or an ultrasonic sensor. However, such a sensor with the vehicle side direction as the directing direction cannot accurately measure the relative speed in the vehicle front-rear direction. Therefore, it distinguishes road installations on the side of the vehicle from other vehicles. It is relatively difficult to detect.

  Accordingly, an object of the present invention is to accurately and more efficiently detect other vehicles that run in parallel with the host vehicle in a periphery monitoring device that monitors the left and right sides of the vehicle.

  In order to achieve the above object, the present invention provides a periphery monitoring device for detecting a parallel vehicle that is mounted on a vehicle and that is parallel to the vehicle, a camera that captures a side of the vehicle, Edge extracting means for extracting an edge of the image pattern included in the image photographed by the camera with an inclination of the luminance change in the longitudinal direction of the automobile larger than a predetermined threshold; and the edge extracting means The edge is an edge of the image pattern of the parallel vehicle included in the image, and includes a parallel vehicle detection means for detecting the parallel vehicle.

According to such a peripheral monitoring device, it is used that the inclination of the luminance change of the edge of the image pattern obtained by photographing an object having a large relative speed with respect to the own vehicle becomes smaller as the relative speed between the object and the own vehicle becomes larger. By a simple process of extracting an edge having a large absolute value of the magnitude of the front-rear direction inclination of the vehicle, the object is efficiently and appropriately low in relative speed with the vehicle, that is, Further, it is possible to extract an edge of an image pattern obtained by photographing a parallel running vehicle and detect a parallel running vehicle based on the extracted edge.
Here, in this periphery monitoring device, further, a vehicle speed detecting means for detecting the speed of the automobile, and a threshold value for setting the threshold value so as to increase as the vehicle speed detected by the vehicle speed detecting means decreases. A setting unit may be provided. In this way, by increasing the threshold as the vehicle speed decreases, the relative speed changes according to the speed of the vehicle, so that the brightness change at the edge of the captured image pattern A fixed object on the road such as a road marking that changes the inclination of the vehicle can be more reliably prevented from being misidentified as a parallel vehicle regardless of the speed of the host vehicle.

  In order to achieve the above object, the present invention provides a periphery monitoring device for detecting a parallel vehicle that is mounted on a vehicle and that is parallel to the vehicle, and a camera that captures a side of the vehicle. Edge extracting means for extracting an edge of an image pattern included in an image photographed by the camera; and symmetry detecting means for detecting the presence or absence of symmetry of the edge extracted by the edge extracting means in the longitudinal direction of the automobile; When the symmetry detection means detects the symmetry of the edge with respect to the longitudinal direction of the automobile, the position of the symmetry axis of symmetry with respect to the longitudinal direction of the automobile with respect to the longitudinal direction of the automobile, The position of the axle in the image pattern of the wheels of the parallel vehicle included in the image in the longitudinal direction of the automobile is configured to include the parallel vehicle detection means for detecting the parallel vehicle. Than is.

  According to such a periphery monitoring device, the symmetry axis of the symmetry is obtained by utilizing the fact that the image pattern obtained by photographing the wheels of the parallel running vehicle becomes an image pattern having symmetry in the front-rear direction of the own vehicle. With the simple processing to detect, the position of the parallel running vehicle in the front-rear direction can be detected efficiently.

  Note that such a periphery monitoring device is configured such that, in the parallel vehicle detection means, when the symmetry detection means detects the symmetry of the edge in the front-rear direction of the automobile, the symmetry of the edge in the front-rear direction of the automobile. Of the positions of the edge of the image pattern on the axis of symmetry, the position closest to the vehicle in the left-right direction of the vehicle is the left and right of the vehicle at the lower end of the wheel in the image pattern of the wheels of the parallel vehicle included in the image. You may make it detect the said parallel running vehicle as a position about a direction. In addition, the periphery monitoring device includes vehicle speed detection means for detecting the speed of the automobile, and threshold setting means for setting a threshold value so as to increase as the vehicle speed detected by the vehicle speed detection means increases. In the edge extraction means, the magnitude of the gradient of the luminance change in the front-rear direction of the automobile of the image pattern included in the image photographed by the camera is greater than the threshold value set by the threshold value setting means. A large edge may be extracted.

  As described above, according to the present invention, in the periphery monitoring device that monitors the left and right sides of the vehicle, it is possible to more accurately and efficiently detect other vehicles that are running in parallel with the host vehicle.

Embodiments of the present invention will be described below.
FIG. 1a shows the configuration of the periphery monitoring system according to the present embodiment.
The peripheral monitoring system is a system mounted on an automobile, and as shown in the drawing, the rear camera 1, the left camera 2, the right camera 3, the rear monitoring unit 4, the side monitoring unit 5, and various states of the vehicle are detected. A vehicle state sensor 6, a surrounding situation presentation unit 7, and a display device 8 are provided.
Here, as shown in FIGS. 1b and 1c, the rear camera 1 is arranged at the rear of the vehicle and photographs the rear of the vehicle. The left camera 2 is disposed on the left side mirror of the vehicle and photographs the left side of the vehicle, and the right camera 3 is disposed on the right side mirror of the vehicle and photographs the right side of the vehicle.
In such a configuration, the rear monitoring unit 4 detects the other vehicle reflected in the image captured by the rear camera 1 by a predetermined image recognition process, and is estimated from the size and position of the other vehicle in the image. Information about the other vehicle, such as the position of the other vehicle, is notified to the surrounding state presenting unit 7 as the rear other vehicle information.

Next, the operation of the side monitoring unit 5 will be described.
The side monitoring unit 5 performs a parallel running vehicle detection process for detecting a parallel running vehicle that is another vehicle running side by side on the left and right sides of the vehicle.
FIG. 2 shows the procedure of the parallel vehicle detection process.
As shown in the figure, in this process, first, the vehicle speed v of the host vehicle detected by the vehicle state sensor 6 is acquired (step 202), and a threshold value Th (v) corresponding to the acquired vehicle speed v is set. (Step 204). Here, the threshold value Th (v) is set to increase as the vehicle speed v decreases.

Next, an image captured by the left camera 2 and an image captured by the right camera 3 are respectively acquired as side images (step 206). Then, an edge included in each side image and having an inclination in the front-rear direction of the own vehicle equal to or greater than a threshold value Th (v) is extracted (step 208).
Then, it is checked in step 208 whether or not an edge whose inclination in the front-and-rear direction is equal to or greater than the threshold Th (v) can be extracted (step 210). Return to the process. On the other hand, when an edge whose inclination in the front-rear direction is greater than or equal to the threshold Th (v) can be extracted in step 208, the edge extracted in step 208 in each side image is set as the recognition target edge. (Step 212).

  Then, it is checked whether or not the set recognition target edge pattern has symmetry in the front-rear direction of the host vehicle (step 214). If there is no symmetry, based on the position of the recognition target edge, The existence range is estimated (step 224), and parallel running vehicle information indicating the estimated existence range of the parallel running vehicle is output to the surrounding situation presentation unit 7 (step 222). Then, the process returns to step 202. Here, in this step 224, for example, a range on the side of the host vehicle corresponding to the range of presence of the recognition target edge in the side image is estimated as the range of presence of the parallel running vehicle.

  On the other hand, if the recognition target edge pattern has symmetry in the front-rear direction of the host vehicle, the front-rear direction position of the host vehicle of the symmetry axis of the symmetry is calculated, and the calculated front-rear direction position is calculated in parallel. As the longitudinal position of the axle of the vehicle, the position of the axle and the distance between the axles are calculated, and the existence range of the parallel running vehicle in the longitudinal direction is estimated based on the calculated position and interval (step 216). Further, an edge that exists on the axis of symmetry calculated in step 216 and whose inclination in the left-right direction of the own vehicle is a predetermined value or more is extracted (step 218). And among the extracted edges, the distance in the left-right direction of the vehicle to the position on the side of the vehicle corresponding to the edge position closest to the own vehicle is the distance in the left-right direction from the own vehicle to the parallel running vehicle. Then, the existence range in the left-right direction of the parallel running vehicle is estimated (step 220).

And the parallel running vehicle information which shows the existence range of the front-and-rear left-right direction of the own vehicle of the parallel running vehicle estimated as mentioned above is output to the periphery condition presentation part 7 (step 222). Then, the process returns to step 202.
Here, a processing example of the parallel running vehicle detection processing as described above is shown in FIG.
Now, as shown in FIGS. 3a1 and a2, when the two-wheeled vehicle 300 exists on the right side of the traveling own vehicle 100, when the two-wheeled vehicle 300 is running in parallel at the same speed as the own vehicle 100, The image captured by the right camera 3 includes the image pattern of the two-wheeled vehicle 300 as shown in c1 on the assumption that the right camera 3 is installed so that the horizontal direction of the image matches the front-rear direction of the host vehicle 100. It will be a thing. On the other hand, even when the two-wheeled vehicle 300 is stopped or trying to pass the own vehicle 100 at a higher speed than the own vehicle 100, the image captured by the right camera 3 includes the image pattern of the two-wheeled vehicle 300. However, the image pattern of the two-wheeled vehicle 300 is larger than the image pattern of the two-wheeled vehicle 300 included in the image photographed when the two-wheeled vehicle 300 is running in parallel as shown in d1 in the front-rear direction of the own vehicle 100. The image is blurred. Also, as shown in FIG. 3b, even when the vehicle 100 is traveling while shooting a road sign on the road surface with the right camera 3, the image taken with the right camera 3 is as shown in e1. The image pattern of the road sign in which the blurring in the front-rear direction of the host vehicle 100 occurs is included.

  Here, as shown in FIGS. 3 d 1 and e 1, the blur in the front-rear direction of the vehicle 100 generated in the image pattern is caused by the presence of a relative speed between the object corresponding to the image pattern and the vehicle 100. In general, images taken by the left camera 2 and the right camera 3 are multiplied by the relative speed by the exposure time of each frame (usually 1/30 seconds for an NTSC camera of 30 frames / second). A blur in the front-rear direction of the vehicle 100 corresponding to the distance is generated.

  FIGS. 3c2, d2, and e2 show the waveform of the change in luminance I of the horizontal line A at the same position in the images c1, c2, and c3. As shown in the figure, the blurring of the image pattern is a rounded waveform. appear. Therefore, as shown in FIGS. 3c3, d3, and e3, the relative speed difference between the vehicle 100 and the absolute value abs (dI) of the differential value of the luminance change waveform of each of the luminance change waveforms in FIGS. 3c2, d2, and e2. An image pattern having an edge with a large absolute value of the gradient of the luminance change is photographed as the image pattern of the object having a smaller number of images. In each figure, the x-axis represents the horizontal coordinate axis of the image.

  Accordingly, in the parallel vehicle detection process, by extracting an edge having a large absolute value of the magnitude of the inclination of the host vehicle 100 in the front-rear direction more than an appropriate threshold value Th (v) as described above, An edge of an image pattern obtained by photographing an object with a small relative speed, that is, an object that is a parallel running vehicle, is extracted as a recognition target edge, and the parallel running vehicle can be appropriately detected from the recognition target edge.

  Further, by increasing the threshold value Th (v) as the speed v of the host vehicle 100 decreases, the blur of the image pattern increases as the speed v of the host vehicle 100 increases. A fixed object on the road such as a road marking in which the magnitude of the inclination of the edge of the vehicle becomes smaller and the inclination of the edge of the image pattern to be photographed becomes larger as the speed v of the car 100 becomes smaller. Regardless of the size, it can be more reliably prevented from being mistaken as a parallel running vehicle.

  Next, as shown in FIG. 4a, when an image including the image pattern of the wheels of the parallel running vehicle is photographed by the right camera 3, the luminance change waveform about the horizontal line A of the image is set as bA, and the horizontal image is displayed. The luminance change waveform for each horizontal line including the image pattern of the wheel is represented by the symmetry axis D as indicated by bB as the luminance change waveform for the line B and bC for the horizontal line C of the image. , E including a horizontally symmetric edge.

  Therefore, in the parallel vehicle detection process, as described above, when there is symmetry in the front-rear direction of the own vehicle 100 in the pattern of the recognition target edge representing the edge of the image pattern of the parallel vehicle, the symmetry The position in the front-rear direction of the own vehicle 100 with the axis of symmetry can be detected as the position in the front-rear direction of the axle. The existence range in the front-rear direction of the parallel running vehicle can be accurately estimated from the detected position of the axle and the distance between the axles. In such an axle position detection, whether or not the detected symmetry axis of the edge is truly an axis passing through the axle is determined in consideration of a horizontal luminance change waveform on the axis. To do.

Now, as shown in FIG. 4c, the waveform of the luminance change of the vertical line of the image passing through the axles D and E (the left-right direction line of the own vehicle 100) is obtained at the lower end of the wheel of the parallel running vehicle. The edge closest to the vehicle 100 appears at the corresponding position. In FIG. 4c, the y-axis represents the vertical coordinate axis of the image.
Therefore, in the parallel vehicle detection process, as described above, the edge of the vehicle 100 that is the same as the symmetry axis in the front-rear direction of the vehicle 100 and whose slope in the left-right direction is greater than or equal to the predetermined value is the most Based on the distance dy to the position of the edge in the image, the position of the edge close to the vehicle 100 is detected, and the distance to the position on the side of the vehicle corresponding to the position of the edge is parallel from the own vehicle 100. It can be obtained as the distance in the left-right direction to the car. And based on this distance, the existence range of the left-right direction of the own vehicle 100 of a parallel running vehicle can be estimated accurately.

The operation of the side monitoring unit has been described above.
Now, referring back to FIG. 1 a, the surrounding situation presentation unit 7 is based on the other vehicle information received from the rear monitoring unit 4 and the parallel running vehicle information received from the side monitoring unit 5. The position and range of the parallel vehicle are displayed on the display device 8, and the possibility of a collision between another vehicle behind the vehicle 100 or the parallel vehicle and the vehicle 100 is estimated. When it exists, the display device 8 performs a process of displaying a warning indicating that there is another vehicle that is dangerous for the vehicle 100 behind or on the side. It is predicted that the possibility of the collision will be performed when a warning is not displayed based on the steering state of the host vehicle 100 detected by the vehicle state sensor 6 and the operation state of the direction indicator. This is performed while considering the course change and lane change behavior of the host vehicle 100.

  The embodiment of the present invention has been described above.

It is a block diagram which shows the structure of the periphery monitoring system which concerns on embodiment of this invention. It is a flowchart which shows the parallel running vehicle detection process which concerns on embodiment of this invention. It is a figure which shows the parallel vehicle detection operation example which concerns on embodiment of this invention. It is a figure which shows the parallel vehicle detection operation example which concerns on embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Back camera, 2 ... Left camera, 3 ... Right camera, 4 ... Back monitoring part, 5 ... Side monitoring part, 6 ... Vehicle state sensor, 7 ... Peripheral condition presentation part, 8 ... Display apparatus.

Claims (7)

A peripheral monitoring device that detects a parallel vehicle that is mounted on an automobile and that is another vehicle that runs in parallel with the automobile,
A camera for photographing the side of the car;
An edge extraction means for extracting an edge in which the magnitude of the gradient of the luminance change in the front-rear direction of the automobile of the image pattern included in the image photographed by the camera is larger than a predetermined threshold;
Peripheral monitoring, comprising: an edge extracted by the edge extraction means as an edge of an image pattern of a parallel running vehicle included in the image, and a parallel running vehicle detection means for detecting the parallel running vehicle apparatus. The periphery monitoring device according to claim 1,
Vehicle speed detecting means for detecting the speed of the automobile;
A periphery monitoring device comprising threshold setting means for setting the threshold so that the vehicle speed detected by the vehicle speed detecting means increases as the vehicle speed decreases. A peripheral monitoring device that detects a parallel vehicle that is mounted on an automobile and that is another vehicle that runs in parallel with the automobile,
A camera for photographing the side of the car;
Edge extraction means for extracting an edge of an image pattern included in an image photographed by the camera;
Symmetry detecting means for detecting the presence or absence of symmetry of the edge extracted by the edge extracting means in the longitudinal direction of the vehicle;
When the symmetry detection means detects the symmetry of the edge in the longitudinal direction of the vehicle, the position of the symmetry axis of symmetry in the longitudinal direction of the vehicle in the longitudinal direction of the vehicle A periphery monitoring device, comprising: a parallel vehicle detection unit that detects the parallel vehicle as a position of an axle in a wheel pattern image of a parallel vehicle included therein in the front-rear direction of the automobile. The periphery monitoring device according to claim 3,
The parallel running vehicle detection means detects the edge of the image pattern on the symmetry axis of symmetry in the longitudinal direction of the automobile when the symmetry detection means detects the symmetry of the edge in the longitudinal direction of the automobile. Of the positions, the position closest to the automobile in the left-right direction of the automobile is the parallel running as the position in the left-right direction of the automobile at the lower end of the wheel in the image pattern of the wheels of the parallel running vehicle included in the image. Perimeter monitoring device characterized by detecting a car. The periphery monitoring device according to claim 3,
Vehicle speed detecting means for detecting the speed of the automobile;
Threshold value setting means for setting a threshold value so as to increase as the vehicle speed detected by the vehicle speed detection means increases,
The edge extracting means has an inclination of a luminance change of an image pattern included in an image photographed by the camera in the front-rear direction of the automobile larger than the threshold set by the threshold setting means. A periphery monitoring device characterized by extracting an edge. In a device mounted on an automobile, a parallel vehicle detection method for detecting a parallel vehicle that is another vehicle parallel to the automobile,
Photographing the side of the car using a camera;
Extracting an edge of an image pattern included in an image photographed by the camera, wherein the magnitude of the gradient of the luminance change in the longitudinal direction of the automobile is greater than a predetermined threshold;
And a step of detecting the parallel vehicle as the edge of the image pattern of the parallel vehicle included in the image. The parallel vehicle detection method according to claim 6,
Detecting the speed of the vehicle;
And a step of setting the threshold value so that the detected vehicle speed increases as the vehicle speed decreases.