JP5464073B2 - Outside environment recognition device, outside environment recognition method - Google Patents

Description

  The present invention relates to a vehicle environment recognition apparatus and a vehicle environment recognition method.

  The front of the host vehicle is imaged by a camera, the speed vector of the object to be determined is calculated based on the captured moving image, the predicted course of the host vehicle is calculated, and the speed vector and the predicted path of the host vehicle are calculated. Thus, there is an apparatus that detects an obstacle that jumps forward in the course of the vehicle (see Patent Document 1).

JP 2005-209019 A

However, when an obstacle is simply detected based on the speed vector, even an object that may be an obstacle for the host vehicle may not be recognized as an obstacle depending on the direction and size of the speed vector. .
An object of the present invention is to improve the detection accuracy of an object that can be an obstacle.

  The external environment recognition device according to the present invention acquires vehicle speed information of the host vehicle, extracts an edge from an image obtained by capturing the front of the host vehicle, and calculates a moving speed and a moving direction of the edge on the image as movement information. Among the extracted edges, vertical edges that are continuous in the vertical direction on the image are extracted based on the movement information. Further, out of the extracted vertical edges, the vertical edges belonging to the stationary object are extracted based on the movement information and the vehicle speed information, and among the extracted vertical edges belonging to the stationary object, the stationary object has a break in the direction of the vehicle. The corresponding vertical edge is extracted based on a predetermined passable width that allows the mobile body to pass through. Further, the vertical edge corresponding to the cut is set as a vertical edge at the time of cut extraction, the position of the vertical edge at the time of cut extraction is set as the cut position, and the vertical edge and the cut position at the time of cut extraction are stored as cut information. Then, after a predetermined time has elapsed after storing the cut information, the vertical edge closest to the cut position among the vertical edges newly extracted is compared with the vertical edge at the time of the cut extraction, so that the cut position It is determined whether or not the vertical edge closest to is a vertical edge belonging to the projecting object from the cut.

  According to the outside environment recognition apparatus according to the present invention, a vertical edge corresponding to a break of a stationary object is extracted, break information of the stationary object is stored, and a projecting object is detected based on a temporal change of the vertical edge at the break position. Therefore, it is possible to improve the detection accuracy of an object that can be an obstacle rather than simply detecting an obstacle based on a velocity vector.

It is a schematic block diagram of an environment recognition apparatus outside a vehicle. It is a figure which shows the example of installation to the vehicle of a camera. It is a figure which shows an example of the image imaged with the camera. It is a figure which shows a binarization process, a thinning process, an expansion process, and a count-up process. It is a figure which shows an example of a speed image. It is a figure which shows the extraction method of a vertical edge. It is the figure which extracted the lower end point from the vertical edge. It is a figure which shows conversion to a bird's-eye view coordinate system. It is a figure which shows the vertical edge determined to be a stationary object. It is the figure which projected the stationary object to the bird's-eye view coordinate. It is a figure which shows a passable determination area. It is a figure which shows a break determination. It is a figure which shows pop-out detection. It is a flowchart which shows a vehicle exterior environment recognition process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
"Constitution"
FIG. 1 is a schematic configuration diagram of an external environment recognition device.
The vehicle exterior environment recognition apparatus 10 includes an image processing unit 100 that performs image processing, a camera 101 that captures an image in front of the vehicle, and a vehicle information input unit 102 that inputs a vehicle speed signal, a steering angle signal, and the like.

The image processing unit 100 includes an image temporary recording unit 103, an edge extraction unit 104, a movement information calculation unit 105, a vertical edge extraction unit 106, a lower end point extraction unit 107, a stationary object edge extraction unit 108, and a break edge. An extraction unit 109 and a pop-up determination unit 110 are provided.
The image temporary recording unit 103 temporarily records an image captured by the camera 101.
The edge extraction unit 104 extracts an edge by performing image processing on the image stored in the image temporary recording unit 103.

The movement information calculation unit 105 calculates the movement speed and movement direction on the image at the edge extracted by the edge extraction unit 104 as edge movement information.
The vertical edge extraction unit 106 extracts vertical edges belonging to the vertical edge from the edge movement information calculated by the movement information calculation unit 105.
The lower end point extraction unit 107 extracts the lower end point on the image in the vertical edge movement information extracted by the vertical edge extraction unit 106.

The stationary object edge extraction unit 108, based on the position of the lower end point of the vertical edge extracted by the lower end point extraction unit 107 on the image and the vehicle speed signal input from the vehicle information input unit 102, the vertical edge belonging to the stationary object To extract.
The cut edge extraction unit 109 calculates a vertical edge corresponding to the cut of the stationary object based on the vertical edge extracted by the stationary object edge extraction unit 108 and at least the steering angle information or the yaw rate information input from the vehicle information input unit 102. Extract.

The pop-out determination unit 110 determines whether there is a pop-out object from the cuts extracted by the cut edge extraction unit 109.
The camera 101 is a camera having an image sensor such as a CCD or a CMOS, for example, and continuously images the front of the vehicle and outputs an image captured for each frame to the image temporary recording unit 103. An image captured by the camera 101 is temporarily recorded in the image temporary recording unit 103.

FIG. 2 is a diagram illustrating an installation example of a camera on a vehicle.
The camera 101 is installed in front of the upper part of the room, its optical axis LS is directed in the front front direction (Z direction) of the vehicle, and the horizontal axis X (not shown) of the imaging surface is parallel to the road surface. The vertical axis Y (not shown) of the surface is set to be perpendicular to the road surface.

FIG. 3 is a diagram illustrating an example of an image captured by the camera.
An image captured by the camera 101 is represented by an xy coordinate system in which the upper left of the image is the origin, the x axis is from left to right, and the y axis is from top to bottom. In FIG. 3, a captured image includes a wall that is a three-dimensional object and a white line that is a flat object.

FIG. 4 is a diagram illustrating binarization processing, thinning processing, expansion processing, and count-up processing.
The edge extraction unit 104 reads an image captured by the camera 101 from the image temporary recording unit 103, and binarizes the read captured image using a predetermined threshold value, thereby detecting an object existing in the image. Extract edges. FIG. 4A shows the extracted vertical edge. Next, as shown in FIG. 4B, thinning processing is performed on each extracted edge to narrow the edge width, and the center of the edge is accurately set. Further, as shown in FIG. 4C, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a constant width, for example, a width corresponding to three pixels. By this operation, the extracted edges are normalized, and an edge image having a uniform width for each edge can be obtained.

  The movement information calculation unit 105 updates the counter value of the pixel counter of the pixel corresponding to the edge. Here, the pixel counter is a counter set for each pixel. When the pixel corresponds to the edge, the counter value is incremented by +1. When the pixel does not correspond to the edge, the counter value becomes 0 and is initialized. Counter. The counter value updating process is performed for each frame continuously captured by the camera 101. By this operation, the counter value of the pixel counter increases as the pixel corresponding to the edge becomes longer, and the counter value of the pixel counter decreases as the pixel corresponds to the edge shorter.

Since the change in the counter value of the pixel counter represents the moving direction and moving amount of the edge, the moving direction and moving speed of the edge on the captured image can be calculated from the counter value. That is, since the x coordinate of the image represents the azimuth, the moving direction and moving speed of the edge can be obtained.
Hereinafter, a method for calculating the edge moving direction and moving speed will be described in detail.

First, binarization processing is performed on the edge image.
The binarization process is a process of setting a pixel at a position where an edge is detected to “1” and setting a pixel at a position where no edge is detected to “0”. By this binarization processing, a binarized image as shown in FIG.

Next, thinning processing is performed on the generated binary image.
The thinning process is a process for reducing the edge width of the detected edge until a predetermined pixel width is reached. For example, as shown in FIG. 4B, the center position of the edge is extracted by thinning until the edge width becomes one pixel. Of course, the edge width may be larger than one pixel.

Next, an expansion process is performed to expand (expand) the edge width of the thinned edge.
The expansion process is a process of expanding the edge width from the center position set by thinning toward the edge movement direction and expanding the edge width from the center position to the opposite direction to the edge movement direction. For example, as shown in FIG. 4C, the edge is expanded from the center position x0 of the edge by one pixel in the edge movement direction (here, the positive direction of the x-axis), and the edge movement direction from the edge center position x0. Is expanded by one pixel in the opposite direction (here, the negative direction of the x-axis), and the edge width is expanded to three pixels. In this way, by performing the thinning process and the expansion process, the edge width of the extracted edge image is standardized to a predetermined width in the edge moving direction.

Next, a count-up process is performed on the edge whose edge width is standardized.
The count-up process is a process for counting up the value of the memory address at the position where the edge is detected and initializing the value of the memory address at the position where the edge is not detected.

  Here, the edge count-up process will be described with reference to FIGS. In order to simplify the description, it is assumed that the edge moves in the positive direction of the x axis. Of course, the edge moves in the x-axis negative direction, the y-axis direction, or two-dimensionally, but basically the same processing is performed.

First, as shown in FIG. 4C, the edge has a center position of the edge at a position [x0] in a certain frame, and a position [x0 + 1] adjacent to one pixel in the moving direction of the edge from the center position. To the position [x0-1] adjacent to one pixel in the direction opposite to the edge moving direction.
In this case, the count values of the positions [x0-1], [x0], and [x0 + 1] where the edge is detected are incremented by “1”, and the count value of the position where the edge is not detected is reset to “0”. Is done.

  For example, as shown in FIG. 4D, since edges are detected at positions [x0-1], [x0], and [x0 + 1] at time t, “1” is counted up at each position. The count value at position [x0 + 1] is “1”, the count value at position [x0] is “3”, and the count value at position [x0-1] is “5”.

  As shown in FIG. 4E, since the edge does not move at time t + 1, the edge is detected at each position of [x0-1], [x0], and [x0 + 1], and the position [x0− 1], [x0], and [x0 + 1] are further incremented by “1”, the count value at position [x0-1] is “2”, the count value at position [x0] is “4”, The count value at the position [x0 + 1] is “6”.

Further, as shown in FIG. 4F, at time t + 2, the edge is shifted by one pixel in the positive direction of the x axis, and the edge is detected at the positions [x0], [x0 + 1], and [x0 + 2]. ing.
In this case, the count values of the positions [x0], [x0 + 1], and [x0 + 2] where the edge is detected are incremented by “1”, and the count values of the position [x0-1] where the edge is not detected are “0”. It will be reset to

  As a result, as shown in FIG. 4F, the count value at position [x0 + 2] is “1”, the count value at position [x0 + 1] is “3”, and the count value at position [x0] is “5”. ing. Further, the count value of the position [x0-1] where no edge is detected is reset to “0”.

  Thus, the count value at the position where the edge is detected is incremented by “1”, and the count value at the position where the edge is not detected is reset to “0”. Therefore, when the count value at the position where the edge is detected is counted up, the count value becomes equal to the time (number of frames) during which the edge is detected at that position. For example, at position [x0], an edge is detected twice at time t and time t + 1, and the time during which an edge is detected at position [x0] corresponds to the number of frames.

  Here, as the position where the count value is detected, the center position [x0] of the edge, the position [x0 + 1] adjacent to one pixel from the center position to the edge movement direction, and the direction opposite to the edge movement direction from the center position. Although the count value is detected with the width of three pixels of the position [x0-1] adjacent to one pixel, it is only necessary to obtain the slope of the count value as will be described later. The width may be two pixels or more.

Next, the moving speed, moving direction, and position of the edge are calculated.
First, as shown in the following formula (1), the inclination of the count value in the moving direction is calculated.
For example, as shown in FIG. 4E, the count value at position [x0-1] is “6”, the count value at position [x0] is “4”, and the count value at position [x0 + 1] is “2”. In this case, the slope of the count value is set to H = (6-2) / 2 = 2 by subtracting the count value “2” of the position [x0 + 1] from the count value “6” of the position [x0-1]. calculate.

  This is because H = {(time from the edge moving to the position [x0-1] to the present) − (time after the edge has moved to the position [x0 + 1])} / (two pixels) This means that the time (number of frames) required for the edge to pass through one pixel having the position [x0] is calculated.

  Accordingly, the slope H of the count value is to determine how many frames are required for the edge to move by one pixel, and the edge moving speed (1 / H) is calculated based on the slope H of the count value. Can do. In FIG. 4 (e), two frames are required to move one pixel (H = 2), so the edge moving speed can be calculated as 1/2 (pixel / frame).

Further, the edge moving direction can be determined by the magnitude of the count value.
The count value at the position where the edge is moved and the edge is newly detected is “1”, which is the smallest value among the count values at each position. Therefore, the count value in the direction in which the edge moves is small, and the count value in the direction opposite to the direction in which the edge moves is large.

  As described above, the count value of the position where the edge is detected is counted up, and the moving speed and moving direction of the edge can be calculated based on the slope of the counted-up count value.

  Next, a velocity image is generated in which edge movement information existing on the captured image is classified into predetermined class values.

FIG. 5 is a diagram illustrating an example of a speed image.
As shown in FIG. 5, the speed image represents the edge pixels from which the movement information is detected as round dots, and the pixels having a higher movement speed indicate the points larger. Further, the moving speed toward the right is represented by a black dot, and the moving speed toward the left is represented by a white dot, thereby representing the moving direction.
The vertical edge extraction unit 106 extracts a vertical edge that is a candidate for a three-dimensional object based on the characteristic that pixels having the same speed exist continuously in the vertical direction as the characteristic of the speed of the three-dimensional object.

FIG. 6 is a diagram illustrating a vertical edge extraction method.
For example, as shown in FIG. 6, when a speed image is searched in the vertical direction (y-axis direction) and a pixel A having speed information is found, the next pixel B is searched and the pixel is set to speed information. And the direction of speed is the same as that of the pixel A, and the difference in speed magnitude between the pixel A and the pixel B (V2−V1) is within the threshold value Tv, it is determined that they are vertically continuous. . Next, similarly for the pixel C, it is determined whether or not the presence / absence of speed information, the direction of speed, and the difference in speed magnitude (V3−V1) are within the threshold value Tv.

  Then, when the condition is not satisfied at the pixel next to the pixel N, the number of pixels that have satisfied the condition is counted, and if the count value is equal to or greater than the threshold value TH, the vertical edge from the pixel A to the pixel N is Extract as vertical edges.

  The lower end point extraction unit 107 extracts the lower end point of the vertical edge extracted by the vertical edge extraction unit 106. That is, the pixel N counted last when the vertical edge extraction unit 106 extracted is extracted as the lower end point.

FIG. 7 is a diagram in which the lower end point is extracted from the vertical edge.
For example, the lower end points 1 to 3 are extracted from the vertical edges 1 to 3, respectively.
The stationary object edge extraction unit 108 extracts the vertical edge belonging to the stationary object from the information of the lower end point of the vertical edge extracted by the lower end point extraction unit 107 and the vehicle speed signal of the host vehicle input from the vehicle information input unit 102. To do.

  First, from the position on the image of the lower end points 1 to 3, the position on the bird's eye coordinate centered on the camera 101 mounted on the vehicle is calculated.

FIG. 8 is a diagram illustrating conversion to the overhead view coordinate system.
A known method may be used to convert the coordinates on the image to the overhead view coordinates, for example, by using information such as the height of the camera from the ground, the mounting angle of the camera, and the like.

  Then, after acquiring the position of the lower end point on the overhead view coordinates, the speed of the lower end point on the image when it is assumed to be a stationary object is calculated. Specifically, when the lower end point is assumed to be a stationary object, the speed of the lower end point in the overhead coordinate system can be regarded as the speed having the same magnitude as the host vehicle speed Vv and the reverse direction. Furthermore, if one frame time (for example, 33 msec) is Δt and the vehicle motion is approximated by a straight-ahead motion, the speed direction of the lower end point can be regarded as the −Z direction on the overhead view coordinates.

  Therefore, as shown in the following equation (1), the lower end of Δt is based on the own vehicle speed Vv, the frame time Δt, the x coordinate value X of the lower end point on the overhead coordinate, and the z coordinate value Z of the lower end point on the overhead coordinate. A horizontal angle change Δθ of the point is obtained.

Further, the estimated image speed Vg of the vertical edge including the lower end point when assumed to be a stationary object can be calculated by the following equation (3) when K is defined by the following equation (2).
K = Effective horizontal angle of view of camera / Effective number of horizontal pixels of camera ......... (2)
Vg = K × Δθ (3)

  Assuming that the image speed of the actual vertical edge calculated from the acquired image is V with respect to the estimated image speed Vg, the vertical edge is a stationary object when the difference (absolute value) between Vg and V is equal to or less than the threshold value TH2. Is determined.

  The cut edge extraction unit 109 determines whether the vertical edge determined to be a stationary object is a candidate for a cut of a continuous stationary object, the vehicle information input from the vehicle information input unit 102, and the width of the movable object that can be passed. Judgment is made from the assumed determination area.

FIG. 9 is a diagram illustrating a vertical edge determined as a stationary object.
First, the coordinates on the image of the vertical edge determined as a stationary object are converted into overhead coordinates with the vehicle-mounted camera as the origin.

FIG. 10 is a diagram in which a stationary object is projected onto overhead coordinates.
After the conversion, it is determined whether each stationary object has another stationary object in a direction perpendicular to the vehicle traveling direction. For example, with respect to the stationary object 1 in FIG. 10, if the coordinate values on the overhead coordinates of the stationary object 1 are (X1, Z1), does the stationary object exist in the x-axis direction within the width Zh in the z direction? Search for it. Zh is determined in consideration of, for example, a conversion error to overhead coordinates and a coordinate measurement error in image coordinates.

  In this case, when the overhead coordinates of the stationary object 2 are (X2, Z2), in the example shown in FIG. 10, Z1−Zh ≦ Z2 ≦ Z1 + Zh, and the stationary object 2 exists in the search range of the stationary object 1 To do. As a result of the search, if there are a plurality of stationary objects in the same search range, only stationary objects whose x coordinate of the stationary object is closer to the own vehicle, that is, whose absolute value of the X coordinate is small are selected. For example, in the example of the stationary object 1 and the stationary object 2, since | X2 | <| X1 | as shown in FIG. 10, the stationary object 2 is selected here.

  The same processing is performed for the stationary object 3 and the subsequent ones, and the one having the smallest absolute value of the X coordinate is selected from the search range of each stationary object. In the example shown in FIG. 10, stationary objects 2 to 6 are selected.

Next, a determination area is set assuming a passable width of the moving body.
FIG. 11 is a diagram showing a passable determination area.
For example, consider a rectangular region R in which the threshold value for the length in the z-axis direction in the overhead view is W and the threshold value for the length in the x-axis direction is L. Here, the threshold value W may be determined assuming a width that allows the object to be determined to fly out to pass. If a pedestrian or the like is assumed, for example, the threshold value W is set to about W = 0.6 m. The threshold value L may be determined in consideration of the conversion error to the overhead view coordinates and the coordinate measurement error in the image coordinates, similar to Zh described above. Further, the angle θ made with the Z axis of the overhead view coordinates of the region R is determined based on the vehicle information. For example, the rudder angle information is acquired, the actual rudder angle θv of the host vehicle is obtained, and θ = θv may be obtained, or the road curvature of the own vehicle course is obtained, and the angle calculated from the road curvature may be θ.

  Next, the region R is applied to the selected stationary object, and when there is no other stationary object in the region R, the stationary object to which the region R is applied is determined to be a continuous stationary object break candidate. .

FIG. 12 is a diagram illustrating break determination.
For example, the region R is set for the selected stationary object 2. Here, assuming that the vehicle is traveling straight and θ = 0, when region R is set, X2−L / 2 ≦ X ≦ X2 + L with respect to the position (X2, Z2) of the overhead coordinate of stationary object 2 / 2, Z2 ≦ Z ≦ Z2 + W. In this range, it is determined whether or not there is another screened stationary object. In FIG. 12, since the stationary object 3 exists in the range set for the stationary object 2 and the stationary object 4 exists in the range set for the stationary object 3, the stationary objects 2 to 4 are continuous stationary objects. Judge that there is. Moreover, since the stationary object 6 exists in the range set to the stationary object 5, it determines with the stationary objects 5 and 6 being a continuous stationary object. On the other hand, since the stationary object 5 is out of the range set for the stationary object 4, the stationary object 4 is determined as a break candidate.

  The pop-out determination unit 110 performs a pop-out determination process on the stationary object extracted as the break candidate, and outputs a detection result when the pop-out is detected.

FIG. 13 is a diagram illustrating pop-out detection.
FIG. 13A shows a schematic diagram of a vertical edge corresponding to a break at a certain time t = t1. Here, when the velocity value of the vertical edge A at the time of cut extraction when Vo = t1 is Vo, the movement time ts during which the vertical edge A at the time of cut extraction moves one pixel or more can be obtained by ts = 1 / Vo. it can.

  FIG. 13B shows a schematic diagram when the moving object does not appear behind the stationary object at time t = t1 + ts. When the vertical edge B corresponding to the break is extracted again after the movement time ts has elapsed from t1, it moves one pixel in the direction of the image speed observed when t = t1. In FIG. 13B, one pixel is moved to the left side of the image. At this time, if the velocity value of the vertical edge B corresponding to the break is Vp1, the relationship Vp1 ≧ Vo holds.

  FIG. 13C shows a schematic diagram when a moving object appears behind a stationary object at a certain time t = t1 + ts. When the vertical edge C corresponding to the break is extracted again after the movement time ts has elapsed from t1, it is observed at substantially the same position as observed when t = t1. If the velocity value of the vertical edge C is Vp2, the relationship Vp2 ≦ Vo holds.

  Therefore, the pop-out determination unit 110 first calculates a movement time ts during which the vertical edge A corresponding to the break moves by one pixel or more on the image at the time t = t1. For example, it is calculated as a movement time ts = 2 / Vo for moving two pixels or more. Further, the position Xo of the vertical edge A on the image and the speed information Vo when t = t1 are stored in the storage area. For example, a vertical edge A when t = t1 is set as an edge at the time of cut extraction, the x coordinate value on the image is Xo, the minimum value of the y coordinate on the image is Yo, and these (A, Xo, Yo, Vo) ) As a set in a storage area on the memory. At this time, Xo is set as a break position.

  Next, after the movement time ts has elapsed from the time t1 (t = t1 + ts), the position and speed value of the vertical edge are observed again to determine whether or not there is a projecting object. First, the vertical edge closest to the cut-out vertical edge A stored in the above process is extracted. When the cut edge at t1 extracted when t = t1 + ts is extracted, the x coordinate value on the image of the vertical edge closest to the vertical edge A is Xp, and when | Xp−Xo | ≧ Xth, the projecting object is Judge that there is no. On the other hand, in the case of | Xp−Xo | <Xth, a determination process using a speed value is performed. Here, Xth is a threshold value for position change determination, and is calculated by Xth = Vo × Ts.

  For example, when the velocity value of the vertical edge A at the time of cut extraction is Vo and the velocity value of the vertical edge extracted when t = t1 + ts is Vp, the determination process of the velocity value is | Vp | When − | Vo | ≦ Vth, it is determined that there is a flying object, and when | Vp | − | Vo |> Vth, it is determined that there is no flying object. Here, Vth is a threshold for speed value determination. For example, assuming that Vp is equal to or lower than the same speed as Vo, Vth may be set to 0, or stays at a certain image position until the movement time ts elapses. In this case, Vf may be calculated as the image speed when Vth = (| Vf | − | V0 |) / 2.

  With the above processing, after the popping-out determination unit 110 performs the popping-out determination processing, the determination result is output.

FIG. 14 is a flowchart showing a vehicle outside recognition process.
This process is executed as a program that is activated when an ignition switch (not shown) is turned on.

First, in step S101, an image in front of the host vehicle captured by the camera 101 is output to the edge extraction unit 104 at predetermined intervals.
In subsequent step S102, the edge extraction unit 104 performs edge extraction processing on the image, extracts the contour of the object existing in the captured image as an edge image, and normalizes the edge image (binarization processing, thin line). Processing, expansion processing).

In the subsequent step S103, after the edge image count-up process is performed, the movement information calculation unit 105 calculates the speed of the edge, and calculates a speed image in which the calculated movement information is represented by a predetermined gradation.
In subsequent step S104, a vertical edge that is a candidate for a three-dimensional object is extracted from the calculated velocity image.

In subsequent step S105, it is determined whether or not extraction processing has been performed on all pixels. If all extraction has been completed, the process proceeds to step S106. On the other hand, when all the extractions are not completed, the process proceeds to step S104.
In step S106, the image coordinate position of the lower end point of the vertical edge extracted in step S104 is extracted.

In subsequent step S107, the image coordinate position of the lower end point of the vertical edge extracted in step S106 is coordinate-transformed into an overhead coordinate system with the camera 101 as the origin and the own vehicle traveling direction as the Z axis.
In the subsequent step S108, it is determined whether or not the vertical edge is a stationary object based on the position of the lower end point of the vertical edge subjected to coordinate conversion and the speed information of the host vehicle input from the vehicle information input unit 102.

In subsequent step S109, among the vertical edges determined as stationary objects in step S108, those which are close to the vehicle path on the overhead coordinates are extracted.
In the subsequent step S110, the region R assuming the width of travel of the moving object is applied to the vertical edge determined as a stationary object close to the path of the host vehicle extracted in step S109, and two or more regions are applied to the applied region R. When there is no vertical edge determined to be a stationary object, the vertical edge to which the region R is applied is determined as a continuous stationary object break candidate.

In the subsequent step S111, the moving time ts required for the vertical edge to move by a predetermined number of pixels or more is calculated from the speed information of the vertical edge determined as the continuous object break candidate in step S110.
In the subsequent step S112, the vertical edge corresponding to the cut is set as a vertical edge at the time of cut extraction, the x coordinate value of the vertical edge at the time of cut extraction is set as the cut position, and the vertical edge and the cut position at the time of cut extraction are stored as cut information. To store.

In subsequent step S113, it is determined whether or not the movement time ts calculated in step S111 has elapsed. When the movement time ts has elapsed, the process proceeds to step S114. On the other hand, if the movement time ts has not elapsed, the process returns to step S101.
In step S114, the currently observed vertical edge closest to the vertical edge position information stored in step S112 is extracted.

  In the subsequent step S115, the position of the vertical edge when extracting the break at the time t1 stored in step S112 is compared with the position information of the vertical edge after the elapse of ts extracted again in step S114, and there is a position change over the threshold Xth. It is determined whether or not. If there is a position change equal to or greater than the threshold value Xth, the process proceeds to step S118. If there is no position change equal to or greater than the threshold value Xth, the process proceeds to step S116.

In step S116, the vertical edge speed information stored in step S112 is compared with the vertical edge speed information extracted in step S114, and it is determined whether or not the speed is slower than the threshold Vth. If the speed is not slower than the threshold Vth, the process proceeds to step S118. If the speed is slower than the threshold Vth, the process proceeds to step S117.
In step S117, based on the determination results in steps S115 and S116, it is determined that the vertical edge extracted in step S114 is a flying object.

In the subsequent step S118, it is determined whether or not the jump determinations in steps S114 to S117 have been made for all the cut candidates extracted in step S110. If all the cut candidates are processed, the process proceeds to step S119. Transition. On the other hand, if not done for all, the process proceeds to step S114.
In step S119, the determination result of the presence or absence of the projecting object is output.

  In a succeeding step S120, it is determined whether the ignition switch is turned off. If the ignition switch is turned off, the vehicle environment recognition process is terminated. On the other hand, if the ignition switch is not turned off, the process returns to step S101.

<Action>
First, an edge image is extracted by performing binarization processing, thinning processing, and expansion processing on an image captured in front of the host vehicle (S102), and further, an image at the edge extracted by performing count-up processing. The moving speed and moving direction are calculated (S103).

  Then, among the extracted edges, vertical edges that are continuous in the vertical direction at the same speed on the image are extracted (S104), and among the extracted vertical edges, vertical edges belonging to a stationary object are extracted (S108) and extracted. Of the vertical edges belonging to the stationary object, a vertical edge corresponding to a break in the direction of the vehicle in the stationary object is extracted based on a predetermined passable width W (S110). At this time, the vertical edge corresponding to the cut is set as a vertical edge at the time of cut extraction, the position of the vertical edge at the time of cut extraction is set as a cut position, and each is stored as cut information (S112).

  Then, after a predetermined movement time ts has elapsed after storing the break information, the vertical edge closest to the break position Xo among the vertical edges extracted anew is compared with the vertical edge A during break extraction. Then, it is determined whether or not the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut (S113 to S117).

  In this way, the vertical edge corresponding to the break of the stationary object is extracted, the break information of the stationary object is stored, and the projecting object is detected based on the temporal change of the vertical edge at the break position. Rather than detecting an obstacle based on this, it is possible to improve the detection accuracy of an object that can be an obstacle.

  For extraction of the vertical edge corresponding to the cut, first, the lower end point of the vertical edge is extracted (S106), the image is transformed into an overhead image, and the lower end point is projected onto the overhead image to generate an overhead image (S107). )

  And among the vertical edges belonging to the stationary object, when the distance between the first vertical edge and the second vertical edge adjacent to each other along the own vehicle path on the overhead image exceeds the passable width W. The first vertical edge is extracted as a vertical edge corresponding to the break.

  As described above, the vertical edge belonging to the cut can be easily extracted by simply applying the passable width W to the first vertical edge and determining whether or not the second vertical edge is included in the range. it can.

  Then, in the determination of the projecting object, the distance (Xp−Xo) in the x-axis direction between the position Xp of the vertical edge closest to the cut position and the cut position Xo on the image is greater than a predetermined distance threshold value Xth. If it is short (determination in S115 is “Yes”), it is determined that the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut (S117).

As described above, it is possible to easily detect the projecting object simply by comparing the position Xp of the vertical edge newly extracted when the movement time ts has elapsed from t1 and the break position Xo.
Further, in the determination of the projecting object, the moving speed Vp of the vertical edge closest to the cut position is slower than the moving speed Vo of the vertical edge when the cut is extracted, and the speed difference (| Vp | − | Vo |) When it is larger than the predetermined speed difference threshold Vth (determination in S116 is “Yes”), it is determined that it is determined that the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut.

  As described above, it is possible to easily detect the projecting object simply by comparing the moving speed Vp of the vertical edge newly extracted when the moving time ts elapses from t1 and the moving speed Vo of the edge at the time of cut extraction. it can.

"effect"
From the above, the vehicle information input unit 102 corresponds to “vehicle speed information acquisition means”, the camera 101 corresponds to “imaging means”, the processing in step S102 corresponds to “edge extraction means”, and the processing in step S103 is “ Corresponding to “movement information calculating means”, the process of step S104 corresponds to “vertical edge extracting means”. Further, the process of step S108 corresponds to “stationary object edge extraction means”, the process of step S110 corresponds to “break edge extraction means”, the process of step S112 corresponds to “break information storage means”, and step S111. , S113 to S117 correspond to “jumping object determination means”. Further, the processing in step S106 corresponds to “lower end point extraction means”, and the processing in step S107 corresponds to “overhead image generation means”.

(1) Vehicle speed information acquisition means for acquiring vehicle speed information of the own vehicle, imaging means for imaging the front of the own vehicle, edge extraction means for extracting an edge from an image captured by the imaging means, and the edge extraction means The movement information calculation means for calculating the movement speed and movement direction on the image at the extracted edge as movement information, and among the edges extracted by the edge extraction means, vertical edges that are continuous in the vertical direction on the image, Of the vertical edges extracted by the vertical edge extracting means and the vertical edge extracting means that extracts based on the movement information calculated by the movement information calculating means, the vertical edge belonging to a stationary object is calculated by the movement information calculating means. Movement information and stationary object edge extracting means for extracting based on vehicle speed information acquired by the vehicle speed information acquiring means, and the stationary object extracted by the stationary object edge extracting means A vertical edge corresponding to a cut in the direction of own vehicle in the stationary object among the vertical edges belonging to the cut edge extraction means for extracting the vertical edge based on a predetermined passable width at which the mobile body can pass, and Break information storage for storing the vertical edge extracted by the cut edge extraction means as a vertical edge at the time of cut extraction, the position of the vertical edge at the time of cut extraction as a cut position, and storing the vertical edge and the cut position at the time of cut extraction as cut information And the break information stored in the break information storage means among the vertical edges newly extracted by the vertical edge extraction means after a predetermined time has passed since the break information was stored in the break information storage means. By comparing the vertical edge closest to the position with the vertical edge at the time of cut extraction stored in the cut information storage means, Nearest vertical edge position and a object determination means jumping determines whether the vertical edges belonging to the jumping object from the cut.

  In this way, the vertical edge corresponding to the break of the stationary object is extracted, the break information of the stationary object is stored, and the projecting object is detected based on the temporal change of the vertical edge at the break position. Rather than detecting an obstacle based on this, it is possible to improve the detection accuracy of an object that can be an obstacle.

(2) Lower end point extracting means for extracting the lower end point of the vertical edge extracted by the vertical edge extracting means, and lower end obtained by performing coordinate conversion of the image into an overhead image and extracting the image on the overhead image by the lower end point extracting means. An overhead image generation means for generating an overhead image by projecting a point, and the cut edge extraction means is an overhead image generated by the overhead image generation means among the vertical edges extracted by the stationary object edge extraction means The first vertical edge corresponds to the cut when the distance between the first vertical edge and the second vertical edge adjacent to each other along the own vehicle path exceeds the passable width. Extract as vertical edges.

  As described above, the vertical edge belonging to the cut can be easily extracted by simply applying the passable width to the first vertical edge and determining whether or not the second vertical edge is included in the range. .

(3) The pop-out object determination means, when the horizontal distance between the position of the vertical edge closest to the cut position and the cut position on the image is shorter than a predetermined distance threshold, It is determined that the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut.

  As described above, the projecting object can be easily detected only by comparing the position of the vertical edge newly extracted when a predetermined time has elapsed and the position of the break.

(4) The projecting object determination means is such that the moving speed of the vertical edge closest to the cut position is slower than the moving speed of the vertical edge when the cut is extracted, and the speed difference is greater than a predetermined speed difference threshold value. Is also determined to be determined that the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut.

  In this way, it is possible to easily detect the projecting object simply by comparing the moving speed of the vertical edge newly extracted when a predetermined time has elapsed and the moving speed of the edge at the time of cut extraction.

(5) Obtaining vehicle speed information of the host vehicle, extracting an edge from an image obtained by imaging the front of the host vehicle, calculating a moving speed and a moving direction on the image at the edge as movement information, and extracting the edge Among them, vertical edges that are continuous in the vertical direction on the image are extracted based on the movement information, and among the extracted vertical edges, vertical edges belonging to a stationary object are extracted based on the movement information and vehicle speed information. Of the vertical edges belonging to the stationary object extracted, the vertical edge corresponding to the cut in the direction of the vehicle in the stationary object is based on a predetermined passable width at which the mobile body can pass. The vertical edge corresponding to the cut is defined as a vertical edge at the time of cut extraction, the position of the vertical edge at the time of the cut extraction is defined as a cut position, and the vertical edge and the cut position at the time of the cut extraction are cut. After storing a predetermined time after storing the cut information, the vertical edge closest to the cut position among the newly extracted vertical edges is compared with the vertical edge at the time of cut extraction. Thus, it is determined whether or not the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut.

DESCRIPTION OF SYMBOLS 10 Outside-vehicle environment recognition apparatus 100 Image processing part 101 Camera 102 Vehicle information input part 103 Image temporary recording part 104 Edge extraction part 105 Movement information calculation part 106 Vertical edge extraction part 107 Lower end point extraction part 108 Stationary object edge extraction part 109 Eye edge extraction Part 110 Pop-up judgment part

Claims (5)

Vehicle speed information acquisition means for acquiring vehicle speed information of the host vehicle;
Imaging means for imaging the front of the host vehicle;
Edge extraction means for extracting an edge from an image captured by the imaging means;
Movement information calculation means for calculating the movement speed and movement direction of the edge extracted by the edge extraction means on the image as movement information;
A vertical edge extracting unit that extracts vertical edges that are continuous in the vertical direction on the image based on the movement information calculated by the movement information calculating unit, among the edges extracted by the edge extracting unit;
Of the vertical edges extracted by the vertical edge extraction means, the vertical edges belonging to a stationary object are extracted based on the movement information calculated by the movement information calculation means and the vehicle speed information acquired by the vehicle speed information acquisition means. Edge extraction means;
Of the vertical edges belonging to the stationary object extracted by the stationary object edge extracting means, a predetermined passable width that allows the moving body to pass through a vertical edge corresponding to a break in the direction of the vehicle of the stationary object. Cutting edge extraction means for extracting based on
The vertical edge extracted by the cut edge extraction means is a vertical edge at the time of cut extraction, the position of the vertical edge at the time of the cut extraction is set as a cut position, and the cut edge information is stored as the cut edge information. Storage means;
After a predetermined time elapses after the break information is stored in the break information storage means, the vertical edge that is newly extracted by the vertical edge extraction means is the highest in the cut position stored in the break information storage means. Whether or not the vertical edge closest to the cut position is a vertical edge belonging to the projecting object from the cut by comparing the close vertical edge and the vertical edge at the time of cut extraction stored in the cut information storage means A vehicle-outside environment recognition device comprising: a projecting object determination unit that determines Lower end point extracting means for extracting the lower end point of the vertical edge extracted by the vertical edge extracting means;
A bird's-eye view image generation means for generating a bird's-eye view image by converting the image into a bird's-eye view image and projecting the lower end point extracted by the lower end point extraction means onto the bird's-eye view image;
The cut edge extracting means includes a first vertical edge and a second vertical edge that are adjacent to each other along the own vehicle path on the overhead image generated by the overhead image generating means among the vertical edges extracted by the stationary object edge extracting means. The outside environment recognition according to claim 1, wherein when the distance between the vertical edges exceeds the passable width, the first vertical edge is extracted as a vertical edge corresponding to the cut. apparatus.   The pop-out object determining means, when the horizontal distance between the position of the vertical edge closest to the cut position and the cut position on the image is shorter than a predetermined distance threshold, 3. The external environment recognition device according to claim 1, wherein the vertical edge closest to is determined to be a vertical edge belonging to a projecting object from the cut.   The popping-out object determination unit is configured such that the moving speed of the vertical edge closest to the cut position is slower than the moving speed of the vertical edge when the cut is extracted, and the speed difference is larger than a predetermined speed difference threshold value. The vehicle exterior environment according to any one of claims 1 to 3, wherein it is determined that the vertical edge closest to the cut position is a vertical edge belonging to a projecting object from the cut. Recognition device.   Obtaining vehicle speed information of the host vehicle, extracting an edge from an image obtained by capturing the front of the host vehicle, calculating a moving speed and a moving direction of the edge on the image as moving information, and out of the extracted edge, Vertical edges continuous in the vertical direction on the image are extracted based on the movement information, and among the extracted vertical edges, vertical edges belonging to a stationary object are extracted based on the movement information and vehicle speed information. The vertical edge corresponding to the cut in the direction of the vehicle in the stationary object is extracted based on a predetermined width that allows the mobile body to pass, among the extracted vertical edges belonging to the stationary object, The vertical edge corresponding to the cut is defined as a vertical edge at the time of cut extraction, the position of the vertical edge at the time of cut extraction is defined as a cut position, and the vertical edge and the cut position at the time of cut extraction are defined as break information. Remember, after a predetermined time has elapsed since the storing of the cut information, the vertical edge closest to the cut position among the vertical edges extracted again is compared with the vertical edge at the time of the cut extraction. Then, it is determined whether or not the vertical edge closest to the cut position is a vertical edge belonging to a projecting object from the cut.

Images