JP5724446B2 - Vehicle driving support device - Google Patents

Description

  The present invention relates to an apparatus for supporting a driver who is driving a vehicle by displaying the vehicle and the vicinity of the vehicle and the front of the vehicle on a monitor screen so that the driver can see the vehicle. It is.

  In order to support driving in this manner, a technique that combines a plurality of viewpoint videos and displays them on a single screen and can help driving by visually checking this screen, for example, It is particularly known as described in Patent Document 1.

  This proposed technology is located at the upper edge of the overhead view screen that displays the surrounding area of the former vehicle, among the screen that displays the overhead view image of the vicinity of the vehicle viewed from above the vehicle and the screen that displays the road environment image in front of the vehicle. Both screens are displayed on the monitor side by side in the vertical direction so that the lower edge of the front road screen that reflects the latter road environment is continuous.

JP 2002-109697 A

However, in the proposed technique described in Patent Document 1, the vehicle surroundings overhead image viewed from above the vehicle and the road environment image viewed from the front of the vehicle are displayed on the monitor side by side in the vertical direction. Lack of feeling.
Therefore, a single obstacle on the front road is temporarily displayed on both the vehicle surroundings overhead image and the road environment image arranged in the vertical direction, or this obstacle is arranged around the vehicle in the vertical direction. It may not be displayed on either the overhead view image or the road environment image.

  Furthermore, after the same obstacle disappears from the road environment image above, it appears difficult to grasp the environment around the vehicle from the display in both images, such as appearing from a shifted position in the overhead image around the vehicle below. The problem also arises.

  The present invention provides a driving support apparatus for a vehicle that can solve the above-mentioned problem by making the image displaying the periphery of the vehicle and the road environment image ahead have continuity. For the purpose.

For this purpose, the vehicle driving support apparatus according to the first invention is configured as follows.
First, a vehicle driving support device that is a premise of the present invention displays the host vehicle and the vicinity of the host vehicle and the front of the host vehicle on a monitor screen so that the driver can visually recognize the vehicle, thereby supporting driving of the vehicle. It is a thing.

The present invention provides the following driving assistance apparatus for a vehicle, including the following vehicle periphery image creation means, forward travel road image creation means, image connection means, and travel path passable width display line setting means. Characterized by the composition.

The vehicle surrounding image creation means creates a vehicle surrounding image displayed as a bird's-eye view of the traveling path around at least the front end of the vehicle and the front end of the vehicle as if looking down at the vehicle.
The forward travel path image creation means is converted so that the flow backward to the side portion at a predetermined position in the forward travel path of the host vehicle during straight traveling is along the longitudinal direction of the host vehicle, A forward traveling road image displayed as a bird's-eye image viewed from the own vehicle with a predetermined look-down angle is created.

Further, the image connecting means uses the vehicle surrounding image and the forward traveling road image created by the own vehicle surrounding image creating means and the forward traveling road image creating means as a traveling direction crossing line on the front side of the traveling direction of the own vehicle surrounding image. to, to overlap the traveling direction transverse line relative to the vehicle traveling direction rear side of the front traveling road image, without a continuous image by interconnected state, and are not for displaying the successive image on the monitor screen,
The travel path passable width display line setting means is a substantially straight travel path passable width display line separated from each other and indicating the passable width of the front travel path displayed in the vehicle periphery image or the front travel path image. , said Ru der shall be set on the vehicle peripheral image and so along the travel path in front the travel path image arranged within said continuous image.
By the way, the travel path passage width display line setting means described above can set the travel path passage width display line, and can pass the travel path provided in the own vehicle pass width display line and the forward travel path image provided in the vehicle periphery image. Width display lines shall be set individually and superimposed,
Further, the image connecting means is configured such that when the own vehicle periphery image and the forward traveling road image are connected, the own vehicle passage width display line or the traveling road passage width display line of the own vehicle periphery image and the forward traveling road image are substantially mutually omitted. It is corrected so as to be connected in a straight line, and the vehicle periphery image and the forward traveling road image are connected.

In the vehicle driving support device according to the present invention described above,
The vehicle periphery image related to the travel path around at least the front end of the vehicle and the front end of the vehicle is an overhead image viewed as if looking down directly on the vehicle,
The front road image related to the road ahead of the host vehicle is converted so that the backward flow of the side portion at a predetermined position in the front road of the host vehicle when traveling straight ahead is along the longitudinal direction of the host vehicle. Because the road ahead is a bird's-eye view seen from the vehicle with a predetermined angle,
The vehicle periphery image and the front travel path image are mutually connected so that the travel direction cross line on the front side in the vehicle travel direction of the host vehicle periphery image overlaps the travel direction cross line on the rear side in the vehicle travel direction of the front travel path image. When a continuous image is obtained by connecting to the vehicle, the continuity between the vehicle periphery image and the front traveling road image can be easily made excellent.

  Therefore, a single obstacle on the front road is temporarily displayed on both the vehicle periphery image and the front road image connected in the vertical direction. There is no problem that it is not displayed on any of the traveling road images.

Furthermore, if the obstacle is the same, this obstacle will appear from the same position in the surrounding image of the vehicle below after disappearing from the upper front road image.
After the same obstacle disappears from the upper front road image, it appears from a shifted position of the vehicle periphery image below, and it is difficult to grasp the environment around the vehicle from the display in both images Can also be avoided.
In addition, when the travel path passable width display line setting means sets the travel path passable width display line, the vehicle passable width display line provided in the surrounding image of the host vehicle and the travel path passable provided in the forward travel path image are possible. Width display lines are set individually and superimposed,
In addition, when the image connecting means connects the vehicle periphery image and the forward travel road image, the vehicle passable width display line or the travel path passable width display line of the vehicle peripheral image and the forward drive road image are substantially linear with each other. In order to correct the connection and connect the vehicle periphery image and the front road image,
In addition to being able to judge intuitively whether there are obstacles that prevent the passage of the vehicle on the forward road, it is possible to grasp the boundary of the forward road area where the presence of obstacles should be checked when the vehicle is going straight ahead. It becomes easy.

1 is a functional block diagram showing an overall configuration of a driving assistance apparatus according to a first embodiment of the present invention. FIGS. 2A and 2B are diagrams showing the installation location of the camera shown in FIG. 1 with respect to the vehicle, in which FIG. 1A is a plan view seen from above the vehicle, and FIG. 2B is a side view seen from the left side of the vehicle. 2 is an optical flowchart showing a driving assistance image creation program executed by the control unit in FIG. FIG. 3 is a diagram showing a virtual camera position of a camera installed as shown in FIG. 2, (a) is a plan view seen from above the vehicle, (b) is a side view seen from the left side of the vehicle, c) is a side view showing the imaging region at the upper virtual camera position when viewed from the left side of the vehicle. It is explanatory drawing about the connection of the bird's-eye view image and the bird's-eye view image constituting the driving support image, (a) is an explanatory diagram when the connection boundary line is a straight line, (b) is the connection boundary line is curved It is explanatory drawing in the case. (A) is an explanatory diagram showing the bird's-eye view image and the bird's-eye view image individually in a state before connection, when the bird's-eye view image and the bird's-eye view image constituting the driving support image are distorted. b) is an explanatory diagram of a method for matching a bird's-eye view image and a bird's-eye view image at a connection boundary line. (A) is an overall plan view of the front view of the area around the front of the vehicle viewed from directly above the vehicle, and (b) is the area around the front of the vehicle. An explanatory diagram of a bird's-eye view of the bird's-eye view image, not a viewpoint from directly above, but with a certain bird's-eye view angle (looking-down angle). An explanatory diagram of the bird's-eye view image displayed so as to be perpendicular to the screen. (D) is an explanation of the bird's-eye image displayed such that the vehicle-passable width display line in the upper right region of the bird's-eye view image is perpendicular to the monitor screen. FIG. 4E is an explanatory diagram showing a composite image of the bird's-eye view images in (c) and (d). FIG. 5 is an explanatory diagram showing the positional relationship among three positions of the own vehicle in a narrow path, a road (wall), and a three-dimensional obstacle such as a utility pole as viewed from above. Fig. 8 shows the monitor screen when the vehicle is in position 3 in Fig. 8, (a) is a front view showing the monitor screen when the vehicle is in the first position in Fig. 8, and (b) is FIG. 9C is a front view showing the monitor screen when in the second position in FIG. 8, and FIG. 9C is a front view showing the monitor screen when the vehicle is in the third position in FIG. It is an operation explanatory view when the own vehicle is in the center of the road width direction, (a) is a plan view looking down from above the own vehicle, (b) is a front view showing a monitor screen. It is an operation explanatory view when the own vehicle is in the right position in the road width direction, (a) is a plan view looking down from the upper side of the own vehicle, (b) is a front view showing a monitor screen. FIG. 5 is a functional block diagram showing an overall configuration of a driving assistance apparatus according to a second embodiment of the present invention. 13 is an optical flowchart showing a driving assistance image creation program executed by the control unit in FIG. It is an operation explanatory view showing the positional relationship between two positions of the own vehicle on a narrow path, a road (wall), and a three-dimensional obstacle such as a utility pole as viewed from above. 14 shows a monitor screen when the vehicle is in the two positions in FIG. 14, (a) is a front view showing the monitor screen when the vehicle is in the first position in FIG. 14, and (b) is a view of the own vehicle. FIG. 15 is a front view showing a monitor screen when in the second position of FIG. It is an operation explanatory view when the own vehicle is in the center of the road width direction, (a) is a plan view looking down from above the own vehicle, (b) is a front view showing a monitor screen. It is an operation explanatory view when the own vehicle is in the road width direction left side position, (a) is a plan view looking down from the upper side of the own vehicle, (b) is a front view showing a monitor screen.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<Configuration of the first embodiment>
FIG. 1 is a functional block diagram showing the overall configuration of the driving support apparatus according to the first embodiment of the present invention, where 1 is a front end camera, 2 is a right camera, and 3 is a left camera.

As shown in FIGS. 2 (a) and 2 (b), the front end camera 1 is disposed at the front end portion 10F of the host vehicle 10 near the center in the vehicle width direction.
For this installation, the front-end camera 1 is directed in front of the vehicle traveling direction and 45 ° forward and downward.
The front end camera 1 is a wide-angle camera (the angle of view is, for example, 180 °), and the front peripheral image (the first peripheral image) relating to the front end portion 10F of the own vehicle including the left and right corners of the front end portion 10F of the own vehicle and the surrounding traveling road portion. A bird's-eye view image) is taken.
Here, the bird's-eye view image means a front image viewed from the own vehicle with a predetermined look-down angle as shown in FIG. 2 (b).

As shown in FIGS. 2 (a) and 2 (b), the right camera 2 is a wide-angle camera (angle of view is 180 °, for example) installed in the vicinity of the right door mirror 10R of the host vehicle 10 and the right door mirror 10R. It is assumed that a right peripheral image (overhead image) including a side surface of the vehicle 10 in the vicinity and a surrounding traveling road portion is taken.
Here, the bird's-eye view image means a planar image as seen when looking down at the own vehicle 10.
As shown in FIGS. 2 (a) and 2 (b), the left camera 3 is a wide-angle camera (angle of view is 180 °, for example) installed in the vicinity of the left door mirror 10L of the host vehicle 10, and the left door mirror 10L It is assumed that a left peripheral image (overhead image) including a side surface of the own vehicle 10 in the vicinity and a surrounding traveling road portion is taken.

  However, the installation location, orientation, angle of view, etc. of the cameras 1 to 3 are not limited to those described above, and the previous peripheral image and right peripheral as described above using existing technologies such as viewpoint conversion. Needless to say, an image and a left peripheral image may be obtained.

The captured images of the cameras 1 to 3 are respectively input to the control unit 100. The control unit 100 creates a driving assistance image (to be described later) and displays it on the monitor 5 when the start switch 4 is turned on by the driver's operation.
The monitor 5 is installed in the vicinity of the driver's seat such as an in-vehicle instrument panel so that the driver can visually recognize the driving assistance image on the monitor screen.

  The control unit 100 is as shown in the functional block diagram of FIG. 1, and executes the control program shown in FIG. 3 to create the driving assistance image to be displayed on the monitor 5.

  First, the control unit 100 will be described with reference to the functional block diagram of FIG. 1, which includes a right-side bird's-eye viewpoint conversion unit 110, a left-side bird's-eye viewpoint conversion unit 115, a bird's-eye image synthesis unit 120, and a bird's-eye image superimposition. Unit 130, mask icon storage unit 132, optical flow position enhancement line storage unit 134, image connection unit 140, front end overhead view point conversion unit 150, right side overhead view point conversion unit 152, and left side overhead view point conversion The unit 154, the bird's-eye view image synthesis unit 160, the bird's-eye view image superimposing unit 170, and the host vehicle icon storage unit 175 are configured.

The right-side bird's-eye viewpoint conversion unit 110 converts the bird's-eye image captured by the front-end camera 1 into an image from the position of the right side of the vehicle toward the traveling direction (as shown in FIG. When the vehicle front portion 10F is disposed obliquely downward in front of the front in the vehicle width direction center, for example, from the front end camera 1 at the same vehicle front-rear direction position as the front end camera 1 as shown in FIGS. 4 (a) and 4 (b), for example, In other words, the optical flow of the pixels flowing from the infinity point to the right side of the vehicle when the vehicle is going straight on is on the right side of the monitor 5 from top to bottom. The viewpoint is converted so as to be substantially vertical, a second bird's-eye image is created, and this second bird's-eye image is output to the bird's-eye image synthesis unit 120.
The virtual camera 7 related to the second bird's-eye image captures an image of a range including obstacles far in the traveling direction, and particularly, the position where the optical axis center of the virtual camera 7 intersects the road surface as shown in FIG. 4 (b). A second bird's-eye image is created as the lower end of the second bird's-eye image (the end closest to the vehicle 10).

The left side bird's-eye viewpoint conversion unit 115 converts the bird's-eye image captured by the front-end camera 1 into an image from the position of the left side of the vehicle toward the traveling direction (as shown in FIG. When the vehicle front portion 10F is disposed obliquely downward in front of the front in the vehicle width direction center, for example, from the front end camera 1 at the same vehicle front-rear direction position as the front end camera 1 as shown in FIGS. 4 (a) and 4 (b), for example, In other words, the optical flow of the pixels flowing from the infinity point to the left side of the vehicle from the top to the bottom on the left side of the monitor 5 The viewpoint is converted so as to be substantially vertical, a third bird's-eye image is created, and this third bird's-eye image is output to the bird's-eye image synthesis unit 120.
The virtual camera 8 related to the third bird's-eye image captures an image of a range including obstacles far in the traveling direction, and in particular, the position where the center of the optical axis of the virtual camera 8 intersects the road surface as shown in FIG. A second bird's-eye image is created as the lower end of the second bird's-eye image (the end closest to the vehicle 10).

The bird's-eye image synthesis unit 120 is subjected to the viewpoint conversion as described above by the right-side bird's-eye viewpoint conversion unit 110 and the left-side bird's-eye viewpoint conversion unit 115, and the optical flow of pixels flowing from infinity to the left and right sides of the vehicle is substantially vertical. From the bird's-eye view image, a part of the image including the vicinity of the left and right side extension lines of the own vehicle is cut out and synthesized, and the synthesized image is output to the bird's-eye view image superimposing unit 130.
The synthesized bird's-eye image corresponds to the forward traveling road image in the present invention. Therefore, the right-side bird's-eye viewpoint conversion unit 110, the left-side bird's-eye viewpoint conversion unit 115, and the bird's-eye image synthesis unit 120 are the forward traveling road image in the present invention. Corresponds to creating means.

  Of the images synthesized by the bird's-eye view image synthesis unit, the mask icon storage unit 132 has an image near the right side extension line and an image near the left side extension line on the road surface in which the vertical position Z is Z = 0. An icon that masks one of the pixels appearing in both is masked and the stored icon is output to the bird's-eye image superimposing unit 130.

The optical flow position emphasis line storage unit 134 stores an extension line icon in the front direction of the host vehicle at the position of the left and right side surfaces of the host vehicle 10 or the front end position of the left and right door mirrors 10L and 10R. Output to 130.
The bird's-eye image superimposing unit 130 adds the mask icon from the mask icon storage unit 132 and the extension from the optical flow position enhancement line storage unit 134 to the synthesized image of the second bird's-eye image and the third bird's-eye image from the bird's-eye image synthesis unit 120. Superimpose line icons.

  The extension line icon in the forward direction of the vehicle described above corresponds to the width display line through which the vehicle can pass in the present invention. Therefore, the bird's-eye image superimposing unit 130 and the optical flow position emphasis line storage unit 134 pass through the vehicle in the present invention. This corresponds to the possible width display line setting means.

  The front end bird's-eye view conversion unit 150 converts the image captured by the front end camera 1 into a bird's-eye view looking down from above the vehicle 10 (as shown in FIG. 2, the two-sided cameras 2 and 3 are connected to the door mirror 10R. , 10L, when facing downward, the upper virtual camera at the center upper position between these cameras 2 and 3 at the same vehicle longitudinal direction position as the two-sided cameras 2 and 3 as shown in FIGS. 4 (a) and 4 (c) 9), a first overhead image is created and output to the overhead image synthesis unit 160.

  The right overhead view conversion unit 152 for the right side converts the image of the right side surface of the own vehicle photographed by the right camera 2 into a second overhead view image looking down from the upper virtual camera 9 directly above the own vehicle 10, The second overhead image is output to the overhead image composition unit 160.

  The left bird's-eye view conversion unit 154 converts the image of the left side image of the own vehicle taken by the left camera 3 into a third bird's-eye view image looking down from the upper virtual camera 9 directly above the own vehicle 10, This third overhead image is output to the overhead image synthesis unit 160.

  From the first to third overhead images converted by the overhead view viewpoint conversion unit 150 for the front end, the overhead view viewpoint conversion unit 152 for the right side, and the overhead view point conversion unit 154 for the left side, the overhead image synthesis unit 160 Part of the image including part of the left and right side surfaces is cut out and combined to create an overhead view image around the front end of the vehicle 10, and output to the overhead image superimposing unit 170.

The own vehicle icon storage unit 175 stores the icons of the front and left and right side positions of the own vehicle 10 and outputs them to the overhead image superimposing unit 170.
The overhead image superimposing unit 170 superimposes the vehicle icon stored in the vehicle icon storage unit 175 on the overhead image around the front of the vehicle synthesized by the overhead image synthesis unit 160, A bird's-eye view image is created by looking down from the top to the bottom of the vehicle including the vehicle, and is output to the image connection unit 140.

  This bird's-eye view image corresponds to the vehicle periphery image in the present invention, and therefore, the front-end bird's-eye view conversion unit 150, the right-side bird's-eye view conversion unit 152, the left-side bird's-eye view conversion unit 154, the bird's-eye view image synthesis unit 160, The unit 175 and the bird's-eye view image superimposing unit 170 correspond to the vehicle periphery image creating means in the present invention.

The image connecting unit 140 includes a front bird's-eye image related to the front of the vehicle created by the bird's-eye image superimposing unit 130, a front part of the own vehicle created by the bird's-eye image superimposing unit 170, and the vicinity of the front of the vehicle related to the periphery thereof. The overhead image is connected, and the connected image is output to the monitor 5 and displayed on the monitor 5.
Therefore, the image connection unit 140 corresponds to the image connection means in the present invention.

  By the way, when the above-mentioned connection between the bird's-eye image from the bird's-eye image superimposing unit 130 and the bird's-eye front image from the bird's-eye image superimposing unit 170 is connected, the three-dimensional coordinates (X axis: Explaining based on the vehicle width direction, Y axis: vehicle longitudinal direction, Z: vehicle vertical direction), the traveling direction crossing line 51a on the rear side in the traveling direction of the vehicle's front bird's-eye view image 51 The vehicle front bird's-eye view image 51 and the vehicle front periphery surrounding bird's-eye view image 52 are connected to each other by a boundary line 53 so that the vehicle crossing line 52a on the front side in the vehicle running direction of 52 overlaps the vertical direction of the monitor screen. Then, a continuous image as shown in FIG.

  Note that the lines “54” and “55” in these drawings respectively indicate the vehicle forward extension line icon (vehicle passable width display line) from the optical flow position emphasis line storage unit 134, and are displayed on the monitor screen. To the vehicle front bird's-eye view image 51 and the vehicle front area surrounding bird's-eye view image 52 so as to extend below the monitor screen.

By the way, when the own vehicle front bird's-eye view image 51 and the own vehicle front periphery bird's-eye view image 52 are both distorted, when these images are connected, the image boundary line 53 is curved as shown in FIG. The image 51 cannot be maintained in the plane as shown in FIG. 5A, and is deformed into a curved shape or the like.
In this case, the distortion in the vehicle front bird's-eye image 51 and the vehicle front periphery surrounding bird's-eye view image 52 is image-converted so that the same portion has the same shape on the ground of Z = 0, and the vehicle is automatically converted based on the portion. Interconnection is made between the bird's-eye view image 51 in front of the vehicle and the bird's-eye view image 52 around the front of the vehicle.
For this reason, the connection boundary line 53 does not become a straight line but has an arc shape whose distance from the own vehicle 10 is substantially equal, or a shape close to the lens distortion of the camera.

FIG. 6 shows a connection method when the curved bird's-eye image 51 and the bird's-eye image 52 are connected as described with reference to FIG. 5 (b).
As shown in FIG. 6 (a), the bird's-eye view image 51 is distorted due to lens distortion or the like (the degree of distortion is indicated by the deformation of the grid line), and the bird's-eye view image 52 is distorted due to lens distortion or the like (the degree of distortion). When the connection boundary lines 51a and 52a serving as the reference for connecting the images 51 and 52 are curved as shown in the figure, the images 51 and 52 are connected as follows. .

That is, as shown in FIG. 6 (b), the bird's-eye image 51 is enlarged in the vertical and horizontal directions in the figure to form a deformed image, and the connection boundary line 51a in the bird's-eye image 51 is connected to the connection boundary line 52a in the bird's-eye image 52. The bird's-eye view image 51 and the bird's-eye view image 51 and the bird's-eye view image 51 at the location of the connection boundary line 51a in the bird's-eye view image 51 and the connection boundary line 52a in the bird's-eye view image 52 in a state where the bird's-eye view image 51 is enlarged and deformed in this way The image 52 is connected to each other.
Thereby, the bird's-eye view image 51 and the bird's-eye view image 52 can be connected to each other at the boundary lines 51a and 52a even if they are curved.

When connecting the bird's-eye view image 51 and the bird's-eye view image 52, the bird's-eye view image 51 and the bird's-eye view 51 are seen in the line with the least distortion in the bird's-eye view image 51. A connection with the image 52 may be made.
However, considering the distortion characteristics of the lens in an actual camera, as shown in Fig. 6, the vertical and horizontal directions with respect to the center of the optical axis of the camera are curved except for the straight line in the horizontal direction. If you want to shoot the entire circumference of your vehicle, you need to widen the shooting range per camera, so use a camera with a wide-angle lens that has a wide shooting range. It tends to cause distortion.

Therefore, with regard to the bird's-eye view image 51, when it is desired to obtain an arbitrary shooting range (for example, an image of a portion reflected far away) other than the horizontal direction of the optical axis center reference without changing the installation position of the camera, the bird's-eye view image 51 In many cases, the connection (boundary line) with 52 does not become linear.
The bird's-eye view image 52 is an image looking down in a direction perpendicular to the ground from an arbitrary distance directly above the host vehicle, and is based on the ground surface (Z = 0 plane), up and down, left and right directions (XY of vehicle reference) As the bird's-eye view image 51, the left and right straight lines on the XY plane are not displayed on the curved line.

As shown in FIG. 6, when one (upper part of the figure) is the bird's-eye view image 51 and the other (lower part of the figure) is the bird's-eye view image 52, the boundary lines 51a and 52a based on the ground surface are the same in both figures. The vertical scale and horizontal scale of each image are changed so that the upper figure is approximately 2.7 times in the height direction and 1.2 times in the horizontal direction, and the boundary line 51a in the upper figure and the boundary line 52a in the lower figure are When the bird's-eye view image 51 and the bird's-eye view image 52 are connected, a continuous image on the Z = 0 plane is obtained.
In this case, the connection points in the upper and lower figures are not linear, but are curved, and there is some distortion in each image, but the image is at a level sufficient for the driver to check the blind spot. It becomes.

Of course, it is possible to make the connection portion 53 between the bird's-eye image 51 and the bird's-eye image 52 straight by changing the pixel mapping of the bird's-eye image 51 using a method such as image conversion or making the pixel mapping nonuniform. ,
When the vehicle moves forward, the fixed obstacle on the road surface slowly approaches the vehicle 10 in the screen at a certain pixel location in the monitor, even though the vehicle 10 is at a constant speed in the depth direction. At another location in the same monitor, there is a problem that a fixed obstacle on the road surface is displayed at an approach speed different from the actual speed, such as approaching the host vehicle 10 rapidly and rapidly.

Therefore, as described above, creating the connection portion of the bird's-eye view image 51 along the left-right straight line with the camera optical axis as a reference means that the approach speed of the road surface fixed obstacle in the monitor becomes uniform. Is preferable.
However, in this embodiment, as shown in FIG. 5, the optical flows 54 and 55 in the traveling direction at the left and right places away from the host vehicle 10 extend over both the bird's-eye image 51 and the bird's-eye image 52. In order to display it in a straight line parallel to the traveling direction of the vehicle, as shown in FIG. 5 (b), the vehicle is connected to the road fixed obstacle at the connection boundary line 53 between the bird's-eye image 51 and the overhead image 52. Even if the display is non-uniform in the approach speed (even if the connection boundary line 53 is curved), it is parallel to the traveling direction at a predetermined distance from the host vehicle in the left-right direction and is linear. It is possible to create an optical flow.

  As a result, the non-uniformity of the approach speed with the road surface fixed obstacle occurs, but the left and right ends of the own vehicle traveling lane, or the straight portion formed by the roots of the left and right walls are parallel to the traveling direction in the monitor, and Since it is displayed in a straight line, it is possible to obtain an effect that it is easy to perform an action of bringing the vehicle parallel to the road and approaching the end.

Compared to a simple bird's-eye view (AVM), the bird's-eye view image also displays a distant image, so that it appears suddenly in the monitor, or the obstacle that the driver actually viewed and the obstacle in the monitor. Correspondence between obstacles confirmed with the naked eye during actual driving and distant obstacles displayed in the monitor, without taking time to compare with the objects and avoiding obstacles suddenly in the monitor (Relationship) is easy to grasp.
Therefore, the time required to compare the image in the monitor with the obstacle visually recognized by the driver can be shortened, and even if the monitor is closely watched, the distance between the distant and the blind spot closest to the vehicle Since the information can be continuously grasped in the monitor, it is possible to obtain an effect that a highly accurate driving operation can be immediately executed.

A control program in which the control unit 100 in FIG. 1 creates a driving support image to be displayed on the monitor 5 as described above will be described below with reference to FIG.
The control program in FIG. 3 is started when an ignition switch (not shown) is turned on. First, in step S10, the cameras 1 to 3 installed in the host vehicle 10, various sensors such as a laser range finder and an ultrasonic sensor (not shown) are installed. to start.

In the next step S20, the width in the front direction of the vehicle is measured using the images taken by the cameras 1 to 3 and signals from the laser range finder and the ultrasonic sensor.
In step S30, it is determined whether or not the start switch 4 of FIG. 1 is turned on. If the start switch 4 is turned on, the driver does not wish to operate the system, and the control is returned to step S20. If the start switch 4 is turned on, the driver desires to operate the system, so that the control is sequentially advanced from step S40.

In step S40, the monitor is located at a location separated by a predetermined distance in the left-right direction with respect to the center in the vehicle width direction of the host vehicle, specifically, two locations, a substantially right door mirror tip and a substantially left door mirror tip. 5. Set a vertical optical flow (viewpoint) in which the driver who generates the calculation is invisible vertical.
In step S50, a distant pixel portion is cut out from the image of the front end camera 1 installed at the center in the vehicle width direction at the front end of the vehicle to create a first bird's-eye view image.

In step S60, the first bird's-eye view image created in step S50 is converted into a viewpoint (shear) so as to be the optical flow location (right door mirror tip) set in step S40 to create a second bird's-eye view image (FIG. 1 is a bird's-eye view conversion unit 110 for the right side.
In step S60, the first bird's-eye view image created in step S50 is converted to the viewpoint (shear) so as to be the optical flow location (left door mirror tip) set in step S40 to create a third bird's-eye view image (FIG. 1 is a left-eye bird's-eye viewpoint conversion unit 115).

  In step S80, in the second bird's-eye image (right viewpoint) created in step S60 and the third bird's-eye image (left viewpoint) created in step S70, the same portion is photographed on the Z = 0 plane (ground). Pixels are masked (bird's-eye view image superimposing unit 130 and mask icon storage unit 132 in FIG. 1).

The second bird's-eye image and the third bird's-eye image are obtained by converting the viewpoint of the first bird's-eye image from different viewpoints (virtual camera positions 7 and 8 in Fig. 4). For the vicinity of the point), the same location exists in both bird's-eye view images.
For this reason, even if there is a single obstacle in front of the vehicle, this single obstacle will be present in both the second bird's-eye image and the third bird's-eye image, and the exact number of obstacles will be calculated. Difficult to grasp.
In order to solve this problem, it is assumed that the pixels appearing in both the second bird's-eye image and the third bird's-eye image are subjected to masking processing with other colors.

  In step S90, the images of the front-end camera 1, the right-side camera 2, and the left-side camera 3 are subjected to viewpoint conversion into a bird's-eye view image looking down from the vehicle directly above (from the virtual camera position 9 in FIG. 4) (see FIG. 1). The bird's-eye view conversion units 150, 152, and 154) further synthesize these three bird's-eye views and create a vehicle surroundings bird's-eye view image that includes the vehicle and its surroundings.

  In step S100, the optical flow position (right door mirror tip) of the second bird's-eye image created in step S60, the optical flow position (left door mirror tip) of the third bird's-eye image created in step S70, and in step S90 The images are connected and combined so that the position in the vehicle width direction matches the optical flow position (tips of the left and right door mirrors) of the created vehicle surroundings overhead view image (image connection unit 140 in FIG. 1).

  In step S110, the driving support image obtained by combining as described above in step S100 is output to the monitor 5 in the vicinity of the driver's seat in the passenger compartment and displayed on the monitor 5.

In step S120, it is checked whether or not the start switch 4 turned on in step S30 is turned off.
While the start switch 4 is not turned OFF, the “start switch 4 OFF determination” in step S120 is continued, and the above-described driving support image is continuously displayed on the monitor.
When the start switch 4 is turned off, step S120 selects step S130. In this step S130, the driving support image on the monitor 5 is erased and hidden, and the control program of FIG. 3 is terminated.

<Drive support image>
The driving assistance image created as described above and displayed on the monitor 5 will be described in detail below with reference to FIG.
FIG. 7 (a) shows the entire front-view image 52 around the front of the vehicle viewed from the viewpoint from directly above to the vehicle, so that the direction of the viewpoint conversion and the optical flow can be easily understood. The lattice mesh on the surface (Z = 0 plane) is also shown.

In the lower half area 56 of the front direction of the subject vehicle around the bird's eye view image 52 in the lower half area 56 of the traveling direction crossing line 52a on the front side in the traveling direction of the subject vehicle, an image including the subject vehicle icon 57 of the subject vehicle 10 and its surroundings Is displayed, and the upper part of the figure is the front of the vehicle.
In the present embodiment, the lower half area 56 of the vehicle front periphery top-down image 52 from the traveling direction crossing line 52a is displayed in the driving support image together with the vehicle icon 57 and its peripheral image displayed here. It is used as a bird's-eye view of the front of the vehicle

  In the lower half area 56 of the vehicle front-periphery bird's eye view image 52, a predetermined distance from the left side of the vehicle when the vehicle goes straight in a steering neutral state (50cm to 1m with a door mirror tip and further margin) An image flow direction (optical flow) 54a at a distant position and an image flow direction (optical flow) 55a at a position away from the vehicle right side by a predetermined distance are displayed.

  FIG. 7 (b) shows a bird's-eye view image obtained by looking down the front-view image 52 around the front part of the vehicle in FIG. 7 (a) with a certain bird's-eye angle (looking-down angle) instead of a viewpoint from directly above.

  FIG. 17 (c) converts the bird's-eye view image of FIG. 17 (b) from the left optical flow 54a of FIG. 17 (b) so that the left optical flow 54b continuous in the vehicle traveling direction is perpendicular to the monitor screen 5. For example, an image of the front-end camera 1 that has been subjected to shear conversion or perspective conversion) is shown, and an area 51b is an area that is used as the second bird's-eye image in the driving assistance image.

  FIG. 17D shows a bird's-eye view image of FIG. 17B with respect to the right optical flow 55a of FIG. 17B so that the right optical flow 55b continuous in the vehicle traveling direction is perpendicular to the monitor screen 5 ( For example, an image of the front-end camera 1 subjected to shear conversion or perspective conversion) is shown, and an area 51c is an area used as the third bird's-eye image in the driving assistance image.

FIG. 7 (e) shows that the left and right optical flows 54a, 54b and 55a, 55b (rear flow) at a position (predetermined side) away from the left and right side surfaces of the host vehicle In the vertical direction, that is, the flow to the rear of the predetermined side part in the traveling path in front of the host vehicle is converted along the longitudinal direction of the host vehicle, and the monitor screen 5 A driving assistance image that flows vertically from top to bottom is shown.
In this image conversion, if the vanishing point of the road (not shown) is a normal bird's-eye image, the vanishing point is located at the center of the image. It is an image converted so as to be arranged at a distant position (predetermined side).

The configuration of the driving support image displayed on the monitor screen 5 is that the lower half area 56 is a bird's-eye view image, the upper right area 51c is a third bird's-eye image after viewpoint conversion, and the upper left area 51b is a second image after viewpoint conversion. It is a bird's-eye view image.
In order to ensure continuity on the Z = 0 plane (the ground surface) and to make the size of the connection area the same for each of the images in these areas, the magnification of the second bird's-eye image and the third bird's-eye image is adjusted. ing.

<Operation of the first embodiment>
The operation of the first embodiment will be described in detail below with reference to FIGS.
Figure 8 shows three-dimensional obstacles such as the own vehicle 10 (shown at three positions 10-1, 10-2, and 10-3), roads (walls) 21 and 22, and power poles 23 and 24 on a narrow path. And shows the positional relationship with
FIGS. 9 (a), (b), and (c) show the monitor screen 5 when the vehicle 10 is at the positions 10-1, 10-2, and 10-3 in FIG. 8, respectively. .

First, FIG. 8 will be described. 21 is the left boundary (ridgeline) of the road with respect to the traveling direction of the own vehicle, or the left boundary of the vehicle's travelable area at the left wall or side groove.
Reference numeral 22 denotes a right boundary (ridgeline) of the road with respect to the traveling direction of the host vehicle, or a right wall or a side groove, which indicates the right boundary of the travelable area of the host vehicle.
23 shows a power pole (stereoscopic obstacle, sign, pole, guardrail, gutter, etc.) installed on the left side with respect to the traveling direction of the vehicle on the road,
Reference numeral 24 denotes a right utility pole (three-dimensional obstacle, sign, pole, guardrail, gutter, etc.) installed on the right side with respect to the traveling direction of the vehicle on the road.

The own vehicle position 10-1 is the own vehicle position immediately before the own vehicle reaches the utility pole 3 (narrow part), and the own vehicle position 10-2 is the own vehicle position directly beside the utility pole 3 (narrow part). The own vehicle position 10-3 is the own vehicle position immediately before reaching the right utility pole 24 (narrow portion).
Note that the L1, L2, and L3 positions in FIG. 8 are respectively those shown in FIGS. 9 (a) and 9 (b) when the host vehicle 10 is at the 10-1, 10-2, and 10-3 positions. ), (c) shows the front position of the vehicle displayed on the image connection boundary line 53 (the boundary between the bird's-eye view image and the bird's-eye view image) of the monitor screen 5 shown in FIG.

8 and 9, respectively, 54 and 55 indicate the optical flows in the traveling direction at the left and right locations (the left and right door mirror tips) that are separated from the host vehicle 10 by a predetermined distance.
In the present embodiment, the optical flows 54 and 55 are set as the vehicle-passable width display lines as the optical flows in the traveling direction at the distal end portions of the left and right door mirrors of the own vehicle 10, but the present invention is not limited to this.
Moreover, you may provide the own vehicle passable width display line (traveling road passable width display line) which a driver | operator can recognize so that it may overlap on this optical flow.

FIGS. 9 (a), (b), and (c) show the monitor screen 5 when the vehicle 10 is at the positions 10-1, 10-2, and 10-3, as described above. However, the upper half of the image connection boundary line 53 on the monitor screen 5 is a bird's-eye view image, and the lower half is a bird's-eye view image.
In the bird's-eye view image section, the image of the front-end optical float camera 1 is processed, and when the vehicle goes straight in the steering neutral state, the vertical direction of the monitor screen 5 at the left and right connection positions of the optical flows 54 and 55 To be parallel to.

Accordingly, the left and right road ridgelines and the wall root portions 21 and 22 on the ground (Z = 0 plane) are displayed in a substantially straight line in the monitor screen 5 in both the bird's-eye view image and the overhead view image (boundary 53).
On the other hand, on the bird's-eye view image side, the viewpoint conversion based on the Z = 0 plane is performed using shear conversion, perspective conversion, etc., so that three-dimensional obstacles such as utility poles 23 and 24 and parked vehicles are distorted and displayed. Will be.

In addition, since two optical flows 54 and 55 that are perpendicular to the monitor screen 5 are set on the bird's eye view image side, the two screens are combined.
At this time, with respect to the Z = 0 plane, either one of the two screens is used for the pixel portion, or masking is performed as shown by the black triangular portion in FIG.
As a result, it is possible to prevent a plurality of obstacles that actually exist only from being displayed in the monitor screen 5.

In FIG. 9 (a) showing the monitor screen 5 when the host vehicle 10 is at the position indicated by 10-1 in FIG. 8, the connection boundary line 53 between the bird's-eye image and the bird's-eye view image displays the L1 position in FIG. ing.
Therefore, both the left utility pole 23 and the right utility pole 24 are displayed on the bird's-eye view image side.
Further, in FIG. 9 (b) showing the monitor screen 5 when the host vehicle 10 is at the position indicated by 10-2 in FIG. 8, the connection boundary line 53 between the bird's-eye image and the bird's-eye view image displays the L2 position in FIG. doing.
Therefore, the left utility pole 23 is displayed on the bird's-eye view image side, and the right utility pole 24 is displayed on the bird's-eye view image side.
Further, in FIG. 9 (c) showing the monitor screen 5 when the vehicle 10 is at the position indicated by 10-3 in FIG. 8, the connection boundary line 53 between the bird's-eye image and the bird's-eye view image displays the L3 position in FIG. doing.
Therefore, the left utility pole 23 is not displayed on the bird's-eye view image side, and the right utility pole 24 is displayed on the bird's-eye view image side.

As is clear from the above description, as the host vehicle 10 travels (straight), the road edges, walls 21 and 22 and utility poles 23 and 24 move on the monitor screen 5, but the host vehicle 10 and The left and right optical flows (width display lines through which the vehicle can pass) do not change in the bird's-eye view image and the bird's-eye view image.
Therefore, in the action of moving the vehicle 10 to the left and right road ends 21 and 22 (width adjustment), the road edge becomes radial like in the past, and the driver grasps the relationship with the extension line on the side of the vehicle. There is no problem that it cannot be performed, and there is no problem that the environment information (angle between the road edge and the extension of the side surface of the vehicle) used for width adjustment is distorted.
Further, since it is displayed in the monitor screen 5 up to a distance from the own vehicle, the width-shifting action can be started in advance, which is very useful for safe driving support.
Furthermore, compared to the overhead image alone, the road edge is displayed from a distance of the host vehicle, so that it is possible to prepare for action and provide a sense of security.

<Effects of the first embodiment>
The effect of the first embodiment will be described below with reference to FIGS.
FIGS. 10 (a) and 11 (a) show two examples of the positional relationship between the own vehicle 10 in a narrow path, the roads (walls) 21 and 22, and the utility poles 23 and 24, respectively.
FIG.10 (b) shows the monitor screen 5 when the own vehicle 10 is in the narrow left side of the narrow road as shown in Fig.10 (a).
FIG. 11 (b) shows the monitor screen 5 when the vehicle 10 is in a narrow path on the right side in the road width direction as shown in FIG. 11 (a).

When the host vehicle 10 is at the left position as shown in FIG. 10 (a), it is possible to pass the optical flow on the monitor screen 5 in FIG. It can be easily grasped from the relative position between the own vehicle passable width display line 54 and the image of the left power pole 23.
Specifically, from the monitor screen 5 in FIG. 10 (b), the ridgelines (walls) 21 and 22 of the road are within the runway width between the optical flows (width indication lines that can pass through the vehicle) 54 and 55 perpendicular to the monitor screen 5. Can be recognized that there is no image of the poles 23 and 24 that are obstacles, and it is easy to grasp that there is no contact with the surrounding environment even if the host vehicle 10 goes straight ahead as it is .

When the host vehicle 10 is in the right-side position as shown in FIG. 11 (a), the monitor screen 5 in FIG. 11 (b) indicates that it is impossible to pass through the electric pole 24 that forms the narrowest part when going straight. It can be easily grasped from the relative position between the optical flow (passage width display line) 55 on the upper side and the image of the utility pole 24.
Specifically, from the monitor screen 5 in FIG. 11 (b), an image of the utility pole 24, which is an obstacle, is shown in the width of the road between the optical flows 54 and 55 that are perpendicular to the vehicle. It can be recognized that it is present, and if the host vehicle 10 is moved straight ahead as it is, it can be easily grasped that it is in contact with the utility pole 24.

By the way, in the first embodiment, as described above, the surrounding image of the own vehicle is a bird's-eye view image seen as if looking down directly on the own vehicle, and the front road image is viewed from the own vehicle with a predetermined looking-down angle. Because it ’s a bird’s-eye view,
The vehicle periphery image and the forward travel are set so that the travel direction cross line 52a on the front side in the vehicle travel direction of the host vehicle periphery image 52 overlaps the travel direction cross line 51a on the rear side in the vehicle travel direction of the front travel path image 51. When the road images are connected to each other to obtain a continuous image, the continuity between the vehicle periphery image and the forward traveling road image can be easily made excellent.

  Therefore, a single obstacle on the front road is temporarily displayed on both the vehicle periphery image and the front road image connected in the vertical direction. There is no problem that it is not displayed on any of the traveling road images.

Furthermore, if the obstacle is the same, this obstacle will appear from the same position in the surrounding image of the vehicle below after disappearing from the upper front road image.
After the same obstacle disappears from the upper front road image, it appears from a shifted position of the vehicle periphery image below, and it is difficult to grasp the environment around the vehicle from the display in both images Can also be avoided.

Furthermore, in the first embodiment, since the optical flows 54 and 55 perpendicular to the monitor screen 5 are set as the width indication lines through which the vehicle can pass as the optical flow in the traveling direction at the tip of the left and right door mirrors,
When there is an obstacle between the left and right optical flows 54, 55, it is necessary to take avoidance action (steering), and when there is no obstacle between the optical flows 54, 55, The screen is displayed so that it can be easily understood that the vehicle can go straight as it is, which is very useful for driving support.
Also, for the same reason, when going straight ahead with the current vehicle position, the contact judgment with the road ridgeline (wall) or utility pole is determined from the stage where the narrowest part is far away (bird's-eye video part) in the monitor screen 5 The determination can be made, and an effect that the determination can be made by an intuitive display such as checking whether there is an obstacle between the left and right vertical optical flows 54 and 55 can be achieved.

Further, in the first embodiment, optical flows (passage width display lines) 54 and 55 are set to be superimposed on the bird's-eye image and the bird's-eye image separately, and these optical flows (passage width display lines for own vehicle) are set. ) To correct 54,55 so that they are connected to each other in a straight line, and connect the bird's-eye view image and the bird's-eye view image,
When the host vehicle goes straight, it is easy to grasp the boundary of the area (between the left and right optical flows 54 and 55) where the presence or absence of an obstacle should be confirmed, and the above effect can be made more remarkable.

And when correcting so that the optical flow (vehicle passable width display line) 54, 55 is connected to each other in a substantially straight line, in order to perform the correction by shear transformation of the overhead image,
There is little distortion of the information in the depth direction in the bird's-eye view image, it is easy to grasp the sense of depth from the display screen, and the above effect can be made more remarkable.

  In addition, since the bird's-eye view image and the bird's-eye view image are connected on the ground surface of Z = 0, even a three-dimensional object can be reliably displayed in the monitor screen 5, and an approach locus from far to near can be displayed. It is easy to recall and the above effects can be made more remarkable.

  In addition, when connecting a bird's-eye view image and a bird's-eye view image, the pixel portions that overlap each other are masked so that they are not displayed, so the number and arrangement of obstacles will not be different from the actual display, and the above effects will be more prominent. can do.

  In addition, when connecting the bird's-eye view image and the bird's-eye view image, the bird's-eye view image is used because the connection is made at the intersection between the optical axis of the front-end camera 1 that captures the bird's-eye view image and the ground surface (positions L1, L2, and L3 in FIG. 8). The connection boundary line 53 between the image and the bird's-eye view image becomes a horizontal straight line as shown in FIG. 9, and there is little distortion of the image, it is easy to grasp the traveling environment from the image, and the above effect can be made more remarkable.

<Configuration of the second embodiment>
FIG. 12 is a functional block diagram showing the overall configuration of the driving support apparatus according to the second embodiment of the present invention, in which 1 is a front end camera, 2 is a right camera, 3 is a left camera, 6 is a laser range finder group, Reference numeral 7 denotes an ultrasonic sensor group.

  As described above with reference to FIGS. 2 (a) and 2 (b), the front end camera 1 is installed 45 ° forward and downward near the center of the front end portion 10F of the own vehicle, and the left and right corners of the front end portion 10F of the own vehicle are arranged. A wide-angle camera (angle of view is 180 °, for example) that captures a front peripheral image (first bird's-eye view image) related to the front end portion 10F of the host vehicle including the vehicle and its surrounding traveling road portion.

As described above with reference to FIGS. 2 (a) and 2 (b), the right camera 2 is installed substantially downward in the vicinity of the right door mirror 10R, and includes the side surface of the own vehicle 10 in the vicinity of the right door mirror 10R and the surrounding traveling road portion. A wide-angle camera (viewing angle is 180 °, for example) that captures the right peripheral image (overhead image).
As described above with reference to FIGS. 2 (a) and 2 (b), the left camera 3 is installed substantially downward in the vicinity of the left door mirror 10L, and includes the side surface of the host vehicle 10 in the vicinity of the left door mirror 10L and the surrounding roadway portion. A wide-angle camera (viewing angle is 180 °, for example) that captures the left peripheral image (overhead image).

The laser range finder group 6 is installed near the left and right front corners of the own vehicle, and detects the distance to the three-dimensional obstacle existing in front of the own vehicle and its direction (angle).
The ultrasonic sensor group 7 is installed in the vicinity of the front corners of both the left and right sides of the own vehicle with a predetermined interval and angle, and detects the distance to the three-dimensional obstacle existing in front of the own vehicle. Shall.

The captured images of the cameras 1 to 3 and the detection results of the laser range finder group 6 and the ultrasonic sensor group 7 are respectively input to the control unit 200. The control unit 200 creates a driving support image, which will be described later, on the monitor 5. indicate.
The monitor 5 is installed in the vicinity of the driver's seat such as an in-vehicle instrument panel so that the driver can visually recognize the driving assistance image on the monitor screen.

  The control unit 200 is as shown in the functional block diagram of FIG. 12, and executes the control program shown in FIG. 13 to create the driving assistance image to be displayed on the monitor 5.

  First, the control unit 200 will be described with reference to the functional block diagram of FIG. 12, which includes a right-side bird's-eye viewpoint conversion unit 250, a left-side bird's-eye viewpoint conversion unit 255, a bird's-eye image synthesis unit 120, and a bird's-eye image superimposition. Unit 130, mask icon storage unit 132, optical flow position enhancement line storage unit 134, image connection unit 140, front end overhead view point conversion unit 150, right side overhead view point conversion unit 152, and left side overhead view point conversion Unit 154, bird's-eye view image composition unit 160, bird's-eye view image superimposition unit 170, own vehicle icon storage unit 175, surrounding environment detection unit 210, image processing unit 220, narrowest part width detection unit 230, display automatic The activation unit 235, the right gap detection unit 240, the left gap detection unit 241, the right optical flow position determination unit 244, and the left optical flow position determination unit 246 are configured.

  The control unit 200 in FIG. 12 is operated based on the detection values of the laser range finder group 6 and the ultrasonic sensor group 7 and the width detected by the image processing unit 220 based on the images taken by the cameras 1 to 3. A support image is created, and this driving support image is output to the monitor 5 and displayed.

  The right-side bird's-eye viewpoint conversion unit 250 converts the image captured by the front-end camera 1 into an image from a predetermined position on the right side of the vehicle toward the traveling direction (output of a right optical flow position determination unit 244 described later). That is, when the vehicle is traveling straight ahead, the viewpoint conversion is performed so that the optical flow of the pixels flowing from the infinity point to the predetermined position on the right side of the vehicle becomes substantially vertical from the top to the bottom on the monitor 5, and the bird's-eye image synthesis unit 120 Output to.

  The left bird's-eye viewpoint conversion unit 255 converts the image captured by the front-end camera 1 into an image from a predetermined position on the left side of the vehicle (output of the left optical flow position determination unit 246 described later) in the traveling direction. That is, when the vehicle travels straight, the viewpoint conversion is performed so that the optical flow of the pixel flowing from the infinity point to a predetermined position on the left side of the vehicle becomes substantially vertical from top to bottom on the monitor 5, and the bird's-eye image synthesis unit 120 Output to.

  The bird's-eye image synthesis unit 120 is subjected to viewpoint conversion by the right-side bird's-eye viewpoint conversion unit 250 and the left-side bird's-eye viewpoint conversion unit 255, and the optical flow of pixels flowing from infinity to a predetermined position on the left and right sides of the vehicle is substantially vertical. A part of the image including the vicinity of the extension line from the left and right predetermined positions of the own vehicle is cut out and synthesized from the above bird's-eye view image and output to the bird's-eye view image superimposing unit 130.

  Of the images synthesized by the bird's-eye image synthesis unit 120, the mask icon storage unit 132 has an image near the extension line from the right predetermined position and an image near the extension line from the left predetermined position on the Z = 0 plane. An icon for masking one of the pixels appearing in both is stored and output to the bird's-eye view image superimposing unit.

The optical flow position emphasis line storage unit 134 stores an optical flow position emphasis icon in the forward direction of the host vehicle at a position determined by a right optical flow position determination unit 244 and a left optical flow position determination unit 246 described later. And output to the bird's-eye view image superimposing unit 130.
The bird's-eye image superimposing unit 130 adds the mask icon from the mask icon storage unit 132 and the extension from the optical flow position enhancement line storage unit 134 to the synthesized image of the second bird's-eye image and the third bird's-eye image from the bird's-eye image synthesis unit 120. Superimpose line icons.

The above-described extension line icon in the forward direction of the host vehicle corresponds to the travel path possible width display line in the present invention, and thus the bird's-eye image superimposing unit 130 and the optical flow position emphasis line storage unit 134 are configured to pass the travel path in the present invention. This corresponds to the possible width display line setting means.
The image connecting unit 140 connects the bird's-eye image created by the bird's-eye image superimposing unit 130 and the bird's-eye image created by the bird's-eye image superimposing unit 170 as will be described later, and outputs them to the monitor 5.

The front-end overhead view conversion unit 150 converts the viewpoint of the image captured by the front-end camera 1 into a first overhead image viewed from directly above the host vehicle, and outputs the first overhead image to the overhead image synthesis unit 160.
The overhead view viewpoint conversion unit 152 for the right side converts the image of the right side surface of the host vehicle photographed by the right camera 2 into a second overhead view image looking down from directly above the host vehicle, and outputs it to the overhead image synthesis unit 160.
The left bird's-eye view conversion unit 154 converts the image of the left side surface of the host vehicle taken by the left camera 3 into a third bird's-eye view image looking down from directly above the host vehicle, and outputs it to the bird's-eye view image synthesis unit 160.

The bird's-eye view image compositing unit 160 is configured to display the front and left and right side surfaces of the host vehicle from the bird's-eye view images converted by the front-view overhead view point conversion unit 150, the right-side overhead view point conversion unit 152, and the left-side overhead view point conversion unit 154. A part of the image including the part is cut out and synthesized to create a bird's-eye view image around the own vehicle and output it to the bird's-eye view image superimposing unit 170.
The bird's-eye view image composition unit 170 superimposes the vehicle icon stored in the vehicle icon storage unit 175 on the bird's-eye view image around the vehicle synthesized by the bird's-eye view image synthesis unit 160, and displays the surroundings of the vehicle including the vehicle. This is combined with the image looked down from directly above and output to the image connection unit 140.
The own vehicle icon storage unit 175 stores the icons of the front and left and right side positions of the own vehicle and outputs them to the overhead image superimposing unit 170.

The image processing unit 220 performs image processing on an image in front of the host vehicle taken by the front end camera 1, acquires environmental information such as the arrangement of obstacles and width, and outputs the environmental information to the surrounding environment detection unit 210.
The surrounding environment detection unit 210 obtains environmental information (disposition of obstacles and specific information on the travelable area) ahead of the host vehicle from the laser range funder group 6, the ultrasonic sensor group 7, and the image processing unit 220 to obtain the maximum information. It outputs to the narrow part width detection part 230.

The narrowest part width detection unit 230 detects from the output of the surrounding environment detection unit 210 where a gap or width between three-dimensional obstacles is less than a predetermined value (for example, 3.5 m or less), and automatically displays the result. Output to the unit 235.
The display automatic activation unit 235 outputs a signal for activating the driving support device of the present embodiment based on the distance to the narrowest part (for example, the distance to the narrowest part is 6 m or more and 20 m or less). .

The right gap detection unit 240 indicates a gap (distance) between the right side surface of the own vehicle and the right obstacle constituting the narrowest part when the own vehicle goes straight to the narrowest part detected by the narrowest part detection unit 230. The result is detected and the result is output to the right optical flow position determination unit 242.
The left gap detection unit 242 indicates a gap (distance) between the left side surface of the own vehicle and the left obstacle constituting the narrowest part when the own vehicle goes straight to the narrowest part detected by the narrowest part detection unit 230. Then, the result is output to the left optical flow position determination unit 246.

The right optical flow position determination unit 244 receives the output from the right gap detection unit 240 and determines the optical flow position that extends vertically in the monitor 5 at a position separated from the right side of the vehicle by a gap (distance). And output to the right-side bird's-eye view conversion unit 250.
The left optical flow position determination unit 246 receives the output from the left gap detection unit 242 and determines the optical flow position extending vertically in the monitor 5 at a position separated from the left side surface of the vehicle by a gap (distance). And output to the left bird's-eye view conversion unit 255.

A control program in which the control unit 200 of FIG. 12 creates a driving support image to be displayed on the monitor 5 as described above will be described below with reference to FIG.
The control program shown in FIG. 13 is started when an ignition switch (not shown) is turned on. First, in step S10, various sensors such as the cameras 1 to 3 installed in the own vehicle 10, the laser range finder group 6 and the ultrasonic sensor group 7 are used. Start up.

In the next step S20, the width in the front direction of the vehicle is measured using the captured images of the cameras 1 to 3 and the detection results of the laser range finder group 6 and the ultrasonic sensor group 7.
In step S200, it is checked whether or not there is a narrowest portion (for example, a portion having a width of 3.5 m or less) whose width in the front direction measured in step S20 is a predetermined value or less,
If there is no width portion below the predetermined value (3.5m), the system operation (driving support) is unnecessary, so the control returns to step S20 and waits.
If there is a width portion that is equal to or less than the predetermined value (3.5 m), it is necessary to operate the system (driving support), and thus control is sequentially advanced to step S210 and subsequent steps.

In step S210, it is confirmed whether or not the maximum width portion of the predetermined value (3.5 m) or less determined in step S200 is further continued within a second predetermined distance (for example, 6 m) in the rear (depth direction). If it is continuous, the narrowest part within the range of the second predetermined distance (6 m) is output.
In step S220, the right gap from the right side of the vehicle to the right obstacle on the narrowest part when the host vehicle goes straight to the narrowest part within the range of the second predetermined distance (6 m) detected in step S210. The left gap from the left side of the vehicle to the left side obstacle at the narrowest part is calculated.

In step S230, the right gap position to the right obstacle at the narrowest part calculated in step S220 is set to the right optical flow (viewpoint) position that is vertical on the monitor 5 on the right side of the vehicle,
The left gap position to the left obstacle at the narrowest part calculated in step S220 is set to the left optical flow (viewpoint) position that is vertical on the monitor 5 on the left side of the vehicle.

In step S50, a distant pixel portion (specifically, the upper half of the image of the front-end camera 1) is cut out from the image of the front-end camera 1 installed at the center in the vehicle width direction at the front end of the vehicle, and the first bird's-eye view image is obtained. create.
In step S240, the first bird's-eye image is converted to the viewpoint (shear) so as to be the right gap position (right optical flow position) set in step S220, and the fourth bird's-eye image on the right side of the vehicle is created. .
In step S250, the first bird's-eye image is subjected to viewpoint (shear) conversion so as to be the left gap position (left optical flow position) set in step S220, and a fifth bird's-eye image on the left side of the vehicle is created. .

In step S260, the same part is photographed on the Z = 0 plane (ground) in the fourth bird's-eye image (right viewpoint) created in step S240 and the fifth bird's-eye image (left viewpoint) created in step S250. Mask the pixel.
Although depending on the type of viewpoint conversion, in the case of shear conversion, the intersection between the left and right optical flow positions set in step S230 and the upper side of the monitor is the bottom, the connection between the bird's-eye image and the overhead image, and The triangular portion having the intersection with the extension line of the center (0X) in the left-right direction of the host vehicle corresponds to the same-portion shooting area on the Z = 0 plane of the fourth bird's-eye image and the fifth bird's-eye image.

  In step S90, the images of the front-end camera 1, the right-side camera 2, and the left-side camera 3 are subjected to viewpoint conversion into a bird's-eye view image looking down from the vehicle directly above (from the virtual camera position 9 in FIG. 4) (see FIG. 1). The bird's-eye view conversion units 150, 152, and 154) further synthesize these three bird's-eye views and create a vehicle surroundings bird's-eye view image that includes the vehicle and its surroundings.

  In step S100, the optical flow position of the fourth bird's-eye image created in step S240 (right gap to the right-side obstacle at the narrowest part) and the optical flow position of the fifth bird's-eye image created in step S250 (left side of the narrowest part). The left gap to the obstacle) is combined with the fourth and fifth bird's-eye images so that the optical flow position on the corresponding side of the bird's-eye image created in step S90 matches in the horizontal direction. (Image connection unit 140 in FIG. 12).

  In step S110, the driving support image obtained by combining as described above in step S100 is output to the monitor 5 in the vicinity of the driver's seat in the passenger compartment and displayed on the monitor 5.

  In step S270, it is checked whether the lateral position in the left-right direction between the narrowest portion in the driving support image displayed in the monitor display in step S110 and the own vehicle has changed, and the lateral position of the own vehicle has changed. In this case, since it is necessary to reset the position of the left and right vertical optical flow in the monitor 5, the control is returned to step S20, and the position of the left and right vertical optical flow in the monitor 5 is reset by the above-described processing. While there is no change in the lateral position of the host vehicle, the control proceeds to step S280.

If there is a change in the lateral position of the vehicle (step S270), the reason why it is necessary to reset the position of the left / right vertical optical flow in the monitor 5 is
In the present embodiment, the right and left vertical optical flow position is not fixed at a predetermined position (the position of the front end of the vehicle door mirror) as in the first embodiment, but is determined by the position of the left and right obstacles at the narrowest portion. In the present embodiment, even when the lateral position of the own vehicle changes, the left and right optical flow positions relative to the road do not change.

In step S280, which is selected when it is determined in step S270 that there is no change in the lateral position of the host vehicle, it is determined whether or not the host vehicle has passed through the narrowest portion (a portion having a width of 3.5 m or less) detected in step S210. To check.
While it is determined in step S280 that the vehicle has not yet passed through the narrowest portion, the check in step S280 is repeated, and the vehicle waits until the vehicle passes through the narrowest portion.
When it is determined in step S280 that the vehicle has passed through the narrowest part, the control proceeds to step S130, the driving support image on the monitor 5 is erased and not displayed, and the control program in FIG. 13 is terminated.

<Operation of Second Embodiment>
The operation of the second embodiment will be described in detail below with reference to FIGS.
Fig. 14 shows the positional relationship between the vehicle 10 (shown at two positions 10-4 and 10-5), roads (walls) 21 and 22, and three-dimensional obstacles such as utility poles 25 in the narrow street. Show from
FIGS. 15 (a) and 15 (b) show the monitor screen 5 when the vehicle 10 is at the position 10-4 and the position 10-5 in FIG. 14, respectively.

First, FIG. 14 will be described. 21 is the left end (ridgeline) of the road with respect to the traveling direction of the host vehicle, or the left wall or a side groove, and shows the left boundary of the area where the host vehicle can travel,
Reference numeral 22 denotes a right boundary (ridgeline) of the road with respect to the traveling direction of the host vehicle, or a right wall or a side groove, which indicates the right boundary of the travelable area of the host vehicle.
Reference numeral 25 denotes a utility pole (stereoscopic obstacle, sign, pole, guardrail, gutter, etc.) installed on the left side with respect to the traveling direction of the vehicle on the road.

The own vehicle position 10-4 is the own vehicle position far away (for example, 20 m or more) from the utility pole 25 (narrow part), and the own vehicle position 10-5 is the vehicle immediately before reaching the utility pole 3 (narrow part). It is a car position.
Note that the L4 position and the L5 position in FIG. 14 are the image connection boundary lines of the monitor screen 5 shown in FIGS. 15 (a) and 15 (b) when the vehicle 10 is at the positions 10-4 and 10-5, respectively. 53 indicates the front position of the vehicle displayed in 53 (the boundary between the bird's-eye view image and the bird's-eye view image).

In FIGS. 14 and 15 (a), 61 and 62 are respectively straight lines for the left and right optical flows in the bird's-eye view image and the left and right optical flows in the bird's-eye view image when the vehicle is at the vehicle position 10-4. The left and right optical flows (running road passable width display lines) for displaying the passable width of the forward running road are shown.
These left and right optical flows (running road passage width display lines) 61, 62 are not located near the obstacle at the vehicle position 10-4, so the left end of the road that forms the narrowest part (width) ( Ridgeline) or left and right boundary lines 21 and 22 such as a left wall and a side groove, and the left and right gaps are set to the inside from these left and right boundary lines 21 and 22.

In FIGS. 14 and 15 (b), 63 and 64 are straight lines respectively for the left and right optical flows in the bird's-eye view image and the left and right optical flows in the bird's-eye view image when the vehicle is at its own position 10-5. The left and right optical flows (running road passable width display lines) for displaying the passable width of the forward running road are shown.
These left and right optical flows (running road passable width display lines) 63 and 64 are located near the left utility pole 25 that defines the narrowest part (width) because the utility pole 25 is close to the vehicle position 10-5. The right boundary line 22 of the road is determined, and the left and right gaps are set inward from the utility pole 25 and the right boundary line 22.

FIGS. 15 (a) and 15 (b) show the monitor screen 5 when the host vehicle 10 is at the positions 10-4 and 10-5, as described above. The upper half is a bird's-eye view image, and the lower half is a bird's-eye view image.
In the bird's-eye view image section, the image of the front-end optical float camera 1 is processed, and when the vehicle goes straight in the steering neutral state, the vertical direction of the monitor screen 5 at the left and right connection positions of the optical flows 54 and 55 To be parallel to.

That is, in this embodiment, the left and right optical flows 61, 61 and 62, 63 are set based on the narrowest part (width) defined object in front of the host vehicle, and are used as the travel path passable width display lines.
Accordingly, the left and right optical flows (running path passage width display lines) 61, 61 and 62, 63 in this embodiment are determined by the road environment and remain fixed even if the left and right positions of the host vehicle change.

In FIG. 15 (a) showing the monitor screen 5 when the host vehicle 10 is at the position indicated by 10-4 in FIG. 14, the connection boundary line 53 between the bird's-eye image and the bird's-eye view image displays the L4 position in FIG. ing.
Accordingly, the left and right road boundaries 21 and 22 and the left and right optical flows (running path passage width display lines) 61 and 61 on the inner side by a predetermined gap are displayed on the bird's eye image side and the bird's eye image side.

In FIG. 15 (b) showing the monitor screen 5 when the host vehicle 10 is at the position indicated by 10-5 in FIG. 14, the connection boundary line 53 between the bird's-eye image and the bird's-eye view image displays the L5 position in FIG. ing.
Accordingly, the left and right road boundaries 21 and 22 are displayed on the bird's-eye view image side, the left power pole 25 and the right road boundary 22 are displayed on the bird's-eye view image side, and a predetermined gap from the left power pole 25 on both the bird's-eye view image side and the bird's-eye view image side. Only the left optical flow (running road passable width display line) 63 on the inner side and the right optical flow (running road passable width display line) 64 on the inner side by a predetermined gap from the right road boundary 22 are displayed.
The left and right optical flows (running path passable width display lines) 63 and 64 at the left and right ends of the passable path in the narrowest part are parallel to the monitor screen 5, and the overhead view image and the bird's eye view image are displayed. Since it is displayed as a straight line passing through, it is easy to recall the route from the vehicle position 10-5 to the narrowest part of the road (electric pole 25) in the bird's-eye view image.

  In addition, since it is easy to confirm the presence or absence of obstacles in the road route even from a distance in the bird's-eye view image, if the lateral position of the vehicle is adjusted (steered) before the narrowest part, it is desirable even in the narrowest part. For example, it passes through the outermost part where the driver side gap is small and the passenger side gap is large, or conversely, the passenger side gap is small and the driver side gap is large. It is possible to easily pass through the outermost part.

As is clear from the above-mentioned place, by changing the position where the optical flow is a straight line based on the width of the road that is currently running and the distance between the narrowest gaps in front, the narrowest part, Since it is easy to make the left and right end portions of the travelable route close to each other, it is easy to recall the route through which the vehicle can pass even in an environment where there are discontinuous utility pole arrangements, parked vehicles, etc. .
Therefore, in the case of the first embodiment, it was difficult to predict how far the narrowest part can be approached in the lateral direction, but in the second example, the narrowest part is displayed. In order to set the straight optical flow at the left and right ends of the travelable path, with the environment side as the reference,
It is easy to recall the travelable route portion, and it is possible to bring the vehicle close to the desired route, for example, the driver's seat side or the passenger seat side within the route.

<Effect of the second embodiment>
The effects of the second embodiment will be described below with reference to FIGS.
FIGS. 16 (a) and 17 (a) respectively show the positions of the own vehicle 10, the roads (walls) 21 and 22, and the utility pole 25 when the own vehicle 10 is at the position indicated by 10-5 in FIG. The relationship is shown from above when the vehicle 10 has a different lateral position,
FIG.16 (b) shows the monitor screen 5 when the host vehicle 10 is in the narrow side of the narrow road, as shown in Fig.16 (a).
FIG. 17 (b) shows the monitor screen 5 when the host vehicle 10 is further to the right in the road width direction as shown in FIG. 17 (a) in a narrow path.

  When the vehicle is in the lateral position shown in FIGS. 16 (a) and 16 (b), it is possible to pass through the narrowest part without contacting the utility pole 25 and the road right boundary 22 when traveling straight. It can be easily grasped from the own vehicle position 10-5 within the area between the optical flows (running path passage width display lines) 63 and 64 on the monitor screen 5 of b).

  When the vehicle is in the lateral position shown in FIGS. 17 (a) and 17 (b), it is impossible to pass through the utility pole 25 and the right boundary 22 of the road that make up the narrowest part when going straight ahead. It can be easily grasped from the own vehicle position 10-5 that protrudes from the area between the optical flows (running path passage width display lines) 63 and 64 on the monitor screen 5 in FIG.

By the way, in this embodiment, in addition to the above-described effects of the first embodiment, the following effects can be obtained.
In other words, since the left and right optical flows 63 and 64 (61 and 62) are set based on the road environment that defines the most extreme part of the road, the calculation load of image processing can be reduced, which is advantageous in terms of cost.

Also, these left and right optical flows 63 and 64 (61, 62) are formed as width indication lines that can pass through the road, so that it can be determined whether or not the vehicle can pass through the most extreme part from the position of the vehicle in the lateral direction. Because we decided to support driving,
The widths of the left and right optical flows 63 and 64 (61 and 62) are displayed wider than the own vehicle passable width display lines 54 and 55 in the first embodiment.

  For this reason, in this embodiment, it is possible to know in advance the width of the path that can pass through the narrowest part, and it is possible to freely select the passing path of the narrowest part within a limited range. A route passing through the center can be selected, or a route passing through the right side or the left side of the narrowest part can be selected, and the allowable range of the driving strategy can be widened.

Also, with regard to steering to avoid obstacles, it is possible to predict in advance how much the obstacle can be avoided, and the amount of steering until the obstacle is removed from the inside of the optical flow can be easily made. I can recall.
Therefore, in the present embodiment, it is possible to obtain an effect that the width of a route that can be passed through in the narrowest portion on the road environment side can be determined from a stage far away (bird's eye view image portion) in the monitor 5.

DESCRIPTION OF SYMBOLS 1 Front end camera 2 Right camera 3 Left camera 4 Orbit switch 5 Monitor 6 Laser range finder group 7 Ultrasonic sensor group
7-9 Virtual camera position
10 Own car
21,22 Left and right road boundaries
23,24,25 telephone pole
51 Bird's-eye view of the front of the vehicle
51a Crossing direction
52 Overhead view of the front of the vehicle
52a Crossing line
53 Image connection border
54,55 Optical Flow (Vehicle width indication line)
57 Car icon
61,62 Optical flow (running path width indication line)
63,64 optical flow (width indication line that can pass through the road)
100 control unit
110 Bird's eye view conversion unit for right side
115 Bird's eye view conversion unit for left side
120 Bird's-eye image composition unit
130 Bird's eye image overlay
132 Mask icon storage
134 Optical flow position emphasis line storage
140 Image connection
150 Overhead viewpoint conversion unit for front end
152 Right-side view conversion unit
154 Overhead viewpoint converter for left side
160 Overhead image composition unit
170 Overhead image overlay
175 Car icon storage
200 control unit
210 Ambient environment detector
220 Image processor
230 Narrowest part width detection part
235 Automatic display startup unit
240 Right gap detector
241 Left gap detector
244 Right optical flow position determination unit
246 Left optical flow position determination unit
250 Bird's eye view conversion unit for right side
255 Bird's eye view conversion unit for left side

Claims (5)

In the device that supports the driving of the vehicle by displaying the own vehicle and the vicinity of the own vehicle and the front of the own vehicle on the monitor screen so that the driver can visually observe the vehicle,
A vehicle periphery image creation means for creating a vehicle periphery image displayed as an overhead image viewed so that at least the front end portion of the vehicle and the vicinity of the vehicle front end portion are viewed directly below the vehicle;
When the vehicle travels straight ahead, the rearward flow of the side located at a predetermined position in the vehicle's forward travel path is converted along the longitudinal direction of the vehicle, and the vehicle's forward travel path has a predetermined look-down angle from the vehicle. A forward traveling road image creation means for creating a forward traveling road image displayed as a bird's eye view image,
The vehicle periphery image and the forward travel path image created by these means are set to the travel direction crossing line on the front side in the vehicle traveling direction of the host vehicle peripheral image, and the travel direction crossing line on the rear side in the vehicle traveling direction of the front travel path image. Are connected to each other so that they overlap each other, and image connection means for displaying on the monitor screen ,
A substantially straight traveling path passable width display line that is separated from each other and indicates a width that can be passed through the forward traveling path displayed in the surrounding image of the own vehicle or in the forward traveling path image, A travel path passable width display line setting means that is arranged along the travel path in the travel path image and set in the continuous image ;
The travel path passable width display line setting means sets the vehicle passable width display line provided in the surrounding image of the host vehicle and the travel path passable width display line provided in the forward travel path image by superimposing them individually. Is,
When the image connection means connects the vehicle periphery image and the forward travel road image, the vehicle passable width display line or the travel path passable width display line of the vehicle periphery image and the forward travel road image are mutually connected. A driving support apparatus for a vehicle characterized in that the vehicle periphery image and the forward traveling road image are connected so as to be connected in a substantially straight line . In the vehicle driving support device according to claim 1 ,
The image connecting means is adapted to correct the own vehicle periphery image and the forward traveling road image so that the own vehicle passage width display line or the traveling road passage width display line is connected to each other in a substantially straight line. A driving support apparatus for a vehicle, wherein the correction is performed by shear conversion of an image. In the vehicle driving support device according to claim 1 or 2 ,
The vehicle driving support device according to claim 1, wherein the image connecting means performs the connection of the vehicle periphery image and the forward traveling road image on the ground surface. In the vehicle driving support device according to any one of claims 1 to 3 ,
The image connection means is configured to mask and display pixel portions that overlap each other at the time of connection of the vehicle periphery image and the forward traveling road image. In the vehicle driving support device according to any one of claims 1 to 4 ,
The image connecting means performs the connection at the intersection between the optical axis of the camera that captures the vehicle periphery image and the ground surface when the connection between the vehicle periphery image and the forward traveling road image is performed. A vehicle driving support device.

Images