JP4595902B2 - Vehicle periphery image display system and vehicle periphery image display method - Google Patents

Description

The present invention is related to the vehicle periphery image display system and a vehicle periphery image display method for displaying the overhead image looking down the peripheral vehicle from above the vehicle.

In recent years, with the spread of navigation systems, the number of vehicles equipped with an image display system that displays a situation around the vehicle on a display device in a vehicle compartment is increasing. The situation around the vehicle includes, for example, the position of other vehicles, the situation of obstacles, road markings such as the center line and stop line, etc., but the blind spots such as the rear of the vehicle and under the front bumper and corners are It is difficult for ordinary drivers to recognize. Accordingly, the image display system captures the surrounding situation in the blind spot with an imaging device such as a camera installed in the vehicle, and displays it on the display device when the vehicle is turned to the right, left, back, etc. Is to assist.
Such a display system is intended to reduce the burden on the driver when driving the vehicle. In view of the large amount of information that the driver needs to acquire during driving, the display system is excellent in visibility and without a sense of incongruity. Then, it is desirable to display an image from which the driver can acquire an image from which necessary information can be acquired.

In view of this, there has been conventionally known a technique in which the periphery of a vehicle is imaged by a camera, and an overhead image in which the periphery of the vehicle is viewed from above is generated and displayed based on the captured image. For example, in Japanese Patent Laid-Open No. 2003-115100, an overhead image around the subject vehicle that is parked is displayed when parking, and the display range and scale of the overhead image are automatically changed as the subject vehicle moves. An image display system is described. According to such a technique, the user can intuitively grasp the relationship between the vehicle and the surrounding features and obstacles as well as the situation around the vehicle from the overhead image.
JP 2003-115100 A (page 3 to page 5, FIG. 6)

However, in the image display system described in Patent Document 1 described above, the user can only visually recognize a limited range around the vehicle, particularly in a situation where the scale is enlarged. Therefore, even if an obstacle such as a person or a bicycle approaches the vehicle, the driver cannot confirm the appearance of the obstacle. Often, they were unaware of the presence of obstacles, increasing the likelihood of contact.
On the other hand, if the scale is reduced and only a wide bird's-eye view image is displayed, it is difficult to grasp the exact position and direction of the host vehicle from the bird's-eye view image, and driving operation becomes difficult.

The present invention has been made in order to solve the above-described conventional problems, and when a parking operation is performed, an overhead view including obstacles within an appropriate range in consideration of obstacles located around the vehicle. An object of the present invention is to provide a vehicle periphery image display system and a vehicle periphery image display method capable of displaying an image and allowing a user to easily grasp an accurate surrounding environment.

In order to achieve the above object, a vehicle periphery image display system (1) according to claim 1 of the present application includes an image pickup means (3) for picking up an image of the periphery of the vehicle, and an upper part of the vehicle based on an image picked up by the image pickup means. A bird's-eye view image generation means (4) for generating a bird's-eye view image of the vehicle surroundings, an image display means (5), and a display control means for displaying the bird's-eye view image generated by the bird's-eye view image generation means on the image display means ( 4), a parking determination means (4) for determining whether or not a parking operation has been performed, and a parking space detection means (4) for detecting a parking space in which the vehicle is parked. The display control means includes an entry determination means (4) for judging whether or not the vehicle has entered the parking space, and an obstacle detection means (7-9) for detecting an obstacle around the vehicle. Is When it is determined that the parking operation is performed by the parking determination unit, the overhead image of the first display range is displayed on the image display unit, and the vehicle has entered the parking space by the entry determination unit. When it is determined, the overhead image in the second display range narrower than the first display range is displayed, and when an obstacle is detected by the obstacle detection means, it is wider than the second display range. The overhead image in the third display range including the detected obstacle and the vehicle is displayed.
The “second display range” and the “third display range” may be the same range.

The vehicle periphery image display system (1) according to claim 2 is the vehicle periphery image display system according to claim 1, wherein the third display range is detected by the obstacle detection means (7-9). When an object is detected, the detected obstacle and the vehicle can be displayed on the image display means (5) and are in the narrowest range.

Further, the vehicle periphery image display method according to claim 3 is generated by the overhead image generation step of generating an overhead image in which the vehicle periphery is viewed from above the vehicle based on the captured peripheral image of the vehicle, and the overhead image generation step. A vehicle-peripheral image display method executed by a computer, comprising: a display control step of displaying the overhead image that is displayed on the image display device;
A parking determination step for determining whether or not a parking operation has been performed, a parking space detection step for detecting a parking space in which the vehicle is parked, and an entry determination step for determining whether or not the vehicle has entered the parking space; An obstacle detection step for detecting obstacles around the vehicle, wherein the display control step is the first for the image display device when it is determined that the parking operation is performed by the parking determination step. The bird's-eye view image in the display range is displayed, and the bird's-eye view image in the second display range narrower than the first display range is displayed when it is determined in the approach determination step that the vehicle has entered the parking space. When an obstacle is detected by the obstacle detection step, the third display range is wider than the second display range and includes the detected obstacle and the vehicle. And displaying the瞰image.

In the vehicle periphery image display system according to claim 1 having the above configuration, when it is determined that a parking operation has been performed by generating an overhead view image of the vehicle periphery from above the vehicle based on the image captured by the imaging unit. Displaying an overhead image of the first display range on the image display means, and displaying an overhead image of the second display range narrower than the first display range when it is determined that the vehicle has entered the parking space; When an obstacle is detected, an overhead image of the third display range that is wider than the second display range and includes the detected obstacle and the vehicle is displayed, so that it is easy to see according to the current vehicle situation An overhead image can be displayed only in the range. When an obstacle is located around the vehicle, it is possible to display an overhead image including the obstacle in an appropriate range in consideration of the obstacle. Therefore, the user can easily grasp the accurate surrounding environment.

In the vehicle periphery image display system according to claim 2, since the detected obstacle and the vehicle can be displayed on the image display means and the overhead image is displayed in the narrowest range, the enlarged overhead image is displayed. Therefore, it is possible to grasp the exact position and direction of the own vehicle and to check obstacles located around the vehicle.

In the vehicle periphery image display method according to claim 3, an overhead image in which the vehicle periphery is viewed from above the vehicle is generated based on the captured vehicle periphery image, and the image is displayed when it is determined that the parking operation is performed. An overhead image of the first display range is displayed on the device, and when it is determined that the vehicle has entered the parking space, an overhead image of the second display range narrower than the first display range is displayed, and an obstacle is displayed. When an image is detected, an overhead image of the third display range that is wider than the second display range and includes the detected obstacle and the vehicle is displayed, so that only an easy-to-view range that matches the current vehicle situation is displayed. It becomes possible to display an overhead image. When an obstacle is located around the vehicle, it is possible to display an overhead image including the obstacle in an appropriate range in consideration of the obstacle. Therefore, the user can easily grasp the accurate surrounding environment.

DESCRIPTION OF EMBODIMENTS Hereinafter, a vehicle periphery image display system according to the present invention will be described in detail with reference to the drawings based on an embodiment that is embodied.
First, a schematic configuration of the vehicle periphery image display system 1 according to the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic configuration diagram of a vehicle periphery image display system 1 according to the present embodiment. FIG. 2 is a block diagram schematically showing a control system of the vehicle periphery image display system 1 according to the present embodiment.

  As shown in FIGS. 1 and 2, a vehicle surrounding image display system 1 according to this embodiment includes a rear camera (imaging unit) 3 installed on a vehicle 2 and an image display ECU (overhead image generation unit, display). Control means, parking determination means, parking space detection means, entry determination means) 4, liquid crystal display (image display means) 5, speaker 6, clearance sonar 7-9 connected to image display ECU 4, vehicle speed sensor 10, The sensor includes various sensors such as a steering sensor 11, a gyro sensor 12, and a shift position sensor 13.

  The rear camera 3 uses a solid-state image sensor such as a CCD, for example, and is installed near the upper center of the license plate mounted on the rear side of the vehicle 2 and installed with the line of sight 45 degrees below the horizontal. The Then, the rear of the vehicle, which is the traveling direction of the vehicle 2 when reversing, is imaged, and the captured image (see FIG. 10) is an overhead image (FIG. 11) and displayed on the liquid crystal display 5.

  Further, the image display ECU (Electronic Control Unit) 4 generates a bird's-eye view of the vehicle 2 from above the vehicle 2 from the image captured by the rear camera 3 when the parking operation is performed, When entering the space or when an obstacle is detected, processing for changing the trimming range of the overhead view image displayed on the liquid crystal display 5 or estimating the predicted course of the vehicle 2 at the time of reversing and superimposing the locus on the overhead view image This is an electronic control unit that performs processing and the like for display. The image display ECU 4 may also be used as an ECU used for controlling the navigation device. The detailed configuration of the image display ECU 4 will be described later.

  The liquid crystal display 5 is provided on the center console or the panel surface in the room of the vehicle 2 and displays a bird's-eye view of the periphery of the vehicle 2 when reversing. Note that the liquid crystal display 5 may also be used for a navigation device.

  The speaker 6 is provided on the center console or the panel surface of the vehicle 2 and outputs guidance voices, warning sounds, and the like related to driving assistance.

  Further, the clearance sonars 7 to 9 are respectively attached to a total of three locations of the left corner, the center, and the right corner of the rear bumper of the vehicle 2, and are basically composed of a sound wave transmitting unit and a sound wave receiving unit. And the ultrasonic wave is radiated from the sound wave transmission unit to the left and right diagonal directions and the rear three directions of the vehicle 2 and the reflected wave reflected by the object (specifically, parked vehicle, bicycle, pedestrian, etc.) Received by the sound wave receiver. As a result, it is possible to detect the distance to the obstacle located around the vehicle 2 based on the intensity and wavelength of the received reflected wave.

The vehicle speed sensor 10 is a sensor that generates a vehicle speed pulse in accordance with the rotation of the vehicle wheel and detects the travel distance and the vehicle speed of the vehicle. The steering sensor 11 is attached to the inside of the steering device, and is a sensor that detects a steering angle when the steering wheel is steered. The gyro sensor 12 is a sensor that detects the turning angle of the vehicle 2. The shift position sensor 13 is built in a shift lever (not shown), and the shift position is “P (parking)”, “N (neutral)”, “R (reverse)”, “D (drive)”, It is detected whether the position is “2 (second speed)” or “L (low)”.
As a result, the image display ECU 4 detects the current vehicle speed of the vehicle 2 based on the output signal of the vehicle speed sensor 10, and detects the steering angle of the vehicle 2 based on the output signal of the steering sensor 11. Further, the vehicle direction is detected by integrating the turning angle detected by the gyro sensor 12. Further, the shift position sensor 13 detects the current shift position of the vehicle 2.

Next, the details of the image display ECU 4 will be described with reference to FIG. 2. The image display ECU 4 is configured with the CPU 31 as a core, and a ROM 32 and a RAM 33 which are storage means are connected to the CPU 31. And the parking image display control which produces | generates a bird's-eye view image from the image imaged with the back camera 3 at the time of parking operation in ROM32, and displays the produced bird's-eye view image with respect to the liquid crystal display 5 in the suitable range according to the present condition A processing program (see FIGS. 3 to 8) and other various programs necessary for controlling the liquid crystal display 5 and the like are stored. The RAM 33 is a memory for temporarily storing various data calculated by the CPU 31.
Further, the ROM 32 has various types such as the wheel radius of the vehicle 2, the vehicle width, the minimum turning radius, the optical axis direction of the rear camera 3 (downward 45 degrees in this embodiment), the installation position of the rear camera 3 with respect to the vehicle 2, and the like. Stored for parameters. Then, the CPU 31 estimates the predicted course of the vehicle 2 when reversing from the image captured by the rear camera 3 by using various parameter information stored as will be described later.
The RAM 33 stores various flags (“STT”, “FLGM”) and variables (“NSW”, “STR”, “ΔDM”, “Δd”, “Dx”) which will be described later. Further, a display image to be displayed on the liquid crystal display 5 is temporarily stored. A V-RAM for storing the display image may be provided separately from the RAM.

  Next, a parking image display control processing program executed by the image display ECU 4 of the vehicle periphery image display system 1 according to this embodiment having the above-described configuration will be described with reference to FIG. FIG. 3 is a flowchart of a parking image display control processing program in the vehicle surrounding image display system 1 according to the present embodiment. Here, the parking image display control processing program generates an overhead image from an image captured by the rear camera 3 during the parking operation, and generates the overhead image generated for the liquid crystal display 5 in an appropriate range according to the current situation. It is a program that performs processing to display. 3 to 8 are stored in the ROM 32 and the RAM 33 provided in the image display ECU 4, and are repeatedly executed by the CPU 31 at predetermined intervals (for example, every 200 ms).

  In the parking image display control processing program, first in step (hereinafter abbreviated as S) 1, the CPU 31 switches the display screen of the liquid crystal display 5 based on the determination as to whether or not the parking operation is being performed. A start management process (FIG. 4) is performed.

  Next, in S <b> 2, the CPU 31 performs vehicle signal input processing (FIG. 5) described later for inputting signals from various sensors such as the clearance sonars 7 to 9, the vehicle speed sensor 10, and the steering sensor 11.

  Subsequently, in S <b> 3, the CPU 31 performs an image composition process (FIG. 7) described later, which generates an overhead image in which the surroundings of the own vehicle are viewed from above the own vehicle based on the image captured by the rear camera 3.

  In S4, the CPU 31 performs an image display process (FIG. 8) to be described later for displaying the overhead image generated in S3 on the liquid crystal display 5 within an appropriate trimming range according to the current situation. The process of S4 corresponds to the process of the display control unit.

  Next, a sub-processing program of the system activation management process executed by the CPU 31 in S1 will be described with reference to FIG. FIG. 4 is a flowchart of a sub-processing program for system activation management processing according to the present embodiment.

  In the system activation management process, first, in S11, the CPU 31 inputs a shift position sensor signal “NSW” for specifying the shift position from the shift position sensor 13. In S12, the CPU 31 determines whether or not NSW = R (reverse), that is, whether or not the parking operation is performed with the shift position set to “R”. The process of S11 corresponds to the process of the parking determination unit.

As a result, when it is determined that NSW = R (S12: YES), it is further determined whether or not the flag “STT” indicating that the vehicle peripheral image display system 1 is being activated is ON. (S13).
If it is determined that STT = ON (S13: YES), since the vehicle peripheral image display system 1 has already been activated, the system activation management process is terminated, and the vehicle signal input process of S2 is started. Transition.

  On the other hand, when it is determined that STT = OFF (S13: NO), the current display screen displayed on the liquid crystal display 5 is switched from the normal screen to the parking guidance screen (S14). Here, the normal screen corresponds to, for example, a screen displaying a map image, route information, and the like by a navigation device. In addition, the parking guidance screen corresponds to a screen displaying a bird's-eye view image (see FIGS. 11 to 16) in a predetermined range from above the own vehicle to be parked as described later.

  Next, in S15, the CPU 31 turns on the flag “STT” based on the activation of the vehicle periphery image display system. Further, in S16, a variable “ΔDM” indicating the distance traveled by the vehicle per unit time is initialized (0 is substituted). Thereafter, the process proceeds to the vehicle signal input process of S2.

On the other hand, if it is determined in the determination process of S12 that NSW ≠ R (S12: NO), it is further determined whether or not the flag “STT” is ON (S17).
And when it determines with STT = OFF (S17: NO), a process is complete | finished, without starting the vehicle periphery image display system 1. FIG.

  On the other hand, when it is determined that STT = ON (S17: YES), the current display screen displayed on the liquid crystal display 5 is switched from the parking guidance screen to the normal screen (S18). Next, in S <b> 19, the CPU 31 turns off the flag “STT” based on the termination of the vehicle surrounding image display system 1. In S20, various variables (“NSW”, “STR”, “ΔDM”, “Δd”, “Dx”) are initialized.

  Next, a sub-processing program for vehicle signal input processing executed by the CPU 31 in S2 will be described with reference to FIG. FIG. 5 is a flowchart of a sub-processing program for vehicle signal input processing according to this embodiment.

  In the vehicle signal input process, first, in S31, the CPU 31 inputs a steering sensor signal “STR” for specifying the steering angle of the steering from the steering sensor 11. Then, in S32, the CPU 31 counts the vehicle speed pulse signal “Vp” output from the vehicle speed sensor 10, and calculates the travel distance “Δd” that the host vehicle travels backward per unit time.

  Subsequently, in S33, the CPU 31 updates the value of ΔDM by adding Δd calculated in S32 to the total distance “ΔDM” that the vehicle has traveled in the back after the system is started. Thereafter, in S34, a clearance sonar signal input process (FIG. 6) described later for detecting an obstacle located around the host vehicle is performed, and the process proceeds to an image composition process (S3).

  Next, a sub processing program of clearance sonar signal input processing executed by the CPU 31 in S34 will be described with reference to FIG. FIG. 6 is a flowchart of a sub-processing program of clearance sonar signal input processing according to this embodiment.

  In the clearance sonar signal input process, first, in S41, the CPU 31 inputs a sonar signal “CSDn (n = 1, 2, 3)” for specifying the distance [cm] from the clearance sonar 7-9 to the obstacle. Here, as shown in FIG. 9, in the vehicle periphery image display system 1 according to the present embodiment, the clearance sonar 7 transmits an ultrasonic wave to the left rear of the vehicle 2 to obtain a sonar signal CSD1. Further, the clearance sonar 8 sends an ultrasonic wave to the rear of the vehicle 2 to obtain a sonar signal CSD2. Further, the clearance sonar 9 sends an ultrasonic wave to the right rear of the vehicle 2 to obtain a sonar signal CSD3.

  In S42, the CPU 31 divides the sonar signal input in S41 by 150, and adds a value obtained by adding 1 to the maximum integer not exceeding the value “CSDnP (the value based on the signal from the clearance sonar 7 is CSD1P, The value based on the signal from the clearance sonar 8 is calculated as CSD2P, and the value based on the signal from the clearance sonar 9 is calculated as CSD3P) ".

  Then, in S43, the CPU 31 determines whether 2 <CSDnP ≦ 3 holds for the CSDnP calculated in S42, that is, whether the distance from the vehicle to the detected obstacle is more than 150 cm and less than 300 cm. Is determined. As a result, when it is determined that 2 <CSDnP ≦ 3 is satisfied (S43: YES), “2” is substituted into the obstacle distance flag “DAn” indicating the distance to the obstacle (S44).

  On the other hand, if it is determined that 2 <CSDnP ≦ 3 is not satisfied for the CSDnP calculated in S42 (S43: NO), whether or not 3 <CSDnP is satisfied in S45, that is, the own vehicle It is determined whether the distance to the detected obstacle is more than 300 cm or cannot be detected. As a result, when it is determined that 3 <CSDnP is established (S45: YES), “0” is substituted for “DAn” which is an obstacle distance flag (S46).

  On the other hand, if it is determined that 3 <CSDnP does not hold for CSDnP calculated in S42 (S45: NO), CSDnP = 1 holds. That is, the distance from the vehicle to the detected obstacle is shorter than 150 cm, and the value of CSDnP (ie, “1”) is substituted into “DAn” that is the obstacle distance flag (S47).

  Thereafter, in S48, it is determined whether or not n = 3, that is, whether or not the processing of S41 to S47 has been performed for all the clearance sonars 7 to 9.

  As a result, when it is determined that n ≠ 3 (S48: NO), 1 is added to the value of n (S49), and then the process returns to S41 to specify DAn for the remaining clearance sonars 7-9. I do. On the other hand, when it is determined that n = 3 (S48: YES), the process proceeds to S50, and the value of n is initialized. Thereafter, the process proceeds to the image composition process of S3.

  Next, a sub-processing program of the image composition process executed by the CPU 31 in S3 will be described with reference to FIG. FIG. 7 is a flowchart of a sub-processing program for image composition processing according to this embodiment.

  In the image composition process, first, in S51, the CPU 31 obtains a camera image of the rear environment of the own vehicle captured by the rear camera 3. Thereafter, in S52, the CPU 31 performs a distortion correction calculation process for correcting distortion generated in the vicinity of the outer edge of the camera image acquired in S51.

  Subsequently, in S53, the CPU 31 performs a predetermined overhead conversion process on the camera image whose distortion has been corrected in S52, thereby generating an overhead image vertically looking down from the upper viewpoint as the current image IM-pr. To do. Note that the overhead conversion process for performing viewpoint conversion of the captured image is a known technique (see, for example, Japanese Patent Laid-Open No. 10-211849), and the description thereof is omitted here.

  Further, in S54, a distance index ΔDM for specifying the total travel distance since the system is started is attached to the current image IM-pr generated in S53 (S54), and the current image IM to which ΔDM is attached is attached. Write -pr to the memory (for example, RAM 33) (S55).

  Next, in S56, the CPU 31 calculates an index IND of the past image to be read with respect to the current position. Specifically, a value obtained by subtracting Dx (a constant (fixed value) for calculating an index of a past image to be read with respect to the current position) from the travel distance ΔDM after the system is started is calculated as IND. To do.

  In S57, the CPU 31 reads out the corresponding past image IM-ind from the memory based on the IND value calculated in S56. Further, in S58, the CPU 31 performs a rotation conversion process for rotating and converting the past image IM-ind read out in S57 based on the steering sensor signal input in S31, thereby generating IM-rot. Specifically, when the host vehicle is turning, the overhead view image is corrected according to the turning angle.

Subsequently, in S59, the CPU 31 synthesizes the current image IM-pr generated in S53 and the rotationally converted past image IM-rot generated in S58 to generate a combined overhead image IM-dsp. Thereby, an overhead image obtained by imaging the periphery of the vehicle from above the vehicle is generated.
For example, FIG. 10 is a diagram illustrating a camera image 40 captured by the rear camera 3, and FIG. 11 is an overhead image displayed on the liquid crystal display 5 of the own vehicle when the camera image 40 illustrated in FIG. 10 is captured. It is the figure which showed a certain parking guidance screen. As shown in FIGS. 10 and 11, the other parked vehicles 52 and 53 and the parking lot lines 54 to 56 captured in the camera image 40 are captured and viewed when viewed from above the host vehicle 51. Is displayed on the parking guidance screen 50. The image of the host vehicle 51 that is not captured in the camera image 40 is also displayed at a predetermined position on the parking guidance screen 50 in a shape looking down from above. Note that the position where the host vehicle 51 is displayed on the parking guidance screen 50 is always fixed before entering the parking space 58, and an overhead view image in which the image around the host vehicle 51 changes as the host vehicle 51 moves backward. Updated to Note that the processing of S52 to S59 corresponds to the processing of the overhead image generation means.

  Next, a sub-processing program for image display processing executed by the CPU 31 in S4 will be described with reference to FIG. FIG. 8 is a flowchart of a sub-processing program for image display processing according to this embodiment.

  In the image display process, first, in S61, the CPU 31 determines whether or not a flag “FLGM” indicating that the vehicle has entered the parking space is ON. If it is determined that “FLGM” is ON (S61: YES), the process proceeds to S72 and continues to display a bird's-eye view image in which only the range around the host vehicle is enlarged as described later. .

  On the other hand, if it is determined that “FLGM” is OFF (S61: YES), the left and right partition lines of the parking frame are recognized as line segments LL and LR from the composite overhead image IM-dsp generated in S59. To extract (S62). Specifically, first, the synthesized overhead image IM-dsp is input using analog communication means such as NTSC or digital communication means such as i-link, and converted into a digital image format such as jpeg or mpeg. Next, using the fact that the parking frame is generally a white line or a yellow line, luminance correction is performed based on the luminance difference between the road surface on which the parking frame is drawn in the captured image and the other road surface. After that, binarization processing to separate the target parking frame from the image, geometric processing to correct distortion, smoothing processing to remove image noise, etc., and detect the boundary line between the parking frame and other road surface To do. For example, parking partition lines 54 and 55 are extracted as line segments LL and LR from the overhead view image shown in FIG. Then, the range of the parking space 58 is specified and detected by the extracted line segments LL and LR. The process of S62 corresponds to the process of the parking space detection unit.

  Next, in S63, the CPU 31 calculates formulas LNL and LNR in the image display coordinate system by performing a linear approximation process on the line segments LL and LR of the parking frame partition line extracted in S62.

Subsequently, in S64, the CPU 31 determines whether or not the steering sensor signal STR is “0”, that is, whether or not the host vehicle is receding linearly without turning. When it is determined that STR ≠ 0 (S64: NO), that is, when the host vehicle is turning while turning, the first trimming range (first range) is set (S65). ).
In S66, the CPU 31 trims the synthesized overhead image IM-dsp generated in S59 by the first trimming range set in S66, and further trims it according to the display range that can be displayed on the liquid crystal display 5. The enlarged image is enlarged or reduced and transferred to the V-RAM provided in the image display ECU 4 (S66). Then, the trimmed image is displayed on the liquid crystal display 5. As a result, as shown in FIG. 11 and FIG. 12, a bird's-eye view image overlooking a wide area around the host vehicle 51 from above the host vehicle 51 trimmed in the first trimming range is displayed on the liquid crystal display 5. The V-RAM may be shared with the RAM 33.

  In S67, the travel locus of the vehicle is predicted based on the steering sensor signal STR. Then, the guide lines 56 and 57 indicating the travel trajectory and the space line 59 indicating the parking space 58 are drawn on the screen so as to be superimposed on the overhead image.

  On the other hand, if it is determined in S64 that STR = 0 (S64: YES), that is, if it is determined that it is receding linearly, then the coordinates LNLe of the end points of LNL and LNR are LNRe is calculated (S68).

  Thereafter, in S69, whether the Y coordinate of any of the LNL and LNR end points LNLe and LNRe is greater than the Y coordinate of the straight line Ys including the rear end of the own vehicle (see FIGS. 11 and 12), that is, It is determined whether or not the own vehicle has entered the parking space 58 partitioned by the line segments LL and LR of the parking lot line. The process of S69 corresponds to the process of the entry determination unit.

  As a result, when it is determined that both the Y coordinates of the coordinates LNLe and LNRe are smaller than Ys (S69: NO), it is determined that the vehicle has not entered the parking space, and the process proceeds to S65 and continues. The overhead image trimmed in the first trimming range is displayed.

  On the other hand, if it is determined that one of the coordinates LNLe and LNRe is greater than Ys (S69: YES), the process proceeds to S70, where a flag “indicating that the vehicle has entered the parking space 58” It is determined whether or not “FLGM” is ON. If it is determined that “FLGM” is OFF (S70: NO), FLGM is turned ON (S71). On the other hand, if it is already determined that “FLGM” is ON (S70: YES), the process proceeds to S71.

  In S72, the CPU 31 sets a second trimming range (second range). Note that the second trimming range is narrower than the first trimming range.

  Subsequently, in S73, whether or not the obstacle distance flag “DA2” specified in S44, S46, S47 is “1” based on the sonar signal of the clearance sonar 8 disposed in the center, that is, the own vehicle. It is determined whether an obstacle has been detected within 150 cm behind. If it is determined that DA2 = 1 (S73: YES), an obstacle detection flag “FLGDA2” indicating that an obstacle has been detected within 150 cm behind the host vehicle is set to “ON”. On the other hand, when it is determined that DA2 ≠ 1 (S73: NO), the process proceeds to S75.

  In S75, whether or not the obstacle distance flag “DA1” specified in S44, S46, S47 is “1” based on the sonar signal of the clearance sonar 7 arranged on the left side, that is, the vehicle's left rear It is determined whether an obstacle is detected within 150 cm. If it is determined that DA1 = 1 (S75: YES), an obstacle detection flag “FLGDA1” indicating that an obstacle has been detected within 150 cm on the left rear side of the host vehicle is set to “ON”. On the other hand, when it is determined that DA1 ≠ 1 (S75: NO), the process proceeds to S77.

  Furthermore, in S77, whether or not the obstacle distance flag “DA3” specified in S44, S46, S47 is “1” based on the sonar signal of the clearance sonar 9 arranged on the right side, that is, the own vehicle. It is determined whether an obstacle has been detected within 150 cm on the right rear. If it is determined that DA3 = 1 (S77: YES), the obstacle detection flag “FLGDA3” indicating that an obstacle has been detected within 150 cm on the right rear side of the host vehicle is set to “ON”. On the other hand, if it is determined that DA3 ≠ 1 (S77: NO), the process proceeds to S79.

  In S79, a third trimming range (third range) obtained by expanding the second trimming range set in S72 based on the setting of “FLGDA1 to 3” is newly set (S79). Specifically, when “FLGDA1” is set to ON, it is extended by 1 to 5 m to the left rear, and when “FLGDA2” is set to ON, it is extended by 1 to 5 m to the rear and “FLGDA3” If "" is set to ON, a third trimming range extended by 1 to 5 m is set to the right rear (S80). If a plurality of flags are set to ON, a third trimming range is set by extending the second trimming range in a plurality of directions corresponding to the flags set to ON. On the other hand, when it is determined that DA2 ≠ 1 (S73: NO), the process proceeds to S75.

  Next, in S80, the CPU 31 determines whether the second trimming range set in S72 or, if the third trimming range is set in S79, the combined overhead generated in S59 by the third trimming range. The image IM-dsp is trimmed, and the trimmed image is enlarged or reduced in accordance with the display range that can be displayed on the liquid crystal display 5 and transferred to the V-RAM included in the image display ECU 4. Then, the trimmed image is displayed on the liquid crystal display 5.

  As a result, when no obstacle is detected within 150 cm by any of the clearance sonars 7 to 9, the periphery of the own vehicle is looked down from above the own vehicle 51 trimmed in the second trimming range shown in FIG. A parking guidance screen 60 based on the overhead image is displayed on the liquid crystal display 5. Here, as shown in FIG. 13, the parking guidance screen 60 is displayed in a range where the own vehicle 51 is the largest, and as a result, only the own vehicle 51 and the extremely narrow range around it are displayed.

  Further, when an obstacle is detected within 150 cm by the clearance sonar 7, parking guidance based on an overhead image obtained by looking down on the periphery of the own vehicle 51 from above the own vehicle 51 trimmed in the third trimming range shown in FIG. The screen 61 is displayed on the liquid crystal display 5. Here, as shown in FIG. 14, the parking guidance screen 61 displays the bird's-eye view image in the narrowest range that includes the host vehicle 51 and the detected obstacle (the bicycle 65 in FIG. 14) and can be displayed on the liquid crystal display 5. The Note that the third trimming range becomes wider than the second trimming range as the range is expanded by 1.5 m to the left rear.

  Further, when an obstacle is detected within 150 cm by the clearance sonar 8, parking guidance based on the bird's-eye view of the surroundings of the vehicle from above the vehicle 51 trimmed in the third trimming range shown in FIG. The screen 62 is displayed on the liquid crystal display 5. Here, as shown in FIG. 15, the parking guidance screen 62 displays the bird's-eye view image in the narrowest range that includes the host vehicle 51 and the detected obstacle (bicycle 65 in FIG. 15) and can be displayed on the liquid crystal display 5. The Note that the third trimming range becomes wider than the second trimming range as the range is expanded by 1.5 m backward.

  Further, when an obstacle is detected within 150 cm by the clearance sonar 9, parking guidance based on an overhead view image of the surroundings of the own vehicle 51 from above the own vehicle 51 trimmed in the third trimming range shown in FIG. The screen 63 is displayed on the liquid crystal display 5. Here, as shown in FIG. 16, the parking guidance screen 63 displays the bird's-eye view image in the narrowest range that includes the host vehicle 51 and the detected obstacle (the bicycle 65 in FIG. 16) and can be displayed on the liquid crystal display 5. The Note that the third trimming range becomes wider than the second trimming range as the range is expanded by 1.5 m to the right rear.

As described in detail above, in the vehicle periphery image display system 1 according to the present embodiment, when it is determined that the parking operation is performed in the vehicle 2 (S12: YES), from the camera image captured by the rear camera 3. A bird's-eye view image of the vehicle periphery is generated from above the vehicle 2 (S52 to S59), and in particular, when an obstacle is detected within a predetermined distance of the vehicle 2 (within 150 cm in this embodiment) by the clearance sonars 7 to 9 Displays the overhead image trimmed in the third trimming range including the detected obstacle and the vehicle 2 on the liquid crystal display 5, so that the obstacle located in the appropriate range in consideration of the obstacle located around the vehicle. It is possible to display a bird's-eye view image including Therefore, the user can easily grasp the accurate surrounding environment.
When an obstacle is detected, the detected obstacle and the vehicle 2 can be displayed on the liquid crystal display 5 and the overhead image is displayed in the narrowest trimming range. The correct position and direction of the vehicle can be grasped from the image, and obstacles located around the vehicle can be confirmed.
In addition, since it is possible to display an overhead image on the liquid crystal display 5 when it is determined that the parking operation has been performed, an obstacle positioned around the vehicle is displayed at the time of the parking operation that particularly requires information on the surrounding environment for the user. In consideration, an overhead image including an obstacle can be displayed in an appropriate range. Therefore, the user can easily perform the parking operation.
Before the vehicle 2 enters the parking space, an overhead image (see FIGS. 11 and 12) based on the first trimming range in which the periphery of the vehicle 2 is displayed in a wide range is displayed as the parking guidance screen 50. When the vehicle enters the parking space, a bird's-eye view image (see FIG. 13) based on the second trimming range narrower than the first trimming range is displayed as the parking guidance screen 60, and further, the vehicle 2 by the clearance sonar 7-9. When the obstacle is detected within a predetermined distance (within 150 cm in the present embodiment), the overhead image (see FIGS. 14 to 16) is displayed in the parking guidance screen 62 by the third trimming range wider than the second trimming range. Since it is displayed as .about.64, it is possible to display a bird's-eye view image in an easy-to-view range that matches the current vehicle conditions.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, when an obstacle is detected behind the vehicle 2 by the clearance sonars 7 to 9, an overhead image trimmed in the third trimming range including the vehicle and the obstacle is displayed. By arranging the clearance sonar in front or side of the vehicle, even when an obstacle is detected in front or side of the vehicle 2, an overhead view is trimmed in the third trimming range including the vehicle and the obstacle. An image may be displayed. Further, the obstacle detection means is not limited to the clearance sonar, and for example, an obstacle may be detected based on an image captured by a camera.

  Further, the second trimming range and the third trimming range may be the same range. In that case, the detected obstacle is newly displayed only by changing the display area of the display.

  Moreover, in this embodiment, when an obstacle is detected behind the vehicle 2 by the clearance sonars 7 to 9, the vehicle and the obstacle are displayed in the same display window. You may make it display on the window different from the window which displays a vehicle.

  When the parking operation is started, a map screen around the vehicle by the navigation device is displayed on the left half of the liquid crystal display 5, and the parking guidance screens 50 and 60 are displayed only on the right half (FIG. 11). To FIG. 13) may be displayed. When an obstacle is detected behind the vehicle 2, the map screen is deleted, and the display area of the parking guidance screens 62 to 64 (FIGS. 14 to 16) including the vehicle 2 and the obstacle is enlarged. May be displayed.

It is a schematic block diagram of the vehicle periphery image display system which concerns on this embodiment. It is a block diagram which shows typically the control system of the vehicle periphery image display system which concerns on this embodiment. It is a flowchart of the parking image display control processing program which concerns on this embodiment. It is a flowchart of the sub process program of the system starting management process which concerns on this embodiment. It is a flowchart of the sub process program of the vehicle signal input process which concerns on this embodiment. It is a flowchart of the sub process program of the clearance sonar signal input process which concerns on this embodiment. It is a flowchart of the sub-processing program of the image composition processing according to the present embodiment. It is a flowchart of the sub-processing program of the image display process which concerns on this embodiment. It is the schematic diagram which showed the detection direction of the obstruction of a vehicle. It is the figure which showed the camera image imaged with the back camera. It is the figure which showed the bird's-eye view image displayed on the liquid crystal display of the own vehicle at the time of imaging the camera image shown in FIG. It is the figure which showed the bird's-eye view image based on the 1st trimming range displayed on a liquid crystal display. It is the figure which showed the bird's-eye view image based on the 2nd trimming range displayed on a liquid crystal display. It is the figure which showed the bird's-eye view image based on the 3rd trimming range displayed on a liquid crystal display, when an obstruction is detected in the left back of a vehicle. It is the figure which showed the bird's-eye view image based on the 3rd trimming range displayed on a liquid crystal display, when an obstruction is detected behind the vehicle. It is the figure which showed the bird's-eye view image based on the 3rd trimming range displayed on a liquid crystal display, when an obstruction is detected in the right rear of a vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle periphery image display system 2 Vehicle 3 Rear camera 4 Image display ECU
5 Liquid crystal display 7-9 Clearance sonar 31 CPU
32 ROM
33 RAM

Claims (3)

Imaging means for imaging the periphery of the vehicle;
An overhead image generation means for generating an overhead image that overlooks the periphery of the vehicle from above the vehicle based on the image captured by the imaging means;
Image display means;
A vehicle surrounding image display system comprising: a display control unit that displays the overhead image generated by the overhead image generation unit on the image display unit;
Parking determination means for determining whether or not a parking operation has been performed;
Parking space detecting means for detecting a parking space where the vehicle parks;
Entry determination means for determining whether a vehicle has entered the parking space;
Obstacle detection means for detecting obstacles around the vehicle,
The display control means displays the overhead image in the first display range on the image display means when it is determined that the parking operation is performed by the parking determination means, and the vehicle is When it is determined that the vehicle has entered a parking space, the overhead image in the second display range narrower than the first display range is displayed, and when the obstacle is detected by the obstacle detection means, the second The vehicle periphery image display system characterized by displaying the said bird's-eye view image of the 3rd display range wider than the display range of this, including the detected obstacle and a vehicle. The third display range is the narrowest range in which the detected obstacle and the vehicle can be displayed on the image display means when an obstacle is detected by the obstacle detection means. The vehicle periphery image display system according to claim 1 . A bird's-eye view image generating step for generating a bird's-eye view image of the vehicle surroundings from above the vehicle based on the imaged surrounding image of the vehicle;
A vehicle-peripheral image display method executed by a computer, comprising: a display control step for displaying an overhead image generated by the overhead image generation step on an image display device;
A parking determination step for determining whether or not a parking operation has been performed;
A parking space detection step for detecting a parking space in which the vehicle parks;
An entry determination step for determining whether a vehicle has entered the parking space; and
An obstacle detection step for detecting obstacles around the vehicle,
The display control step displays the overhead image in the first display range on the image display device when it is determined that the parking operation is performed in the parking determination step, and the vehicle is When it is determined that the vehicle has entered the parking space, the overhead image in the second display range that is narrower than the first display range is displayed, and when the obstacle is detected by the obstacle detection step, the second And displaying the bird's-eye view image in a third display range that is wider than the display range and includes the detected obstacle and the vehicle.

Images