JP2012188058A - Display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device which intuitively indicates the presence of an obstacle to a user in correspondence to the obstacle's level from a road surface, thereby prevents his/her own vehicle from coming in contact with the obstacle.SOLUTION: The display device includes an image-capturing means 1 which captures an image of the surroundings of a user's vehicle, a sensor 4 which detects an obstacle at the side of the user's vehicle, an image processing unit 2 which makes an overlooking image based on a taken image obtained from the image-capturing means 1 and superposes graphic on the overlooking image based on information obtained from the sensor 4, and a display means 3 which displays an image on which graphic is superposed by the image processing unit 2. When the sensor 4 detects the obstacle at the side of the user's vehicle, the image processing unit 2 superposes a boundary guide for an area ranging vertically from a road surface to a door mirror at the side where the obstacle is detected.

Description

  The present invention relates to a display device that displays a vehicle boundary on a display.

  2. Description of the Related Art Conventionally, there has been known an all-around imaging means system that creates a bird's-eye view image by projecting an image captured by an in-vehicle camera onto a road projection model (see, for example, Patent Document 1). In the bird's-eye view image using the road surface projection model, a three-dimensional object such as a vehicle is reflected in a planar shape.

JP 2005-12465 A

  However, there is a discrepancy between the position of the three-dimensional object displayed in the overhead view image using the road surface projection model and the actual three-dimensional object. For this reason, even if a user drives so that a solid thing and a self-vehicle may not overlap on a bird's-eye view picture, a self-vehicle and a solid thing may collide. This will be specifically described with reference to the drawings.

  FIG. 6 is a diagram illustrating an image of the situation around the host vehicle during a parking operation. As shown in FIG. 6, the vehicle 101 is parked on the right side of the host vehicle 100. The vehicle 101 is parked between the parking frame lines 102 and 103. The vehicle body of the host vehicle 100 is provided with imaging means (not shown). This imaging means images the surroundings of the own vehicle. In particular, the imaging means provided on the right side of the host vehicle 100 among the imaging means is indicated by an imaging range 104.

  FIG. 7 is a diagram for explaining the state of the periphery of the vehicle 101 in FIG. 6 as viewed from above. As shown in FIG. 7, when the periphery of the vehicle 101 is viewed from above, the side ends of the vehicle 101 are located on the parking frame lines 102 and 103. In particular, the left end of the vehicle 101 extends outward from the inner boundary of the parking frame line 102. Furthermore, the door mirror 103 on the left side of the vehicle 101 protrudes outward in the vehicle width direction from the outer boundary of the parking frame line 102.

  FIG. 8 is a diagram illustrating an overhead view image around the host vehicle 100 in FIG. 6 as an image. As shown in FIG. 8, it is a figure explaining the bird's-eye view image | video using the road surface projection model from right above the own vehicle 100 by an image. As shown in FIG. 8, the vehicle 101 is parked on the right side of the host vehicle 100. The vehicle 101 is displayed in a shape that collapses in the upper part of the screen. A parking frame line 102 is displayed under the tire of the vehicle 101.

Here, it is assumed that the user of the host vehicle 100 determines that the parking frame line 102 on the screen is a boundary line and drives the host vehicle 100 so as not to overlap the parking frame line 102. However, when the host vehicle 100 is driven with the parking frame line 102 as a boundary line, the left door mirror of the vehicle 101 actually located outside the parking frame line 102 may collide with the right end of the host vehicle 100. In addition, the right door mirror of the host vehicle 100 may collide with the left end of the vehicle 101 located on the parking frame line 102. When a bird's-eye view image using a road projection model is created from directly above the host vehicle 100 as shown in FIG. 8, the display of the positional relationship between the vehicle 101 and the parking frame line 102 is shown in FIG. This is different from the display of the actual positional relationship with the frame line 102. Then, the display in FIG. 8 may cause the host vehicle 100 to come into contact with the vehicle 101. In particular, when the side end position changes according to the height of the vehicle 100, such as a door mirror, the possibility of a collision is further increased.

  Therefore, an object of the present invention is to prevent the contact between the host vehicle and the obstacle by intuitively showing the user the presence of the obstacle according to the height from the road surface.

  In order to achieve the above object, when the sensor detects a side obstacle, the image processing unit according to the present invention has a height boundary guide from the road surface to the door mirror on the side where the obstacle is detected. It is characterized by superimposing a graphic indicating.

  According to the present invention, it is possible to intuitively show the presence of an obstacle according to the height from the road surface to the user and to prevent contact between the host vehicle and the obstacle.

The block diagram which shows the structure of the display apparatus in embodiment of this invention The flowchart figure explaining the process by CPU which is the principal part of the same FIG. The flowchart figure explaining the graphic superimposition process by the signal processing part which is the principal part of the same FIG. The figure explaining the state which the signal processing part which is the principal part of the same FIG. The figure explaining the other state of the same figure 4 with the image A diagram explaining the situation around the host vehicle when parking The figure explaining the state which looked at the circumference of the vehicles of Drawing 6 from the top with an image The figure explaining the bird's-eye view image of the periphery of the own vehicle of FIG.

  Hereinafter, a display device according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, a display device when the vehicle is moving backward will be described.

  FIG. 1 is a block diagram showing a configuration of a display device according to an embodiment of the present invention.

  As shown in FIG. 1, the display device includes an image pickup unit 1 that is attached to a vehicle body and picks up an image of the periphery of the host vehicle, an image processing unit 2 that performs image processing on a picked-up image input from the image pickup unit 1, and an image processing unit 2. Display means 3 for displaying the display image inputted from the vehicle, a sensor 4 provided on the vehicle body for detecting an obstacle outside the vehicle, and an input means 5 for the user to input information.

  The image pickup means 1 is composed of a plurality of cameras and picks up an image around the host vehicle. For example, the vehicle body is provided with one camera on each of the front, rear, and left and right sides to capture the entire periphery of the host vehicle. The imaging means 1 converts the captured image into an electrical signal and inputs it to the image processing unit 2.

  The image processing unit 2 is configured by an ECU (Electric Control Unit) and is provided in the vehicle interior. The image processing unit 2 corrects the lens distortion of the captured image input from the imaging unit 1. The image processing unit 2 generates a viewpoint converted image obtained by performing viewpoint conversion on the corrected image. Further, the image processing unit 2 superimposes a graphic such as a guideline or the own vehicle on the overhead image. In order to make the captured image of the portion where the graphic is superimposed visible, it is desirable that the image processing unit 2 performs α blending on each graphic and then superimposes the captured image on the captured image.

The display means 3 is provided in a vehicle that is visible to the user. For example, the display means 3 is also used as a display of a navigation device provided in the instrument panel. The display means 3 displays the video signal input from the image processing unit 2. For example, the display unit 3 displays a captured image and various graphics superimposed on the captured image.

  The sensor 4 is composed of a plurality of ultrasonic sonars, and is provided at the front, rear, side, corner, etc. of the vehicle body. At least two sonars are provided on each of the front, rear, left and right sides. The sensor 4 emits ultrasonic waves and acquires a reflection signal from an obstacle. As a result, the sensor 4 detects an obstacle approaching the host vehicle. The sensor 4 inputs the reflection signal from the obstacle to the image processing unit 2.

  The input means 5 is composed of at least one of a switch, a button, and a touch panel, for example. The user operates the input unit 5 to give various instructions to the image processing unit 2.

  The image processing unit 2 includes an AD converter 21, a signal processing unit 22, a video memory 23, a nonvolatile memory 24, a CPU 25, and a DA converter 26.

  The AD converter 21 is composed of a plurality of AD converters, and is connected to each camera constituting the imaging means 1. The AD converter 21 acquires a captured image from each camera of the imaging unit 1 and performs analog-digital conversion, and outputs the converted captured image to the signal processing unit 22.

  The signal processing unit 22 performs lens distortion correction processing on the captured image input from the AD converter 21. Then, the signal processing unit 22 performs viewpoint conversion processing and graphic superimposition processing on the corrected image. For example, the signal processing unit 22 converts the viewpoint from an upper side of the host vehicle into an overhead image using a road surface projection model.

  The video memory 23 temporarily stores the captured image input to the signal processing unit 22 and the processing result by the signal processing unit 22.

  The nonvolatile memory 24 includes a correction data table necessary for lens distortion correction processing, a data table such as a mapping table necessary for viewpoint conversion processing, graphic data of the host vehicle, and a graphic indicating a boundary guide of the height from the road surface to the door mirror. Store the data.

  The CPU 25 controls the entire image processing unit 2. For example, the CPU 25 calculates the distance between the host vehicle and the obstacle based on the reflection signal input from the sensor 4. The CPU 25 instructs the signal processing unit 22 to perform graphic superimposition processing indicating the boundary guide of the height from the road surface to the door mirror based on the information on the distance between the host vehicle and the obstacle. For example, when the CPU 25 determines that the distance between the host vehicle and the obstacle is equal to or less than a predetermined threshold, the CPU 25 superimposes a graphic indicating the boundary guide on the signal processing unit 22.

  Next, processing performed by the CPU 25 on the signal processing unit will be described. FIG. 2 is a flowchart illustrating processing performed by the CPU 25 on the signal processing unit 22.

  As shown in step S <b> 1, the CPU 25 first detects an obstacle on the side of the host vehicle using each sonar of the sensor 4. For example, the CPU 25 instructs each sonar of the sensor 4 to output an ultrasonic wave. Then, the CPU 25 acquires sound wave reflection information from the obstacle from each sonar. Based on this acquired information, the CPU 25 acquires distance information between the host vehicle and the obstacle according to the height from the road surface for each predetermined height.

Next, as shown in step S <b> 2, the CPU 25 determines address information to be read from the nonvolatile memory 24 when an obstacle is detected on the side of the host vehicle based on information acquired from each sonar of the sensor 4. For example, the CPU 25 determines that an obstacle has been detected on the side of the host vehicle when the distance information acquired from at least one sonar of the sensor 4 is equal to or less than a predetermined threshold. On the other hand, when the distance information acquired from any sonar of the sensor 4 is larger than the predetermined threshold, the CPU 25 determines that no obstacle is detected around the host vehicle. In this case, the CPU 25 repeats the processing from step S1 again after a predetermined time. When the CPU 25 determines that there is an obstacle on the side of the host vehicle, the CPU 25 determines the address where the graphic data of the boundary guide of the height from the road surface to the door mirror is stored in the nonvolatile memory 24 as the address information to be read. To do.

  Then, as shown in step S <b> 3, the CPU 25 outputs the address information determined in step S <b> 2 to the signal processing unit 22.

  Next, graphic superimposition processing by the signal processing unit 22 will be described.

  FIG. 3 is a flowchart for explaining graphic superimposition processing by the signal processing unit 22.

  As shown in step S <b> 11, the signal processing unit 22 temporarily stores the captured image input from the AD converter 21 in the video memory 23.

  As shown in step S <b> 12, the signal processing unit 22 takes out the captured image temporarily stored in step S <b> 11 from the video memory 23. The signal processing unit 22 performs distortion correction processing on the captured image based on the correction data table stored in the nonvolatile memory 24. Then, the signal processing unit 22 creates a display video to be output to the display unit 3 from the processed image that has been subjected to the distortion correction processing.

  Next, as shown in step S13, the signal processing unit 22 superimposes a graphic such as a guideline read from the nonvolatile memory 24 on the display image in step S12.

  Next, as shown in step S14, the signal processing unit 22 reads the boundary guide graphic data from the non-volatile memory 24 based on the address information input from the CPU 25 in S3 and superimposes it on the display image in step S12. .

  Next, as shown in step S <b> 15, the signal processing unit 22 outputs the superimposed video that is superimposed on the display video in steps S <b> 13 and S <b> 14 to the DA converter 26. The display means 3 displays this superimposed video input from the DA converter 26.

  Next, the boundary guide graphic superimposed in step S14 will be described.

  FIG. 4 is a diagram illustrating the state in which the signal processing unit 22 overlaps the boundary guide. In FIG. 4, an image of a state in which the guard rail 50 that is an obstacle exists on the left side of the host vehicle 40 is viewed from the front of the host vehicle 40, and an image of an overhead view image created by the signal processing unit 22 at this time is shown. Indicated.

When the sensor 4 detects the guard rail 50, the overhead image of FIG. 4 is superimposed with a graphic 43 of the host vehicle and a guard rail 51 having a shape falling down in a direction away from the graphic 43 of the host vehicle. Further, a boundary guide 44 is superimposed on the left side of the own vehicle graphic 43 (to the right of the own vehicle graphic 43 in FIG. 4). As shown in FIG. 4, the boundary guide 44 has a trapezoidal shape. The side 44a of the boundary guide 44 is a graphic 4 of the host vehicle.
3 side. The position of the side 44 a is uniquely determined according to the distance between the host vehicle 40 and the guard rail 50 detected by the sensor 4. A side 44a of the boundary guide 44 indicates the position of the guardrail 50 on the overhead view video. The user can avoid a collision with the guardrail 50 by driving the host vehicle 40 so that the graphic 43 of the host vehicle does not exceed the side 44a of the boundary guide 44 on the overhead view video.

  On the other hand, the side 44 b of the boundary guide 44 indicates the height from the road surface of the host vehicle 40 to the door mirror 41. As shown in FIG. 4, the user can recognize that the guard rail 50 is lower than the door mirror 41 when the side 44 b of the boundary guide 44 is positioned to the right of the right end of the guard rail 51 on the overhead view image. At this time, the user can recognize that the door mirror 41 and the guard rail 50 do not collide. Further, the user can recognize that the host vehicle 40 may be driven so that the left end 42 of the host vehicle 40 does not collide with the guard rail 50. At this time, the user operates the vehicle 40 and the guardrail by driving so that the vehicle body left end 46 does not exceed the side 44a even if the door mirror 45 of the graphic 43 of the vehicle exceeds the side 44a of the boundary guide 44 on the overhead view video. 50 collisions can be prevented.

  As shown in FIG. 4, the signal processing unit 22 superimposes a first guide line 47 extending along the vehicle body left end 46 on the overhead view video. Since the first guide line 47 serves as an index line indicating the vehicle body left end 46, the user can easily drive the vehicle so that the vehicle left end 46 does not exceed the side 44a of the boundary guide 44. Further, the signal processing unit 22 may further superimpose the guide mark 48 on the left side of the side 44 a of the boundary guide 44. This guide mark 48 means that the side 44a of the boundary guide 44 has a margin value. The guide mark 48 is located further on the side of the host vehicle 40 than the side 44a that is the end of the boundary guide 44 on the side of the host vehicle 40. The guide mark 48 indicates that the host vehicle can move until the first guide line 47 comes into contact. Since the user can recognize that the first driving line 47 is driven until it matches the guide mark 48, the collision between the host vehicle 40 and the guard rail 50 can be prevented more reliably.

  The signal processing unit 22 may further superimpose a height index line 49. The height index line 49 includes a plurality of index lines 49a to 49c. The signal processing unit 22 may add a distance numerical value to each of the index lines 49a to 49c. This makes it easier for the user to intuitively recognize that the boundary guide 44 is a graphic indicating the height.

  As shown in FIG. 4, the guide mark 48 and the height index line 49 are superimposed on the front side of the graphic 43 of the host vehicle. The signal processing unit 22 determines positions before and after the guide mark 48 and the height index line 49 are superimposed on the basis of information on the traveling direction of the host vehicle acquired from a traveling direction detection unit (not shown). For example, when the signal processing unit 22 obtains drive position information from a shift lever (not shown), the guide mark 48 and the height index line 49 are superimposed in front of the graphic 43 of the host vehicle as shown in FIG. This prevents a collision in the traveling direction of the host vehicle 40 that easily collides with an obstacle, reduces the overlapping area of the guide mark 48 and the height index line 49, and maintains the visibility of the boundary guide 44. Can be compatible.

  Next, another state of the boundary guide graphic superimposed in step S14 will be described. FIG. 5 is a diagram for explaining another state of FIG. 4 with an image. In FIG. 5, an image of a state in which a wall 52 that is an obstacle exists on the left side of the host vehicle 40 is viewed from the front of the host vehicle 40, and an image of an overhead view image created by the signal processing unit 22 at this time Indicated. In FIG. 5, the same components as those in FIG. 4 are denoted by the same reference numerals, and detailed description thereof is omitted.

When the sensor 4 detects the wall 52, the overhead image of FIG. 5 is superimposed with the graphic 43 of the host vehicle and the wall 53 having a shape that falls down in a direction away from the graphic 43 of the host vehicle. Further, a boundary guide 44 is superimposed on the left side of the graphic 43 of the host vehicle. As shown in FIG. 5, the boundary guide 44 has a trapezoidal shape similar to that in FIG. 4. The position of the side 44 a of the boundary guide 44 is uniquely determined according to the distance between the vehicle 40 and the wall 52 detected by the sensor 4. A side 44a of the boundary guide 44 indicates the position of the wall 53 on the overhead view video. The user can avoid a collision with the wall 52 by driving the host vehicle 40 so that the graphic 43 of the host vehicle does not exceed the side 44a of the boundary guide 44 on the overhead view video.

  On the other hand, the side 44 b of the boundary guide 44 is positioned to the left of the right end of the wall 53 on the overhead view image of FIG. At this time, the user can recognize that the wall 52 is higher than the door mirror 41. Then, the user can recognize that the door mirror 41 and the wall 52 may collide. Then, the user can recognize that it is only necessary to drive the host vehicle 40 so that the door mirror 41 does not collide with the wall 52. At this time, the user can prevent the collision between the host vehicle 40 and the guardrail 50 by driving so that the door mirror 45 of the graphic 43 of the host vehicle does not exceed the side 44a of the boundary guide 44 on the overhead view video.

  As illustrated in FIG. 5, the signal processing unit 22 superimposes a second guide line 54 extending along the side edge 45 of the door mirror on the overhead view video. Since the second guide line 54 serves as an index line indicating the side end 45 of the door mirror, the user can easily drive so that the side end 45 of the door mirror does not exceed the side 44 a of the boundary guide 44. Further, the signal processing unit 22 may further superimpose the guide mark 55 on the left side of the side 44 a of the boundary guide 44. Since it can be recognized that the user drives the second guide line 54 until it matches the guide mark 55, the collision between the host vehicle 40 and the guard rail 50 can be prevented more reliably. Note that the signal processing unit 22 may further superimpose the height index line 56. The height index line 56 includes a plurality of index lines 56a to 56c. The signal processing unit 22 may add a distance numerical value to each of the index lines 56a to 56c. This makes it easier for the user to intuitively recognize that the boundary guide 44 is a graphic indicating the height.

  As shown in FIG. 5, the guide mark 55 and the height index line 56 are superimposed on the rear side of the graphic 43 of the host vehicle. When the signal processing unit 22 obtains reverse position information from a shift lever (not shown), the signal processing unit 22 superimposes the guide mark 55 and the height index line 56 behind the graphic 43 of the host vehicle. This prevents a collision in the backward direction of the host vehicle 40 that easily collides with an obstacle when moving backward, and reduces the overlapping area of the guide mark 55 and the height index line 56 to maintain the visibility of the boundary guide 44. Both.

  As shown in FIGS. 4 and 5, when the boundary guide 44 is superimposed on the overhead view image, either the first guide line 47 or the second guide line 54 is selected by the input unit 5. The The user visually recognizes the positional relationship between the side 44b of the boundary guide 44 and the obstacle. Then, the input means 5 inputs which of the vehicle body side end 46 or the door mirror side end 45 of the host vehicle is passed. Thus, the user can select a guide line suitable for guidance from the first guide line 47 and the second guide line 54. The signal processing unit 22 superimposes either the first guide line 47 or the second guide line 54 on the overhead view video based on the input result from the input unit 5. As a result, it is possible to guide collision prevention according to the height of the obstacle.

  4 and 5, the case where the sensor 4 detects an obstacle on the left side of the host vehicle 40 has been described. However, when the sensor 4 detects an obstacle on the right side of the host vehicle 40, the boundary guide 44. Is superimposed on the right side of the vehicle. That is, the boundary guide 44 is superimposed on the side of the host vehicle 40 on the side where the sensor 4 detects the obstacle.

As described above, according to the present invention, the presence of an obstacle according to the height from the road surface can be intuitively shown to the user to prevent contact between the host vehicle and the obstacle.

  The present invention is useful in preventing a collision between a three-dimensional object and the host vehicle even when the three-dimensional object such as a vehicle is displayed in a planar shape collapsed in an overhead view image using a road surface projection model.

DESCRIPTION OF SYMBOLS 1 Imaging means 2 Image processing unit 3 Display means 4 Sensor 5 Input means

Claims (6)

Imaging means for imaging the surroundings of the vehicle;
A sensor for detecting obstacles on the side of the vehicle,
An image processing unit that creates an overhead video based on the captured video acquired from the imaging means and superimposes a graphic on the overhead video based on information acquired from the sensor;
Display means for displaying a superimposed video on which graphics are superimposed by the image processing unit,
When the sensor detects a side obstacle, the image processing unit superimposes a boundary guide having a height from a road surface to a door mirror on the side where the obstacle is detected. Display device.   A guide line indicating that the image processing unit extends along the vehicle length direction through either the vehicle body side end or the door mirror side end of the host vehicle and is movable until it contacts the host vehicle side end of the boundary guide. The display device according to claim 1, further superimposing on the overhead view video.   The boundary guide is formed in a trapezoidal shape, the side of the boundary guide on the own vehicle side indicates the position of the obstacle detected by the sensor, and the other side of the boundary guide indicates the height to the door mirror. The display device according to claim 2, wherein the display device is characterized.   4. A guide mark located further on the host vehicle side than the end of the boundary guide on the host vehicle side and indicating that the host vehicle can move until the guide line comes into contact with the boundary guide. The display device described in 1.   The image processing unit superimposes the guide mark on the front side when the host vehicle is moving forward, and superimposes the guide mark on the rear side when the host vehicle is moving backward. The display device described.   6. The display device according to claim 2, further comprising input means for inputting whether the guide mark passes through a vehicle body side end or a door mirror side end of the host vehicle.