WO2014109030A1

WO2014109030A1 - Virtual image-displaying device, control method, program and memory medium

Info

Publication number: WO2014109030A1
Application number: PCT/JP2013/050297
Authority: WO
Inventors: 憲彦福島
Original assignee: パイオニア株式会社
Priority date: 2013-01-10
Filing date: 2013-01-10
Publication date: 2014-07-17

Abstract

In the present invention, a control unit (55) in a heads-up display (2) recognizes the travelling state of a vehicle (Ve) on the basis of gear information (Ig). In addition, the control unit (55) displays a parking-assisting image (Is) on a combiner (5) in a case in which the travelling state of the vehicle (Ve) is a travelling state in which the vehicle is being parked, the parking-assisting image (Is) illustrating the contour of an obstacle and the positional relationship between the obstacle and the vehicle (Ve) in a rearview image (Ib) obtained from a rear view camera (3). The combiner (5) displays a virtual image of the parking-assisting image (Is) in a location in parallel with a rearview mirror (16F) in the vehicle (Ve) within the front view area of a driver of the vehicle (Ve).

Description

Virtual image display device, control method, program, and storage medium

The present invention relates to a technique for displaying an image that supports driving during parking.

Conventionally, technology that supports driving during parking has been known. For example, in Patent Document 1, when a sensor that detects an obstacle behind the vehicle detects the obstacle, a technique for adding a telop or the like that notifies the presence of the obstacle to the video of the camera that captures the rear of the vehicle. Is disclosed. Patent Document 2 discloses a technique for processing image data behind a car imaged by a stereo camera, generating an image in which a wall of a building, a human outer shape, and the like are modeled into a three-dimensional shape, and displaying the image on a display device. Is disclosed.

JP 2009-169991 A JP 2002-104117 A

In

Patent Documents

1 and 2, the presence of an obstacle behind is displayed on the display. However, at the time of driving when parking, the driver needs to pay close attention to the rearview mirror (rear mirror) for confirming the rear. On the other hand, since the rearview mirror and the display are separated from each other, it is difficult for the driver to visually recognize them at the same time, and the line of sight needs to be frequently switched.

The present invention has been made in order to solve the above-described problems, and has as its main object to provide a virtual image display device capable of suitably supporting driving when parking.

The invention described in the claims is a virtual image display device mounted on a vehicle, wherein a first acquisition unit that acquires a traveling state of the vehicle and an image behind the vehicle are acquired from an imaging unit installed in the vehicle. Second acquisition means for displaying, a display means for displaying a virtual image superimposed on a front landscape of the vehicle, and an obstacle in the video acquired by the second acquisition means in accordance with the running state acquired by the first acquisition means Display control means for causing the display means to display information indicating the outer shape of the object and the positional relationship between the obstacle and the vehicle, and the display means within the front view area of the driver of the vehicle, The virtual image is displayed at a position aligned with the rear-view mirror of the vehicle.

The invention described in claim is a control method executed by a virtual image display device that is mounted on a vehicle and has a display unit that displays a virtual image superimposed on a front landscape of the vehicle, and acquires a traveling state of the vehicle According to the first acquisition step, the second acquisition step of acquiring the video behind the vehicle from the imaging means installed in the vehicle, and the second acquisition step according to the traveling state acquired in the first acquisition step. A display control step for causing the display means to display information indicating the outer shape of the obstacle in the video acquired by the above and the positional relationship between the obstacle and the vehicle, and the display control step includes driving the vehicle. In the front view area of the person, the virtual image is displayed on the display means at a position aligned with the rear-view mirror of the vehicle.

The invention described in the claims is a program executed by a computer that controls a virtual image display device that is mounted on a vehicle and has a display unit that displays a virtual image superimposed on a front landscape of the vehicle, First acquisition means for acquiring a state; second acquisition means for acquiring an image behind the vehicle from an imaging means installed in the vehicle; display means for displaying a virtual image superimposed on a front landscape of the vehicle; In accordance with the traveling state acquired by the first acquisition means, information indicating the outer shape of the obstacle in the video acquired by the second acquisition means and the positional relationship between the obstacle and the vehicle is displayed on the display means. The computer functions as display control means for causing the display control means to display the virtual image at a position aligned with an interior rearview mirror of the vehicle in a front view area of the driver of the vehicle. Characterized in that to be displayed on the shown means.

1 shows a schematic configuration of a display system according to an embodiment. 1 shows a schematic configuration of a navigation device. 1 shows a schematic configuration of a head-up display. The schematic structure of a light source unit is shown. It is the figure which showed the space used as a blind spot of a driver | operator. It is a flowchart which shows the display process of a parking assistance image. The example of a display of a parking assistance image is shown. The field of view of the driver who is displaying the parking assistance image is shown.

In one preferred embodiment of the present invention, the virtual image display device is mounted on a vehicle, and includes a first acquisition unit that acquires a traveling state of the vehicle, and an imaging unit installed on the vehicle. Second acquisition means for acquiring video, display means for displaying a virtual image superimposed on the front landscape of the vehicle, and video acquired by the second acquisition means in accordance with the running state acquired by the first acquisition means Display control means for causing the display means to display information indicating the outer shape of the obstacle inside and the positional relationship between the obstacle and the vehicle, wherein the display means is a front view area of the driver of the vehicle. The virtual image is displayed at a position in line with the rear-view mirror of the vehicle.

The virtual image display device includes first acquisition means, second acquisition means, display means, and display control means. The first acquisition means acquires the traveling state of the vehicle. A 2nd acquisition means acquires the image | video of a vehicle back from the imaging means installed in the vehicle. The display means displays a virtual image superimposed on the front scenery of the vehicle. The display control means displays information indicating the outer shape of the obstacle in the video acquired by the second acquisition means and the positional relationship between the obstacle and the vehicle according to the traveling state acquired by the first acquisition means. Display. Here, the display means displays the virtual image at a position aligned with the vehicle rear-view mirror in the front view area of the driver of the vehicle. According to this aspect, the virtual image display device can simultaneously accommodate the indoor rearview mirror and the virtual image in the driver's field of view. Therefore, it is possible to allow the driver to recognize the outer shape of the obstacle and the positional relationship between the vehicle and the obstacle preferably while suppressing an increase in the driver's line-of-sight switching frequency.

In one aspect of the virtual image display device, the display unit includes a combiner that reflects light constituting the virtual image and visually recognizes the driver, and the combiner is arranged in front of the driver side by side with the indoor rear-view mirror. Is done. According to this aspect, the virtual image display device can suitably display a virtual image at a position aligned with the vehicle interior rear-view mirror in the front view area of the driver of the vehicle.

In another aspect of the virtual image display device, the virtual image display device further includes means for attaching to a sun visor equipped in front of the ceiling of the vehicle. According to this aspect, the virtual image display device can suitably display a virtual image at a position aligned with the vehicle interior rear-view mirror in the front view area of the driver of the vehicle.

In another aspect of the virtual image display device, when the display control unit recognizes a line defining a parking position from the video acquired by the second acquisition unit, information on the position of the line is displayed on the display unit. Let According to this aspect, the virtual image display device can make the driver appropriately recognize the position to be parked.

In another aspect of the virtual image display device, the display control unit causes the display unit to display information on a parking space with a line defining the parking position as a boundary. According to this aspect, the virtual image display device can cause the driver to appropriately recognize the target range for parking.

In another aspect of the virtual image display device, the display control unit causes the display unit to display information indicating a position away from the vehicle position toward the rear of the vehicle at predetermined intervals. According to this aspect, the virtual image display device can make the driver appropriately grasp the distance interval behind the vehicle.

In another preferred embodiment of the present invention, there is provided a control method executed by a virtual image display device that is mounted on a vehicle and has display means for displaying a virtual image superimposed on a front landscape of the vehicle, wherein the traveling state of the vehicle is determined. According to the first acquisition step of acquiring, the second acquisition step of acquiring the video behind the vehicle from the imaging means installed in the vehicle, and the second acquisition according to the traveling state acquired in the first acquisition step A display control step of causing the display means to display information indicating the outer shape of the obstacle in the video acquired by the step and the positional relationship between the obstacle and the vehicle, and the display control step includes: In the front view area of the driver, the virtual image is displayed on the display means at a position aligned with the rear-view mirror of the vehicle. By executing this control method, the virtual image display device can simultaneously bring the indoor rear-view mirror and the virtual image into the driver's field of view. Therefore, it is possible to allow the driver to recognize the outer shape of the obstacle and the positional relationship between the vehicle and the obstacle preferably while suppressing an increase in the driver's line-of-sight switching frequency.

In another preferred embodiment of the present invention, there is provided a program that is executed by a computer that controls a virtual image display device that is mounted on a vehicle and has a display unit that displays a virtual image superimposed on a front landscape of the vehicle. First acquisition means for acquiring a running state; second acquisition means for acquiring an image behind the vehicle from an imaging means installed in the vehicle; and display means for displaying a virtual image superimposed on a front landscape of the vehicle; In accordance with the traveling state acquired by the first acquisition means, information indicating the outer shape of the obstacle in the video acquired by the second acquisition means and the positional relationship between the obstacle and the vehicle is displayed on the display means. The computer functions as display control means for displaying, and the display control means displays the virtual image at a position aligned with the vehicle interior rearview mirror in the front view area of the driver of the vehicle. To be displayed on the serial display means. The computer can load and execute the indoor rear-view mirror and the virtual image displayed by the virtual image display device at the same time in the driver's field of view by installing and executing this program. Therefore, it is possible to allow the driver to recognize the outer shape of the obstacle and the positional relationship between the vehicle and the obstacle preferably while suppressing an increase in the driver's line-of-sight switching frequency. Preferably, the program is stored in a storage medium.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Hereinafter, the “driving operation” refers to an operation in which the vehicle is moved backward for parking or temporarily moved forward for turning back.

[System configuration]
FIG. 1 shows a configuration example of a display system 100 according to the present embodiment. As shown in FIG. 1, the display system 100 is mounted on a vehicle Ve and includes a navigation device 1, a head-up display 2, and a rear camera 3 that captures the rear. In FIG. 1, the vehicle Ve is moving backward in order to stop at a predetermined parking position determined by the car stopper 15A. Instead of the configuration shown in FIG. 1, the head-up display 2 may incorporate a function corresponding to the navigation device 1.

The navigation device 1 has a function of performing route guidance from the departure point to the destination. The navigation device 1 can be a mobile phone such as a stationary navigation device installed in the vehicle Ve, a PND (Portable Navigation Device), or a smartphone.

The head-up display 2 generates an image (also referred to as “guidance image”) that displays map information including the current position, route guidance information, travel speed, and other guidance information for assisting driving, and the guidance image is displayed on the driver. This is a device that allows a user to visually recognize a virtual image from the eye position (eye point). Various information used for navigation processing such as the position of the vehicle Ve, the traveling speed of the vehicle Ve, map information, and facility data is supplied to the head-up display 2 from the navigation device 1. Thus, the head-up display 2 functions as a “virtual image display device” in the present invention.

The rear camera 3 is installed in the rear part of the vehicle Ve, and transmits an image (also referred to as “rear image Ib”) obtained by photographing the scenery behind the vehicle Ve to the head-up display 2. In the example of FIG. 1, the rear camera 3 includes a car stopper 15 A that determines the parking position of the vehicle Ve at the time of parking and a utility pole 15 B that exists in the vicinity of the car stopper 15 A. In addition, the rear camera 3 may be a stereo camera or a camera composed of a plurality of cameras so that shooting can be performed simultaneously from a plurality of viewpoints. The rear camera 3 is an example of the “imaging unit” in the present invention.

If the navigation device 1 is a mobile phone such as a smartphone, the navigation device 1 may be held by a cradle or the like. In this case, the navigation device 1 may exchange information with the head-up display 2 via a cradle or the like. Further, the rear camera 3 may be electrically connected to the navigation device 1 and transmit the rear image Ib to the navigation device 1. In this case, the navigation device 1 transfers the received rear image Ib to the head-up display 2.

[Configuration of navigation device]
FIG. 2 shows the configuration of the navigation device 1. As shown in FIG. 2, the navigation device 1 includes a self-supporting positioning device 10, a GPS receiver 18, a system controller 20, a disk drive 31, a data storage unit 36, a communication interface 37, a communication device 38, an interface 39, and a display unit 40. A voice output unit 50 and an input device 60.

The self-supporting positioning device 10 includes an acceleration sensor 11, an angular velocity sensor 12, and a distance sensor 13. The acceleration sensor 11 is made of, for example, a piezoelectric element, detects the acceleration of the vehicle Ve, and outputs acceleration data. The angular velocity sensor 12 is composed of, for example, a vibrating gyroscope, detects the angular velocity of the vehicle Ve when the direction of the vehicle Ve is changed, and outputs angular velocity data and relative azimuth data. The distance sensor 13 measures a vehicle speed pulse composed of a pulse signal generated with the rotation of the wheel of the vehicle Ve.

The GPS receiver 18 receives radio waves 19 carrying downlink data including positioning data from a plurality of GPS satellites. The positioning data is used to detect the absolute position (also referred to as “current position”) of the vehicle Ve from latitude and longitude information.

The system controller 20 includes an interface 21, a CPU (Central Processing Unit) 22, a ROM (Read Only Memory) 23, and a RAM (Random Access Memory) 24, and controls the entire navigation device 1.

The interface 21 performs an interface operation with the acceleration sensor 11, the angular velocity sensor 12, the distance sensor 13, and the GPS receiver 18. From these, vehicle speed pulses, acceleration data, relative azimuth data, angular velocity data, GPS positioning data, absolute azimuth data, and the like are input to the system controller 20. The CPU 22 controls the entire system controller 20. The ROM 23 includes a nonvolatile memory (not shown) in which a control program for controlling the system controller 20 is stored. The RAM 24 stores various data such as route data preset by the user via the input device 60 so as to be readable, and provides a working area to the CPU 22.

A system controller 20, a disk drive 31 such as a CD-ROM drive or a DVD-ROM drive, a data storage unit 36, a communication interface 37, a display unit 40, an audio output unit 50 and an input device 60 are mutually connected via a bus line 30. It is connected to the.

The disk drive 31 reads and outputs content data such as music data and video data from a disk 33 such as a CD or DVD under the control of the system controller 20. The disk drive 31 may be either a CD-ROM drive or a DVD-ROM drive, or may be a CD and DVD compatible drive.

The data storage unit 36 is configured by, for example, an HDD or the like, and stores various data used for navigation processing such as map data. The map data includes road data represented by links corresponding to roads and nodes corresponding to road connecting portions (intersections), facility information about each facility, and the like.

The communication device 38 is composed of, for example, an FM tuner, a beacon receiver, a mobile phone, a dedicated communication card, and the like, and via a communication interface 37, traffic jams distributed from a VICS (registered trademark, Vehicle Information Communication System) center Receive road traffic information such as traffic information and other information. Further, the communication device 38 performs navigation processing such as information on the guide route determined by the system controller 20, information on the current position acquired from the GPS receiver 18, information such as vehicle speed pulses acquired from the autonomous positioning device 10, and map data. Various information to be used is transmitted to the head-up display 2.

The display unit 40 displays various display data on a display device such as a display under the control of the system controller 20. Specifically, the system controller 20 reads map data from the data storage unit 36. The display unit 40 displays the map data read from the data storage unit 36 by the system controller 20 on the display screen. The display unit 40 includes a graphic controller 41 that controls the entire display unit 40 based on control data sent from the CPU 22 via the bus line 30 and a memory such as a VRAM (Video RAM), and can display image information that can be displayed immediately. A buffer memory 42 that temporarily stores, a display control unit 43 that controls display of a display 44 such as a liquid crystal or a CRT (Cathode Ray Tube) based on image data output from the graphic controller 41, and a display 44 are provided. . The display 44 functions as an image display unit, and includes, for example, a liquid crystal display device having a diagonal size of about 5 to 10 inches and is mounted near the front panel in the vehicle.

The audio output unit 50 performs D / A (Digital to Analog) conversion of audio digital data sent from the CD-ROM drive 31, DVD-ROM 32, RAM 24, or the like via the bus line 30 under the control of the system controller 20. A D / A converter 51 to perform, an amplifier (AMP) 52 that amplifies the audio analog signal output from the D / A converter 51, and a speaker 53 that converts the amplified audio analog signal into sound and outputs the sound into the vehicle. It is prepared for.

The input device 60 includes keys, switches, buttons, a remote controller, a voice input device, and the like for inputting various commands and data. The input device 60 is disposed around the front panel and the display 44 of the main body of the in-vehicle electronic system mounted in the vehicle. When the display 44 is a touch panel system, the touch panel provided on the display screen of the display 44 also functions as the input device 60.

[Configuration of head-up display]
FIG. 3 schematically shows a state in which the head-up display 2 is installed in the vehicle interior. FIG. 3 is a side view of the driver's seat of the vehicle, and the driver is sitting on the seat in the passenger compartment. Above the driver's head is a roof (sheet metal) 27 that forms the outer frame of the vehicle, and below that is a ceiling 28 that is the interior of the passenger compartment. In front of the driver, there is a windshield 25 and a sun visor 29 of the vehicle. In FIG. 1, the sun visor 29 is fixed in a state of being opposed to the ceiling 28 (also referred to as “storage state”).

The head-up display 2 is installed diagonally upward in front of the driver. The head-up display 2 mainly includes a main body portion 4 in which the light source unit 6 is accommodated, a combiner 5, a reflection portion 7, a support portion 8, and a clip portion 9.

The light source unit 6 accommodated in the main body unit 4 emits light constituting an intermediate image indicating information to be visually recognized by the observer toward the reflection unit 7. A specific configuration of the light source unit 6 will be described later with reference to FIG.

The combiner 5 projects the light constituting the intermediate image generated by the reflecting unit 7 and partially reflects the projected light to the driver's eye point “Pe”, thereby giving the virtual image “Iv” to the observer. An optical member to be visually recognized. Note that the tip of the arrow in the virtual image Iv indicates the upward direction. The support 8 extends from the main body 4 toward the windshield 25 and supports the combiner 5. The support portion 8 is, for example, a pair of arms, and one end is attached to each side surface of the main body portion 4, and the combiner 5 is sandwiched by the other end. As will be described later, the support unit 8 holds the combiner 5 so that the combiner 5 is positioned side by side with the room mirror (indoor rearview mirror).

The reflection unit 7 is a reflective optical member that generates an intermediate image, and functions as an exit pupil expander (EPE). For example, the reflection unit 7 is formed with a microlens array in which a plurality of microlenses are arranged on a surface on which light from the light source unit 6 is incident, and a reflection surface is formed on a surface opposite to the microlens array. Is done.

The clip portion 9 is attached to the upper surface of the main body portion 4 facing the ceiling 28, and the main body portion 4 is attached to the sun visor 29 with the sun visor 29 sandwiched therebetween. The clip portion 9 is a plate-like elastic body that is curved in a substantially J shape, and has an elastic force that biases the sun visor 29 in the direction of nipping.

[Configuration of light source unit]
FIG. 4 is a diagram schematically showing the configuration of the light source unit 6. As shown in FIG. 4, the light source unit 6 includes a light source 54, a control unit 55, and a communication unit 56.

The light source 54 includes, for example, red, blue, and green laser light sources, and emits light (also referred to as “display light”) that constitutes a display image to be irradiated to the combiner 5 based on the control of the control unit 55. . The light source 54 and the combiner 5 function as “display means” in the present invention.

The communication unit 56 receives various information used for the navigation processing from the navigation device 1 based on the control of the control unit 55. For example, the communication unit 56 receives guide route information from the navigation device 1 when a destination is set. Further, the communication unit 56 receives position information (also referred to as “gear information Ig”) of the shift lever (including a select lever) from the control unit (ECU) of the vehicle Ve based on the control of the control unit 55. To do. The communication unit 56 receives the rear image Ib from the rear camera 3 at predetermined or indefinite intervals based on the control of the control unit 55.

The control unit 55 includes a CPU, a ROM that stores a control program executed by the CPU, data, and the like, and a RAM that sequentially reads and writes various data as a work memory when the CPU operates. Control.

For example, when it is determined from the gear information Ig received by the communication unit 56 that the vehicle Ve moves backward, the control unit 55 determines that the vehicle Ve is in a traveling state in which parking is performed. In this case, the control unit 55 generates an image (also referred to as “parking support image Is”) for assisting driving during parking based on the rear image Ib acquired from the rear camera 3, and The display light is emitted to the light source 54. The parking assistance image Is is an image showing the outer shape of an obstacle such as the car stopper 15A and the utility pole 15B and the positional relationship between the obstacle and the vehicle Ve. Here, the obstacle is assumed to have a three-dimensional shape that exists in or near the space to be parked. The parking assistance image Is will be described in detail in the section [Parking assistance image display processing]. The control unit 55 functions as a “first acquisition unit”, “second acquisition unit”, “display control unit”, and a computer that executes a program in the present invention.

[Shooting range of rear camera]
The rear camera 3 captures a range that is a blind spot in the driver's field of view through a rearview mirror for confirming the rear of a room mirror, a door mirror, and the like. This will be described with reference to FIG.

Fig. 5 (A) shows a top view of the vehicle Ve, and Fig. 5 (B) shows a side view of the vehicle Ve. In FIG. 5A, lines L1 and L2 indicate boundaries of a range that can be visually recognized by the driver through the left door mirror 16L. Similarly, the lines L3 and L4 in FIG. 5A indicate the boundaries of the range that the driver can visually recognize through the room mirror 16F, and the lines L5 and L6 can be visually recognized by the driver through the right door mirror 16R. Indicates the boundaries of a range. In addition, lines L7 and L8 in FIG. 5B indicate boundaries of a range that can be visually recognized by the driver through the room mirror 16F.

As shown in FIGS. 5 (A) and 5 (B), the driver cannot visually recognize the interior of the space 17 behind the vehicle Ve indicated by the oblique lines even through the door mirrors 16R and 16L and the room mirror 16F. . That is, the space 17 becomes a driver's blind spot.

The rear camera 3 includes a range including a space 17 that is a blind spot of the driver in the shooting range. Thereby, as will be described later, the control unit 55 can suitably generate the parking assistance image Is displaying the obstacles in the blind spot of the driver based on the rear image Ib received from the rear camera 3.

[Parking support image display processing]
Next, the display process of the parking assistance image Is performed by the control unit 55 will be described. Schematically, the control unit 55 recognizes white lines (also referred to as “parking white lines”) that define the range of an obstacle or a parking space for the vehicle Ve from the rear image Ib in a running state in which parking is performed. The parking assistance image Is illustrated in FIG. Thereby, the control part 55 supports the driving | operation at the time of parking suitably.

(1) Processing Flow FIG. 6 is a flowchart showing the procedure of the parking assist image Is display process executed by the control unit 55. The control unit 55 repeatedly executes the process of the flowchart shown in FIG.

First, the control unit 55 determines whether or not the traveling state of the vehicle Ve is a traveling state in which parking is performed (step S101). For example, the control unit 55 determines whether or not the position of the shift lever is the reverse position (that is, “R”) based on the gear information Ig transmitted from the vehicle Ve. If the position of the shift lever is not the reverse position, the control unit 55 determines that the vehicle is not in a traveling state in which parking is performed (step S101; No). Subsequently, in step S101, the vehicle Ve travels based on the gear information Ig. It is determined whether the state is a traveling state in which parking is performed.

Next, when it is determined that the traveling state of the vehicle Ve is a traveling state in which parking is performed (step S101; Yes), the control unit 55 acquires the rear image Ib from the rear camera 3 (step S102). In the example of FIG. 1, the car stopper 15 A and the utility pole 15 B are displayed in the rear image Ib acquired by the control unit 55.

Thereafter, the controller 55 recognizes the parking white line and the three-dimensional obstacle from the acquired rear image Ib (step S103). In the example of FIG. 1, the control unit 55 recognizes the car stopper 15A and the utility pole 15B as a three-dimensional obstacle.

Here, a supplementary explanation will be given of step S103. In step S103, the control unit 55 stores in advance information that characterizes the white line for parking, such as the color and shape of the white line for parking and the display position in the assumed rear image Ib, and refers to the information. Then, the parking white line is recognized from the rear image Ib by a known image recognition technique. Similarly, the control unit 55 stores in advance information that is a feature of an object that can be an obstacle in advance, such as a stopper or a utility pole, and refers to this information to perform the rear image Ib using a known image recognition technique. 3D obstacles are recognized. In addition, when the rear camera 3 is a stereo camera or the like that can be simultaneously photographed from a plurality of viewpoints, the control unit 55 can display the plurality of rear images Ib photographed from the different viewpoints using the same technique as that described in Patent Document 2, for example. The presence of a three-dimensional obstacle may be recognized based on the three-dimensional measurement of the obstacle.

Next, the control part 55 produces | generates the parking assistance image Is by step S104 and step S105. Specifically, first, the control unit 55 generates an image in which each figure indicating the parking white line and the obstacle recognized in step S103 and the parking space defined by the parking white line is arranged (step S104). . In this case, for example, the control unit 55 arranges these figures so as to be in the same positional relationship as the parking white line and the obstacle display positions in the rear image Ib acquired in step S102. In consideration of drawing the figure of the own vehicle drawn in step S105 described later, the control unit 55 sets the display position of each figure drawn in step S104 in the rear image Ib by the drawing area of the figure of the own vehicle. You may shift from each display position.

After execution of step S104, the control unit 55 adds a figure indicating the vehicle Ve and an equidistant line indicating a position away from the own vehicle position at a predetermined interval backward from the vehicle position to the image generated in step S104. The parking support image Is is generated (step S105). The equidistant lines will be described in detail in the section “(2) Display example”.

And the control part 55 displays the parking assistance image Is on the combiner 5 by making the light source 54 emit the light which comprises the produced | generated parking assistance image Is (step S106). As a result, the driver can appropriately grasp the presence of an obstacle that cannot be visually recognized by each rearview mirror. In addition, since the parking assistance image Is is displayed on the combiner 5, even when the driver temporarily moves the vehicle Ve forward for switching, the driver keeps his / her line of sight when moving backward and moving forward. There is no need to move a lot. Further, as described in the section “(3) Display position”, the control unit 55 displays the parking assist image Is at a position aligned with the room mirror 16F, thereby switching the driver's line of sight during driving during parking. An increase in frequency can be suitably suppressed. Thus, the control part 55 can support the driving | operation at the time of parking suitably by displaying the parking assistance image Is.

Next, the control unit 55 determines whether or not parking is completed (step S107). And when it is judged that parking was completed (step S107; Yes), the control part 55 complete | finishes the process of a flowchart. For example, based on the gear information Ig, the control unit 55 determines that parking is completed when the reverse lever position is changed to the parking lever position. In another example, the control unit 55 determines that the parking is completed when the control unit 55 determines that a predetermined time or more has elapsed since switching from the reverse lever position to another lever position based on the gear information Ig. To do. In this case, the above-mentioned predetermined time is set in consideration of the time to move forward temporarily by switching back at the time of parking. On the other hand, when the control unit 55 determines that the parking is not completed (step S107; No), the process of steps S102 to S106 is performed again, and the parking support image Is displayed on the combiner 5 as the vehicle Ve moves. Update the image.

(2) Display Example Next, a display example of the parking assistance image Is shown in FIG. 7 will be described with reference to the flowchart of FIG. 6 as appropriate.

FIG. 7 shows the parking assistance image Is displayed on the combiner 5 in the state of FIG. As shown in FIG. 7, the parking assistance image Is includes white

line display portions

90L and 90R that indicate white lines for parking,

obstacle display portions

91, 92L, and 92R that indicate obstacles, and a vehicle display that indicates the rear portion of the vehicle Ve. Part 93, equidistant line 94 indicating the distance from the vehicle position, and parking space display part 95.

First, after recognizing the white line for parking and the obstacle from the rear image Ib in step S103 of FIG. 6, the control unit 55 performs these based on the display position of the white line for parking and the obstacle in the rear image Ib in step S104. An image in which the white

line display units

90L and 90R and the

obstacle display units

91, 92L, and 92R that are simplified figures are arranged is generated. Here, the control unit 55 draws the

obstacle display units

92L and 92R corresponding to the left and right car stoppers 15A recognized from the rear image Ib and the obstacle display unit 91 corresponding to the utility pole 15B. Furthermore, the control part 55 recognizes the part pinched by the white

line display parts

90L and 90R as a parking space, and arrange | positions the parking space display part 95 in the said part. Thereby, the control part 55 can represent suitably the presence of the obstruction which should be careful, and the presence of the white line for parking used as a standard in driving | operation, and a parking space.

Further, in step S105, the control unit 55 arranges a vehicle display unit 93 corresponding to the rear part of the vehicle Ve in the image generated in step S104. In this way, the control unit 55 can appropriately grasp the positional relationship between the vehicle position and the obstacle by virtually drawing the rear part of the vehicle Ve that is not actually displayed in the rear image Ib. it can.

Further, in step S105, the control unit 55 arranges equidistant lines 94 indicating equal distance intervals in the backward direction from the vehicle position in the image generated in step S104. In this case, for example, the control unit 55 determines each drawing position of the equidistance line 94 indicating equal intervals from the own vehicle position experimentally or based on the installation position, installation angle, etc. of the rear camera 3 and stores it in the memory. Store in advance. Thus, by drawing the equidistant line 94, the control unit 55 can cause the driver to appropriately grasp the distance from the own vehicle position such as an obstacle. In addition, the control part 55 may further attach | subject the display which shows the numerical value of the distance from the own vehicle position with respect to each equidistant line 94. FIG.

Moreover, the control part 55 displays each display part of the parking assistance image Is mentioned above with a different color for every classification. In FIG. 7, for convenience, each display unit of the parking assistance image Is is hatched with a different pattern for each type. Thereby, the control part 55 can make a driver | operator recognize the classification of each display part easily.

(3) Display Position FIG. 8 shows a driver's front view area while the parking assistance image Is is being displayed. As illustrated in FIG. 6, the control unit 55 causes the light source 54 to emit display light of the generated parking assistance image Is, thereby displaying the parking assistance image Is via the combiner 5.

As shown in FIG. 8, the combiner 5 is disposed at a position aligned with the rearview mirror 16 F in the driver's front view area. Therefore, in this case, the combiner 5 causes the driver to visually recognize the parking assistance image Is at a position aligned with the room mirror 16F.

Here, a supplementary explanation of the position of the combiner 5 will be given. As described with reference to FIG. 3, the head-up display 2 has the clip portion 9 and is attached to the sun visor 29 in a state where the clip portion 9 sandwiches the sun visor 29. Thereby, the head-up display 2 can hold the combiner 5 at substantially the same height as the room mirror 16F.

And as shown in FIG. 8, the driver | operator can confirm the room mirror 16F and the parking assistance image Is simultaneously by making a driver | operator visually recognize the parking assistance image Is in the position aligned with the room mirror 16F. . Therefore, the head-up spray 2 can prevent an increase in the line-of-sight switching frequency by visually recognizing the parking assistance image Is when the driver performs driving during parking while appropriately viewing the rearview mirror.

As described above, the control unit 55 of the head-up display 2 according to the present embodiment recognizes the traveling state of the vehicle based on the gear information Ig. Then, when the traveling state of the vehicle Ve is a traveling state where parking is performed, the control unit 55 determines the outer shape of the obstacle in the rear image Ib acquired from the rear camera 3 and the positional relationship between the obstacle and the vehicle Ve. Is displayed on the combiner 5. Here, the combiner 5 displays a virtual image of the parking assist image Is in a position aligned with the vehicle rearview mirror 16F in the front view area of the driver of the vehicle Ve. According to this aspect, the head-up display 2 can simultaneously accommodate the room mirror 16F and the virtual image of the parking assistance image Is in the driver's field of view. Therefore, it is possible to allow the driver to recognize the outer shape of the obstacle and the positional relationship between the vehicle Ve and the obstacle preferably while suppressing an increase in the driver's line-of-sight switching frequency.

<Modification>
Hereinafter, modified examples suitable for the embodiments will be described. The following modifications may be applied to the embodiments in any combination.

(Modification 1)
In FIG. 7, the control unit 55 displays a simplified figure of an obstacle or the like on the parking assistance image Is. However, the display method of the parking assistance image Is to which the present invention is applicable is not limited to this.

Alternatively, the control unit 55 may cause the driver to recognize the presence of an obstacle or the like by generating and displaying a parking assistance image Is by superimposing a predetermined display on the rear image Ib. In this case, the control unit 55 draws a white line for parking in the rear image Ib recognized in step S103 of FIG. 6 and a border line for emphasizing the outer shape of the display area indicating the obstacle so as to overlap the rear image Ib. Then, the white line for parking and the display area of the obstacle are highlighted. Furthermore, the control unit 55 causes the equidistant line from the vehicle position to be superimposed on the above-described rear image Ib, and causes the rear image Ib to be displayed on the combiner 5 as the parking assistance image Is. Also by this, the control part 55 can make a driver | operator grasp | ascertain suitably the presence of the obstruction etc. which become a driver | operator's blind spot, and the positional relationship of the vehicle Ve and an obstruction.

In the embodiment, the control unit 55 simplifies the obstacle and displays it on the parking support image Is. Instead, when the obstacle is measured three-dimensionally, the three-dimensional measurement is performed. The obstacle may be displayed in three dimensions by computer graphics based on the information acquired by the above.

(Modification 2)
The head-up display 2 shown in FIG. 3 has a combiner 5 and makes the driver visually recognize the virtual image Iv based on the light emitted from the light source unit 6 reflected by the combiner 5. However, the configuration to which the present invention is applicable is not limited to this. Instead of this, the head-up display 2 may not have the combiner 5 and may allow the driver to visually recognize the virtual image Iv based on the emitted light of the light source unit 6 reflected by the windshield 25.

Even in this case, the light source 54 emits display light to a predetermined range of the windshield 25 so that the parking assistance image Is is displayed at a position aligned with the room mirror 16F when observed from the driver. Also by this, the head-up display 2 can preferably reduce the switching frequency of the driver's line of sight during driving during parking. In this case, the windshield 25 and the light source 54 are examples of the “display unit” in the present invention.

(Modification 3)
In the description of FIG. 6, the control unit 55 has executed all the processes in the flowchart of FIG. 6. However, the aspect to which the present invention is applicable is not limited to this. Instead of this, the system controller 20 of the navigation device 1 may execute processes other than step S106 instead of the control unit 55.

In this case, the system controller 20 performs the determination in step S101 and step S107 by receiving the gear information Ig from the vehicle Ve. In addition, after receiving the rear image Ib from the rear camera 3 in step S102, the system controller 20 generates a parking assistance image Is in steps S103 to S105, similarly to the control unit 55. Then, the system controller 20 transmits the generated parking assistance image Is to the head-up display 2 and causes the head-up display 2 to display the parking assistance image Is. Also by this, the display system can generate | occur | produce the parking assistance image Is which assists the driving | operation at the time of parking suitably, and can make a driver | operator visually recognize it. In this case, the system controller 20 is an example of the “computer” in the present invention.

(Modification 4)
In step S101 and step S107 of FIG. 6, the control unit 55 determines whether or not the traveling state of the vehicle Ve is a traveling state in which parking is performed based on the gear information Ig transmitted from the vehicle Ve. However, the determination method to which the present invention is applicable is not limited to this.

Alternatively, the control unit 55 may determine whether or not the vehicle is in a traveling state where parking is performed by detecting the pressing of a predetermined button provided in the navigation device 1 or the main body unit 4 or the like. In this case, when the above-described button is pressed, the control unit 55 repeatedly executes the processing of step S102 to step S106 until the button is pressed again, and causes the combiner 5 to display the generated parking assistance image Is. .

DESCRIPTION OF SYMBOLS 1 Navigation apparatus 2 Head-up display 4 Main-body part 5 Combiner 6 Light source unit 8 Support part 25 Windshield 100 Display system

Claims

A virtual image display device mounted on a vehicle,
First acquisition means for acquiring a traveling state of the vehicle;
Second acquisition means for acquiring an image behind the vehicle from an imaging means installed in the vehicle;
Display means for displaying a virtual image superimposed on the front landscape of the vehicle;
In accordance with the traveling state acquired by the first acquisition means, information indicating the outer shape of the obstacle in the video acquired by the second acquisition means and the positional relationship between the obstacle and the vehicle is displayed on the display means. Display control means for displaying;
With
The virtual image display device, wherein the display means displays the virtual image at a position aligned with an interior rearview mirror of the vehicle in a front view area of the driver of the vehicle.
The display means includes a combiner that reflects light constituting a virtual image and makes the driver visually recognize the light,
The virtual image display device according to claim 1, wherein the combiner is arranged in front of the driver along with the indoor rearview mirror.
The virtual image display device according to claim 2, further comprising means for attaching to a sun visor equipped in front of the ceiling of the vehicle.
The display control means, when recognizing a line defining a parking position from the video acquired by the second acquisition means, displays information on the position of the line on the display means. The virtual image display device according to any one of the above.
5. The virtual image display device according to claim 4, wherein the display control means causes the display means to display information on a parking space with a line defining the parking position as a boundary.
5. The display unit according to claim 1, wherein the display unit displays information indicating a position away from the position of the vehicle toward the rear of the vehicle at predetermined intervals on the display unit. The virtual image display device described.
A control method executed by a virtual image display device mounted on a vehicle and having display means for displaying a virtual image superimposed on a front landscape of the vehicle,
A first acquisition step of acquiring a running state of the vehicle;
A second acquisition step of acquiring an image behind the vehicle from imaging means installed in the vehicle;
According to the traveling state acquired in the first acquisition step, information indicating the outer shape of the obstacle in the video acquired in the second acquisition step and the positional relationship between the obstacle and the vehicle is displayed on the display means. A display control process for displaying,
The display control step displays the virtual image on the display means at a position aligned with the rear-view mirror of the vehicle in a front view area of the driver of the vehicle.
A program executed by a computer that controls a virtual image display device that is mounted on a vehicle and has a display unit that displays a virtual image superimposed on a front landscape of the vehicle,
First acquisition means for acquiring a traveling state of the vehicle;
Second acquisition means for acquiring an image behind the vehicle from an imaging means installed in the vehicle;
Display means for displaying a virtual image superimposed on the front landscape of the vehicle;
In accordance with the traveling state acquired by the first acquisition means, information indicating the outer shape of the obstacle in the video acquired by the second acquisition means and the positional relationship between the obstacle and the vehicle is displayed on the display means. Causing the computer to function as display control means for displaying;
The display control means causes the display means to display the virtual image at a position aligned with an interior rearview mirror of the vehicle in a front view area of the driver of the vehicle.
A storage medium storing the program according to claim 8.