JP7400356B2 - Superimposed image display device and computer program - Google Patents

Description

The present invention relates to a superimposed image display device and a computer program that support driving of a vehicle.

2. Description of the Related Art Conventionally, various means have been used as information providing means for providing vehicle occupants with various types of information for providing vehicle driving support, such as route guidance and warning of obstacles. For example, it is a display on a liquid crystal display installed in a vehicle, a sound output from a speaker, etc. In recent years, as one of such information providing means, there has been a device that provides information by displaying an image superimposed on the surrounding environment (scenery, real scene) of the passenger. For example, in addition to a head-up display, a window shield display, a method of displaying an image superimposed on a captured image of the surroundings of the vehicle displayed on a liquid crystal display, etc. are applicable.

Here, when guiding the vehicle occupant by displaying an image superimposed on the surrounding environment, it is important to make the occupant understand the position where the image is superimposed. For example, when providing guidance to an obstacle or a guidance junction, by superimposing an image at the position of the obstacle or guidance junction, the passenger who visually recognizes the image can see the obstacle or guidance junction at the position where the image is superimposed. It becomes possible to recognize that there is a point. Therefore, for example, Japanese Patent Application Laid-Open No. 2018-136698 discloses that when a head-up display is used to generate a virtual image to be superimposed on the scenery around the vehicle, the shadow caused by sunlight on the virtual image is also estimated from the position of the sun, and the shadow A technique is disclosed that generates a virtual image in consideration of the above, and makes the sense of distance to the virtual image more clear.

JP 2018-136698 (pages 7-10, Figure 4)

Here, when adding shadow information to an image superimposed on the surrounding environment (hereinafter referred to as a superimposed image), as in the technology described in Patent Document 1, the position of the light source that causes the shadow is important. becomes. The superimposed image basically faces the front of the vehicle occupants in order to provide information to the occupants, that is, the cross section of the superimposed image faces upward, so setting a light source vertically above the superimposed image will cause shadows. There is a problem that the shadow is not displayed or becomes extremely thin, making it difficult to visually recognize the shadow. In addition, the superimposed image is not necessarily superimposed near the ground, but may be superimposed above the ground away from the ground. In that case, if the light source is set lower than the horizontal direction of the superimposed image, shadows may appear. There is also a problem with not being displayed on the ground.

The present invention has been made to solve the above-mentioned conventional problems, and when displaying a guide object superimposed on the scenery around the vehicle, it is possible to reliably display a shadow corresponding to the guide object. An object of the present invention is to provide a superimposed image display device and a computer program that enable a passenger to clearly recognize the position where a guide object is superimposed based on the displayed shadow.

In order to achieve the above object, a superimposed image display device according to the present invention is a superimposed image display device that is mounted on a vehicle and allows a guide object that guides information to a passenger of the vehicle to be visually recognized by superimposing it on the scenery around the vehicle. a guide object arranging means for arranging the guide object to be displayed in a three-dimensional space corresponding to a periphery of an area on which the guide object is superimposed; and a light source arranging means for arranging a light source in the three-dimensional space; object display means for displaying the guide object based on the guide object arranged in the three-dimensional space; and object display means for displaying the guide object based on the guide object arranged in the three-dimensional space and the light source. and a shadow display means for displaying a shadow, the light source arrangement means disposing the light source at a position excluding the vertical direction above the guide object and the horizontal direction below the guide object, and the shadow display means When a plurality of guide objects are displayed by the object display means, the shadow is displayed only for the guide object located closest to the ground .
Note that the term "landscape" includes not only the scenery that is actually seen from the vehicle (actual scene), but also images of scenery, images that reproduce scenery, and the like.
Further, the term "shadow" includes at least one of a shadow caused by the guide object on another object (for example, the ground), and a shadow (ie, shadow) caused by the guide object itself.

Further, the computer program according to the present invention is a program that supports driving of a vehicle. Specifically, a superimposed image display device that is mounted on a vehicle and that superimposes and makes visible a guide object that guides information to the occupants of the vehicle on the scenery around the vehicle is provided around the area where the guide object is superimposed. a guide object arranging means for arranging the guide object to be displayed in a three-dimensional space; a light source arranging means for arranging a light source in the three-dimensional space; and a shadow display means that displays a shadow corresponding to the guide object based on the guide object arranged in the three-dimensional space and the light source. , wherein the light source arrangement means arranges the light source at a position excluding vertically above the guide object and below the guide object in the horizontal direction, and the shadow display means disposes the light source at a position excluding the position above the guide object in the horizontal direction, and the shadow display means When a plurality of guide objects are displayed, the shadow is displayed only for the guide object located closest to the ground .

According to the superimposed image display device and computer program according to the present invention having the above-mentioned configuration, the shadow of the guide object is displayed for the occupant of the vehicle to be visible in a manner superimposed on the scenery, and the shadow is caused to appear on the guide object. Regarding the position of the light source, the light source is placed at a position excluding vertically above the guide object and below the guide object in the horizontal direction. As a result, it becomes possible to reliably display the shadow corresponding to the guide object, and it becomes possible to make the occupant clearly recognize the position where the guide object overlaps based on the displayed shadow. Furthermore, it is possible to prevent the shadow of the guide object located above from being displayed on the guide object located below, making it difficult to visually recognize the guide object below. Furthermore, by reducing the number of displayed shadows, it is possible to prevent a decrease in visibility.

FIG. 1 is a block diagram showing a navigation device according to a first embodiment. 3 is a flowchart of a driving support processing program according to the first embodiment. FIG. 3 is a diagram showing an example of a travel guide screen displayed on a liquid crystal display. 12 is a flowchart of a sub-processing program of guide object display position determination processing. FIG. 3 is a diagram showing the displacement of a guide object to be displayed as the vehicle travels. FIG. 3 is a diagram showing an example of the arrangement (horizontal direction) of guide objects. FIG. 7 is a diagram showing an example of arrangement of guide objects (in the vertical direction). FIG. 3 is a diagram showing an example of a visually recognized image of a guide object arranged in a three-dimensional space. 3 is a flowchart of a sub-processing program of light source arrangement processing. FIG. 3 is a diagram showing a prohibited area (in the vertical direction) for placing a light source. FIG. 3 is a diagram showing a prohibited area (horizontal direction) for placing a light source. It is a figure showing an example of arrangement (horizontal direction) of a light source. FIG. 2 is a schematic configuration diagram of a superimposed image display device according to a second embodiment. FIG. 7 is a diagram illustrating a display example of a guide object in a superimposed image display device according to a second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a first embodiment and a second embodiment in which a superimposed image display device according to the present invention is embodied in a navigation device will be described in detail with reference to the drawings.

[First embodiment]
First, a schematic configuration of a navigation device 1 according to a first embodiment will be described using FIG. 1. FIG. 1 is a block diagram showing a navigation device 1 according to the first embodiment.

As shown in FIG. 1, the navigation device 1 according to the first embodiment includes a current position detection unit 11 that detects the current position of a vehicle in which the navigation device 1 is mounted, and a data recording unit 12 that records various data. , a navigation ECU 13 that performs various calculation processes based on input information, an operation unit 14 that accepts operations from the user, and a liquid crystal display 15 that displays to the user an image of the actual scene taken in the forward direction of travel. Communication is performed between a speaker 16 that outputs voice guidance regarding route guidance, a DVD drive 17 that reads a DVD as a storage medium, and an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center. It has a communication module 18. Further, the navigation device 1 is connected to a front camera 19 and various sensors installed in the vehicle in which the navigation device 1 is mounted via an in-vehicle network such as CAN.

Below, each component included in the navigation device 1 will be explained in order.
The current position detection unit 11 includes a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current vehicle position, direction, vehicle speed, current time, etc. . Here, in particular, the vehicle speed sensor 22 is a sensor for detecting the travel distance and vehicle speed of the vehicle, generates pulses according to the rotation of the drive wheels of the vehicle, and outputs the pulse signal to the navigation ECU 13. Then, the navigation ECU 13 calculates the rotational speed and travel distance of the drive wheels by counting the generated pulses. Note that it is not necessary for the navigation device 1 to include all of the above four types of sensors, and the navigation device 1 may be configured to include only one or more types of sensors.

The data recording unit 12 also includes a hard disk (not shown) as an external storage device and recording medium, and a driver for reading map information DB 31 and predetermined programs recorded on the hard disk and writing predetermined data on the hard disk. A recording head (not shown) is provided. Note that the data recording section 12 may be configured with a flash memory, a memory card, or an optical disk such as a CD or DVD instead of a hard disk. Alternatively, the map information DB 31 may be stored in an external server and acquired by the navigation device 1 through communication.

Here, the map information DB 31 includes, for example, link data 32 related to roads (links), node data 33 related to node points, branch point data 34 related to branch points, point data related to points such as facilities, and a map display for displaying a map. It is a storage means in which data, search data for searching for routes, search data for searching for points, etc. are stored.

In addition, the link data 32 includes the width, slope, cant, bank, road surface condition, number of lanes on the road, places where the number of lanes decreases, and width of each link that constitutes the road. Data representing narrowing points, railroad crossings, etc., regarding corners, data representing radius of curvature, intersections, T-junctions, entrances and exits of corners, etc., regarding road attributes, data representing downhill roads, uphill roads, etc. Regarding the type, data representing expressways and general roads (national roads, prefectural roads, narrow streets, etc.) are recorded.

In addition, the node data 33 includes coordinates (positions) of actual road branch points (including intersections, T-junctions, etc.) and node points set at predetermined distances on each road according to the radius of curvature, etc. Node attributes that indicate whether a node corresponds to an intersection, etc., a connected link number list that is a list of link numbers of links that connect to the node, and adjacency that is a list of node numbers of nodes that are adjacent to the node via links. A list of node numbers, data regarding the height (altitude) of each node point, etc. are recorded.

In addition, the branch point data 34 includes the intersection name of the branch point, corresponding node information that specifies the node forming the branch point, connecting link information that specifies the link connected to the branch point, and information on the link connected to the branch point. The corresponding direction name, information specifying the shape of the branch point, etc. are stored. Further, structures that can serve as landmarks when providing left/right turn guidance at a junction are also stored.

On the other hand, the navigation ECU (electronic control unit) 13 is an electronic control unit that controls the entire navigation device 1, and includes a CPU 41 as an arithmetic unit and a control device, and a working memory when the CPU 41 performs various arithmetic processes. In addition to the RAM 42 which stores route data etc. when a route is searched, the ROM 43 which stores a driving support processing program (see FIG. 2) to be described later in addition to the control program, which is read from the ROM 43 It includes an internal storage device such as a flash memory 44 that stores programs. Note that the navigation ECU 13 has various means as processing algorithms. For example, the guide object arrangement means arranges the guide object to be displayed in a three-dimensional space corresponding to the periphery of the area where the guide object is to be superimposed. The light source arrangement means arranges the light sources in a three-dimensional space. The object display means displays the guide object based on the guide object arranged in the three-dimensional space. The shadow display means displays a shadow corresponding to the guide object based on the guide object and the light source arranged in the three-dimensional space.

The operation unit 14 is operated when inputting a departure point as a travel start point and a destination as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 13 performs control to execute various corresponding operations based on switch signals output by pressing each switch. Note that the operation unit 14 may have a touch panel provided on the front surface of the liquid crystal display 15. Alternatively, a configuration including a microphone and a voice recognition device may be used.

The liquid crystal display 15 also displays map images including roads, traffic information, operation guidance, operation menus, key guidance, a guidance route from the departure point to the destination, guidance information along the guidance route, news, weather forecasts, etc. The time, email, TV programs, etc. are displayed. In particular, in the first embodiment, during normal driving, the liquid crystal display 15 displays the captured image taken by the front camera 19, that is, the scenery (actual scene image) around the vehicle at the current moment (especially in front of the vehicle), and Guide objects are displayed superimposed on the scenery according to the situation.

Here, the guide objects displayed superimposed on the scenery include information regarding the vehicle and various types of information used to support the driving of the passenger. For example, warnings to the occupants about objects (other vehicles, pedestrians, guide signs), guidance information based on the guide route or guide route set in the navigation device 1 (arrows indicating right/left turn directions, guide branches) icons indicating point markers, distance to guidance junctions, etc.), warnings displayed on the road (beware of rear-end collisions, speed limits, etc.), lane markings in which the vehicle is traveling, current vehicle speed, shift position, remaining energy, advertisements These include images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, and the screen of a connected smartphone. In the first embodiment described below, the guide object is guide information for providing guidance at a guide branch point located ahead in the direction of travel of the vehicle, and more specifically, information for providing guidance at a guide branch point along the guide route. Use an arrow, etc. to indicate the direction.

Furthermore, the speaker 16 outputs audio guidance for guiding travel along the guide route and traffic information based on instructions from the navigation ECU 13.

Further, the DVD drive 17 is a drive that can read data recorded on recording media such as DVDs and CDs. Then, based on the read data, music and video are played back, and the map information DB 31 is updated. Incidentally, instead of the DVD drive 17, a card slot for reading and writing a memory card may be provided.

Further, the communication module 18 is a communication device for receiving traffic information including various information such as congestion information, regulation information, traffic accident information, etc. transmitted from a traffic information center, such as a VICS center or a probe center, For example, this applies to mobile phones and DCMs.

Further, the front camera 19 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed with its optical axis facing forward in the direction of travel of the vehicle. The captured image captured by the front camera 19 is displayed on the liquid crystal display 15 as a scenery (actual scene image) around the vehicle (particularly in front of the vehicle), as described above.

Next, a driving support processing program executed by the navigation ECU 13 in the navigation device 1 having the above configuration will be described based on FIG. 2. FIG. 2 is a flowchart of the driving support processing program according to the first embodiment. Here, the driving support processing program is executed after the vehicle's ACC power supply (accessory power supply) is turned on, and the driving support processing program is executed by making the guide object superimposed on the scenery around the vehicle displayed on the liquid crystal display 15 visible. This is a program that provides support. Note that the programs shown in flowcharts in FIGS. 2, 4, and 9 below are stored in the RAM 42 and ROM 43 included in the navigation device 1, and are executed by the CPU 41.

In the following description, an example will be described in which driving guidance of a vehicle along a guidance route set by the navigation device 1 is performed as driving support for a vehicle using a guide object. In addition, the guidance object to be displayed is the guidance information for providing guidance at the guidance junction located ahead in the direction of travel of the vehicle, and in particular the arrow indicating the exit direction of the guidance junction and the name of the intersection, and the landmark of the guidance junction. The following describes an example of a process in which an icon shown in the figure is displayed as a guide object. However, the navigation device 1 can also use guide objects to provide guidance and information other than the above-mentioned driving support. Further, the guide object to be displayed can be information other than the above-mentioned arrows and icons. For example, warnings for objects (other vehicles, pedestrians, guide signs) that are subject to warnings for occupants as guide objects, warnings displayed on the road surface (beware of rear-end collisions, speed limits, etc.), lane markings in which the vehicle is traveling. It is also possible to display current vehicle speed, shift position, remaining energy, advertising images, facility information, guide signs, map images, traffic information, news, weather forecast, time, connected smartphone screen, etc.

First, in step (hereinafter abbreviated as S) 1 of the driving support processing program, the CPU 41 specifies the current position of the vehicle based on the detection result of the current position detection section 11 and map information. Note that when identifying the current location of the vehicle, map matching processing is also performed to match the current location of the vehicle with map information. Thereafter, the guidance route set by the navigation device 1 is read out, and the distance from the specified current position of the vehicle to the next guidance branch point along the guidance route is calculated. Note that the guidance branch point is a branch point (intersection) to which the navigation device 1 provides guidance such as right/left turn instructions when the navigation device 1 provides driving guidance according to the guidance route set in the navigation device 1.

Next, in S2, the CPU 41 determines whether the distance to the next guidance branch point calculated in S1 is less than or equal to a predetermined guidance start distance. Note that the guidance start distance is determined by the type of road on which the vehicle is traveling; for example, an expressway is 1 km, and a general road is 300 m, which is shorter than the expressway.

If it is determined that the distance to the next guidance branch point calculated in S1 is less than or equal to the guidance start distance (S2: YES), the process moves to S3. On the other hand, if it is determined that the distance to the next guidance branch point calculated in S1 is not less than the guidance start distance (S2: NO), the process returns to S1.

In S3, the CPU 41 acquires the coordinates of a point to be guided by the guide object, that is, a point on which the guide object should be superimposed (arranged) (hereinafter referred to as a guide point). In the first embodiment, the guide object is used to provide right/left turn guidance at the guide branch point, so the guide point corresponds to the guide branch point, and therefore the coordinates of the guide point correspond to the coordinates X of the guide branch point. In addition, especially when providing right/left turn guidance at a guidance junction using landmarks, guidance objects are also superimposed on the landmarks, so landmarks (facilities, structures, etc.) around the guidance junction are also included in the guidance point. Correspondingly, the coordinates of the guide point also include the coordinates Y of the landmarks around the guide branch point. Note that the coordinates X of the guide branch point and the coordinates Y of the landmarks around the guide branch point are specified from the map information possessed by the navigation device 1, and image recognition processing is performed on the image captured by the front camera 19. It may also be specified by

Next, in S4, the CPU 41 performs a guide object display position determination process (FIG. 4), which will be described later. The guide object display position determination process determines the shape of the guide object to be displayed on the liquid crystal display 15 and the position (range) in which the guide object is displayed based on the current position of the vehicle and the coordinates of the guide point acquired in S3. This is a process to specifically determine. Note that the shape of the guide object and the position at which the guide object is displayed determined in step S4 serve as conditions for superimposing the guide object on a guide point in the scenery and making it visible to the passenger, for example. However, depending on the current position of the vehicle and the type of the guide object, the guide object may be superimposed and visually recognized in addition to the guide points in the scenery.

Next, in S5, the CPU 41 transmits a control signal to the liquid crystal display 15, and displays the image of the guide object having the shape determined in S4 on the liquid crystal display 15 at the position (range) determined in S4. Display against. Note that the liquid crystal display 15 displays an image taken in advance by the front camera 19, that is, a scene (actual scene image) around the vehicle (especially in front of the vehicle) at the current moment. As a result, it becomes possible for the vehicle occupant to visually recognize the guide object superimposed on the scenery.

FIG. 3 is a diagram showing an example of the travel guide screen 51 displayed on the liquid crystal display 15 in S5. As shown in FIG. 3, the liquid crystal display 15 displays a current scene 52 in front of the vehicle captured by the front camera 19. Then, images 53 to 55 of guide objects are displayed superimposed on the scenery 52 in front of the vehicle.

Here, in the first embodiment, there are a plurality of types of guide objects used for guidance, and one or more types of guide objects selected depending on the content of the guide and the current situation are displayed. Furthermore, multiple types of guide objects may be displayed at the same time. For example, in the example shown in FIG. 3, there is an image 53 of a first guide object consisting of three arrows indicating the exit direction of the guide junction, and a second guide object consisting of arrows indicating the exit direction of the guide junction and the name of the intersection. An image 54 of , and an image 55 of a third guide object consisting of an icon indicating a landmark of a guide branch point are displayed simultaneously. Note that for the images 53 to 55 of each guide object, the timing to start displaying and the timing to end displaying may be set to different timings. Further, the images 53 to 55 of the guide objects are displayed superimposed on guide points (that is, guide junctions and facilities serving as landmarks of guide junctions) within the scenery 52. Furthermore, particularly for the image 53 of the first guide object, an image 56 of a shadow created on the road surface below by a virtually set light source is also displayed.

Furthermore, it is desirable that the shadow image 56 of the first guide object be displayed in a hue or brightness that corresponds to the brightness of the area on which the shadow is to be superimposed. Specifically, first, the brightness of the area on which the shadow is to be superimposed is obtained by performing image processing on the image captured by the front camera 19 or by using an illuminance sensor. For example, when the area on which the shadow is to be superimposed is bright, such as during the daytime on a clear day, the shadow is displayed with a hue and brightness that allows it to be more clearly distinguished from its surroundings. On the other hand, if the area where the shadow is to be superimposed is in a dark situation, such as when it is cloudy or at night, the shadow is displayed with a hue and brightness that makes it distinguishable from the surrounding area. Furthermore, shadows may not be displayed in situations where shadows cannot occur, such as at night. Note that details of the shadow of the guide object will be described later.

Therefore, when the vehicle occupant visually confirms the travel guidance screen 51, he or she can accurately grasp the position of the guidance branch point to be turned left or right and the exit direction at the guidance intersection. In particular, by displaying the shadow image 56 of the arrow of the first guide object, it becomes easier to understand the distance to the first guide object, that is, the distance to the guide branch point.

In addition, as guidance objects, in addition to the above-mentioned arrows indicating the exit direction of the vehicle guidance junction, the arrow indicating the intersection name, and the icon of the guidance junction landmark, an image indicating the distance to the guidance intersection is also displayed superimposed on the scenery. You can do it like this. The position where the image indicating the distance to the guidance intersection is superimposed is, for example, immediately in front of the vehicle, and is displayed while the distance from the vehicle to the guidance junction is within a predetermined distance section (for example, from 300 m to 150 m).

Thereafter, in S6, the CPU 41 calculates the distance from the current position of the vehicle to the next guidance branch point along the guidance route in the same manner as in S1 above, and determines whether the calculated distance to the next guidance junction point corresponds to the predetermined guidance branch point. Determine whether the distance is less than or equal to the end distance. Note that the guidance end distance is determined by the type of road on which the vehicle is traveling; for example, it is 30 m for an expressway, and 10 m for a general road, which is shorter than the expressway.

If it is determined that the distance to the next guidance branch point is less than or equal to the guidance end distance (S6: YES), the process moves to S7. On the other hand, if it is determined that the distance to the next guidance branch point is not less than the guidance end distance (S6: NO), the process returns to S4 and the guidance object continues to be displayed.

Subsequently, in S7, the CPU 41 transmits a control signal to the liquid crystal display 15, and changes the transmittance of the image of the guide object (including the shadow image) displayed on the liquid crystal display 15 to the guide branch point. Increase gradually depending on distance. Specifically, the transmittance is increased so that the transmittance reaches 100% at the timing when the vehicle reaches the guidance branch point. Note that the transmittance of the guide object image is set to, for example, 20% instead of 0% at the time of display start so that the vehicle occupant can grasp the scenery hidden behind the guide object image. Therefore, the image of the guidance object is displayed with a transparency of 20% until the distance from the vehicle to the guidance junction becomes less than the guidance end distance, and then increases from 20% to 30% to 40% as it approaches the guidance junction. →... The transmittance increases step by step, and finally reaches 100% when the vehicle reaches the guidance branch point.

Thereafter, in S8, the CPU 41 determines whether the vehicle has passed through a guidance branch point. For example, the determination is made based on the current position of the vehicle detected by the current position detection unit 11 and map information.

If it is determined that the vehicle has passed the guidance branch point (S8: YES), a control signal is sent to the liquid crystal display 15, and the guidance object displayed on the liquid crystal display 15 is hidden. (S9). Note that the captured image captured by the front camera 19, that is, the current scenery (actual scene image) around the vehicle (particularly in front of the vehicle) is continuously displayed. However, the landscape image may also be hidden and the display screen of the liquid crystal display 15 may be switched to display a map image or the like.

On the other hand, if it is determined that the vehicle has not passed the guidance branch point (S8: NO), the process returns to S4 and the guidance object continues to be displayed.

Next, the sub-processing of the guide object display position determining process executed in S4 will be described based on FIG. 4. FIG. 4 is a flowchart of a sub-processing program for guide object display position determination processing.

First, in S11, the CPU 41 determines the type of guide object to be displayed. Here, in the first embodiment, there are a plurality of types of guide objects used for guidance, and one or more types of guide objects selected depending on the content of the guide and the current situation are displayed. In the following explanation, an example will be described in which a guide object to be displayed is determined from the following three types of guide objects (1) to (3). However, it is also possible to include guidance objects (for example, distance to a guidance branch point) other than the following (1) to (3) in the display target.
(1) "First guide object"... Three arrows indicating the exit direction of the guide branch point (2) "Second guide object"... Arrow indicating the exit direction of the guide branch point and the intersection name ( 3) "Third guide object"...Icon indicating a landmark of a guide branch point

Then, in S11, the CPU 41 selects a guidance object to be displayed based on the distance from the current position of the vehicle to the guidance junction and whether or not there is a facility or structure that serves as a guidance landmark at the guidance junction. Determine. Note that information specifying whether or not there is a facility or structure that serves as a guide mark at a guidance junction is included in the junction data 34 of the map information. An example of the determination method when the vehicle travels on a general road will be described below with reference to FIG. 5.
(A) When there is no other branch point within 150m in front of the guide branch point If the distance to the guide branch point is less than 300 m and not less than 150 m, only the "second guide object" is displayed. set to target.
If the distance to the guidance branch point is 150 m or less, the "first guidance object" and "second guidance object" are displayed. Furthermore, if there is a facility or structure that serves as a guidance landmark at a guidance branch point, the "third guidance object" is also displayed.
(B) When there is another branch point within 150m in front of the guide branch point If the distance to the guide branch point is 300 m or less and before reaching the other branch point, the "second guide object" Only the following items will be displayed.
After reaching another branch point, the "first guide object" and "second guide object" are displayed. Furthermore, if there is a facility or structure that serves as a guidance landmark at a guidance branch point, the "third guidance object" is also displayed.

Next, in S12, the CPU 41 sets "the position at which the guide object is superimposed on the scenery" for each guide object determined to be displayed in S11. For example, for the "first guide object" and "second guide object", the "position at which the guide object is superimposed on the scenery" is guided to the guide branch point (including its surroundings), which is the guide point. This is a position where at least a part of the object is visually recognized as being superimposed on the object. However, the "first guide object" may be set at a position where the relative position with respect to the vehicle is fixed for the occupant when the distance from the vehicle to the guide branch point is long. Specifically, a position a predetermined distance ahead (for example, 10 m ahead) of the current position of the vehicle may be set as the superimposed position of the guide object.
On the other hand, regarding the "third guide object", the "position at which the guide object is superimposed on the scenery" is set relative to the landmarks around the guide branch point (including the surrounding area) of the guide object. The position is such that at least a portion of the images overlaps and is visually recognized.

Subsequently, in S13, the CPU 41 uses the current position of the vehicle and the map information stored in the map information DB 31 to generate a three-dimensional space corresponding to the area around the current position of the vehicle (particularly the area ahead in the direction of travel of the vehicle). Note that map information can also be acquired from an external server. Basically, only roads (including intersections) specified based on map information are modeled in the three-dimensional space, but in addition to roads, buildings, road signs, etc. may also be modeled. For example, by performing point cloud matching on the captured image captured by the front camera 19, it is possible to generate a three-dimensional space in which buildings, road signs, etc. around the road are also modeled. Alternatively, the three-dimensional space may be simply a blank three-dimensional space containing only the ground without modeling roads. Alternatively, the three-dimensional space may be stored in the map information DB 31 in advance as three-dimensional map information, and in step S13, the three-dimensional map information around the corresponding vehicle position may be read from the map information DB 31.

Furthermore, in S13, the CPU 41 also specifies the current position and orientation of the host vehicle in the generated three-dimensional space based on the parameters detected by the current position detection unit 11. In particular, the position of the front camera 19 installed on the vehicle is taken as the current position of the own vehicle, and the optical axis direction of the front camera 19 is taken as the orientation of the own vehicle. The position of the front camera 19 also corresponds to the position of the vehicle occupant, and the optical axis direction of the front camera 19 also corresponds to the line of sight direction of the vehicle occupant. Additionally, the current position and orientation of the vehicle in the three-dimensional space are updated as appropriate as the vehicle moves.

Next, in S14, the CPU 41 generates a guide object of the type to be displayed on the liquid crystal display 15 determined in S11.
For example, if the guide object determined to be displayed in step S11 includes a "first guide object" indicating the exit direction of the guide branch point, the guide object indicates the exit direction of the guide branch point located ahead in the direction of travel of the vehicle. Arrows 61 to 63 are generated. In particular, three arrows 61 to 63 having an isosceles triangular shape are generated as shown in FIGS. 6 to 8. Note that if the guide route is a route that turns right at a guide intersection located ahead in the direction of travel, the arrow will indicate the right direction. If the guide route is a route that turns left at a guide intersection ahead in the direction of travel, the arrow will indicate the left direction.
Further, if the guidance object determined to be displayed in S11 includes a "second guidance object" indicating the exit direction of the guidance branch point and the intersection name, an arrow 64 with a character string written inside is generated. do. The character string written inside is the intersection name of the guidance branch point. Note that instead of the intersection name, the direction name of the exit direction or the road name of the exit road may be used.
In addition, if the guide object determined to be displayed in S11 includes a "third guide object" indicating a guide branch point mark, a mark icon 65 with a mark indicating the genre of the mark displayed inside is displayed. generate. The "third guide object" is an icon that indicates a landmark of a guidance branch point in front of the vehicle in the direction of travel, and in particular, a landmark icon with a mark indicating the genre of the landmark displayed inside, as shown in Figures 6 to 8. Generate 65.

Further, it is assumed that the guide object generated in S14 is a two-dimensional polygon, and basically has no thickness. However, a thick three-dimensional polygon may also be used. Further, the shape of the generated guide object can be changed as appropriate, and may be a shape other than an arrow or an icon.

Subsequently, in S15, the CPU 41 arranges the guide object generated in S14 above in the three-dimensional space generated in S13. The position at which the guide object is placed in the three-dimensional space is determined based on the "position at which the guide object is superimposed on the scenery" set in S12.

Below, three types of guide objects are set as the types of guide objects to be displayed on the liquid crystal display 15 in S11: "first guide object", "second guide object", and "third guide object". The process of S15 will be explained by taking as an example the case where the above-mentioned information is displayed. Here, FIG. 6 is a diagram showing an example of the arrangement of each guide object in a case where a guide route exiting a guide branch point 60 of a general road diagonally to the left is set.
First, regarding the "first guide object", three arrows 61 to 63 are arranged at equal intervals along the traveling direction of the vehicle 66, as shown in FIG. In particular, of the three arrows 61 to 63, the left end of the arrow 63 closest to the traveling direction (or the right end in the case of a rightward arrow) coincides with the coordinates X of the guidance branch point (horizontal direction match). Although the coordinates X of the guidance branch point are specified from the map information possessed by the navigation device 1, they may also be specified by performing image recognition processing on an image captured by the front camera 19. In that case, the point where the branch point connects with the road in the exit direction is taken as the coordinate X of the guidance branch point. Further, the arrangement interval of the three arrows 61 to 63 can be changed as appropriate, but is set to, for example, an interval of 5 m. Further, the three arrows 61 to 63 are not arranged parallel to the direction of travel, but are arranged so that the closer the arrows are to the guide branch point, the closer to the exit direction the arrows are. However, they may be arranged parallel to the traveling direction. On the other hand, in the vertical direction, the lower ends of the arrows 61 to 63 are placed at a predetermined distance (for example, 1 m) from the road surface, as shown in FIG.
On the other hand, regarding the "second guide object", one arrow 64 is arranged along the exit direction of the guide branch point 60 of the vehicle 66, as shown in FIG. Therefore, as shown in FIG. 6, when the exit direction of the guidance junction 60 is not perpendicular to the direction of entry into the guidance junction 60, the arrow 64 does not face in front of the traveling direction of the vehicle 66. It will be placed at an angle. Further, since a character string corresponding to the intersection name of the guidance branch point 60 is drawn inside the arrow 64, the character string of the intersection name is also arranged along the exit direction. Further, the tip of the arrow 64 is arranged at a position where it coincides with the coordinates X of the guide branch point (coinciding in the horizontal direction). Further, the length of the arrow 64 is determined in consideration of the length of the intersection name drawn inside the arrow and the angle of the exit direction. On the other hand, in the vertical direction, as shown in FIG. 7, the lower end of the arrow 64 is placed at a predetermined distance (for example, 5 m) from the road surface.
Regarding the "third guide object", as shown in FIG. 6, a landmark icon 65 is placed above the facility or structure that serves as a guidance landmark for the guidance branch point 60. That is, unlike the "first guide object" and the "second guide object," the "third guide object" is arranged not above the road surface but above the facility or structure. In particular, the center of the landmark icon 65 is placed at a position where it matches (in the horizontal direction) the coordinate Y of the landmark associated with the guide branch point. Note that the coordinate Y of the landmark is specified from map information possessed by the navigation device 1, but may be specified by performing image recognition processing on an image captured by the front camera 19. On the other hand, in the vertical direction, as shown in FIG. Place it directly above the structure.

Subsequently, in S16, the CPU 41 performs a light source arrangement process (FIG. 9), which will be described later. The light source arrangement process is a process of arranging a virtual light source 67 imitating the sun in the three-dimensional space generated in S13. Note that the light source 67 is placed at a position in the three-dimensional space that avoids a predetermined placement prohibited area.

Next, in S17, the CPU 41 first displays an image (hereinafter referred to as a visible image) of the three-dimensional space in which the guide object is arranged, which is viewed from the position of the vehicle (corresponding to the viewpoint) identified in S13 in the traveling direction of the vehicle. ) to obtain. For example, FIG. 8 is a diagram showing a visible image 69 obtained when the vehicle and each guide object are arranged in the manner shown in FIG. 6. In particular, since the position of the vehicle is the position of the front camera 19, the acquired visible image 69 is an image that can be visually recognized when each guide object arranged in a three-dimensional space is viewed from the viewpoint of the front camera 19 in the direction of travel of the vehicle. However, it also corresponds to the field of view of vehicle occupants. It also includes the shadow of the guide object caused by the light source 67. Note that the light source 67 is excluded from the visible image 69.

Thereafter, the CPU 41 stores the shape and position of each guide object included in the visible image 69 as the shape and position of the guide object to be displayed on the liquid crystal display 15. Here, the shape of the guide object stored in S17 is the shape of the guide object that can be visually recognized when the guide object placed in the three-dimensional space is viewed from the viewpoint of the vehicle (more precisely, the front camera 19). . Further, the position of the guide object stored in S17 is the position of the guide object that can be visually recognized when the guide object arranged in the three-dimensional space is viewed from the viewpoint of the vehicle (more precisely, the front camera 19).

Subsequently, in S18, the CPU 41 determines whether there are multiple types of each guide object determined to be displayed in S11.

If it is determined in S11 that there are multiple types of guide objects determined to be displayed (S18: YES), the process moves to S20. On the other hand, if it is determined in S11 that each guide object determined to be displayed is of only one type (S18: NO), the process moves to S19.

In S19, the CPU 41 calculates the shadow of the guide object caused by the light source 67 based on the position of the virtual light source 67 placed in the three-dimensional space in S16 and the position and shape of the guide object also placed in the three-dimensional space. Memorize the shape and position of. Basically, shadows are formed on the road surface, but if there are structures on the road surface, shadows may be formed on those structures. Note that, as will be described later, the light source 67 is basically placed farther away from the current position of the vehicle than the position corresponding to the vertically upper position of the guide object. Therefore, the position of the shadow is formed slightly closer to the vehicle than vertically below the guide object. Further, the guide objects whose shadows are to be stored in S19 are basically limited to guide objects located above the road surface. That is, in the first embodiment, the display target is limited to either the "first guide object" or the "second guide object". After that, the process moves to S21.

On the other hand, in S20, the CPU 41 compares the positions of the plurality of types of guide objects determined to be displayed in S11 in the three-dimensional space in S15, and selects the type of guide object that is placed closest to the ground. Identify guidance objects. For example, if "first guide object", "second guide object", and "third guide object" are set to be displayed, the type of guide placed closest to the ground The object becomes the "first guide object." Based on the position of the virtual light source 67 placed in the three-dimensional space in S16 and the position and shape of the guide object also placed in the three-dimensional space, among the shadows of each guide object caused by the light source 67, The shape and position of the shadow of the guide object placed closest to the ground is memorized.

For example, in the visible image 69 shown in FIG. 8, shadows 71 to 73 corresponding to the arrows 61 to 63 that are the first guide objects, shadows 74 corresponding to the arrow 64 that is the second guide object, and shadows 74 that correspond to the arrows 64 that are the second guide objects, and shadows 71 to 73 that correspond to the arrows 61 to 63 that are the first guide objects, A shadow 75 corresponding to the landmark icon 65, which is an object, is generated. Therefore, in S20, the shapes and positions of the shadows 71 to 73 corresponding to the arrows 61 to 63, which are the first guide objects, are stored. Further, the guide objects whose shadows are to be stored in S20 are basically limited to guide objects located above the road surface. That is, in the first embodiment, the shadows are limited to those corresponding to either the "first guide object" or the "second guide object". After that, the process moves to S21.

Thereafter, in S21, the CPU 41 determines the range in which the guide object is displayed on the liquid crystal display 15, based on the shape of the guide object and the position of the guide object stored in S17. The display range of the guide object is a range in which the guide object having the shape stored in S17 is displayed at the same position as the guide object placed in the three-dimensional space. Further, the CPU 41 similarly determines the range in which the shadow of the guide object is displayed on the liquid crystal display 15 based on the shape and position of the guide object's shadow stored in S19 or S20. After that, the process moves to S5, where a control signal is sent to the liquid crystal display 15 as described above, and the image of the guide object and shadow having the shape stored in the above S18 is determined on the liquid crystal display 15 in the above S21. Display in the specified display range.

As a result, the travel guide screen 51 displayed on the liquid crystal display 15 as the vehicle travels becomes a screen as shown in FIG. 3. In particular, by displaying the image 56 showing the shadow of the first guide object located closest to the ground, it becomes easier to understand the distance to the first guide object, that is, the distance to the guide branch point. On the other hand, since the shadows of the second and third guide objects located above the first guide object are not displayed, the shadows of the guide objects located above are displayed on the guide objects located below. Therefore, it is possible to prevent a phenomenon in which it becomes difficult to visually recognize the guide object below. Furthermore, by reducing the number of displayed shadows, it is possible to prevent a decrease in visibility.

Next, the sub-processing of the light source placement process executed in S16 will be described based on FIG. 9. FIG. 9 is a flowchart of a sub-processing program for light source arrangement processing.

First, in S31, the CPU 41 sets a placement prohibited area in which placement of the light source 67 is prohibited in the three-dimensional space generated in S13. Here, in the vertical direction, the prohibited placement area is set below the horizontal direction of the guide object (arrow 61 in the example shown in FIG. 10) whose shadow is to be displayed, as shown in FIG. As mentioned above, in this embodiment, the shadow is displayed only for the guide object placed closest to the ground, so if the shadow is displayed below the horizontal direction of the guide object placed closest to the ground Set a prohibited area for the area. However, it is also possible to display shadows for a plurality of guide objects, and in that case, a vertical placement prohibited area is set for each of the plurality of guide objects. When a plurality of placement prohibited areas are set, the light source may be set avoiding all placement prohibited areas, or the light source may be set for each guide object.

On the other hand, as shown in FIG. 11, the horizontal direction of the placement prohibited area is set to an area that includes at least the vertical direction above the guide object (arrow 61 in the example shown in FIG. 10) whose shadow is to be displayed. Specifically, an area excluding a rectangular area adjacent to the guide object on the opposite side of the vehicle 66 (an area that does not overlap with the guide object and may be located a predetermined distance from the guide object) is arranged. This is a prohibited area. Note that the rectangular area has a width of 400 m x 200 m in the example shown in FIG. 11, but it can be changed as appropriate depending on the height of the light source 67. As mentioned above, in this embodiment, only the guide object placed closest to the ground is subject to displaying a shadow, so the guide object placed closest to the ground is shown in FIG. Set the prohibited area for placement. However, it is also possible to display shadows for a plurality of guide objects, and in that case, a horizontal placement prohibited area is set for each of the plurality of guide objects. When a plurality of placement prohibited areas are set, the light source may be set avoiding all placement prohibited areas, or the light source may be set for each guide object.

Subsequently, in S32, the CPU 41 calculates the position of the sun at the current time based on the latitude and longitude of the current location of the vehicle and the current date and time. Note that if the current time is after sunset, the process of S32 is unnecessary.

Thereafter, in S33, the CPU 41 determines whether the position of the sun at the current time is outside the placement prohibited area set in S31. However, in S33, it is preferable to use the position where the sun is assumed to be at a certain distance (for example, 1 km) from the ground, rather than the actual position of the sun. In this case, it is desirable to change the distance to the sun by changing the position of the sun without changing the direction of the sun with respect to the guide object. Note that if the current time is after sunset, the position of the sun may be within the placement prohibited area, or the light source may not be placed (that is, no shadow will be displayed).

If it is determined that the position of the sun at the current time is outside the placement prohibited area set in S31 (S33: YES), the process moves to S34. On the other hand, if it is determined that the position of the sun at the current time is within the placement prohibited area set in S31 (S33: NO), the process moves to S35.

In S34, the CPU 41 places the light source 67 at a position corresponding to the position of the sun at the current time in the three-dimensional space. Specifically, the light source 67 is arranged so that the direction of the sun and the direction of the light source 67 with respect to the guide object displaying a shadow are the same direction. The vertical position of the light source is set at a sufficiently high distance from the ground (for example, 1 km) or at an infinite distance (infinite light source). As a result, it becomes possible to display a more natural shadow that corresponds to the position of the sun at the current time. Furthermore, since the position of the sun is outside the prohibited area, the light source 67 is also placed outside the prohibited area. Therefore, the light source 67 is disposed above the guide object in the horizontal direction and further away from the current position of the vehicle 66 than the position corresponding to the guide object in the vertical direction.

On the other hand, in S35, the CPU 41 arranges the light source 67 based on the current position of the vehicle in the three-dimensional space and the position of the guide object that displays the shadow. For example, as shown in FIG. 12, the guide object (arrow 61 in the example shown in FIG. 12) is located 100 m further away from the current position of the vehicle 66 than the position corresponding to the vertical direction upward, and 1 km above the ground. do. However, the light source may be installed at another position as long as it is outside the placement prohibited area set in S31. Here, in S35, the light source 67 is placed outside the placement prohibited area, so the light source 67 is located above the guide object in the horizontal direction and closer to the current position of the vehicle 66 than the position corresponding to the vertical direction above the guide object. It will be placed on the far side.

Note that the position of the light source set in S35 may be changed depending on the type of guide object. For example, when displaying the shadow of a small guide object, the light source may be set at a lower position to make the shadow larger. Furthermore, when displaying the shadow of a guide object on the edge of the road or outside the road, the light source may be set at an angle at which the shadow of the guide object is displayed on the road surface. Further, the number of light sources to be set does not necessarily have to be one, and when displaying the shadows of a plurality of guide objects, a light source may be set for each guide object.

After that, the process returns to S19, and the light generated by the light source 67 is determined based on the position of the virtual light source 67 placed in the three-dimensional space in S34 or S35, and the position and shape of the guide object also placed in the three-dimensional space. The shadow of the guided guide object is displayed. Note that, as described above, the light source 67 is arranged farther from the current position of the vehicle than the position corresponding to the vertically upper position of the guide object. Therefore, the position of the shadow is formed slightly closer to the vehicle than vertically below the guide object.

As explained in detail above, according to the navigation device 1 according to the first embodiment and the computer program executed by the navigation device 1, when there is a guidance target point to be guided in front of the vehicle in the traveling direction, A guide object for guiding the point is placed in a three-dimensional space corresponding to the surrounding area (S15), while a virtual light source for creating a shadow on the guide object is also placed in the three-dimensional space (S16). ), guide objects and shadows are displayed based on the arranged guide objects and light sources (S5). In addition, since the light source is placed at a position excluding the vertically above the guide object and the horizontally below the guide object (S34, S35), it is possible to reliably display the shadow corresponding to the guide object, and the guide object This makes it possible for the occupant to clearly recognize the position where the images overlap.

[Second embodiment]
Next, a superimposed image display device according to a second embodiment will be described based on FIGS. 13 and 14. In the following description, the same reference numerals as those in the structure of the superimposed image display device according to the first embodiment shown in FIGS. It shows.

The general configuration of the superimposed image display device according to the second embodiment is almost the same as the superimposed image display device according to the first embodiment. In addition, various control processes are almost the same as those of the superimposed image display device according to the first embodiment.
However, the superimposed image display device according to the first embodiment displays the captured image captured by the front camera 19 on the liquid crystal display 15 of the navigation device 1, and further displays the guide object on the liquid crystal display 15. , the guide object is displayed superimposed on the scenery around the vehicle, whereas the superimposed image display device according to the second embodiment uses a head-up display system as a means for displaying an image superimposed on the scenery around the vehicle. It's different.

A schematic configuration of a superimposed image display device according to a second embodiment will be described below with reference to FIG. 13. FIG. 13 is a schematic configuration diagram of a superimposed image display device 101 according to the second embodiment.
As shown in FIG. 13, the superimposed image display device 101 basically includes a navigation device 103 mounted on a vehicle 102, and a front display 104 also mounted on the vehicle 102 and connected to the navigation device 103. Note that the front display 104 functions as a head-up display together with the windshield 105 of the vehicle 102, and serves as information providing means for providing various information to the occupants 106 of the vehicle 102.

Here, the front display 104 is a liquid crystal display that is installed inside the dashboard 107 of the vehicle 102 and has a function of displaying an image on an image display surface provided at the front. As the backlight, for example, a CCFL (cold cathode fluorescent lamp) or a white LED is used. Note that as the front display 104, in addition to a liquid crystal display, an organic EL display or a combination of a liquid crystal projector and a screen may be used.

The front display 104 functions as a head-up display together with the windshield 105 of the vehicle 102, and the image output from the front display 104 is reflected on the windshield 105 in front of the driver's seat so that the occupant 106 of the vehicle 102 can see it. It is configured as follows. Note that guide objects are displayed on the front display 104 as necessary. Note that in the second embodiment described below, the guidance object is guidance information for providing guidance at a guidance junction located ahead in the direction of travel of the vehicle, as in the first embodiment, and more specifically, a guidance junction. An arrow or the like indicating the direction of exit.

Further, when the passenger 106 views the image displayed on the front display 104 by reflecting off the windshield 105, the passenger 106 may see the front display not at the position of the windshield 105 but at a distant position beyond the windshield 105. The image displayed on 104 is configured to be visually recognized as a virtual image 110. In addition, the virtual image 110 is displayed superimposed on the surrounding environment (landscape, real scene) in front of the vehicle, and for example, is superimposed on any object located in front of the vehicle (road surface, building, object to be warned, etc.). It is also possible to display the information in the same way.

Here, the position where the virtual image 110 is generated, more specifically the distance L from the occupant 106 to the virtual image 110 (hereinafter referred to as imaging distance), is determined by the position of the front display 104. For example, the imaging distance L is determined by the distance along the optical path (optical path length) from the position where the image is displayed on the front display 104 to the windshield 105. For example, the optical path length is set so that the imaging distance L is 1.5 m.

Further, a front camera 111 is installed above the front bumper of the vehicle, behind the room mirror, etc. The front camera 111 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed with its optical axis facing forward in the direction of travel of the vehicle. Then, by performing image processing on the captured image captured by the front camera 111, the situation of the front environment (that is, the environment on which the virtual image 110 is superimposed) that is visually recognized by the occupant 106 through the windshield is detected. Ru. Note that a sensor such as a millimeter wave radar may be used instead of the front camera 111.

Furthermore, an in-vehicle camera 112 is installed on the top surface of the instrument panel of the vehicle. The in-vehicle camera 112 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed with its optical axis directed toward the driver's seat. The range where the face of the passenger is generally expected to be located inside the vehicle is set as a detection range (imaging range of the in-vehicle camera 112), and the face of the passenger 106 sitting in the driver's seat is imaged. Then, image processing is performed on the captured image captured by the in-vehicle camera 112, thereby detecting the eye position (line-of-sight starting point) and line-of-sight direction of the occupant 106.

The superimposed image display device according to the second embodiment displays images 120, 121, and 122 of guide objects on the front display 104 as shown in FIG. indicate. As a result, as shown in FIG. 14, when the vehicle occupant visually recognizes images 120, 121, and 122 of the guide object displayed on the front display 104, the image 122 of the guide object is superimposed on the scenery through the windshield 105. A virtual image 123 of is visually recognized. Similarly, a virtual image 124 of the image 121 of the guide object is visually recognized superimposed on the scenery through the windshield 105. Further, a virtual image 125 of the image 122 of the guide object is visually recognized superimposed on the scenery through the windshield 105.

Furthermore, as shown in FIG. 14, an image 126 of the shadow of the guide object is also displayed on the front display 104. As a result, as shown in FIG. 14, when the vehicle occupant visually recognizes the guide object's shadow image 126 displayed on the front display 104, the guide object's shadow image 126 is superimposed on the scenery through the windshield 105. A virtual image 127 of is visually recognized. Note that if the shadow image 126 is black, the virtual image cannot be visually recognized, so it is preferable that the shadow image 126 be gray.

As a result, similarly to the superimposed image display device according to the first embodiment, it is possible to accurately grasp the position of the guide branch point to be turned left or right and the exit direction at the guide branch point. In the superimposed image display device according to the second embodiment, in the guide object display position determination process of S4, the shape of the guide object to be displayed on the front display 104 and the position (range) at which the guide object is displayed are determined. . Similarly, the shape and position (range) of the shadow of the guide object to be displayed on the front display 104 are also determined. Further, it is preferable that the current position and orientation of the vehicle specified in the three-dimensional space in S13 be the position of the vehicle occupant and the direction of the passenger's line of sight detected using the in-vehicle camera 112.

It should be noted that the present invention is not limited to the above-described embodiments, and it goes without saying that various improvements and modifications can be made without departing from the gist of the present invention.
For example, as a means for displaying an image superimposed on the scenery around the vehicle, the first embodiment uses a liquid crystal display 15 displaying an actual scene image, and the second embodiment uses a head-up display system. A window shield display (WSD) that displays an image may also be used. In WSD, the windshield may be used as a screen to display images from a projector, or the windshield may be used as a transmissive liquid crystal display. The image displayed on the windshield by the WSD becomes an image superimposed on the scenery around the vehicle.

Further, in the first and second embodiments, the guide object is an image of an arrow indicating the direction of travel of the vehicle along the guide route set by the navigation device, but it may be another image. For example, it may be a warning image for an object (for example, another vehicle, a pedestrian, a guide sign) that is a warning target for the occupants of the vehicle, or an image of the marking line of the lane in which the vehicle is traveling.

In addition, in the first and second embodiments, guidance objects are used to provide guidance to guidance junctions, but the points to be guided by guidance objects are not limited to guidance junctions, for example, lane reduction points, etc. It is also possible to use other points such as a merging section to warn vehicle occupants. Further, it is desirable to change the content of the displayed guidance object depending on the type of guidance target point.

In addition, in the first and second embodiments, when multiple types of guide objects are displayed, only the type of guide object whose shadow is displayed is the one closest to the ground. Shadows may also be displayed for other types of guide objects. For example, shadows may be displayed for all displayed guide objects.

Furthermore, in the first and second embodiments, the shadow caused by the guide object on another object (for example, the ground) is displayed as the shadow caused by the light source on the guide object, but the shadow (i.e., shadow) caused by the guide object itself is displayed. It may be displayed.

In addition, in the first and second embodiments, driving support using guide objects is performed both when driving on an expressway and when driving on a general road, but when driving only on an expressway or when driving on a general road. You may choose to do it only in time.

Further, in the first embodiment, the actual view image and the guide object captured by the front camera 19 are displayed on the liquid crystal display 15 of the navigation device 1, but the display for displaying the actual view image and the guide object is not provided inside the vehicle. Any display other than the liquid crystal display 15 may be used as long as the display is arranged.

Further, in the second embodiment, a virtual image is generated in front of the windshield 105 of the vehicle 102 by the front display 104, but a virtual image may be generated in front of a window other than the windshield 105. Further, the object on which the image is reflected by the front display 104 may be a visor (combiner) installed around the windshield 105 instead of the windshield 105 itself.

Further, in the first and second embodiments, the navigation ECU 13 of the navigation device 1 executes the processing of the driving support processing program (FIG. 2), but the execution entity can be changed as appropriate. For example, it may be configured to be executed by a control unit of the liquid crystal display 15, a vehicle control ECU, or other on-vehicle equipment.

Further, although the embodiments embodying the superimposed image display device according to the present invention have been described above, the superimposed image display device can also have the following configuration, and in that case, the following effects are achieved.

For example, the first configuration is as follows.
A superimposed image display device (1) for superimposing and visualizing guide objects (61 to 65) mounted on a vehicle and guiding information to occupants of the vehicle, superimposed on the scenery around the vehicle, wherein the guide objects are superimposed. guide object placement means (41) for arranging the guide object to be displayed in a three-dimensional space corresponding to the periphery of the area to be displayed; and a light source placement means (41) for arranging a light source (67) in the three-dimensional space. , object display means (41) for displaying the guide object based on the guide object arranged in the three-dimensional space; and object display means (41) for displaying the guide object based on the guide object arranged in the three-dimensional space and the light source. a shadow display means (41) for displaying a shadow (56) corresponding to the object; A light source is arranged, and when a plurality of guide objects are displayed by the object display means, the shadow display means displays the shadow only for the guide object located closest to the ground .
According to the superimposed image display device having the above configuration, the shadow of the guide object that is superimposed on the scenery and visible to the occupants of the vehicle is displayed, and the position of the light source that causes the shadow with respect to the guide object is displayed on the guide object. Place the light source at a position excluding the vertically above the guide object and the horizontally below the guide object. As a result, it becomes possible to reliably display the shadow corresponding to the guide object, and it becomes possible to make the occupant clearly recognize the position where the guide object overlaps based on the displayed shadow. Furthermore, it is possible to prevent the shadow of the guide object located above from being displayed on the guide object located below, making it difficult to visually recognize the guide object below. Furthermore, by reducing the number of displayed shadows, it is possible to prevent a decrease in visibility.

Further, the second configuration is as follows.
The object display means (41) includes a viewpoint setting means (41) for setting a viewpoint at a position in the three-dimensional space corresponding to the current position of the vehicle, and the object display means (41) is configured to display the guide arranged in the three-dimensional space. The objects (61 to 65) are displayed in a shape that is visible from the viewpoint, and the shadow display means (41) is configured to display a shadow (56) corresponding to the guide object caused by the light source in a shape that is visible from the viewpoint. Display in .
According to the superimposed image display device having the above configuration, by displaying the guide object and shadow based on an image from a viewpoint set in a three-dimensional space, the guide object and shadow can be superimposed at an appropriate position in the landscape. This makes it possible for passengers to visually recognize the vehicle. Furthermore, it is possible to accurately specify the position and shape of the shadow that appears on the guide object based on the virtual light source. As a result, it becomes possible to display guide objects and shadows that do not give a sense of discomfort to vehicle occupants.

Moreover, the third configuration is as follows.
The light source arranging means (41) arranges the light source (67) further away from the viewpoint than a position corresponding to a vertically upper position of the guide object (61 to 65).
According to the superimposed image display device having the above configuration, the position of the shadow is located in front of the guide object vertically below, so that the position of the shadow can be set to be a natural position that does not give a sense of discomfort to the passenger. , it is possible to more accurately grasp the distance to the guide object than when displaying it vertically downward. Further, even if the guide object does not have a thickness shape, it is possible to create a shadow.

Further, the fourth configuration is as follows.
The light source arrangement means (41) arranges the light source (67) at a position corresponding to the position of the sun at the current time.
According to the superimposed image display device having the above configuration, a shadow generated based on a light source located at a position corresponding to the current position of the sun is displayed, so that the position of the shadow is natural and does not give a sense of discomfort to the occupants. becomes possible.

Further, the fifth configuration is as follows.
There are a plurality of types of guide objects (61 to 65), and the light source placement means (41) arranges them at positions corresponding to the types of guide objects that cause shadows by the light source (67).
According to the superimposed image display device having the above configuration, the position of the light source can be set according to the position and shape of the guide object, so it is possible to display a shadow in a position and shape that is easily visible to vehicle occupants. becomes.

Further, the sixth configuration is as follows.
It has an environment acquisition means (41) for acquiring the brightness of an area on which the shadow (56) is to be superimposed, and the shadow display means displays the shadow with a hue or brightness depending on the brightness of the area. do.
According to the superimposed image display device having the above configuration, it is possible to display a shadow corresponding to the brightness of the scenery on which the shadow is superimposed. As a result, it is possible to create a natural shadow that does not give a sense of discomfort to the occupants.

1 Navigation device 15 Liquid crystal display 19 Front camera 41 CPU
42 RAM
43 ROM
51 Driving guidance screen 52 Scenery 53-55 Image of guidance object 56 Image of shadow 60 Guidance junction 61-64 Arrow (an example of guidance object)
65 Landmark icon (an example of a guide object)
66 Vehicle 67 Light source

Claims (7)

A superimposed image display device that is mounted on a vehicle and allows a guide object that guides information to a passenger of the vehicle to be visually recognized by superimposing it on the scenery around the vehicle,
Guide object placement means for arranging the guide object to be displayed in a three-dimensional space corresponding to the periphery of an area on which the guide object is superimposed;
light source arrangement means for arranging a light source in the three-dimensional space;
object display means for displaying the guide object based on the guide object arranged in the three-dimensional space;
a shadow display means for displaying a shadow corresponding to the guide object based on the guide object arranged in the three-dimensional space and the light source;
The light source arranging means arranges the light source at a position other than above the guide object in the vertical direction and below the guide object in the horizontal direction ,
The shadow display means is a superimposed image display device that displays the shadow of only the guide object located closest to the ground when a plurality of the guide objects are displayed by the object display means. comprising a viewpoint setting means for setting a viewpoint at a position in the three-dimensional space corresponding to the current position of the vehicle;
The object display means displays the guide object arranged in the three-dimensional space in a shape that is visible from the viewpoint,
The superimposed image display device according to claim 1, wherein the shadow display means displays a shadow corresponding to the guide object caused by the light source in a shape that is visible from the viewpoint. 3. The superimposed image display device according to claim 2, wherein the light source placement means places the light source farther from the viewpoint than a position corresponding to a vertically upper position of the guide object. 4. The superimposed image display device according to claim 1, wherein the light source arrangement means arranges the light source at a position corresponding to the position of the sun at the current time. There are multiple types of guide objects,
4. The superimposed image display device according to claim 1 , wherein the light source arrangement means arranges the light source at a position corresponding to the type of the guide object causing a shadow by the light source. comprising an environment acquisition means for acquiring the brightness of the area on which the shadow is to be superimposed;
6. The superimposed image display device according to claim 1, wherein the shadow display means displays the shadow in a hue or brightness that corresponds to the brightness of the area. A superimposed image display device that is mounted on a vehicle and allows a guide object that guides information to a passenger of the vehicle to be visually recognized by superimposing it on the scenery around the vehicle,
Guide object placement means for arranging the guide object to be displayed in a three-dimensional space corresponding to the periphery of an area on which the guide object is superimposed;
light source arrangement means for arranging a light source in the three-dimensional space;
object display means for displaying the guide object based on the guide object arranged in the three-dimensional space;
A computer program for functioning as a shadow display means for displaying a shadow corresponding to the guide object based on the guide object arranged in the three-dimensional space and the light source,
The light source arranging means arranges the light source at a position other than above the guide object in the vertical direction and below the guide object in the horizontal direction ,
The shadow display means is a computer program for displaying the shadow of only the guide object located closest to the ground when a plurality of the guide objects are displayed by the object display means.

Images