JP5658507B2 - Image display system, image generation apparatus, and image display method - Google Patents

Description

  The present invention relates to a technique for displaying an image in a vehicle.

  2. Description of the Related Art Conventionally, there is known an image display system that is mounted on a vehicle such as an automobile and displays a peripheral image indicating a region around the vehicle on a display in a vehicle interior based on a captured image obtained by an in-vehicle camera. By using this image display system, a user (typically a driver) can grasp the situation around the vehicle almost in real time.

  For example, the outer region of the front fender that is on the opposite side of the driver's seat is likely to be a blind spot from the driver's seat, and it is difficult for the user to grasp the clearance between the vehicle body and the obstacle. On the other hand, if an image display system is used, a peripheral image showing an outer region of the front fender can be displayed on a display in the vehicle interior based on a photographed image of an in-vehicle camera arranged on the side of the vehicle. As a result, the user can easily confirm the clearance between the vehicle body on the opposite side of the driver's seat and the obstacle, for example, when the vehicle is brought close.

  Note that there is Patent Document 1 as a document disclosing a technology related to the technology described in this specification.

JP 2007-210458 A

  In recent years, an image display system has been proposed in which vehicle-mounted cameras are arranged at a plurality of positions such as the front, rear, and side of a vehicle, and peripheral images based on captured images obtained by the plurality of vehicle-mounted cameras are displayed. In such an image display system, it is possible to display surrounding images viewed from different viewpoints. For example, it is possible to display a peripheral image viewed from the front viewpoint of the vehicle, a peripheral image viewed from the rear viewpoint of the vehicle, and the like.

  However, in such an image display system that can display a peripheral image viewed from a plurality of viewpoints, when the display content is switched to the peripheral image, from which viewpoint the displayed peripheral image is the peripheral image viewed It is difficult for the user to grasp intuitively.

  For example, in the display state of the map image around the vehicle, when the display content is switched to the peripheral image viewed from the viewpoint behind the vehicle, the user determines from which viewpoint the peripheral image displayed after the switching is viewed May be difficult to grasp intuitively. The same applies to the case where the viewpoint of the peripheral image to be displayed is changed, such as when the display content is switched to the peripheral image viewed from the rear viewpoint of the vehicle in the display state of the peripheral image viewed from the front viewpoint of the vehicle. .

  The present invention has been made in view of the above problems, and in the case of switching display contents to a peripheral image, the user can intuitively grasp from which viewpoint the peripheral image displayed after switching is viewed from which viewpoint. The purpose is to provide technology.

In order to solve the above-described problems, an invention according to claim 1 is an image display system mounted on a vehicle, wherein input means for inputting captured images of a plurality of cameras arranged in the vehicle, and the captured image Based on the image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint, and a first peripheral image viewed from the first viewpoint based on the captured image, the first peripheral image is different from the first viewpoint. When switching the display content from the second peripheral image viewed from two viewpoints , a plurality of the composite images are displayed continuously in time, via a reference viewpoint that is a viewpoint overlooking the vehicle from the second viewpoint. position of the virtual viewpoint to the first viewpoint is provided with a display means for performing an animation expression to move stepwise, the Te.

Further, the invention of claim 2 is an image display system mounted on a vehicle, wherein an input means for inputting captured images of each of a plurality of cameras arranged in the vehicle, and an arbitrary one based on the captured images Image generation means for generating a composite image showing a region around the vehicle viewed from a virtual viewpoint, display means for displaying a map image around the vehicle, and photo acquisition for acquiring a photographic image of the ground taken from the sky And when the display content is switched from the map image to a first peripheral image viewed from a first viewpoint based on the photographed image, the display unit of the vehicle is approximately the same scale as the map image. after displaying the photographic image showing the periphery of the region, the said image in the synthesized image substantially the same scale as viewed from the reference viewpoint is a bird's-eye-view of the vehicle to show the area around the vehicle Expanding the body image in stages, further, to display continuously a plurality of the composite image temporally, animation representation the position of the virtual viewpoint from the reference viewpoint to the first viewpoint is moved stepwise Do.

Further, the invention of claim 3 is an image generation device that generates an image to be displayed on a display device mounted on a vehicle, and an input unit that inputs captured images of each of a plurality of cameras arranged in the vehicle; Image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image, and output means for outputting and displaying an image on the display device, When switching display content from a second peripheral image viewed from a second viewpoint different from the first viewpoint to a first peripheral image viewed from a first viewpoint based on the captured image , the display device displays a plurality of the synthesized images as a time. displaying continuous and performs animation representation the position of the virtual viewpoint from the second viewpoint to the first viewpoint via the standard viewpoint is looking down perspective the vehicle moves stepwise.
The invention of claim 4 is an image generation device for generating an image to be displayed on a display device mounted on a vehicle, and an input means for inputting captured images of each of a plurality of cameras arranged in the vehicle; Based on the photographed image, an image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint, an output means for outputting and displaying an image on the display device, and the ground from above Photograph acquisition means for acquiring a photographed photograph image, and the display device switches a display content from a map image around the vehicle to a first peripheral image viewed from a first viewpoint based on the photographed image. After displaying the photographic image showing the area around the vehicle at approximately the same scale as the map image, the front at approximately the same scale as the composite image viewed from a reference viewpoint that is a viewpoint overlooking the vehicle The subject image of the photographic image is enlarged step by step until it shows an area around the vehicle, and a plurality of the composite images are displayed continuously in time, and the virtual image from the reference viewpoint to the first viewpoint is displayed. Animated expression in which the position of the viewpoint moves step by step.

Further, the invention of claim 5 is an image display method for displaying an image on a display device mounted on a vehicle, and (a) a step of inputting captured images of each of a plurality of cameras arranged in the vehicle; (B) generating a composite image indicating a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image; and (c) displaying the image on the display device, In the step (c), when the display content is switched from the second peripheral image viewed from a second viewpoint different from the first viewpoint to the first peripheral image viewed from the first viewpoint based on the photographed image, a plurality of the composite images Is displayed continuously in time, and an animation expression is performed in which the position of the virtual viewpoint moves stepwise from the second viewpoint to the first viewpoint via a reference viewpoint that is a viewpoint overlooking the vehicle. .
The invention of claim 6 is an image display method for displaying an image on a display device mounted on a vehicle, wherein (a) a step of inputting captured images of each of a plurality of cameras arranged in the vehicle; (B) based on the captured image, generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint, (c) acquiring a photographic image of the ground photographed from above, d) the display device displaying an image, wherein the step (d) displays a display content from a map image around the vehicle to a first peripheral image viewed from a first viewpoint based on the captured image. In the case of switching, after displaying the photographic image showing the area around the vehicle at approximately the same scale as the map image, the scale image is approximately the same as the composite image viewed from a reference viewpoint that is a viewpoint overlooking the vehicle. Area around the vehicle The subject image of the photographic image is enlarged stepwise until it is shown, and a plurality of the synthesized images are continuously displayed in time, and the position of the virtual viewpoint is stepped from the reference viewpoint to the first viewpoint. Animated expression to move to.

According to the first , third, and fifth aspects of the present invention, when the viewpoint of the surrounding image to be displayed is switched, an animation expression is performed in which the virtual viewpoint moves stepwise from the viewpoint overlooking the vehicle to the viewpoint of the surrounding image. The For this reason, the user can intuitively grasp from which viewpoint the peripheral image displayed after the switching is viewed.

According to the invention of 請 Motomeko 2, 4 and 6, in the case of switching the display content to the peripheral image from the map image, to display the image indicating the area around the vehicle in the map image substantially the same scale, Thereafter, the subject image of the photographic image is enlarged step by step until the area around the vehicle is shown at substantially the same scale as the synthesized image viewed from the reference viewpoint. Furthermore, an animation expression is performed in which the virtual viewpoint moves stepwise from the reference viewpoint to the viewpoint of the surrounding image. For this reason, the viewpoint of the image to display can be changed smoothly from the map image before display switching to the peripheral image after display switching. As a result, the user can more intuitively grasp from which viewpoint the peripheral image displayed on the display means is viewed.

FIG. 1 is a block diagram illustrating a configuration of an image display system according to the first embodiment. FIG. 2 is a diagram illustrating a position where the in-vehicle camera is arranged in the vehicle. FIG. 3 is a diagram illustrating a method for generating a composite image. FIG. 4 is a diagram illustrating transition of operation modes of the image display system. FIG. 5 is a diagram illustrating a state where the overhead image is displayed on the display. FIG. 6 is a diagram illustrating an example of transition of virtual viewpoints. FIG. 7 is a diagram illustrating an example of an animation expression. FIG. 8 is a diagram illustrating an example of the transition of the virtual viewpoint. FIG. 9 is a diagram illustrating an example of an animation expression. FIG. 10 is a diagram illustrating a processing flow when switching from the navigation mode to the operation mode in which the surrounding image is displayed. FIG. 11 is a diagram illustrating an example of the transition of the virtual viewpoint. FIG. 12 is a diagram illustrating an example of an animation expression. FIG. 13 is a diagram illustrating a flow of processing when the operation mode is switched between the operation modes for displaying the peripheral image. FIG. 14 is a diagram illustrating how the driver viewpoint image is displayed on the display. FIG. 15 is a diagram illustrating an example of the transition of the virtual viewpoint. FIG. 16 is a diagram illustrating an example of transition of virtual viewpoints. FIG. 17 is a block diagram illustrating a configuration of an image display system according to the third embodiment. FIG. 18 is a diagram showing how a photographic image is displayed on the display. FIG. 19 is a diagram illustrating transition of images displayed on the display. FIG. 20 is a diagram illustrating a processing flow when switching from the navigation mode to the operation mode in which the surrounding image is displayed. FIG. 21 is a diagram illustrating an example of an operation mode for displaying a composite image from a virtual viewpoint.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. System configuration>
FIG. 1 is a block diagram illustrating a configuration of an image display system 120 according to the first embodiment. This image display system 120 is mounted on a vehicle (in this embodiment, an automobile), and has a function of photographing the periphery of the vehicle, generating an image, and displaying the image in the passenger compartment. By using this image display system 120, a user (typically a driver) of the image display system 120 can grasp the state around the vehicle in almost real time.

  As shown in FIG. 1, the image display system 120 mainly includes an image generation device 100 that generates a peripheral image showing a state around the vehicle, and a navigation device 20 that displays various types of information to a user who gets on the vehicle. In preparation. The peripheral image generated by the image generation device 100 is displayed on the navigation device 20.

  The navigation device 20 provides navigation guidance to the user, and includes a display 21 such as a liquid crystal provided with a touch panel function, an operation unit 22 such as a hard switch operated by the user, and a control unit 23 that controls the entire device. And. The navigation device 20 is installed on an instrument panel or the like of the vehicle so that the screen of the display 21 is visible from the user.

  Various user instructions are received by the operation unit 22 and the display 21 as a touch panel. The control unit 23 is a computer including a CPU, a RAM, a ROM, and the like, and various functions including a navigation function are realized by the CPU performing arithmetic processing according to a predetermined program. The navigation apparatus 20 is communicably connected to the image generation apparatus 100, and can transmit and receive various control signals to and from the image generation apparatus 100 and receive images generated by the image generation apparatus 100.

  Under the control of the control unit 23, a map image around the vehicle for navigation guidance is usually displayed on the display 21 by the function of the navigation device 20 alone. The map image is stored in advance in a hard disk or the like provided in the navigation device 20. Further, the position (latitude and longitude) of the vehicle is acquired by the GPS device provided in the navigation device 20, and a map image around the vehicle is displayed on the display 21 based on the position of the vehicle. The scale of the map image to be displayed can be changed according to the operation of the operation unit 22 by the user.

  On the other hand, when the operation mode of the image display system 120 is changed, a peripheral image showing a state around the vehicle generated by the image generation device 100 is displayed on the display 21. As a result, the navigation device 20 also functions as a display device that receives and displays the peripheral image generated by the image generation device 100.

  The image generation apparatus 100 includes a main body 10 that is an ECU (Electronic Control Unit) having a function of generating an image, and an imaging unit 5 that captures the periphery of the vehicle. The main body unit 10 is arranged at a predetermined position of the vehicle, and generates a peripheral image to be displayed on the display 21 based on a captured image obtained by photographing the periphery of the vehicle with the photographing unit 5.

  The photographing unit 5 is electrically connected to the main body unit 10 and operates based on a signal from the main body unit 10. The photographing unit 5 includes a front camera 51, a side camera 52, and a back camera 53 that are in-vehicle cameras. Each of the in-vehicle cameras 51 to 53 includes a lens and an image sensor and electronically acquires an image.

  The plurality of in-vehicle cameras 51 to 53 are respectively arranged at different positions of the vehicle. FIG. 2 is a diagram illustrating positions where the in-vehicle cameras 51 to 53 are arranged on the vehicle 9.

  As shown in FIG. 2, the front camera 51 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 9, and its optical axis 51 a is directed in the straight traveling direction of the vehicle 9. The back camera 53 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 9, and its optical axis 53 a is directed in the direction opposite to the straight traveling direction of the vehicle 9. The front camera 51 and the back camera 53 are preferably attached at the center in the left-right direction, but may be slightly shifted from the center in the left-right direction in the left-right direction. The side cameras 52 are provided on the left and right door mirrors 93, respectively, and their optical axes 52a are directed to the outside of the vehicle 9 along the left-right direction of the vehicle 9 (direction orthogonal to the straight traveling direction).

  A fish-eye lens or the like is adopted as the lens of these in-vehicle cameras 51 to 53, and the in-vehicle cameras 51 to 53 have an angle of view θ of 180 degrees or more. For this reason, the surroundings of the vehicle 9 can be image | photographed by utilizing the four vehicle-mounted cameras 51-53.

  Returning to FIG. 1, the main body 10 of the image generation apparatus 100 includes a control unit 1 that controls the entire apparatus, an input terminal 49 that inputs captured images of each of the four in-vehicle cameras 51 to 53 of the imaging unit 5, and a display Are mainly provided with an image generation unit 3 that generates a peripheral image of the camera and a navigation communication unit 42 that performs communication with the navigation device 20.

  Various instructions from the user received by the operation unit 22 or the display 21 of the navigation device 20 are received by the navigation communication unit 42 as control signals and input to the control unit 1. In addition, the image generation apparatus 100 includes a changeover switch 43 that receives an instruction to switch display contents from a user. A signal indicating a user instruction is also input to the control unit 1 from the changeover switch 43. As a result, the image generating apparatus 100 can operate in response to both a user operation on the navigation apparatus 20 and a user operation on the changeover switch 43. The changeover switch 43 is arranged at an appropriate position of the vehicle separately from the main body unit 10 so that the user can easily operate.

  The image generation unit 3 is a hardware circuit capable of various types of image processing, processes a captured image input from the imaging unit 5 via the input terminal 49, and generates a peripheral image to be displayed on the display 21. . The image generation unit 3 includes a captured image adjustment unit 31 and a composite image generation unit 32 as main functions.

  The photographed image adjusting unit 31 is for adjusting a photographed image acquired by the photographing unit 5 to be used for display. Specifically, the captured image adjustment unit 31 performs image processing such as distortion correction, enlargement / reduction, and clipping on the captured image.

  Based on the plurality of captured images acquired by the plurality of in-vehicle cameras 51 to 53 of the photographing unit 5, the composite image generating unit 32 generates a composite image indicating a region around the vehicle viewed from an arbitrary virtual viewpoint around the vehicle. Generate. A method by which the composite image generation unit 32 generates a composite image will be described later.

  The captured image adjusted for display by the captured image adjustment unit 31 and the composite image generated by the composite image generation unit 32 are output to the navigation device 20 by the navigation communication unit 42. As a result, a peripheral image including at least a partial region around the vehicle as a subject image is displayed on the display 21 of the navigation device 20.

  In this specification, the “peripheral image” means an image showing at least a part of the area around the vehicle, and is a concept that includes both a captured image adjusted for display and a composite image. The captured image adjusted for display is a peripheral image viewed from the viewpoint of the acquired lens position of the in-vehicle cameras 51 to 53 as a viewpoint. Further, the composite image is a peripheral image viewed from a virtual viewpoint set at an arbitrary position around the vehicle.

  The control unit 1 is a computer including a CPU, a RAM, a ROM, and the like, and various control functions are realized by the CPU performing arithmetic processing according to a predetermined program. An image control unit 11 and a display control unit 12 shown in the figure show some of the functions of the control unit 1 realized in this way.

  The image control unit 11 performs control related to image processing executed by the image generation unit 3. For example, the image control unit 11 instructs various parameters necessary for generating a composite image generated by the composite image generation unit 32.

  The display control unit 12 performs control related to display contents displayed on the display 21 of the navigation device 20. For example, the display content of the display 21 is switched according to a change in the operation mode of the image display system 120.

  The main body 10 of the image generation apparatus 100 further includes a nonvolatile memory 40, a card reading unit 44, and a signal input unit 41, which are connected to the control unit 1.

  The nonvolatile memory 40 is a flash memory or the like that can maintain stored contents even when the power is turned off. The nonvolatile memory 40 stores viewpoint data 4a. The viewpoint data 4a is used when determining the virtual viewpoint of the composite image.

  The card reading unit 44 reads a memory card MC that is a portable recording medium. The card reading unit 44 includes a card slot in which the memory card MC can be attached and detached, and reads data recorded on the memory card MC installed in the card slot. Data read by the card reading unit 44 is input to the control unit 1. The memory card MC is a flash memory or the like capable of storing various data, and the image generating apparatus 100 can use various data stored in the memory card MC. For example, a program (firmware) that realizes the function of the control unit 1 can be updated by storing a program in the memory card MC and reading the program.

  Moreover, the signal input part 41 inputs the signal from the various apparatuses provided in the vehicle. A signal from the outside of the image display system 120 is input to the control unit 1 via the signal input unit 41. In the present embodiment, a signal from the shift sensor 71 is input to the control unit 1 via the signal input unit 41. From the shift sensor 71, the position of operation of the shift lever of the transmission of the vehicle, that is, shift such as “P (parking)”, “D (forward)”, “N (neutral)”, “R (reverse)”, etc. Position is entered.

<1-2. Image composition processing>
Next, a method in which the composite image generation unit 32 of the image generation unit 3 generates a composite image that shows how the periphery of the vehicle 9 is viewed from an arbitrary virtual viewpoint based on a plurality of captured images obtained by the imaging unit 5. explain. FIG. 3 is a diagram for explaining a method of generating a composite image.

  When the front camera 51, the side camera 52, and the back camera 53 of the photographing unit 5 perform photographing simultaneously, four photographed images P1 to P4 that respectively indicate the front, left side, right side, and rear of the vehicle 9 are acquired. That is, the four captured images P1 to P4 acquired by the imaging unit 5 include information indicating the entire periphery of the vehicle 9 at the time of shooting.

  Next, each pixel of the four captured images P1 to P4 is projected onto a three-dimensional curved surface TS in a virtual three-dimensional space. The three-dimensional curved surface TS has, for example, a substantially hemispherical shape (a bowl shape), and a center portion (a bottom portion of the bowl) is determined as a position where the vehicle 9 exists. A correspondence relationship is determined in advance between the positions of the pixels included in the captured images P1 to P4 and the positions of the pixels of the solid curved surface TS. For this reason, the value of each pixel of the solid curved surface TS can be determined based on this correspondence and the value of each pixel included in the captured images P1 to P4.

  The correspondence between the positions of the pixels of the captured images P1 to P4 and the positions of the pixels of the three-dimensional curved surface TS is the arrangement of the four in-vehicle cameras 51 to 53 in the vehicle 9 (inter-distance, ground height, optical axis angle, etc. ). Table data indicating the correspondence is stored in advance in the nonvolatile memory 40.

  Further, data indicating the shape and size of the vehicle body stored in advance in the nonvolatile memory 40 is used, and a polygon model indicating the three-dimensional shape of the vehicle 9 is virtually configured. The configured model of the vehicle 9 is arranged in a substantially hemispherical center portion determined as the position of the vehicle 9 in the three-dimensional space in which the three-dimensional curved surface TS is set.

  Furthermore, the virtual viewpoint VP is set by the control unit 1 for the three-dimensional space where the three-dimensional curved surface TS exists. The virtual viewpoint VP is defined by the viewpoint position and the visual field direction, and is set at an arbitrary viewpoint position corresponding to the vicinity of the vehicle 9 in this three-dimensional space toward an arbitrary visual field direction.

  Then, according to the set virtual viewpoint VP, a necessary area on the three-dimensional curved surface TS is cut out as an image. The relationship between the virtual viewpoint VP and a necessary area in the three-dimensional curved surface TS is determined in advance, and is stored in advance in the nonvolatile memory 40 or the like as table data. On the other hand, rendering is performed on a polygon model according to the set virtual viewpoint VP, and the resulting two-dimensional vehicle image is superimposed on the cut-out image. Thereby, the composite image which shows a mode that the vehicle 9 and the periphery of the vehicle 9 were seen from arbitrary virtual viewpoints VP will be produced | generated.

  For example, when a virtual viewpoint VPa is set in which the viewpoint position is just above the center of the position of the vehicle 9 and the viewing direction is directly below, the vehicle 9 and the vehicle 9 A composite image CPa indicating the peripheral area of the image is generated. Further, as shown in the figure, when the virtual viewpoint VPb is set in which the viewpoint position is the left rear of the position of the vehicle 9 and the visual field direction is the substantially forward direction in the vehicle 9, the entire periphery from the left rear of the vehicle 9 is set. As a result, a composite image CPb indicating the vehicle 9 and the area around the vehicle 9 is generated.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface TS, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint VP are photographed. By determining based on the images P1 to P4, the processing speed can be improved.

  In the image display system 120, by using such a function of the composite image generation unit 32, a composite image viewed from an arbitrary viewpoint around the vehicle 9 can be generated and displayed on the display 21.

<1-3. Operation mode>
Next, the operation mode of the image display system 120 will be described. FIG. 4 is a diagram illustrating transition of operation modes of the image display system 120. The image display system 120 has three operation modes: a navigation mode M0, a front mode M1, a side mode M2, and a back mode M3. These operation modes are switched by the control of the control unit 1 in accordance with the driver's operation on the vehicle 9 and the traveling state of the vehicle 9.

  The navigation mode M0 is an operation mode in which navigation guidance is performed by the function of the navigation device 20. In the navigation mode M0, the functions of the image generation apparatus 100 are not used, and various displays are performed on the display 21 with the functions of the navigation apparatus 20 alone. Specifically, a map image NP around the vehicle 9 is mainly displayed on the display 21.

  On the other hand, the front mode M1, the side mode M2, and the back mode M3 use the function of the image generation device 100 to display a peripheral image on the display 21 and to show the user the situation around the vehicle in real time. Mode.

  The front mode M1 is an operation mode for displaying the front of the vehicle 9, and is used when entering an intersection with poor visibility. On the display 21, a captured image (hereinafter referred to as “front image”) SP1 acquired by the front camera 51 and adjusted for display is displayed. The front image SP1 is a peripheral image viewed from the viewpoint in front of the vehicle 9 (the lens position of the front camera 51). In the front mode M1, an icon C1 indicating that the viewpoint of the displayed peripheral image (that is, the front image) SP1 is the front of the vehicle 9 is displayed.

  The side mode M2 is an operation mode for displaying the side of the vehicle 9, and is used when performing width alignment. On the display 21, a captured image (hereinafter referred to as “side image”) SP2 acquired by the side camera 52 and adjusted for display is displayed. The side image SP2 is a peripheral image viewed from the side viewpoint of the vehicle 9 (the lens position of the side camera 52). In the side mode M2, an icon C2 indicating that the viewpoint of the displayed peripheral image (that is, the side image) SP2 is the side of the vehicle 9 is displayed.

  The back mode M3 is an operation mode for displaying the rear of the vehicle 9, and is used when the vehicle is moving backward. On the display 21, a captured image (hereinafter referred to as “back image”) SP3 acquired by the back camera 53 and adjusted for display is displayed. This back image SP3 is a peripheral image viewed from the viewpoint behind the vehicle 9 (the lens position of the back camera 53). In the back mode M3, an icon C3 indicating that the viewpoint of the displayed peripheral image (that is, the back image) SP3 is behind the vehicle 9 is displayed.

  In the navigation mode M0, when a predetermined operation is performed on the operation unit 22 of the navigation device 20, the operation mode (the so-called last mode) that is last activated among the front mode M1 and the side mode M2 is switched. The front mode M1 and the side mode M2 are switched each time the changeover switch 43 is pressed. In addition, when a predetermined operation is performed on the operation unit 22 of the navigation device 20 in the front mode M1 or the side mode M2, the navigation mode M0 is restored.

  In the operation mode other than the back mode M3, when the shift position input from the shift sensor 71 is “R (reverse)”, the mode is switched to the back mode M3. That is, when the shift position is “R (reverse)”, the vehicle 9 is in a reverse state, and therefore, the back mode M3 that mainly indicates the rear of the vehicle 9 is switched. Further, in the case of the back mode M3, when the shift position is other than “R (reverse)”, the operation mode immediately before switching to the back mode M3 is returned.

  As described above, the image display system 120 can switch the peripheral images viewed from a plurality of viewpoints to be displayed on the display 21 by switching the plurality of operation modes. When the peripheral images from a plurality of viewpoints can be displayed in this way, when the display content of the display 21 is switched to the peripheral images, the user can determine from which viewpoint the displayed peripheral images are the peripheral images. It may be difficult to grasp intuitively.

  For example, when the navigation mode M0 for displaying a map image is switched to one of the operation modes M1, M2, and M3 for displaying a peripheral image, which viewpoint is the peripheral image displayed after the switching is viewed from the viewpoint ( It is difficult for the user to intuitively grasp which of the front image SP1, the side image SP2, and the back image SP3). Further, even when the operation mode is switched between the operation modes M1, M2, and M3 for displaying the peripheral image (when the viewpoint of the peripheral image to be displayed is changed to a different viewpoint), the peripheral image displayed after the switching is performed. However, it is difficult for the user to intuitively grasp from which viewpoint.

  Therefore, in the image display system 120, when the display content of the display 21 is switched to the peripheral image, the viewpoint position of the image to be displayed is stepwise and continuous from the predetermined reference viewpoint to the viewpoint of the peripheral image after switching. An animation expression that moves to is displayed on the display 21. Hereinafter, such animation expression will be described.

<1-4. Switching from navigation mode>
First, a case will be described in which the navigation mode M0 is switched to one of the operation modes M1, M2, and M3 for displaying surrounding images.

  In this case, first, as shown in FIG. 5, from the state in which the map image NP in the navigation mode M0 is displayed, the bird's-eye view image CP0, which is a composite image for bird's-eye view of the vehicle 9 and the entire periphery of the vehicle 9, is displayed. The The viewpoint position of the virtual viewpoint of the bird's-eye view image CP0 is directly above the center of the vehicle 9, and the visual field direction is a direct downward direction. A virtual viewpoint defined by such viewpoint position and viewing direction is the reference viewpoint. The virtual viewpoint is moved stepwise and continuously from the reference viewpoint to the viewpoint of the peripheral image displayed after switching.

  FIG. 6 is a diagram illustrating transition of the virtual viewpoint when switching from the navigation mode M0 to the front mode M1. In this case, the virtual viewpoint VP is moved stepwise from the reference viewpoint VP0 to the viewpoint of the front image (lens position of the front camera 51) VP1. Then, a plurality of synthesized images are continuously generated in time with the virtual viewpoint VP being moved stepwise in this way. The plurality of generated composite images are sequentially output to the navigation device 20 and displayed on the display 21 continuously in time.

  As a result, as shown in FIG. 7, on the display 21, an animation expression is made such that the virtual viewpoint of the composite image CPm to be displayed moves stepwise from the reference viewpoint VP0 to the viewpoint VP1 of the front image. That is, the composite image CPm displayed on the display 21 is initially shown as an overview of the entire vehicle 9 (state ST11). Thereafter, the composite image CPm gradually shows an area in front of the vehicle 9 (state ST12), and finally shows an area in front of the vehicle 9 (state ST13).

  When the virtual viewpoint VP moves to the front image viewpoint VP1 and a region similar to the front image is indicated in the composite image CPm, the front image CP1 is displayed on the display 21 instead of the composite image CPm. As a result, the mode is switched to the front mode M1.

  FIG. 8 is a diagram illustrating transition of the virtual viewpoint when switching from the navigation mode M0 to the back mode M3. In this case, the virtual viewpoint VP is moved stepwise from the reference viewpoint VP0 to the viewpoint of the back image (lens position of the back camera 53) VP3. Then, a plurality of synthesized images are continuously generated in time with the virtual viewpoint VP being moved stepwise in this way. The plurality of generated composite images are sequentially output to the navigation device 20 and displayed on the display 21 continuously in time.

  As a result, as shown in FIG. 9, on the display 21, an animation expression is performed such that the virtual viewpoint of the composite image CPm to be displayed moves stepwise from the reference viewpoint VP0 to the viewpoint VP3 of the back image. That is, the composite image CPm displayed on the display 21 is initially shown as an overview of the entire vehicle 9 (state ST21). After that, the subject image indicated by the composite image CPm is turned upside down (state ST22), and the composite image CPm gradually enlarges and shows the area behind the vehicle 9 (state ST23), and finally the area behind the vehicle 9 (State ST24). When the virtual viewpoint VP moves to the viewpoint VP3 of the back image and a region similar to the back image is indicated in the composite image CPm, the back image CP3 is displayed on the display 21 instead of the composite image CPm. As a result, the mode is switched to the back mode M3.

  By performing such an animation expression, the user can intuitively grasp from which viewpoint the peripheral image displayed after switching is the peripheral image as compared to the case where the peripheral image is displayed instantaneously. it can. In addition, since the reference viewpoint VP0 that is the starting point of the movement of the virtual viewpoint is a viewpoint overlooking the vehicle 9, the viewpoint position and the viewing direction of the viewpoint after the movement can be easily grasped based on the entire vehicle.

  FIG. 10 is a diagram illustrating a flow of processing when switching from the navigation mode M0 to any one of the operation modes M1, M2, and M3 for displaying the surrounding images. This process is realized mainly by the control of the display control unit 12.

  First, data indicating the viewpoint position and viewpoint direction of the reference viewpoint VP0 is acquired from the viewpoint data 4a stored in the nonvolatile memory 40 (step S11). Next, data indicating the viewpoint position and viewpoint direction of the destination viewpoint is acquired from the viewpoint data 4a stored in the nonvolatile memory 40 (step S12). The destination viewpoint is the viewpoint of the peripheral image displayed after switching. Specifically, the destination viewpoint is the front image viewpoint when switching to the front mode M1, the side image viewpoint when switching to the side mode M2, and the back image viewpoint when switching to the back mode M3.

  Next, a composite image viewed from the reference viewpoint VP0, that is, the overhead image CP0 is generated in the composite image generation unit 32 and displayed on the display 21 (see FIG. 5) (step S13).

  Subsequently, a plurality of synthesized images CPm are continuously displayed in time, and an animation expression is performed in which the position of the virtual viewpoint VP moves stepwise from the reference viewpoint VP0 to the destination viewpoint. While the virtual viewpoint VP is moving, the viewpoint position and the viewpoint direction of the virtual viewpoint VP are changed in stages.

  Specifically, first, the virtual viewpoint VP is moved toward the destination viewpoint by an amount corresponding to one frame (step S14). The viewpoint position and viewpoint direction of the virtual viewpoint VP during movement are based on the viewpoint position and viewpoint direction of the viewpoint before movement, the viewpoint position and viewpoint direction of the viewpoint after movement, and the number of frames of the composite image used for movement. Derived by linear interpolation. Then, the composite image CPm from the virtual viewpoint VP is generated by the composite image generation unit 32 and displayed on the display 21 (step S15).

  By repeating the processes of step S14 and step S15, an animation expression in which the position of the virtual viewpoint moves stepwise is performed. Steps S14 and S15 are repeated until the virtual viewpoint VP reaches the destination viewpoint. Such animation expression is performed over 0.5 seconds to 1.5 seconds, for example.

  When the virtual viewpoint VP reaches the destination viewpoint (Yes in step S16), instead of the synthesized image CPm on the display 21, the captured images (front image SP1, side image SP2, and back image to be displayed after switching the operation mode) Any one of SP3) is displayed (step S17). In this case, the display content of the display 21 may be gradually switched from the composite image CPm to the captured image to be displayed by using a known method such as alpha blending instead of instantaneously switching.

<1-5. Change of perspective>
Next, a case where the operation mode is switched between the operation modes M1, M2, and M3 for displaying the peripheral image will be described. In this case, the virtual viewpoint is moved stepwise and continuously from the viewpoint of the peripheral image before switching to the reference viewpoint VP0, and further, stepwise and continuously from the reference viewpoint VP0 to the viewpoint of the peripheral image after switching. Moved.

  FIG. 11 is a diagram illustrating the transition of the virtual viewpoint when switching from the front mode M1 to the back mode M3. In this case, the virtual viewpoint VP is moved in stages from the viewpoint VP1 of the front image to the reference viewpoint VP0, and then moved in stages from the reference viewpoint VP0 to the viewpoint VP3 of the back image. Then, a plurality of synthesized images are continuously generated in time with the virtual viewpoint VP being moved stepwise in this way. The plurality of generated composite images are sequentially output to the navigation device 20 and displayed on the display 21 continuously in time.

  As a result, as shown in FIG. 12, the virtual viewpoint of the composite image CPm to be displayed gradually moves from the viewpoint VP1 of the front image to the viewpoint VP3 of the back image via the reference viewpoint VP0 on the display 21. Animation expression will be made.

  That is, the composite image CPm displayed on the display 21 initially indicates a region in front of the vehicle 9 similarly to the front image (state ST31). The composite image CPm gradually shows the front side of the vehicle 9 (state ST32), and then shows the entire vehicle 9 in an overhead view (state ST33). At this time, the bird's-eye view image CP0 that is the composite image CPm from the reference viewpoint VP0 is displayed on the display 21. Subsequently, the top and bottom of the subject image indicated by the composite image CPm are reversed (state ST34). The composite image CPm gradually enlarges and shows the area behind the vehicle 9 (state ST35), and finally shows the area behind the vehicle 9 (state ST36).

  Even when changing the viewpoint of the surrounding image to be displayed, by performing such animation expression, the viewpoint of the peripheral image displayed after switching is viewed from which point of view compared to when the viewpoint of the surrounding image is switched instantaneously. The user can intuitively grasp whether the image is a peripheral image.

  FIG. 13 is a diagram illustrating a flow of processing when the operation mode is switched between the operation modes M1, M2, and M3 for displaying the peripheral image. This process is also realized mainly by the control of the display control unit 12.

  First, data indicating the viewpoint position and viewpoint direction of the reference viewpoint VP0 is acquired from the viewpoint data 4a stored in the nonvolatile memory 40 (step S21). Next, data indicating the viewpoint position and the viewpoint direction of each of the movement source viewpoint and the movement destination viewpoint is acquired from the viewpoint data 4a stored in the nonvolatile memory 40 (step S22). The source viewpoint is the viewpoint of the peripheral image displayed before switching. Specifically, the viewpoint of the movement source is the viewpoint of the front image when switching from the front mode M1, the viewpoint of the side image when switching from the side mode M2, and the viewpoint of the back image when switching from the back mode M3.

  Next, a composite image CPm viewed from the viewpoint of the movement source is generated. Then, the generated composite image CPm is displayed on the display 21 instead of the captured image displayed before the operation mode is switched (step S23). In this case, the display content of the display 21 may be gradually switched from the displayed captured image to the composite image CPm using a known method such as alpha blending instead of instantaneously switching.

  Subsequently, a plurality of synthesized images CPm are continuously displayed in time, and an animation expression in which the position of the virtual viewpoint VP moves in a stepwise manner from the source viewpoint to the reference viewpoint VP0 is performed. First, the virtual viewpoint VP is moved toward the reference viewpoint VP0 by an amount corresponding to one frame (step S24). Then, a composite image CPm from this virtual viewpoint VP is generated by the composite image generation unit 32 and displayed on the display 21 (step S25). The processes of step S24 and step S25 are repeated until the virtual viewpoint VP reaches the reference viewpoint VP0.

  When the virtual viewpoint VP reaches the reference viewpoint VP0 (Yes in step S26), subsequently, a plurality of composite images CPm are continuously displayed in time, and the virtual viewpoint VP from the reference viewpoint VP0 to the destination viewpoint is displayed. An animation expression in which the position moves in stages is made. First, the virtual viewpoint VP is moved by an amount corresponding to one frame toward the destination viewpoint (step S27). Then, the composite image CPm from the virtual viewpoint VP is generated by the composite image generation unit 32 and displayed on the display 21 (step S28). The processes of step S27 and step S28 are repeated until the virtual viewpoint VP reaches the destination viewpoint.

  As a result of such processing, an animation expression in which the position of the virtual viewpoint VP moves stepwise from the source viewpoint to the destination viewpoint via the reference viewpoint VP0 is performed on the display 21. Such animation expression is performed over 0.5 seconds to 1.5 seconds, for example.

  When the virtual viewpoint VP reaches the destination viewpoint (Yes in step S29), a captured image to be displayed after switching the operation mode is displayed on the display 21 instead of the composite image CPm (step S30). Also in this case, the display content of the display 21 may be gradually switched from the composite image CPm to the captured image to be displayed by using a known method such as alpha blending instead of instantaneously switching.

  As described above, in the image display system 120, when the display content of the display 21 is switched to the peripheral image, a plurality of composite images CPm are displayed in time sequence, and the destination is moved from the predetermined reference viewpoint VP0. An animation expression in which the position of the virtual viewpoint VP moves to the viewpoint in stages is made. For this reason, the user can intuitively grasp from which viewpoint the peripheral image displayed on the display 21 after the switching is viewed.

  Also, when switching the display content from the map image to the peripheral image, animation is expressed, so the peripheral image displayed after switching is the peripheral image viewed from which viewpoint compared to the case where the peripheral image is displayed instantaneously. The user can intuitively grasp this.

  In addition, when the viewpoint of the peripheral image to be displayed is switched, an animation expression in which the position of the virtual viewpoint VP moves stepwise from the viewpoint of the peripheral image before switching to the viewpoint of the peripheral image after switching through the reference viewpoint. Is made. For this reason, compared with the case where the viewpoint of the peripheral image is switched instantaneously, the user can intuitively understand from which viewpoint the peripheral image displayed after the switching is viewed.

  Further, the viewpoint overlooking the vehicle 9 is set as a reference viewpoint VP0, and an animation expression in which the virtual viewpoint VP moves in stages from the reference viewpoint VP0 to the viewpoint of the peripheral image after switching is performed. For this reason, the viewpoint of the surrounding image after switching can be easily grasped on the basis of the entire vehicle.

<2. Second Embodiment>
Next, a second embodiment will be described. In the first embodiment, the viewpoint overlooking the vehicle 9 is the reference viewpoint VP0. However, in the second embodiment, the viewpoint of the driver of the vehicle 9 is the reference viewpoint VP0. Since the configuration and operation of the image display system 120 of the second embodiment are substantially the same as those of the first embodiment, the following description will focus on differences from the first embodiment.

  In the second embodiment, when switching from the navigation mode M0 to any one of the operation modes M1, M2, and M3 for displaying surrounding images, first, as shown in FIG. 14, a map image NP in the navigation mode M0 is displayed. From the displayed state, a driver viewpoint image CP5 showing a state viewed from the viewpoint of the driver of the vehicle 9 is displayed.

  In this driver viewpoint image CP5, an area in front of the vehicle 9 as viewed from the driver's viewpoint and the interior of the vehicle 9 as viewed from the driver's viewpoint are shown. The driver viewpoint image CP <b> 5 is generated by the composite image generation unit 32 by superimposing a composite image having the driver's viewpoint as the virtual viewpoint VP and an interior image indicating the interior of the vehicle 9. For the interior image, data such as a bitmap or polygon stored in advance in the nonvolatile memory 40 is used. The virtual viewpoint VP is moved stepwise from the viewpoint of this driver to the viewpoint of the peripheral image displayed after switching.

  For example, when switching from the navigation mode M0 to the back mode M3, as shown in FIG. 15, the virtual viewpoint VP is moved stepwise from the viewpoint VP5 of the driver to the viewpoint VP3 of the back image. Then, a plurality of composite images are generated in such a state that the virtual viewpoint VP is moved stepwise, and are displayed on the display 21 continuously in time.

  As a result, on the display 21, an animation expression is made such that the virtual viewpoint of the composite image to be displayed moves stepwise from the viewpoint VP 5 of the driver to the viewpoint VP 3 of the back image. That is, the composite image CPm displayed on the display 21 initially indicates a region in front of the vehicle from the driver's viewpoint VP5, gradually indicates a region on the side of the vehicle 9, and finally displays a region behind the vehicle 9. Will show.

  As described above, in the second embodiment, since the virtual viewpoint moves in a stepwise manner from the viewpoint VP5 of the driver of the vehicle 9 to the viewpoint of the peripheral image, the viewpoint of the peripheral image after switching is set to the driver's viewpoint. The user can easily grasp it based on the viewpoint.

  When the viewpoint of the peripheral image to be displayed is switched to a different viewpoint, the position of the virtual viewpoint VP moves stepwise from the viewpoint of the peripheral image before switching to the viewpoint of the peripheral image after switching through the viewpoint VP5 of the driver. An animated expression is performed.

  In the synthesized image used for the animation expression, the interior image may be superimposed and the interior image to be superimposed may be changed in accordance with the change of the virtual viewpoint VP. As a result, when the virtual viewpoint VP is changed as shown in FIG. 15, a composite image showing the same state as that visually recognized when the driver turns back to the vehicle 9 can be displayed on the display 21. The user can intuitively grasp the position of the moving virtual viewpoint.

  Further, as shown in FIG. 16, before moving the virtual viewpoint VP, the virtual viewpoint VP may be moved after changing the visual field direction of the virtual viewpoint VP to the visual field direction of the destination viewpoint. Also in this case, if the interior image is superimposed on the composite image used for the animation expression, a composite image showing a state of traveling through the vehicle interior of the vehicle 9 can be displayed on the display 21, and the moving virtual viewpoint can be displayed. The user can intuitively grasp the position.

<3. Third Embodiment>
Next, a third embodiment will be described. In the first embodiment, when switching from the navigation mode M0 to any one of the operation modes M1, M2, and M3 for displaying surrounding images, the display content of the display 21 is switched from the map image NP to the overhead image CP0 as it is. (See FIG. 5). On the other hand, in the third embodiment, a photographic image such as an aerial photograph is displayed between the display of the map image NP and the display of the overhead image CP0. Since the configuration and operation of the image display system 120 of the third embodiment are also substantially the same as those of the first embodiment, the following description will focus on differences from the first embodiment.

  FIG. 17 is a block diagram illustrating a configuration of an image display system 120 according to the third embodiment. In the image display system 120 according to the third embodiment, the main body unit 10 of the image generation apparatus 100 includes a wireless communication unit 45.

  The wireless communication unit 45 is connected to a network 80 such as the Internet by wireless communication, and can acquire data from the server device 81 connected to the network 80. The server device 81 stores a photographic image obtained by photographing the ground from the sky such as an aerial photograph or a satellite photograph.

  The image display system 120 also includes a photo acquisition unit 13 as a function realized by the CPU of the control unit 1 performing arithmetic processing according to a program. The photograph acquisition unit 13 transmits a request signal indicating latitude, longitude, scale, and the like to the server device 81 via the wireless communication unit 45. Thereby, the photo acquisition unit 13 can acquire a photo image of an arbitrary scale at an arbitrary position. The photo acquisition unit 13 acquires a photo image indicating a region around the vehicle 9 based on the position (latitude, longitude) of the vehicle 9 obtained by the GPS device included in the navigation device 20. Other configurations of the image display system 120 of the third embodiment are the same as those of the image display system 120 of the first embodiment shown in FIG.

  In the third embodiment, when switching from the navigation mode M0 to any one of the operation modes M1, M2, and M3 for displaying surrounding images, first, as shown in FIG. 18, a map image NP in the navigation mode M0 is displayed. From the displayed state, the photographic image AP is displayed.

  This photographic image AP shows an area around the vehicle 9 at approximately the same scale as the map image NP displayed before switching. Therefore, the map image NP and the photographic image AP indicate substantially the same region with the vehicle 9 as the center. For this reason, even if the display content of the display 21 is switched from the map image NP to the photographic image AP, the user does not feel uncomfortable. Such a photo image AP is acquired from the server device 81 using the wireless communication unit 45 under the control of the photo acquisition unit 13.

  The subject image shown in this photographic image AP is enlarged step by step until it shows an area around the vehicle 9 at approximately the same scale as the overhead image CP0. FIG. 19 is a diagram illustrating transition of images displayed on the display 21 when the navigation mode M0 is switched to the front mode M1. On the display 21, first, a photographic image AP showing a region around the vehicle 9 is displayed at a scale substantially the same as the map image NP (state ST41). Then, the subject image of the photographic image AP is enlarged stepwise (state ST42), and the photographic image AP shows an area around the vehicle 9 at the same scale as the overhead image CP0 (state ST43).

  In this state, the display content of the display 21 is switched from the photographic image AP to the overhead image CP0 that is the composite image CPm viewed from the reference viewpoint VP0 (state ST44). The photographic image AP and the bird's-eye view CP0 before and after the switching show substantially the same area. For this reason, even if the display content of the display 21 is switched from the photographic image AP to the overhead image CP0, there is no sense of incongruity to the user.

  Thereafter, as in the first embodiment, an animation expression is performed such that the virtual viewpoint of the composite image CPm to be displayed moves stepwise from the reference viewpoint VP0 to the viewpoint VP1 of the front image. (States ST45 and ST46).

  FIG. 20 is a diagram illustrating a flow of processing when switching from the navigation mode M0 to any one of the operation modes M1, M2, and M3 for displaying the surrounding image.

  First, data indicating the viewpoint position and viewpoint direction of the reference viewpoint VP0 is acquired (step S31), and then data indicating the viewpoint position and viewpoint direction of the destination viewpoint is acquired (step S32).

  Next, the photograph acquisition unit 13 acquires the position (latitude, longitude) of the vehicle 9 and the scale of the displayed map image NP from the navigation device 20. Then, a request signal including the position (latitude, longitude) and scale of the vehicle 9 is transmitted to the server device 81 using the wireless communication unit 45. Thereby, the photograph acquisition part 13 acquires the photograph image AP which shows the area | region of the periphery of the vehicle 9 from the server apparatus 81 by the substantially the same scale as the map image NP (step S33).

  The acquired photographic image AP is displayed on the display 21 under the control of the display control unit 12 instead of the map image NP (step S34). In this case, the display content of the display 21 may be gradually switched from the map image NP to the photographic image AP by using a known method such as alpha blending instead of instantaneously switching.

  Next, the subject image of the photographic image AP is enlarged stepwise by the control of the display control unit 12 (step S35). When the scale of the area around the vehicle 9 indicated by the photographic image AP becomes substantially the same as the scale of the overhead image CP0 (Yes in step S36), the enlargement of the subject image of the photographic image AP is stopped.

  Then, the overhead image CP0 is displayed on the display 21 instead of the photographic image AP (step S37). Also in this case, the display contents of the display 21 may be gradually switched from the photographic image AP to the overhead image CP0 by using a known method such as alpha blending without instantaneously switching. The subsequent processing is the same as step S14 shown in FIG.

  As described above, in the third embodiment, when the display content of the display 21 is switched from the map image NP to the peripheral image, the photographic image AP showing the area around the vehicle is displayed at substantially the same scale as the map image NP. Is done. Then, after the subject image of the photographic image AP is enlarged step by step until it shows an area around the vehicle 9 at approximately the same scale as the overhead image CP0 that is the composite image CPm viewed from the reference viewpoint VP0, the first embodiment The same animation expression is made.

  For this reason, the viewpoint of the image displayed on the display 21 can be changed smoothly from the map image NP before switching to the peripheral image after switching. As a result, the user can more intuitively understand from which viewpoint the peripheral image displayed on the display 21 after the switching is viewed.

<4. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Below, such a modification is demonstrated. All forms including those described in the above embodiment and those described below can be combined as appropriate.

  In the above embodiment, the operation modes M1, M2, and M3 for displaying the peripheral image have been described on the assumption that the captured image is displayed as the peripheral image. However, the composite image viewed from the virtual viewpoint is displayed as the peripheral image. It may be. For example, as shown in FIG. 21, there may be an operation mode in which a composite image CP6 from a virtual viewpoint in which the entire vehicle 9 is viewed from behind the vehicle 9 is displayed. Even when switching to the operation mode for displaying the composite image in this way, it is possible to perform an animation expression in which the virtual viewpoint moves stepwise from the reference viewpoint to the viewpoint of the composite image to be displayed after the switching.

  Further, in the third embodiment, it has been described that the photo acquisition unit 13 acquires a photo image from the external server device 81. On the other hand, a photographic image may be stored in advance in a nonvolatile memory 40 or a hard disk provided in the navigation device 20 and the photographic image acquisition unit 13 may acquire the photographic image therefrom.

  In the above-described embodiment, the image generation device 100 and the navigation device 20 are described as separate devices. However, the image generation device 100 and the navigation device 20 are arranged in the same casing and are integrated devices. It may be configured as.

  In the above embodiment, the display device that displays the image generated by the image generation device 100 has been described as the navigation device 20, but a general display device that does not have a special function such as a navigation function. It may be.

  In the above embodiment, some of the functions described as being realized by the control unit 1 of the image generation device 100 may be realized by the control unit 23 of the navigation device 20.

  A signal from the shift sensor 71 may be input to the navigation device 20. In this case, the signal may be input to the control unit 1 of the image generation apparatus 100 via the navigation communication unit 42.

  Further, in the above-described embodiment, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, some of these functions are realized by an electrical hardware circuit. Also good. Conversely, some of the functions realized by the hardware circuit may be realized by software.

DESCRIPTION OF SYMBOLS 5 Image | photographing part 9 Vehicle 12 Display control part 13 Photograph acquisition part 21 Display 32 Composite image generation part 81 Server apparatus AP Photograph image CP0 Overhead image NP Map image VP Virtual viewpoint VP0 Reference viewpoint

Claims (6)

An image display system mounted on a vehicle,
Input means for inputting captured images of each of a plurality of cameras arranged in the vehicle;
Image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
When switching display content from a second peripheral image viewed from a second viewpoint different from the first viewpoint to a first peripheral image viewed from the first viewpoint based on the captured image, a plurality of the composite images are temporally continuous. Display means for performing animation representation in which the position of the virtual viewpoint moves stepwise from the second viewpoint to the first viewpoint via a reference viewpoint that is a viewpoint overlooking the vehicle ;
An image display system comprising: An image display system mounted on a vehicle,
Input means for inputting captured images of each of a plurality of cameras arranged in the vehicle;
Image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
Display means capable of displaying a map image around the vehicle;
Photo acquisition means for acquiring a photographic image of the ground taken from above,
With
When the display means switches the display content from the map image to a first peripheral image viewed from a first viewpoint based on the captured image,
After displaying the photographic image showing the area around the vehicle at approximately the same scale as the map image,
Magnifying the subject image of the photographic image in a stepwise manner until it shows an area around the vehicle at approximately the same scale as the composite image viewed from a reference viewpoint that is a viewpoint overlooking the vehicle;
An image display system characterized in that a plurality of the synthesized images are displayed continuously in time, and animation representation is performed in which the position of the virtual viewpoint moves stepwise from the reference viewpoint to the first viewpoint . An image generation device that generates an image to be displayed on a display device mounted on a vehicle,
Input means for inputting captured images of each of a plurality of cameras arranged in the vehicle;
Image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
Output means for outputting and displaying an image on the display device;
With
When the display device switches display content from a second peripheral image viewed from a second viewpoint different from the first viewpoint to a first peripheral image viewed from a first viewpoint based on the captured image,
A plurality of the composite images are displayed continuously in time, and the position of the virtual viewpoint moves stepwise from the second viewpoint to the first viewpoint via a reference viewpoint that is a viewpoint overlooking the vehicle. An image generation apparatus characterized by performing animation expression. An image generation device that generates an image to be displayed on a display device mounted on a vehicle,
Input means for inputting captured images of each of a plurality of cameras arranged in the vehicle;
Image generating means for generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
Output means for outputting and displaying an image on the display device;
Photo acquisition means for acquiring a photographic image of the ground taken from above,
With
The display device, when switching display content from a map image around the vehicle to a first peripheral image viewed from a first viewpoint based on the captured image,
After displaying the photographic image showing the area around the vehicle at approximately the same scale as the map image,
Magnifying the subject image of the photographic image in a stepwise manner until it shows an area around the vehicle at approximately the same scale as the composite image viewed from a reference viewpoint that is a viewpoint overlooking the vehicle;
An image generating apparatus, wherein a plurality of the synthesized images are displayed continuously in time, and animation expression is performed in which the position of the virtual viewpoint moves stepwise from the reference viewpoint to the first viewpoint. An image display method for displaying an image on a display device mounted on a vehicle,
(A) inputting a captured image of each of a plurality of cameras arranged in the vehicle;
(B) generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
(C) the display device displaying an image;
With
In the case where the display content is switched from the second peripheral image viewed from a second viewpoint different from the first viewpoint to the first peripheral image viewed from the first viewpoint based on the captured image, the step (c)
A plurality of the composite images are displayed continuously in time, and the position of the virtual viewpoint moves stepwise from the second viewpoint to the first viewpoint via a reference viewpoint that is a viewpoint overlooking the vehicle. An image display method characterized by performing animation expression. An image display method for displaying an image on a display device mounted on a vehicle,
(A) inputting a captured image of each of a plurality of cameras arranged in the vehicle;
(B) generating a composite image showing a region around the vehicle viewed from an arbitrary virtual viewpoint based on the captured image;
(C) acquiring a photographic image of the ground taken from the sky;
(D) the display device displaying an image;
With
In the step (d), when the display content is switched from a map image around the vehicle to a first peripheral image viewed from a first viewpoint based on the captured image,
After displaying the photographic image showing the area around the vehicle at approximately the same scale as the map image,
Magnifying the subject image of the photographic image in a stepwise manner until it shows an area around the vehicle at approximately the same scale as the composite image viewed from a reference viewpoint that is a viewpoint overlooking the vehicle;
An image display method, wherein a plurality of the synthesized images are displayed continuously in time, and an animation expression is performed in which the position of the virtual viewpoint moves stepwise from the reference viewpoint to the first viewpoint.

Images