WO2018109991A1 - Display device, electronic mirror, display device control method, program, and storage medium - Google Patents

Abstract

The present invention quickly displays video having a sense of depth or a pop-out effect. A depth determination unit (12) determines a depth indicating how far back or forward to display a visual object in an output video relative to display screens (20, 20') in a video display device (25), and an output video generation unit (13) generates an output video by alternatingly arranging pixels which are offset by a pixel offset amount corresponding to the depth determined by the depth determination unit (12).

Description

Display device, electronic mirror, display device control method, program, and recording medium

The present invention relates to a display device for an electronic mirror that displays an image taken by an in-vehicle camera, and an electronic mirror using the display device.

In recent years, vehicles have been developed that display images around the vehicle taken by a camera installed on the vehicle on a display screen arranged in front of the driver. The combination of the camera and the display screen is called an electronic mirror. For example, a back display screen that displays an image immediately after the vehicle is included in the electronic mirror. The electronic mirror is used as a substitute for a room mirror or a door mirror (side mirror).

Some electronic mirrors display 3D video (stereoscopic video) (see Patent Documents 1 and 2). 3D images have a natural depth. Therefore, it is easy for the driver to intuitively understand the position and speed of the object (viewing target object) reflected on the display screen. In addition, the viewpoint of the driver is farther in the 3D image displayed on the far side than the display screen when viewed from the driver as compared with the 2D image. Therefore, it is easy for the driver to recognize his / her visual object displayed on the display screen by moving his / her line of sight from the front of the vehicle to the display screen.

Japanese Published Patent Publication “JP 2013-026770” (published February 4, 2013) Japanese Patent Publication “Japanese Patent Laid-Open No. 2003-339060” (published on November 28, 2003)

However, since a conventional electronic mirror needs to be equipped with a twin-lens camera in order to display 3D video, the cost is increased. In addition, since the conventional electronic mirror combines two captured images captured by the twin-lens camera by the parallax mapping process, it takes time to display the 3D image on the display screen of the electronic mirror. Since the driver drives the vehicle while watching the 3D video displayed on the display screen, it is a problem that the timing for displaying the 3D video on the display screen is delayed.

An object of one embodiment of the present invention is to quickly display an image having a sense of depth or popping out.

A display device according to one embodiment of the present invention is disposed in front of an occupant including a driver of a vehicle, and a visual target included in a photographed image around the photographed vehicle is farther from the occupant than a display screen. Depth for setting a depth from the display screen for displaying the object to be recognized on the back side or the near side, which is displayed on the back side or the near side close to the occupant. A setting unit, and a generation unit that generates a first output video including pixel data for left eye and pixel data for right eye according to the depth set by the depth setting unit from the captured video. I have.

In addition, a display device according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and the sighted object included in a captured image around the vehicle is displayed on the occupant rather than the display screen. A depth of the display screen that allows the object to be recognized to be recognized on the back side or the near side. The data of the position shifted by the pixel shift amount corresponding to the depth set by the depth setting unit and the depth set by the depth setting unit in the captured image is the pixel data for the left eye and the pixel data for the right eye And generating units that generate alternately arranged first output images, and the display screen is inclined in a direction away from the occupant.

In addition, a display device according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and the sighted object included in a captured image around the vehicle is displayed on the occupant rather than the display screen. A depth of the display screen that allows the object to be recognized to be recognized on the back side or the near side. The data of the position shifted by the pixel shift amount corresponding to the depth set by the depth setting unit and the depth set by the depth setting unit in the captured image is the pixel data for the left eye and the pixel data for the right eye A generation unit that generates first output video arranged alternately, and the display screen uses a narrow viewing angle liquid crystal that shields light in a direction deviated by a predetermined angle or more from the front of the display screen. A liquid crystal display .

In addition, a display device according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and the sighted object included in a captured image around the vehicle is displayed on the occupant rather than the display screen. A depth of the display screen that allows the object to be recognized to be recognized on the back side or the near side. The data of the position shifted by the pixel shift amount corresponding to the depth set by the depth setting unit and the depth set by the depth setting unit in the captured image is the pixel data for the left eye and the pixel data for the right eye A generation unit that generates first alternately output video images, and is not visible when the display screen is viewed from the front of the display screen, and when viewed from a direction deviated by a predetermined angle or more from the front. The visible image It is displayed on the serial display screen.

In addition, a display device according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and the sighted object included in a captured image around the vehicle is displayed on the occupant rather than the display screen. A depth of the display screen that allows the object to be recognized to be recognized on the back side or the near side. The data of the position shifted by the pixel shift amount corresponding to the depth set by the depth setting unit and the depth set by the depth setting unit in the captured image is the pixel data for the left eye and the pixel data for the right eye And a generation unit that generates first output video arranged alternately, and an optical film that blocks light in a direction shifted by a predetermined angle or more from the front of the display screen is pasted on the display screen. .

A display device according to an aspect of the present invention is a display device that is disposed in front of an occupant including a driver of a vehicle and displays a visual target object included in a photographed image around the photographed vehicle. A detection unit that detects a position of the occupant's face with respect to the display screen; and a position at which the trimmed image is cut out in the captured video is determined according to the position of the occupant's face with respect to the display screen detected by the detection unit. A trimming position determination unit; and a generation unit that generates an output video from the trimmed image cut out from the captured video in accordance with the position determined by the trimming position determination unit.

In addition, the display device control method according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and a visual object included in a captured image of the surrounding of the vehicle is displayed on a display screen. Is a control method for a display device that is displayed so as to be recognized on the far side far from the occupant or the near side close to the occupant, from the display screen for recognizing the visible object on the far side or the near side. A depth setting step for setting a depth of the first eye, and a first output including pixel data for the left eye and pixel data for the right eye according to the depth set in the depth setting step in the captured image Generating a video.

In addition, the display device control method according to one embodiment of the present invention is arranged in front of an occupant including a driver of a vehicle, and a visual object included in a captured image of the surrounding of the vehicle is displayed on a display screen. Is a control method for a display device that displays so as to be recognized on the far side away from the occupant or on the near side close to the occupant, the display screen being inclined in a direction away from the occupant, A depth setting step for setting a depth from the display screen for recognizing an object on the back side or the near side, and a pixel shift according to the depth set in the depth setting step in the captured image The generation step of generating the first output video alternately arranged as the pixel data for the left eye and the pixel data for the right eye includes data at positions shifted by the amount.

The display device control method according to one aspect of the present invention is a display device that is disposed in front of an occupant including a driver of a vehicle and displays a visual target object included in a captured image of the surroundings of the vehicle. And detecting the position of the occupant's face relative to the display screen, and trimming the captured video according to the position of the occupant's face relative to the display screen detected in the detection step. A trimming position determining step for determining a position to cut out the image; and a generating step for generating an output video from the trimmed image cut out from the captured video in accordance with the position determined in the trimming position determining step.

According to one embodiment of the present invention, it is possible to quickly display an image having a sense of depth or a feeling of popping out.

1 is a block diagram illustrating a configuration of an electronic mirror according to Embodiment 1. FIG. It is a figure which shows schematically arrangement | positioning of the imaging part of the electronic mirror and video display apparatus which concern on Embodiment 1. FIG. It is a figure which shows a part of vehicle inner side provided with the electronic mirror which concerns on Embodiment 1, and is a figure which shows arrangement | positioning of a video display apparatus. 1 is a diagram illustrating an arrangement of a video display device according to Embodiment 1. FIG. (A)-(c) is a figure which shows the depth of the output image | video respectively displayed on the video display apparatus, (a) is depth zero, (b) is negative depth, (c) Is positive in depth. (A) shows a photographed video, (b) shows a right-eye video included in an output video trimmed from the photographed video, and (c) shows a left-eye video contained in an output video trimmed from the photographed video. (D) shows the output image produced | generated from the image | video for right eyes, and the image | video for left eyes. It is the figure which numbered and showed the pixel of the picked-up image. It is a figure which shows the output image produced | generated from the picked-up image shown in FIG. 7, and is the figure which numbered the pixel of the corresponding picked-up image with respect to the pixel of an output image. 4 is a flowchart showing a flow of video processing executed by the video control device for the electronic mirror according to the first embodiment. 6 is a table showing a correspondence relationship between the depth of an output video and a pixel shift amount in the electronic mirror according to the first and second embodiments. It is a figure which shows arrangement | positioning of the video display apparatus which concerns on Embodiment 2. FIG. It is a block diagram which shows the structure of the electronic mirror which concerns on Embodiment 2. FIG. 10 is a flowchart illustrating a flow of video processing executed by the video control device for an electronic mirror according to the second embodiment. (A)-(c) is a figure which shows the range of the output image trimmed (cut out) from each picked-up image, (a) is the range of the reference output image, (b) is a driver | operator's range. This is the range of the output video when the eye position moves to the left, and (c) is the range of the output video when the driver's viewpoint moves upward. It is a block diagram which shows the structure of the electronic mirror which concerns on Embodiment 3. FIG. (A) (b) is a figure which shows the Example of the viewing angle control in the electronic mirror which concerns on Embodiment 5, (a) shows the Example of the viewing angle control by a directional diffusion film, (b) An example of viewing angle control by a luminance light-shielding film is shown.

Embodiment 1
Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.

(Electronic mirror 1)
The configuration of the electronic mirror 1 according to the present embodiment will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the electronic mirror 1. As shown in FIG. 1, the electronic mirror 1 includes an imaging unit 11 (imaging unit), a video control device 10, a parallax barrier-compatible video display device 25 (display device) (hereinafter referred to as a video display device 25), and a parallax barrier device. 30, a vehicle information acquisition device 40, an eye tracking camera 50, a driver information acquisition device 52, and a parallax barrier control device 55. The video control device 10 and the parallax barrier-compatible video display device 25 according to the present embodiment constitute a display device.

FIG. 2 shows a vehicle to which the electronic mirror 1 according to this embodiment is attached. As shown in FIG. 2, the imaging unit 11 is attached to the rear part of the vehicle so as to face the direction opposite to the traveling direction of the vehicle. However, the imaging unit 11 may be attached so as to face the traveling direction of the vehicle. Specifically, the imaging unit 11 is a monocular camera. The imaging unit 11 includes a plurality of monocular cameras, and includes at least three monocular cameras that respectively capture the front rear, left rear, and right rear of the vehicle. The imaging unit 11 may further include a monocular camera for a drive recorder that captures the front of the vehicle. For example, a monocular camera directed to the rear right side is arranged on the right door of the vehicle. This monocular camera captures the right rear side of the vehicle. While the electronic mirror 1 is activated, that is, while the vehicle is running, the imaging unit 11 captures the periphery of the vehicle (in FIG. 2, the rear of the vehicle) and generates captured video data. The captured image includes a visual target that exists around the vehicle. In some cases, the captured image includes a visual target in the vehicle. For example, when the electronic mirror 1 is used as a substitute for a room mirror, the imaging unit 11 includes a monocular camera that captures the interior of the vehicle. The imaging unit 11 outputs the generated captured video to the video control device 10.

The video control device 10 acquires a captured video from the imaging unit 11 and generates an output video to be output to the video display device 25 from the acquired captured video. The output video includes a video corresponding to the right eye of the driver (hereinafter referred to as a video for the right eye) and a video corresponding to the left eye of the driver (hereinafter referred to as a video for the left eye). As illustrated in FIG. 1, the video control apparatus 10 includes a depth determination unit 12 (depth setting unit) and an output video generation unit 13 (generation unit). Each part of the video control apparatus 10 will be described later.

The video display device 25 displays the output video generated by the video control device 10 on the display screens 20 and 20 ′. The video display device 25 may be a liquid crystal display device, for example. The video display device 25 displays on the display screen 20 an output video based on the captured video taken by the imaging unit 11. The video display device 25 displays an output video based on a digital video including information such as speed, engine speed, and used gear on the display screen 20 ′.

FIG. 3 shows a part of the inside of the vehicle shown in FIG. 2, particularly the instrument panel of the driver's seat (cockpit). As shown in FIG. 3, the display screens 20 and 20 ′ of the video display device 25 are arranged in the vicinity of the driver's seat inside the vehicle. For example, the display screen 20 of the video display device 25 may be incorporated in a dashboard. Alternatively, it may be arranged in the dashboard and / or on the ceiling of the vehicle (position of the room mirror). The video display device 25 displays the output video acquired from the video control device 10 on the display screens 20 and 20 ′ arranged at the positions illustrated in FIG. 3.

The parallax barrier device 30 is disposed in front of each display screen 20, 20 ′ of the video display device 25. The parallax barrier device 30 controls the viewing angle of the output video displayed on the display screens 20 and 20 ′ of the video display device 25 by being controlled by the parallax barrier control device 55. Alternatively, when the video display device 25 is another image directional liquid crystal display device capable of controlling the viewing angle of the video, the viewing angle of the output video is controlled by controlling the directivity of the backlight of the video display device 25. May be. In this case, the electronic mirror 1 may not include the parallax barrier device 30 and the parallax barrier control device 55.

The vehicle information acquisition device 40 acquires various information indicating the state of the vehicle. The information acquired by the vehicle information acquisition device 40 includes, for example, a seat offset amount (corresponding to the seat position), a vehicle speed, and gear information.

The eye tracking camera 50 tracks the driver's face or eye position. The eye tracking camera outputs the detected driver face or eye position information to the driver information acquisition device 52 and the parallax barrier control device 55.

The driver information acquisition device 52 (detection unit) acquires various information indicating the states of the driver and other passengers. The information acquired by the driver information acquisition device 52 includes, for example, information on the driver's face or eye position. The driver information acquisition device 52 acquires information on the driver's face or eye position detected by the eye tracking camera 50. The driver information acquisition device 52 may include, for example, a biosensor attached to a vehicle seat. In this configuration, the driver information acquisition device 52 acquires information such as the body temperature, sweating, and movement of the occupant seated on the seat from the biometric sensor.

The parallax barrier control device 55 controls the parallax barrier device 30 based on information on the driver's face or eye position detected by the eye tracking camera 50.

(Configuration of video control apparatus 10)
As shown in FIG. 1, the video control apparatus 10 includes a depth determination unit 12 and an output video generation unit 13.

The depth determination unit 12 determines the depth of the output video based on at least one of information indicating the vehicle state acquired by the vehicle information acquisition device 40 and information indicating the driver state acquired by the driver information acquisition device 52. To decide. The depth of the output video is a parameter representing the depth of the position of the visual target object in the output video when viewed from the driver (that is, the distance from the display screen 20, 20 'to the visual target object). The depth can be positive or negative or zero. In the present embodiment, the depth determination unit 12 determines the pixel shift amount Δpix between the right-eye pixel data and the left-eye pixel data based on the determined depth of the output video. The depth and the pixel shift amount Δpix will be described later.

Based on the pixel shift amount Δpix determined by the depth determination unit 12, the output video generation unit 13 converts pixel data at a position shifted from each other by the pixel shift amount Δpix from one captured video, pixel data for the left eye, and right As the eye pixel data, an output video is generated by alternately arranging the pixel data. The output video generated in this way has a sense of depth (when the depth is positive) or a pop-out feeling (when the depth is negative) like a 3D video when viewed through the parallax barrier device 30. Further, the processing time for generating the output video is shortened compared to the processing time for generating the 3D video because the parallax mapping processing is not performed. In addition, since there is no parallax like 3D video in the output video, the driver can capture it as a plane image of the distance moved by the depth, and by capturing the image instantly by focusing on that plane, A visual target can be found. In addition, since the viewpoint movement with respect to the front visual field can be reduced depending on the depth, the visual recognition object in the output video can be quickly recognized. Therefore, the driver can make a judgment and act more quickly based on information included in the output video than when viewing a normal 2D video.

The output video generation unit 13 outputs the generated output video to the video display device 25. Note that the output video generation unit 13 may adjust the angle of view or the trimming range of the output video according to which video display device 25 displays the output video. For example, the output video generation unit 13 may display an image on the video display device 25 used as a substitute for a room mirror rather than an output video output to another video display device 25 used as a substitute for a door mirror (side mirror). An output video having a large corner may be output. This is because the rearview mirror has a role of transmitting a wide range of information behind the vehicle to the driver. In addition, when the electronic mirror 1 includes a twin-lens camera, the output video generation unit 13 is a parallax captured by the twin-lens camera based on the depth (the pop-out amount p and the depth amount d) determined by the depth determination unit 12. A 3D image may be generated from two captured images with a general 3D image generation method.

(Arrangement of video display device 25)
FIG. 4 shows an example of the arrangement of the video display device 25 (see FIG. 3) arranged in the dashboard. The video display device 25 is arranged instead of a mechanical meter such as a speedometer. The video display device 25 displays the output video output from the video control device 10 on the display screen 20 ′. Since the entire image display device 25 is smaller than the mechanical meter, the space in the dashboard can be expanded as compared with the conventional case. In the present embodiment, the display screen 20 ′ of the video display device 25 faces the driver. Therefore, when the depth of the output video changes, the viewpoint of the driver who views the output video moves in the traveling direction of the vehicle or in the opposite direction. When the depth of the output video displayed on the video display device 25 is deeper than zero, when the driver moves the viewpoint from the front of the vehicle to the meter displayed on the display screen 20 ′ of the video display device 25, The moving distance is shortened. Therefore, the driver can easily feel the movement of the line of sight.

(Determination method of pixel shift amount Δpix)
A method in which the depth determining unit 12 determines the pixel shift amount Δpix between the right-eye pixel data and the left-eye pixel data will be described with reference to FIGS. . 5A to 5C show the display screen 20 of the video display device 25 displaying the output video. The viewing angle of the output video is controlled by the parallax barrier device 30 and the parallax barrier control device 55. In FIGS. 5A to 5C, the right eye of the driver sees only the right-eye video composed of the right-eye pixel data included in the output video. The driver's left eye sees only the left-eye video composed of pixel data for the right eye included in the output video. The driver recognizes as if the visual target is present at the position where the line of sight of the right eye and the line of sight of the left eye shown in FIGS.

(A) of FIG. 5 shows an output image when the depth is 0, that is, the pixel shift amount Δpix is 0. In this case, the visual target is on the display screen 20 of the video display device 25 as viewed from the driver. That is, the driver views the output video in the same manner as when the 2D video is displayed on the video display device 25. In this case, the parallax barrier control device 55 stops or disables the parallax barrier device 30.

(B) of FIG. 5 shows an output image when the depth is negative, that is, the pixel shift amount Δpix is negative, and the visually recognized object appears to jump out of the display screen 20. In this case, the depth of the output video is represented by the pop-out amount p. As shown in FIG. 5B, when the depth is negative, the visual target is in front of the display screen 20 of the video display device 25 as viewed from the driver. In this case, the driver feels that the visually recognized object is popping out from the display screen 20 of the video display device 25. For example, when the video display device 25 displays the opening screen of the electronic mirror 1, the depth determination unit 12 may make the depth of the output video smaller than zero.

(C) of FIG. 5 shows an output image when the depth is positive, that is, the pixel shift amount Δpix is positive. In this case, the depth of the output video is represented by a depth amount d. As shown in (c) of FIG. 5, when the depth is positive, the visual target is located behind the display screen 20 of the video display device 25 as viewed from the driver. In this case, the viewpoint of the driver is farther than the display screen 20 of the video display device 25.

The depth determination unit 12 may change the depth of the output video based on the vehicle state information acquired from the vehicle information acquisition device 40. For example, when the vehicle is moving at high speed, the driver's viewpoint is usually far from the vehicle. Therefore, the depth determination unit 12 may increase the depth of the output image as the speed of the vehicle increases or as the used gear increases. Further, when the R gear is used, the depth determination unit 12 may make the output depth shallower, that is, reduce the pixel shift amount Δpix than when other gears are used. This makes it easier for the driver to visually recognize the vicinity of the vehicle. Further, the depth determination unit 12 may increase the depth of the output video as the position of the sheet is closer to the display screen 20 of the video display device 25. Thereby, irrespective of the position of the seat, the distance from the driver to the visual recognition object in the output video can be appropriately maintained.

Also, the depth determination unit 12 may change the depth of the output video based on the driver information acquired from the driver information acquisition device 52. For example, when the driver information acquisition device 52 fails in eye tracking, the depth determination unit 12 may set the depth of the output video to a default value, that is, zero (fail safe). In addition, the driver may be able to manually change the depth of the output video to zero. Thereby, it is possible to prevent the output image from being seen twice by the driver and other occupants.

(Modification of depth determination unit 12)
In the present embodiment, the depth of the output video determined by the depth determination unit 12 is constant in the vertical direction (vertical direction) and the horizontal direction (horizontal direction) of the output video. However, the method by which the depth determination unit 12 determines the depth of the output video is not limited to the method described in this embodiment.

In a modification, the depth determination unit 12 may change the depth of the output video in the vertical direction (vertical direction) or the horizontal direction (horizontal direction) of the output video (see Embodiment 2).

In addition, when the depth determination unit 12 sets the depth of the output video to zero, the parallax barrier control device 55 stops, that is, invalidates the operation of the parallax barrier device 30.

In another modification, the depth determination unit 12 may change the depth of the output video according to the surrounding environment detected by a sensor or the like. For example, the depth determination unit 12 may determine the depth of the output video according to the brightness of the environment (for example, backlight or twilight) or the time zone (for example, daytime or night).

In another modification, the depth determination unit 12 may determine the depth of the output video according to information included in the output video. For example, the depth determination unit 12 may analyze which of the sky, road, and other vehicles is included in the output video, and may determine the depth of the output video according to the analysis result. In particular, when the output video includes many roads or other vehicles, the depth determination unit 12 may determine the depth of the output video according to the distance from the vehicle to the road or other vehicles.

In another modification, the depth determination unit 12 may determine the depth of the output video according to the arrangement of the display screens 20 and 20 ′ of the video display device 25. For example, when the video display device 25 arranged near the driver's face or a part of the face (eyes, nose, etc.) displays the output video, the depth determination unit 12 may increase the depth of the output video. Good.

In another modification, the depth determination unit 12 may determine the depth of the output video according to the position of the driver's face or a part of the face (eyes, nose, etc.). For example, when the driver's face moves back and forth, left and right, or up and down (for example, head swing and stretch), the depth determination unit 12 determines the depth of the output video according to the position of the driver after the movement. May be.

In another modification, the depth determination unit 12 may determine the depth of the output video according to the driver's action. For example, when the driver performs an operation for moving the vehicle backward (for example, putting an arm on the passenger seat, opening a window or door of the driver seat), the depth determining unit 12 determines the type of the driver's operation. The depth of the output video may be determined.

(Output video generation method)
A method of generating the output video by the output video generation unit 13 will be described with reference to FIGS. 6A to 6D, FIG. 7, and FIG.

(A) of FIG. 6 shows the imaging | photography image | video which the imaging part 11 image | photographed. The captured image shown in FIG. 6A includes a visual recognition object. The captured video shown in FIG. 6A has a pixel size of (1920 + α) × 1080, for example. Here, α is equal to the pixel shift amount Δpix.

FIG. 6B shows the range of the right-eye video included in the output video trimming range. FIG. 6C shows the range of the left-eye video included in the output video trimming range.

As shown in FIG. 6C, the range of the left-eye video is shifted to the left by the pixel shift amount Δpix determined by the depth determination unit 12 from the range of the right-eye video. The output video generation unit 13 is (1920 + Δpix) × 1080 from the left end of the left eye video range shown in FIG. 6C to the right end of the right eye video range shown in FIG. Trim the pixel data of the area. Then, the output video generation unit 13 extracts pixel data at positions shifted by the pixel shift amount Δpix from the trimmed pixel data, and alternately arranges them as pixel data for the left eye and pixel data for the right eye. , Generate output video.

(D) in FIG. 6 shows an output video in which pixel data are alternately arranged. In FIG. 6D, the position of the visual target object included in the right-eye image and the position of the same visual object included in the left-eye image are shifted from each other by Δpix. That is, the size of the output video is 1920 × 1080, similar to the range of the right-eye video and the range of the left-eye video.

A method for generating the output video by the output video generation unit 13 will be described in more detail with reference to FIGS. FIG. 7 is a diagram showing numbers assigned to pixels of a captured video acquired by the output video generation unit 13. FIG. 8 is a diagram showing an output video generated from the captured video shown in FIG. 7. The output video pixel is associated with the corresponding right-eye video pixel number and the corresponding left-eye video pixel. It is the figure which numbered and showed.

In FIG. 7, the pixels corresponding to the right-eye image are numbered sequentially from the first in the horizontal direction, and the pixels corresponding to the left-eye image are sequentially numbered from the 51st in the horizontal direction. Waving. That is, in the example shown in FIG. 7, the pixel shift amount Δpix between the right-eye video and the left-eye video is 50 pixels. Although FIG. 7 illustrates only the pixels in the top row of the right-eye video and the left-eye video, the pixels in the other rows of the captured video are the same as the pixels in the top row. By rule, a number is given.

As shown in FIG. 8, the output video generation unit 13 includes the odd-numbered pixels (1, 3,...) Of the right-eye video and the even-numbered pixels (2, 4,...) Of the left-eye video. .) Are alternately arranged in the horizontal direction to generate an output video. Although FIG. 8 illustrates only the pixels in the top row of the output video, the pixels in the other rows of the output video are also arranged in the same rule as the pixels in the top row. That is, in this embodiment, the pixel shift amount Δpix in the trimming range does not change in the vertical direction and the horizontal direction of the output video. The viewing angle of the output video displayed by the video display device 25 is controlled by the parallax barrier device 30. Therefore, the right eye of the driver sees the right-eye video (video composed of even-numbered pixel data) included in the output video. On the other hand, the driver's left eye sees a left-eye image (image composed of odd-numbered pixel data) included in the output image. The video for the right eye and the video for the left eye are synthesized in the driver's brain. Therefore, the driver feels as if he is watching only one output image. In addition, since the size of one pixel is very small, the driver hardly recognizes the space between pixels.

(Operation of video control apparatus 10)
The operation of the video control apparatus 10 according to the present embodiment will be described using FIG. FIG. 9 is a flowchart showing the operation of the video control apparatus 10. As illustrated in FIG. 9, the video control device 10 acquires a captured video captured by the imaging unit 11 (S2). In addition, the depth determination unit 12 acquires information indicating the state of the vehicle from the vehicle information acquisition device 40, and acquires information indicating the state of the driver from the driver information acquisition device 52 (S4).

The depth determination unit 12 determines the depth of the output video based on at least one of information indicating the state of the vehicle and information indicating the state of the driver. Further, a pixel shift amount Δpix corresponding to the determined depth of the output video is determined with reference to a table (see FIG. 10) described later (S6, setting step). The depth determination unit 12 notifies the output video generation unit 13 of information on the determined pixel shift amount Δpix.

The output video generation unit 13 generates an output video by the above-described method based on the information of the pixel shift amount Δpix determined by the depth determination unit 12 (S8, generation step). The output video generation unit 13 outputs the generated output video to the video display device 25. The video display device 25 displays the output video received from the output video generation unit 13 on the display screen 20 (S10). Thus, the operation of the video control apparatus 10 is finished.

(Correspondence between depth and pixel shift amount)
FIG. 10 is a table showing a correspondence relationship between the depth (the pop-out amount p and the depth amount d) set in the output video and the pixel shift amount Δpix. In FIG. 10, Δx (unit: mm) is a shift width corresponding to different pixel shift amounts Δpix. In FIG. 10, the pop-out amount p (see FIG. 5B) indicates how much the object to be visually recognized in the output video protrudes forward from the display screen 20, that is, in the direction toward the driver. Represents what looks like. In addition, the depth amount d (see FIG. 5C) indicates how much the object to be visually recognized in the output video is offset from the display screen 20 in the back direction, that is, the driver's line-of-sight direction. Represents what it looks like.

[Embodiment 2]
Another embodiment of the present invention is described below with reference to FIG. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

Depending on the arrangement of the video display device, the display screen of the video display device may not face the driver. In this case, since the driver sees the display screen of the video display device at an angle, it may be difficult to accurately grasp the sense of depth of the output video. In the present embodiment, the depth of the output video is changed according to the tilt angle of the display screen of the video display device. This makes it easy to accurately grasp the sense of depth of the output video.

FIG. 11 shows an arrangement of the display screen 20 ′ of the video display device 25 according to the present embodiment. As can be seen from comparison with the arrangement of the display screen 20 of the video display device 25 according to the first embodiment, the display screen 20 ′ of the video display device 25 according to the present embodiment has an ideal surface facing the driver. It is inclined with respect to it. The angle formed between the display screen 20 ′ of the video display device 25 and the ideal plane is θ. In FIG. 11, the upper end of the display screen 20 ′ comes closest to the front and tilts away from the driver as it approaches the lower end of the display screen 20 ′, but the display screen 20 ′ tilts in the opposite direction. It may be. That is, the upper end of the display screen 20 'may be inclined farther from the driver and closer to the driver as it approaches the lower end of the display screen 20'.

In the arrangement of the video display device 25 in the present embodiment, a space (a triangle indicated by a broken line in FIG. 11) is generated between the dashboard and the display screen 20 ′ of the video display device 25. This space can be used for arranging electronic devices, HUD (Head-Up-Display), and the like, similarly to the space in the dashboard. Moreover, since the incident angle of sunlight with respect to the display screen 20 ′ of the video display device 25 is reduced and the reflection of sunlight on the display screen 20 ′ is reduced, the length of the hood that shields sunlight is made shorter than before. can do.

In the present embodiment, the depth determination unit 12 of the video control apparatus 10 refers to the table shown in FIG. 10, and shifts the pixel shift amount Δpix ((d in FIG. 6) in the vertical direction (the direction indicated by the arrow of the height h). )) Is gradually changed. Specifically, the depth determination unit 12 determines the pixel shift amount Δpix corresponding to the depth d determined according to the position a of the height h by referring to the table. Here, the ideal depth d corresponding to the position a is the distance from the display screen 20 ′ to the ideal plane where the driver wants to recognize the visual target.

In FIG. 11, the ideal depth d is calculated by a calculation formula of d = L × sin θ. Here, L is the length from the lower end of the display screen 20 ′ of the video display device 25 to the position a measured. Thus, in the present embodiment, the pixel shift amount Δpix is determined according to the ideal depth d and position a. Therefore, the visual recognition object in the output video can be seen on the ideal surface for the driver.

[Embodiment 3]
The following will describe another embodiment of the present invention with reference to FIGS. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

In this embodiment, the electronic mirror changes the field angle range of the captured video displayed as the output video according to the position of the driver's face or eyes.

FIG. 12 is a block diagram showing a configuration of the electronic mirror 2 according to the present embodiment. 12, in addition to the configuration of the video control apparatus 10 described in the first embodiment, the video control apparatus 210 of the electronic mirror 2 according to the present embodiment includes an angle-of-view range determination unit 14 (trimming position determination unit). ). The video control device 210 and the parallax barrier-compatible video display device 25 according to the present embodiment constitute a display device.

The angle-of-view range determination unit 14 of the electronic mirror 2 acquires information on the driver's face or eye position from the driver information acquisition device 52. Then, the view angle range determination unit 14 determines the view angle range of the output video according to the position of the driver's face or eyes. More specifically, the angle-of-view range determination unit 14 determines a range of pixel data (trimming image range) to be extracted in order to generate an output video from a captured video.

The output video generation unit 13 according to the present embodiment generates an output video by trimming a part of the captured video according to the angle of view range determined by the angle of view range determination unit 14 ((( a) to (d)).

(Operation of video control device 210)
The operation of the video control apparatus 210 according to the present embodiment will be described with reference to FIG. FIG. 13 is a flowchart showing the operation of the video control apparatus 210.

As shown in FIG. 13, the video control device 210 acquires a captured video captured by the imaging unit 11 (S2).

Next, the angle-of-view range determination unit 14 acquires information indicating the position of the driver's face or eyes detected by the driver information acquisition device 52 (S32, detection step). Subsequently, the angle-of-view range determination unit 14 determines the trimming range of the output video based on the position of the driver's face or eyes (S34, trimming position determination step). The angle-of-view range determination unit 14 outputs information on the determined trimming range to the output video generation unit 13. When the driver information acquisition device 52 fails in the eye tracking process, the view angle range determination unit 14 may return the view angle of the output video to a preset default value (fail safe).

The depth determination unit 12 acquires information indicating the state of the vehicle and information indicating the state of the driver (S4). The specific example of the information acquired by the depth determination unit 12 has been described in the first embodiment. Next, the depth determination unit 12 determines the pixel shift amount Δpix as described in the first embodiment (S6).

In the present embodiment, the output video generation unit 13 generates an output video based on the trimming range information determined by the angle-of-view range determination unit 14 and the pixel shift amount Δpix determined by the depth determination unit 12. (S8 '). The output video generation unit 13 outputs the generated output video to the video display device 25. The video display device 25 displays the output video received from the output video generation unit 13 on the display screens 20 and 20 '(see FIG. 3) arranged in the vicinity of the driver's seat (S10). Thus, the operation of the video control device 210 is completed.

(How to determine the field of view of the output video)
A process executed by the angle-of-view range determination unit 14 to determine the angle-of-view range of the output video will be specifically described with reference to FIGS. (A) to (c) of FIG. 14 are diagrams showing the field angle range of the output video i3 according to the position of the driver's face or eyes.

(A) of FIG. 14 shows the field angle range of the output image i3 when the driver's face or eyes are at a predetermined reference position. Here, the reference position may be set, for example, to the position of the driver's face or eyes when the driver is seated in an appropriate posture on the vehicle seat.

(B) in FIG. 14 shows the field angle range of the output video i3 when the driver's face or eyes move leftward from the reference position shown in (a) in FIG. In this case, the field angle range of the output video i3 in the captured video moves to the right from the position shown in FIG.

(C) in FIG. 14 shows the field angle range of the output video i3 when the driver's face or eyes move upward from the reference position shown in (a) in FIG. In this case, the field angle range of the output video i3 in the captured video moves downward from the position shown in FIG.

According to the configuration of the present embodiment, when the driver's face or eyes move, the field angle range (trimming position) of the output video displayed by the video display device 25 changes. Therefore, the driver can naturally change the viewing area by moving his / her face or eyes as in the case of looking at the mirror. Furthermore, according to the configuration of the present embodiment, the angle of view of the output video can be changed according to the direction that the driver wants to see without changing the shooting direction of the imaging unit 11, that is, the monocular camera. In the present embodiment, when displaying an output video obtained by shifting pixels from a video captured by the imaging unit 11, that is, a monocular camera, the angle of view range of the output video is changed according to the position of the driver's face or eyes. Although the processing has been described, the output video may be a 2D video or a 3D video described in the fourth embodiment later.

[Embodiment 4]
The following will describe another embodiment of the present invention with reference to FIG. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

In the present embodiment, the type of video displayed by the video display device is switched according to at least one of the state of the vehicle and the state of the driver.

FIG. 15 is a block diagram showing a configuration of the electronic mirror 3 according to the present embodiment. As shown in FIG. 15, the video control device 310 of the electronic mirror 3 further includes a display switching unit 15 as compared with the configuration of the video control device 10 of the electronic mirror 1 according to the first embodiment. Although not shown, the electronic mirror 3 further includes a 3D video generation device that generates a normal 3D video. Note that the video control apparatus 310 may further include the angle-of-view range determination unit 14 described in the second embodiment. The video control device 310 and the parallax barrier-compatible video display device 25 according to the present embodiment constitute a display device.

As shown in FIG. 15, the output video generation unit 13 generates an output video by the method described in the first embodiment. The output video generation unit 13 outputs the generated output video to the display switching unit 15.

The 3D video generation device generates a 3D video using two video data having parallax with each other. Specifically, when the electronic mirror 3 includes a twin-lens camera, the 3D video generation device can generate a 3D video from two captured videos with parallax captured by the twin-lens camera. Note that a method for generating a 3D image from two captured images having a parallax is well known, and thus description thereof is omitted here. The 3D video generation device outputs the generated 3D video to the display switching unit 15. The 3D video generation device may generate a 3D video as an opening screen when the electronic mirror 3 is activated.

The display switching unit 15 acquires information indicating the state of the vehicle from the vehicle information acquisition device 40. In addition, the display switching unit 15 acquires information indicating the state of the driver from the driver information acquisition device 52. Then, the display switching unit 15 outputs an image to be output to the image display device 25 according to at least one of the vehicle state and the driver state as an output image (first output image) and a 2D image (second output image). ) And 3D video (third output video).

As described above, the display switching unit 15 also acquires 2D video data. The 2D video data may be, for example, video content data recorded on a recording device (not shown). Alternatively, the 2D video may be generated from a captured video generated by the imaging unit 11. In this case, the depth is not set in the 2D video like the output video.

What kind of video the display switching unit 15 outputs to the video display device 25 is not particularly limited. If a specific example is given, the display switching part 15 may switch the image | video output to the video display apparatus 25 according to the case classification demonstrated below.

First, when the vehicle is traveling, the display switching unit 15 outputs the output video generated by the output video generation unit 13 to the video display device 25. In this case, the video display device 25 can quickly display an image having a sense of depth or a feeling of popping out.

Second, when the vehicle is stopped, the display switching unit 15 causes the video display device 25 to match the captured video (2D video) captured by the imaging unit 11 with the output format of the video display device 25. Output to. In addition, the display switching unit 15 outputs 2D video other than captured video, such as a television program, a navigation screen, and content, to the video display device 25. In these cases, the parallax barrier control device 55 stops the parallax barrier device 30 of the electronic mirror 3.

Third, when the video display device 25 displays the opening screen of the electronic mirror 3, the display switching unit 15 outputs the 3D video generated by the 3D video generation device to the video display device 25.

[Embodiment 5]
Another embodiment of the present invention will be described below with reference to FIG. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted.

In the configuration described in the second embodiment, the field angle range of the output video is dynamically changed according to the position of the driver's face so that the driver can always see a normal output video. In this case, the output image seen by the person (passenger) seated in the passenger seat of the vehicle changes between normal state, reverse view, and double image. Therefore, the passenger may feel such a change in the output video uncomfortable.

In this embodiment, the viewing angle of the output video is controlled so that the passenger cannot see the output video. Passengers may not be able to see any video, or any video other than the output video.

(A) of FIG. 16 is a figure which shows one Example of viewing angle control. In this embodiment, the video display device 25 includes a directional diffusion film that is a kind of optical film. The graph shown in FIG. 16A shows the diffusion viewing angle characteristic of the output video in this embodiment. As shown in FIG. 16A, when the viewing angle θ is small, the output video is hardly diffused, but as the viewing angle increases, the output video is strongly diffused by the directional diffusion film. Accordingly, a normal output image can be seen by the driver D, while a double image of the right hand output image and the left hand output image can always be seen by the person P sitting in the passenger seat. However, since the double image seen by the person P has no luminance flicker, the discomfort of the person P is reduced.

(B) of FIG. 16 is a figure which shows the other Example of viewing angle control. In this embodiment, the video display device 25 includes a viewing angle control film that is a kind of optical film. The graph shown in FIG. 16B shows the luminance viewing angle characteristics of the output video in this embodiment. As shown in FIG. 16B, the output image is more strongly shielded by the viewing angle control film as the viewing angle θ increases. Accordingly, the driver D can normally see the output video, while the person P sitting in the passenger seat does not see the output video and sees a black screen. Therefore, at the viewing angle of the person P, even if the output video changes, the person P hardly senses such a change in the output video.

Note that, as shown in FIG. 16B, the luminance viewing angle control film may block light at both the positive and negative viewing angles θ, or may block output video that enters the eyes of a person seated in the passenger seat. Thus, the output image may be shielded by only one of the positive and negative viewing angles θ.

The video display device 25 may be a liquid crystal display device including a narrow viewing angle liquid crystal. In this case, the video display device 25 can control the range of the viewing angle θ of the output video by controlling the transmittance of the narrow viewing angle liquid crystal. In this configuration, an output image with a large viewing angle θ can be shielded without a viewing angle control film. Therefore, since the person sitting in the passenger seat cannot see the output video, the person sitting in the passenger seat does not feel uncomfortable due to the change in the output image.

The video display device 25 may display a so-called bale view. In this configuration, a veil view image that is not visible when viewed from the front of the display screen 20 and is visible when viewed from a direction deviated by a predetermined angle or more from the front is displayed on the display screen 20. Cannot see the output video. Therefore, the person seated in the passenger seat does not feel uncomfortable due to the change in the output image.

[Example of software implementation]
In the first to fifth embodiments, the electronic mirror device for a vehicle has been described. However, the control blocks (particularly the video control devices 10, 210, 310) of the electronic mirrors 1, 2, 3 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, It may be realized by software using a CPU (Central Processing Unit). For example, a smartphone equipped with a 3D display (video display device 25), an eye tracking camera 50 (driver information acquisition device 52), and a photographing camera (imaging unit 11) is connected to the electronic mirror 1 by a CPU and an application included in the smartphone. , 2 and 3 can be realized.

In the latter case, the video control apparatus 10 includes a CPU that executes instructions of a program that is software for realizing each function, and a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by the computer (or CPU). Alternatively, a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. Note that one embodiment of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[Summary]
The display devices ( video control devices 10, 210, 310 and the parallax barrier video display device 25) according to the first aspect of the present invention are arranged in front of a passenger including a driver of the vehicle and are photographed around the vehicle. A display device for displaying a visual recognition object included in a captured image so as to be recognized on the far side farther from the occupant than the display screen (20, 20 ') or on the near side of the occupant, A depth setting unit (depth determining unit 12) for setting a depth (amount of protrusion d, a depth amount d) from the display screen for recognizing an object on the back side or the front side, and the depth from the captured image. A generation unit (output video generation unit 13) that generates a first output video including pixel data for the left eye and pixel data for the right eye corresponding to the depth set by the height setting unit.

According to the above configuration, the depth (depth) is set in the output video, and the pixel data for the left eye and the pixel for the right eye shifted in the pixel arrangement direction by the pixel shift amount corresponding to the set depth. An output video containing data is generated. For the occupant (user) who sees the output video, depending on the depth set in the output video, the object to be viewed in the output video appears behind the display screen or pops out of the display screen.

The display device according to aspect 2 of the present invention is the display apparatus according to aspect 1, in which the captured image is an image captured by a monocular camera, and the generation unit is set by the depth setting unit in the captured image. Pixel data at a position shifted by a pixel shift amount (Δpix) corresponding to the depth is alternately arranged as the left-eye pixel data and the right-eye pixel data to generate the first output video. May be.

According to the above configuration, an output video having a sense of depth or popping out can be generated from a shot video shot using a monocular camera without requiring a binocular camera as in the prior art.

In the display device according to aspect 3 of the present invention, in the aspect 1 or 2, the depth setting unit may set the depth based on at least one of the state of the vehicle and the state of the occupant. .

According to the above configuration, an appropriate depth can be set in the output video based on at least one of the vehicle state and the occupant state.

The display device according to aspect 4 of the present invention is the display device according to any one of the aspects 1 to 3, wherein the visual object generated from the first output video generated by the generation unit and the captured video is displayed on the display screen. The second output image displayed so as to be recognized at the position of the image and the third output image generated from the photographed image and reproducing the binocular parallax of the human are displayed as the state of the vehicle and the occupant. A display switching unit (15) that selects an output video to be displayed on the display screen based on at least one of the states may be further provided.

According to the above configuration, it is possible to appropriately switch the output video to be displayed based on at least one of the vehicle state and the occupant state.

The display device according to aspect 5 of the present invention is the display apparatus according to aspect 3, wherein the state of the vehicle is one of the speed of the vehicle and the position of a seat included in the vehicle, or the state of the occupant is the occupant. It may be either the position of the face relative to the display screen or the binocular parallax of the occupant.

According to the above configuration, an appropriate depth can be set in the output video based on any one of the vehicle speed, the seat position, the occupant's face position, and the occupant's binocular parallax.

The display devices ( video control devices 10, 210, and 310 and the parallax barrier video display device 25) according to aspect 6 of the present invention are arranged in front of a passenger including a driver of the vehicle and are photographed around the vehicle. A display device for displaying a visual recognition object included in a captured image so as to be recognized on the far side farther from the occupant than the display screen (20, 20 ') or on the near side of the occupant, A depth setting unit (depth determination unit 12) for setting a depth from the display screen for recognizing an object on the back side or the near side, and the depth set by the depth setting unit in the captured image A generation unit (output video generation unit 13) that generates first output video alternately arranged as pixel data for the left eye and pixel data for the right eye, with the data shifted by the pixel shift amount corresponding to With Both the display screen is inclined in a direction away from the occupant.

In the above configuration, corresponding to the tilt of the display screen, the pixel farther from the occupant is set to a shallower depth from the display screen, and the pixel closer to the occupant is set to a deeper depth from the display screen. can do. Therefore, regardless of the pixel position in the tilt direction of the display screen, the output image of the tilted display screen can be shown facing the occupant.

In the display device according to aspect 7 of the present invention, in the aspect 6, the depth setting unit sets the depth from the display screen to a pixel farther from the occupant according to the inclination of the display screen. It may be set shallowly.

According to the above configuration, the distance from the occupant to the image becomes shorter as the image corresponds to the pixel set with a shallow depth from the display screen. Therefore, when the display screen is tilted, set the pixel farther away from the occupant to a shallower depth from the display screen, and the pixel closer to the occupant to set the depth from the display screen deeper. Can do. Therefore, regardless of the pixel position in the tilt direction of the display screen, the output image of the tilted display screen can be shown facing the occupant.

The display device (video control device 310 and parallax barrier video display device 25) according to aspect 8 of the present invention is arranged in front of a passenger including a driver of a vehicle and is included in a captured video of the surroundings of the vehicle. A display device that displays the object to be recognized on the back side farther from the occupant than the display screen (20, 20 ′) or the near side to the occupant. A depth setting unit (depth determination unit 12) for setting a depth from the display screen to be recognized on the side or the near side, and a pixel corresponding to the depth set by the depth setting unit in the captured image The data at the position shifted by the shift amount includes a generation unit (output video generation unit 13) that generates first output video that is alternately arranged as pixel data for the left eye and pixel data for the right eye. ,the above示画 surface is a liquid crystal display device using a narrow viewing angle liquid crystal to block light from the front of the display screen to the predetermined angle or more displacement direction.

According to the above configuration, by limiting the field of view of the output video by the liquid crystal display device using the narrow viewing angle liquid crystal, only the left eye pixel is visible to the occupant's left eye, and the occupant's right eye is visible. Only the pixel for the right eye can be seen, so that the occupant (user) who sees the output video displays the object to be viewed in the output video according to the depth set in the output video. It appears behind the screen or appears to jump out of the display screen.

The display device (video control device 310 and parallax barrier video display device 25) according to aspect 9 of the present invention is disposed in front of the occupant including the driver of the vehicle and is included in the captured video around the vehicle. A display device that displays the object to be recognized so as to be recognized on the back side farther from the occupant than the display screen or on the near side near the occupant, and on the back side or the near side. A depth setting unit for setting a depth from the display screen to be recognized, and data of a position shifted by a pixel shift amount corresponding to the depth set by the depth setting unit in the captured image is the left eye A generation unit that generates first output video arranged alternately as pixel data for the right eye and pixel data for the right eye, and is not visible when the display screen is viewed from the front of the display screen. Or An image visible when viewed from a predetermined angle or more displacement direction, is displayed on the display screen.

According to the above configuration, only the occupant viewing the display screen from a direction other than the front can view the output video. For an occupant who looks at the display screen from a direction other than the front, the object to be visually recognized in the output video appears behind the display screen or jumps out of the display screen.

The display device (video control device 310 and parallax barrier video display device 25) according to aspect 10 of the present invention is disposed in front of an occupant including the driver of the vehicle and is included in the captured video of the surroundings of the vehicle. A display device that displays the object to be recognized so as to be recognized on the back side farther from the occupant than the display screen or on the near side near the occupant, and on the back side or the near side. A depth setting unit for setting a depth from the display screen to be recognized, and data of a position shifted by a pixel shift amount corresponding to the depth set by the depth setting unit in the captured image is the left eye And a generation unit that generates first output video arranged alternately as pixel data for the right eye and pixel data for the right eye, and the display screen is shifted by a predetermined angle or more from the front of the display screen Optical film for blocking light to have been attached.

According to the configuration described above, a passenger viewing the display screen from the front of the display screen can see the visual target in the output video behind the display screen or jump out of the display screen. On the other hand, an occupant viewing the display screen from a direction deviated by a predetermined angle or more from the front of the display screen cannot see the output video. Therefore, it is possible to prevent an occupant who views the display screen from a direction deviated by a predetermined angle or more from the front of the display screen from giving unpleasant feeling due to flickering of the luminance of the output video.

The display device (video control device 210 and parallax barrier video display device 25) according to aspect 11 of the present invention is disposed in front of an occupant including the driver of the vehicle and is included in the captured video of the surroundings of the vehicle. A display device that displays a visual recognition target, a detection unit that detects a position of the occupant's face relative to the display screen, and a position detected by the detection unit relative to the display screen of the occupant's face, A trimming position determining unit that determines a position to cut out a trimmed image in the captured video; a generation unit that generates an output video from the trimmed image cut out from the captured video according to the position determined by the trimming position determining unit; It has.

According to the above configuration, an output image with an appropriate depth can be shown to the occupant according to the position of the occupant's face.

In the display device according to aspect 12 of the present invention, in the aspect 11, the trimming position determination unit is configured to capture the image in a direction opposite to the direction of the movement when the occupant's face moves relative to the display screen. The position where the trimmed image is cut out in the video may be moved. According to the above configuration, the occupant moves the face with respect to the display screen so as to view the normal mirror, so that the trimmed image displayed as the output video displayed on the display screen is displayed. The range (trimming range) can be easily changed.

An electronic mirror according to aspect 13 of the present invention images the display device according to any one of aspects 1 to 12 above, the front of the vehicle, the front rear, the left rear, or the right rear, and displays the captured image as described above. An imaging unit (imaging unit 11) that outputs to the apparatus.

According to the above configuration, it is possible to generate an output video based on a shot video obtained by shooting the front, rear of the front, left rear, or right rear of the vehicle.

According to the control method of the display device according to the fourteenth aspect of the present invention, the visual object included in the photographed video around the vehicle that is disposed in front of the occupant including the driver of the vehicle is displayed on the display screen. A display device control method for displaying on a back side far from an occupant or a near side close to the occupant, wherein the depth from the display screen is set to allow the visual recognition object to be recognized on the back side or the near side. A first output image including pixel data for the left eye and pixel data for the right eye corresponding to the depth set in the depth setting step in the depth setting step for setting the depth and the captured image Generating step. According to said structure, there exists an effect similar to the display apparatus which concerns on the said aspect 1. FIG.

The display device control method according to the aspect 15 of the present invention is arranged in front of a passenger including a driver of the vehicle, and the object to be viewed included in the photographed video around the vehicle is more than the display screen. A display device control method for displaying on a far side away from an occupant or a near side closer to the occupant, wherein the display screen is inclined in a direction away from the occupant, and A depth setting step for setting the depth from the display screen to be recognized on the back side or the near side, and a pixel shift amount corresponding to the depth set in the depth setting step in the captured image A generation step of generating the first output video in which the data at the shifted position is alternately arranged as the pixel data for the left eye and the pixel data for the right eye is included. According to said structure, there exists an effect similar to the display apparatus which concerns on the said aspect 6. FIG.

The display device control method according to aspect 16 of the present invention is a control of a display device that is arranged in front of an occupant including a driver of a vehicle and displays a visual object included in a photographed image around the photographed vehicle. A detection step of detecting a position of the occupant's face relative to the display screen, and a trimming image in the captured video according to the position of the occupant's face relative to the display screen detected in the detection step. A trimming position determining step for determining a position to be cut out, and a generating step for generating an output video from the trimmed image cut out from the captured video in accordance with the position determined in the trimming position determining step. According to said structure, there exists an effect similar to the display apparatus which concerns on the said aspect 11. FIG.

The display device ( video control device 10, 210, 310) and the electronic mirror (1, 2, 3) according to each aspect of the present invention may be realized by a computer. An electronic mirror control program that causes the electronic mirror to be realized by a computer by operating as each part (software element) provided, and a computer-readable recording medium that records the electronic mirror control program also fall within the scope of the present invention.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

1, 2, 3 Electron mirror 10, 210, 310 Video control device (display device)
11 Imaging unit (monocular camera, imaging unit)
12 Depth determination part (depth setting part)
13 Output video generator (generator)
14 Angle-of-view range determination unit (trimming position determination unit)
DESCRIPTION OF SYMBOLS 15 Display switching part 20, 20 'Display screen 25 Parallax barrier corresponding | compatible video display apparatus (display apparatus)
52 Driver information acquisition device (detection unit)

Claims (18)

1. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A display device that displays as
   A depth setting unit for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
   A generating unit configured to generate a first output image including pixel data for left eye and pixel data for right eye corresponding to the depth set by the depth setting unit from the captured image. Display device.
2. The above video is a video taken with a monocular camera,
   The generator generates pixel data at a position shifted by a pixel shift amount corresponding to the depth set by the depth setting unit in the captured video, the pixel data for the left eye and the pixel for the right eye The display device according to claim 1, wherein the first output video is generated by alternately arranging data.
3. 3. The display device according to claim 1, wherein the depth setting unit sets the depth based on at least one of the state of the vehicle and the state of the occupant.
4. The first output video generated by the generation unit;
   A second output video generated from the captured video and displayed so that the visual recognition object is recognized at a position of the display screen;
   One of the third output video generated from the captured video and reproducing human binocular parallax,
   4. The display switching unit according to claim 1, further comprising: a display switching unit that selects an output image to be displayed on the display screen based on at least one of the state of the vehicle and the state of the occupant. Display device.
5. The state of the vehicle is one of the speed of the vehicle and the position of a seat included in the vehicle, or the state of the occupant is the position of the occupant's face relative to the display screen and the binocular parallax of the occupant. The display device according to claim 3, wherein the display device is any one.
6. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A display device that displays as
   A depth setting unit for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
   In the captured image, data at positions shifted by the pixel shift amount corresponding to the depth set by the depth setting unit are alternately arranged as pixel data for the left eye and pixel data for the right eye. A generation unit that generates one output video,
   The display device, wherein the display screen is inclined in a direction away from the occupant.
7. The display device according to claim 6, wherein the depth setting unit sets the depth from the display screen shallower as the pixel is located farther from the occupant according to the inclination of the display screen. .
8. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A display device that displays as
   A depth setting unit for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
   In the captured image, data at positions shifted by the pixel shift amount corresponding to the depth set by the depth setting unit are alternately arranged as pixel data for the left eye and pixel data for the right eye. A generation unit that generates one output video,
   The display device is a liquid crystal display device using a narrow viewing angle liquid crystal that shields light in a direction deviated by a predetermined angle or more from the front of the display screen.
9. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A display device that displays as
   A depth setting unit for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
   In the captured image, data at positions shifted by the pixel shift amount corresponding to the depth set by the depth setting unit are alternately arranged as pixel data for the left eye and pixel data for the right eye. A generation unit that generates one output video,
   A display device that displays on the display screen an image that is not visible when the display screen is viewed from the front of the display screen and that is visible when viewed from a direction deviated by a predetermined angle or more from the front.
10. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A display device that displays as
    A depth setting unit for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
    In the captured image, data at positions shifted by the pixel shift amount corresponding to the depth set by the depth setting unit are alternately arranged as pixel data for the left eye and pixel data for the right eye. A generation unit that generates one output video,
    An optical film that shields light in a direction shifted by a predetermined angle or more from the front of the display screen is attached to the display screen.
11. A display device that is arranged in front of an occupant including a driver of a vehicle and displays a visual target object included in a photographed image around the photographed vehicle,
    A detection unit for detecting a position of the occupant's face with respect to the display screen;
    A trimming position determination unit that determines a position to cut out a trimmed image in the captured video according to the position of the occupant's face relative to the display screen detected by the detection unit;
    A display device comprising: a generation unit configured to generate an output video from the trimmed image cut out from the captured video according to the position determined by the trimming position determination unit.
12. The trimming position determination unit, when the occupant's face moves relative to the display screen, moves a position where a trimmed image is cut out in the captured image in a direction opposite to the movement direction. Item 12. The display device according to Item 11.
13. A display device according to any one of claims 1 to 12,
    An electronic mirror comprising: a photographing unit that photographs the front, rear of the vehicle, rear of the left side, or rear of the right side of the vehicle, and outputs the captured image to the display device.
14. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A method for controlling a display device to display as follows:
    A depth setting step for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
    Generating a first output image including pixel data for left eye and pixel data for right eye corresponding to the depth set in the depth setting step in the captured image. A control method for a display device.
15. The object to be visually recognized, which is placed in front of the occupant including the driver of the vehicle and included in the photographed image around the vehicle, is recognized on the far side farther from the occupant than the display screen or near the occupant. A method for controlling a display device to display as follows:
    The display screen is tilted away from the occupant,
    A depth setting step for setting a depth from the display screen that allows the visual recognition object to be recognized on the back side or the near side;
    In the captured image, data at positions shifted by a pixel shift amount corresponding to the depth set in the depth setting step are alternately arranged as pixel data for the left eye and pixel data for the right eye. And a generation step of generating a first output video.
16. A control method for a display device that displays a visual target object that is arranged in front of an occupant including a driver of a vehicle and is included in a photographed image around the photographed vehicle.
    A detection step of detecting the position of the occupant's face relative to the display screen;
    A trimming position determining step for determining a position to cut out a trimmed image in the captured video according to the position of the occupant's face relative to the display screen detected in the detecting step;
    And a generating step of generating an output video from the trimmed image cut out from the captured video in accordance with the position determined in the trimming position determining step.
17. A program for causing a computer to execute the control method for a display device according to any one of claims 14 to 16.
18. A computer-readable recording medium on which the program according to claim 17 is recorded.

Images