JP2022114602A - Display control device, display device, display system, and image display control method - Google Patents

Abstract

To suppress occurrence of visual discomfort, when a passenger(s) other than a driver is/are not onboard, felt by the driver due to seeing images on which distortion correction suitable for the passenger(s) is performed.SOLUTION: A display control device which is placed in a vehicle and controls image display on display devices which show images to the driver and passengers other than the driver of the vehicle includes a control unit having a function of changing a correction parameter which is used to correct image data of images so that distortions on images seen by the driver or the passengers are reduced, and when a display device's primary viewer is a passenger and a non-primary viewer is the driver, the control unit, as the result of a determination based on at least one of information on whether the passenger is onboard and information on where a passenger is seated, sets the correction parameter value to a first value of M-wp which is suitable for the passenger when it has been determined that the passenger is onboard, and sets the correction parameter value to a second value of S-wp which is suitable for the driver and differs from the first value, when it has been determined that the passenger is not onboard.SELECTED DRAWING: Figure 5

Description

The present invention relates to a display control device, a display device, a display system, an image display control method, and the like.

It is conceivable that an image displayed by a display device mounted on a vehicle (vehicle display device) may be viewed by passengers other than the driver.

Regarding this point, for example, in [0004] of Patent Document 1, "... a fellow passenger is riding (sitting) on the front passenger seat in addition to the driver, and the passenger sitting on the front passenger seat If you want to visually recognize the display image consisting of vehicle information etc., in addition to the above-mentioned head-up display device (that is, the head-up display device for the driver's seat), the inside of the instrument panel that is in front of the passenger sitting in the passenger seat It is conceivable to separately arrange another head-up display device for the front passenger seat."

As an example of processing for correcting the distortion of an image to be displayed, for example, Patent Document 2 discloses warping processing in a HUD device (correction processing in which distortion having characteristics opposite to the distortion caused by an optical system is applied to image data in advance). ) is described.

JP 2012-108470 A JP 2015-87619 A

The inventor's studies have revealed the following problems. It may be the case, for example, that the display device displayed an image for viewing by an occupant, but the occupant was not present.

Since the display image is supposed to be viewed by the passenger, the image data is often subjected to correction processing for correcting the distortion of the image when viewed from the front by the passenger.

When the driver is not present and the driver views the displayed image from an angle, for example, to obtain information, the image assumes that the driver views the image from the front. A person may perceive the image to be highly distorted and may feel uncomfortable.

The aforementioned Patent Documents 1 and 2 do not mention this problem, nor do they describe any countermeasures therefor.

One of the objects of the present invention is to prevent the driver from feeling a sense of incongruity when viewing an image subjected to distortion correction suitable for the passenger when there is no passenger other than the driver.

Other objects of the present invention will become apparent to those skilled in the art by referring to the following exemplary aspects and best mode, as well as the accompanying drawings.

In order to facilitate an understanding of the general outline of the invention, the following illustrates embodiments in accordance with the invention.

In a first aspect, the display control device comprises:
A display control device that controls image display of a display device that is mounted in a vehicle and causes an image to be viewed by a driver of the vehicle or an occupant other than the driver,
a control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image seen by the driver or the passenger;
The control unit
When the display device has the occupant as a primary viewer and the driver as a secondary viewer,
If the occupant is present as a result of determination based on at least one of the occupant presence information and the occupant arrangement information, the value of the correction parameter is set to a first value suitable for the occupant. ,
When the passenger is absent, the value of the correction parameter is set to a second value different from the first value and suitable for the driver.

In the first aspect, when the passenger who is the main viewer of the image displayed by the display device is absent, the correction parameter is changed so that the image is suitable for the driver who is the secondary viewer. .

For example, if the displayed image has been subjected to correction processing for correcting the distortion of the image when viewed from the front by the occupant, the driver will feel uncomfortable when viewing the displayed image from an oblique direction. However, in this aspect, the correction parameters are changed to those suitable for the driver who looks at the image from an oblique angle, and an image corrected to reduce the distortion seen from the driver is displayed. Therefore, the sense of discomfort of the driver is suppressed.

In a second aspect dependent on the first aspect,
The occupant as the primary viewer,
An occupant sitting in the front passenger seat of the vehicle,
or
It may be an occupant sitting on a seat in the back row of the vehicle.

In the second aspect, the ``occupant who is the main viewer of the image displayed by the display device'' described in the first aspect may include an occupant in the passenger seat in the front row of the vehicle. clarifies that occupants in the back row of a vehicle may be included.

For example, there may be a case where a display device for the front passenger seat is installed inside the dashboard that is in front of (preferably in front of) the passenger sitting on the front passenger seat, or on the surface of the dashboard. In other words, the display device in this case is a display device that is arranged in a position directly facing the passenger seated in the front passenger seat (however, the term "directly facing" is interpreted broadly and includes positions other than the exact front). In this case, the occupant sitting on the front passenger seat becomes "the main viewer of the display device."

In addition, for passengers seated in the rear seats, it is preferable that the position in front of the passenger is directly facing the passenger in order to provide, for example, navigation information and entertainment information. , including positions other than the exact front), the display device may be installed inside or on the surface of the dashboard. In this case, an occupant sitting in the rear seat becomes "the main viewer of the display device".

In the above case, when "the passenger who is the main viewer of the display device" is absent, the driver must look at the image displayed by the display device obliquely, and no countermeasures must be taken. distortion will increase. By applying the present invention, image distortion is reduced and discomfort is suppressed.

In a third aspect depending on the first or second aspect,
The correction parameter is
a warping parameter for pre-distortion correction that preliminarily gives the image data a distortion having characteristics opposite to the distortion that occurs when the image is displayed;
as well as,
When the outline of the display area of the image is a rectangle including rectangles and squares, and the rectangle appears to be deformed into a trapezoid or a parallelogram according to the angle at which the viewer views the image, the transformation parameters for deformation suppression correction, which transforms a trapezoid or parallelogram into a rectangle;
as well as,
3D image correction parameters for correcting three-dimensional image distortions in the case of a three-dimensional (3D) display.

The third aspect clarifies that the correction parameter may be a warping parameter, a transformation parameter for deformation suppression correction, or a correction parameter for a 3D image.

By performing the above correction using an appropriate correction parameter, image distortion can be effectively reduced, and discomfort can be suppressed.

In a fourth aspect depending on any one of the first to third aspects,
The control unit
When the operation of the display device is started, the image may be hidden before the correction parameter value is determined, and the image may be displayed after the correction parameter value is determined.

According to the fourth aspect, by displaying the image after determining the correction parameter to be applied, the viewer's sense of discomfort due to display distortion is reduced compared to, for example, the case where the correction parameter is changed after the image is displayed. can do.

In a fifth aspect depending on any one of the first to third aspects,
The control unit
When the operation of the display device is started,
When the display device has the driver as the main viewer, correcting the image data using a correction parameter suitable for the driver and starting image display,
When the display device has the passenger as the main viewer, the image may be displayed after the value of the correction parameter is determined.

In the fifth aspect, for the display device for which the driver is the main viewer, it is possible to immediately start displaying an image and present information useful for driving the vehicle without delay, for example. On the other hand, for a display device in which a passenger is the main viewer, it is possible to display an image with little distortion by opening the display after the presence or absence of the passenger is confirmed and the value of the correction parameter is determined.

In a sixth aspect depending on any one of the first to third aspects,
The control unit
When the operation of the display device is started,
When the display device has the driver as the main viewer, correcting the image data using a correction parameter suitable for the driver and starting image display,
When the occupant is the main viewer, the display device sets the correction parameter to the first value and performs display while determining whether or not the correction parameter needs to be changed. If so, the value of the correction parameter may be changed.

According to the sixth aspect, even when the passenger is the main viewer, the image can be displayed quickly without waiting for the determination of the value of the correction parameter.

In a seventh aspect depending on any one of the first to sixth aspects,
The control unit
When changing the value of the correction parameter during image display, the value before change may be gradually changed to the value after change.

According to the seventh aspect, when it is necessary to change the correction parameter, by gradually changing the value, it is possible to suppress abrupt changes in the appearance of the image. Discomfort can be reduced.

In an eighth aspect, the display device comprises
a display control device according to any one of the first to seventh aspects;
an image data correction unit that corrects the image data of the image using the correction parameter so as to suppress distortion of the image as viewed by the driver or the passenger;
a display that displays an image based on the corrected image data;
have

According to the eighth aspect, when there is no passenger other than the driver, the image corrected for distortion suitable for the driver viewing the image from an oblique direction can be displayed on the display unit, thereby reducing discomfort. be able to. As a result, it is possible to realize a high-performance display device capable of displaying an image with little distortion to both the driver and the passengers.

In a ninth aspect depending on the eighth aspect,
The display device
The head-up device further includes an optical system for projecting display light of an image displayed on the display unit onto a projection target member provided in the vehicle, and has a function of allowing at least one of the driver and the passenger to visually recognize a virtual image. It may be a display device,
or
The display device may display a real image on the display unit as a display panel and allow at least one of the driver and the passenger to visually recognize the real image.

The ninth aspect clarifies that the display device may be a HUD device or a display device. It is possible to realize a high-performance HUD device or display device capable of displaying an image with less distortion to both the driver and the passengers.

As the display device, a liquid crystal device (for example, a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, may be used.

In a tenth aspect, the display system comprises:
A plurality of display devices according to the eighth aspect are arranged along the width direction of the vehicle in the front portion, which is the portion on the windshield side of the vehicle,
one of the plurality of display devices is a first display device with the driver as a viewer;
At least one of the other display devices is a second display device for a passenger in the front passenger seat or a passenger in the rear row as a main viewer and the driver as a secondary viewer.

According to the tenth aspect, a plurality of display devices are arranged in the front part of the vehicle along the width direction of the vehicle (in other words, the left-right direction as seen from the viewer facing the displayed image). to build a display system.

For example, in each display device, information that can specify the main viewer of the display device is input and set in advance. For example, when there are two display devices, for the display device installed at the position corresponding to the driver, inputting, for example, "1" on the initial setting screen indicates that the driver is the main viewer. is set. For the other display device, for example, when "2" is entered on the initial setting screen, it is set that the passenger positioned next to the driver's seat in the front row is the main viewer.

After this setting is applied to all the display devices, if at least one of information on the presence or absence of passengers and information on the arrangement of passengers is input in parallel to each display device, the control unit of each display device changes the correction parameter. is determined, and the correction parameters are automatically optimized. Therefore, it is possible to realize a high-performance display system that is easy to use and can display an image with less distortion for both the driver and the passengers.

In an eleventh aspect depending on the tenth aspect,
The plurality of display devices may be head-up display devices,
or
The plurality of display devices may be a display device,
or
A head-up display device and a display device may be mixed in the plurality of display devices.

According to the aspect of FIG. 11, the display system can be freely constructed by using only the HUD device, only the display device, or a combination of the HUD device and the display device. Therefore, a user-friendly display system can be constructed.

In a twelfth aspect, the image display control method includes:
An image display control method for a display device mounted on a vehicle, comprising:
a first step of detecting at least one of the presence or absence and arrangement of passengers, including passengers or drivers;
a second step of determining an image correction parameter for the displayed content based on the detection result, and changing the image correction parameter if change is necessary;
a third step of correcting the image data of the display content using the determined or changed image correction parameters;
a fourth step of performing at least one of displaying a virtual image by a head-up display device and displaying a real image by a display device;
including.

According to the twelfth aspect, for example, when there is no passenger other than the driver, it is possible to display a distortion-corrected image suitable for a viewer such as a driver viewing the image from an oblique direction. It is possible to reduce the discomfort of the person. As a result, for example, it is possible to realize a high-performance display device capable of displaying an image with little distortion to both the driver and the passengers.

Those skilled in the art will readily appreciate that the illustrated embodiments in accordance with the present invention may be further modified without departing from the spirit of the invention.

FIG. 1(A) shows two display devices (HUD devices) for a driver and a passenger in the front seat of a vehicle that seats two in the front row. FIG. 1B is a diagram showing an example of the path of the display light, and FIG. 1B is a diagram showing an example of the path of the display light of each display device when the passenger is absent. FIG. 2A is a diagram showing a configuration example of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effect of warping correction processing. FIG. 3A is a diagram showing an example of an image display before deformation suppression correction that converts a trapezoid or parallelogram as the contour shape of the image display area into a rectangle (including rectangles and squares); 3B) is a diagram showing an example of an image display after deformation suppression correction, and FIG. 3C is a diagram showing a main part configuration example of a HUD device having a deformation suppression correction function. FIG. 4 is a diagram showing an example of a display system in a vehicle with three seats in the front row. 5A and 5B are diagrams showing examples of basic display control in the display system of FIG. FIGS. 6A and 6B are diagrams showing an example of display control when the display device starts operating. FIGS. 7A and 7B are diagrams showing another example of display control when the display device starts operating. FIGS. 8A and 8B are diagrams showing still another example of display control when the display device starts operating. FIG. 9(A) is a diagram showing an example in which one of three display devices is used to display an image to an occupant seated in the rear seat, and FIG. FIG. 4 is a diagram showing an example in which two display devices located close to the driver are both used to display an image to the driver. FIG. 10A shows an example of a display system using a display device (such as a liquid crystal display device), and FIG. 10B shows an example of a display system in which a HUD device and a display device are mixed. FIG. 4 is a diagram showing; FIG. 11 is a diagram showing an example of gradually changing the correction parameter value when changing the correction parameter. FIG. 12 is a diagram illustrating an example of the main configuration of a display device and an example of a display system. FIG. 13 is a flowchart illustrating an example of a display control procedure.

The best mode described below is used for easy understanding of the invention. Accordingly, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below.

(First embodiment)
FIG. 1(A) shows two display devices (HUD devices) for a driver and a passenger in the front seat of a vehicle that seats two in the front row. FIG. 1B is a diagram showing an example of the path of the display light, and FIG. 1B is a diagram showing an example of the path of the display light of each display device when the passenger is absent.

In FIG. 1 , the X direction indicates the left-right direction of the vehicle 1 , the Y direction indicates the height direction of the vehicle 1 , and the Z direction indicates the front (front-rear direction) of the vehicle 1 . As illustrated, the vehicle 1 is a front row two-seat passenger car with the steering wheel 8 on the right side.

Two display devices (here, HUD devices) 100 a and 100 b are arranged along the width direction (X direction) of the vehicle 1 in the front portion, which is the portion on the windshield 2 side of the vehicle 1 . A HUD device 100 a is arranged at a position corresponding to the driver's seat 6 (a position facing the driver's seat 6 ), and a HUD device 100 b is arranged at a position corresponding to the passenger's seat 9 .

Each HUD device 100a, 100b can be installed inside the dashboard in the front part of the vehicle 1, for example. The display device is not limited to the HUD device, and a display device such as a liquid crystal device (for example, a TFT liquid crystal device) or a self-luminous display device such as a micro LED display device may be used ( An example of using a display device is described below with reference to FIG. 10).

In FIG. 1A, a driver 7 is seated in a driver's seat 6 and a passenger (passenger) 10 is seated in a passenger's seat 9 . No occupant is present in the rear seat 11 .

Further, the presence (seated) of the driver 7, the presence (seated) of the occupant 7, and the absence of the occupant in the rear seat 11 can be detected by, for example, an image captured by the camera 200. It can be determined by image analysis.

The HUD devices 100a and 100b project image display light onto the windshield 2 to allow the driver 7 and the passenger 10 to view the image (virtual image). The HUD device 100a is a display device in which the driver 7 is the main viewer and the passenger 10 is the secondary viewer.

Turning on the ignition switch of the vehicle 1 means that the driver 7 is always present. correction (distortion correction) is performed to suppress the distortion, and the driver 7 visually recognizes the corrected image.

The driver 7 views the display image of the HUD device 100a from the front. In other words, the driver 7 can be said to be a viewer at a position directly facing the HUD device (display device) or the display image of which the driver 7 is the main viewer. In addition, the meaning of "front" or "right facing" should be interpreted in a broad sense, and the range that can be regarded as "front" and the range that can be regarded as "right facing" may be defined, including some fluctuation range.

The HUD device 100b is a display device with the occupant 10 as the main viewer and the driver 7 as the secondary viewer. In other words, the HUD device 100b basically performs correction (distortion correction) to suppress distortion of the image on the assumption that the image is viewed by the occupant 10, and the corrected image is viewed by the driver 7. Let However, when the occupant 10 is absent, the HUD device 100b can perform image distortion correction on the assumption that the driver 7 (subordinate viewer) sees the image (see FIG. 1B). .

The HUD devices 100a and 100b incorporate a display control device (processor: reference numeral 159 in FIG. 12, not shown in FIG. 1) having a correction parameter change function for distortion correction. An example of display control of the display control device will be described later.

Further, image distortion correction includes, for example, "pre-distortion correction processing (warping correction processing)" in which distortion having characteristics opposite to the distortion that occurs when an image is displayed is given to the image data in advance, and image display. When the outline of an area is a rectangle containing rectangles and squares, if the rectangle appears to be deformed into a trapezoid or parallelogram according to the angle at which the image is viewed by the viewer, that trapezoid or parallelogram At least one of "deformation suppression correction processing" for converting to a rectangle and "3D image correction processing" for correcting distortion of a three-dimensional image in the case of three-dimensional (3D) display may be included.

In FIG. 1B, the passenger (fellow passenger) 10 is absent, and only the driver 7 is present in the vehicle. In this state, the driver 7 can visually recognize the image displayed by the HUD device 100b, and the driver 7 sees the image from an oblique direction.

Assuming that the image remains the image after the distortion correction, which is assumed to be viewed by the occupant 10, it is highly likely that the image will appear distorted to the driver 7, and the driver 7 may feel uncomfortable. be.

Therefore, in the present embodiment, in the case shown in FIG. 1B, the correction parameter is changed (for example, the correction parameter is switched) to correct the distortion suitable for the driver 7 who is the secondary viewer. The data is processed and a distortion-corrected image is displayed to the driver 7 (details will be described later). This makes it possible to reduce (suppress) discomfort.

In the example of FIG. 1, no occupant is present in the rear seat 11, but at least one occupant is present in the rear seat 11, and the occupant is displayed in front of the vehicle 1 (or in the front portion). When the occupant is seated in a position where the image can be visually recognized, the occupant can be the main viewer and the target of the image display (this point will be described later with reference to FIGS. 9 and 10). do).

Also, in the example of FIG. 1, two display devices are used, but the number of display devices to be used is not particularly limited. For example, as in the example of FIG. 4, three HUD devices can be used.

By using multiple display devices with built-in display control devices, a high-performance display system that is easy to use and can display images with little distortion to both the driver and passengers is realized. can do.

Next, refer to FIG. FIG. 2 illustrates a case where warping is performed as distortion correction.

FIG. 2A is a diagram showing a configuration example of a HUD device having a warping correction function, and FIG. 2B is a diagram showing the effect of warping correction processing.

The HUD device 100 is mounted on a vehicle (which can be interpreted in a broad sense) 1 . The HUD device 100 includes a display unit (for example, a light transmission screen, a liquid crystal display device, etc.) 101, a reflecting mirror (flat mirror) 103, and a curved mirror (for example, a concave mirror) as an optical member for projecting display light. The surface may be a free-form surface) 105, an image generation unit 150, a control unit 160, a ROM (storage device) 210 having an image conversion table 212 storing warping parameters WP as correction parameters, have

Note that the control unit 160 can be constructed as a functional block included in the display control device (processor: reference numeral 159 in FIG. 12). The controller 160 can selectively retrieve the warping parameters WP from the ROM 210 . In other words, the control unit 160 has a function of appropriately changing the warping parameter WP to be used based on information on the presence or absence of passengers in the vehicle 1 and information on the arrangement of the passengers.

An image displayed on the display unit 101 is projected onto the projectable area 5 of the windshield 2 as a projection target member via the reflecting mirror 103 and the curved mirror 105 . Reference numeral 4 indicates the projection area of the image.

In addition, in the HUD device 100, a plurality of curved mirrors may be provided. In addition to the mirror (reflecting optical element) of this embodiment, or in place of part (or all) of the mirror (reflecting optical element) of this embodiment, a refractive optical element such as a lens, a diffractive optical element, etc. A configuration including a functional optical element may be employed.

A part of the display light of the image is reflected by the windshield 2, and a preset eye box (actually three-dimensional, but for convenience of explanation, it is drawn as a rectangle with a predetermined area in FIG. 2A) EB is incident on the viewpoint (eye) A of the driver or the like located inside (or on the EB), and an image is formed in front of the vehicle 1 to display a virtual virtual image corresponding to the display surface 102 of the display unit 101 A virtual image V is displayed on the surface PS.

The image on the display unit 101 is distorted under the influence of the shape of the curved mirror 105, the shape of the windshield 2, and the like. In other words, distortion occurs due to optical members including the optical system of the HUD device 100 and the windshield 2 . In order to suppress the distortion, the image is preliminarily provided with distortion having characteristics opposite to the distortion (warping processing or warping image correction processing).

FIG. 2(B) shows an example of the virtual image V visually recognized by the driver (reference numeral 7 in FIG. 1) through the windshield 2 . In FIG. 2B, a virtual image V having a rectangular outer shape has, for example, 5 vertically and 5 horizontally, for a total of 25 reference points (reference pixel points or coordinate points) GD(i,j ) (where i and j are both variables that can take values from 1 to 5). Each reference point (each coordinate point) in the image (original image) is preliminarily given a distortion having characteristics opposite to the distortion occurring in the virtual image V due to the warping process. Therefore, the distortion that is given in advance and the distortion that actually occurs cancel each other out, and ideally, a virtual image V without curvature is displayed as shown in FIG. 1(B). Note that the number of reference points GD(i, j) can be appropriately increased by interpolation processing or the like.

Next, refer to FIG. FIG. 3A is a diagram showing an example of an image display before deformation suppression correction that converts a trapezoid or parallelogram as the contour shape of the image display area into a rectangle (including rectangles and squares); 3B) is a diagram showing an example of an image display after deformation suppression correction, and FIG. 3C is a diagram showing a main part configuration example of a HUD device having a deformation suppression correction function.

In FIGS. 3A and 3B, it is assumed that the driver drives, for example, a vehicle with three front seats (and a steering wheel on the right side). 3A and 3B, similarly to the example of FIG. 2A, the contour (outer shape) of the display area of the virtual image (image) V on the virtual image display surface PS is a rectangle including a rectangle and a square. Suppose that It should be noted that the rectangle indicating the outline of the image display area is indicated by a thick solid line in FIGS. 3A and 3B.

As shown in FIG. 3A, the driver 7 as a viewer is in front of the driver's seat (a position directly facing the driver) in an image (indicated by three white circles in the figure). ), the contour G1 of the image area is a rectangle (rectangular). When viewing the image at the center position (the position in front of the front passenger seat in the center), the image is viewed obliquely, so the outline G2 of the image display area is a trapezoid. In addition, it may become a parallelogram.

When viewing the image at the left end position (the position in front of the front passenger seat at the left end), the image is viewed from an even more oblique direction, so the outline G3 of the image display area becomes a more distorted trapezoid.

Thus, a rectangle appears to be deformed into a trapezoid or a parallelogram depending on the angle at which the image is viewed. If this continues, the driver 7, who is a viewer, will feel uncomfortable.

Therefore, the control unit 160 performs deformation suppression correction to convert the trapezoid (or parallelogram) into a rectangle. As a result, as shown in FIG. 3B, contours G2 and G3 (indicated by dotted lines) of the image display area are converted to G2' and G3' (indicated by thick solid lines), respectively. This reduces the distortion of the image and suppresses the driver's sense of discomfort.

The configuration of FIG. 3(C) is the same as the configuration shown in the lower right of FIG. 2(A). However, in the case of FIG. 3C, the image conversion table 212 of the ROM (storage device) 210 contains conversion parameters for deformation suppression correction (in other words, conversion parameters for correction for converting trapezoids and parallelograms into rectangles). JM is stored.

The transformation parameter JM is, for example, a parameter that determines the contents of a projective transformation matrix that transforms a trapezoid into a rectangle, or a parameter that determines the contents of a projective transformation (affine transformation) matrix that transforms a parallelogram into a rectangle.

The control unit 160 can appropriately retrieve the conversion parameters from the ROM 210 and implement the deformation suppression correction process. In other words, the control unit 160 has a function of appropriately changing the conversion parameters.

Moreover, the warping correction process described with reference to FIG. 2 and the deformation suppression correction process described with reference to FIG. In this case, the distortion of the 3D image can be reduced by performing the above-described distortion correction processing on each of the images for the left and right eyes of the viewer (driver).

Note that the warping correction process described with reference to FIG. 2 and the deformation suppression correction process described with reference to FIG. 3 can be used together. In this case, for example, it is preferable to perform deformation suppression correction processing on the image data, and then perform warping correction processing.

(Second embodiment)
Reference is now made to FIG. FIG. 4 is a diagram showing an example of a display system in a vehicle with three seats in the front row. In addition, in the example of FIG. 4, the steering wheel 8 is located on the left side inside the vehicle.

In FIG. 4, three display devices (here, HUD devices) 110a, 110b, and 110c are arranged along the width direction (X direction) of the vehicle 1 in the front portion, which is the portion on the windshield 2 side of the vehicle 1. It is

A HUD device 110a is arranged at a position corresponding to the driver's seat 16 (a position facing the driver's seat), and a HUD device 110b is arranged at a position corresponding to the adjacent passenger's seat (center passenger's seat) 18, and the adjacent passenger's seat ( A HUD device 110c is arranged at a position corresponding to the passenger's seat 20 on the right end.

A driver 15 is seated in a driver's seat 16 , a passenger (fellow passenger) 17 is seated in a passenger seat 18 , and a passenger (fellow passenger) 19 is seated in a passenger seat 20 . No occupant is present in the rear seat 21 . Further, the presence (sitting) of the driver 15, the presence (sitting) of the passengers 17 and 19, and the absence of the passengers in the rear seat 21 can be detected by imaging by the camera 200, for example. It can be determined by image analysis of the image.

Each of the HUD devices 110a to 110c projects image display light onto the windshield 2 to allow the driver 15 and passengers 17 and 19 to visually recognize the image (virtual image).

The HUD device 110a is a display device with the driver 15 as the main viewer and the passengers 17 and 19 as secondary viewers. However, as in the example of FIG. 1, since the driver 15 is always present, the HUD device 110a always performs image distortion correction on the assumption that the driver 15 sees the image. is displayed to the driver 15.

The HUD device 110b is a display device with the occupant 17 as the main viewer and the driver 15 as the secondary viewer. The HUD device 110c is a display device with the occupant 19 as the main viewer and the driver 15 as the secondary viewer.

Information capable of specifying the main viewer of each HUD device is input and set in advance in each of the HUD devices 110a to 110c (the control unit 160 of the display control device thereof).

For example, among the three display devices 110a to 110c, for the display device 110a installed at a position corresponding to the driver 15, for example, if "1" is input on the initial setting screen, the driver 15 is the main display device. It is set to be a viewer.

Similarly, for the display device 110b arranged at the position corresponding to the central passenger seat 18, if for example "2" is entered on the initial setting screen, the display device 110b will be positioned next to the driver's seat 16 in the front row (in other words, , seated in the middle front passenger seat 18) is set to be the main viewer. Similarly, for the display device 110c, by inputting "3" on the initial setting screen, the main viewer is set to be the occupant 19 seated in the rightmost passenger seat 20 in the front row.

After setting all the display devices in this way, if at least one of information on the presence or absence of passengers and information on the arrangement of passengers is input in parallel to each of the display devices 110a to 110c, each of the display devices 110a to 110c The control unit 160 of 110c determines whether or not the correction parameters have been changed, and automatically optimizes the correction parameters. Therefore, it is possible to realize a high-performance display system that is easy to use and can display an image with less distortion for both the driver and the passengers.

Reference is now made to FIG. 5A and 5B are diagrams showing examples of basic display control in the display system of FIG. FIG. 5 is drawn corresponding to each arrangement of the display device, the driver, and the passenger (passenger) in FIG. Further, in the examples of FIGS. 5A and 5B, it is assumed that warping correction is performed as image distortion correction. The above points are the same for FIGS. 6 to 8 as well.

In FIGS. 5A and 5B, an eye box EB10 for the driver 15 and eye boxes EB20 and EB30 for the passengers 17 and 19 are set on the near side. In front of the vehicle 1, image display areas G10, G11, and G12 set on the virtual image display surface PS are shown. The arrows in the drawing indicate the line-of-sight directions of the driver 15 and the passengers (passengers) 17 and 19, respectively.

In the example of FIG. 5A, a driver 15 and passengers (fellow passengers) 17 and 19 are present. Therefore, in each display device (HUD device) 110a to 110c, the correction parameters used for image distortion correction are the main viewers of each display device (in other words, the driver sitting facing each display device). 15 and passengers (passengers) 17 and 19 can display images with little distortion.

Here, the correction parameter specified assuming that the main viewer sees the image is here referred to as the main parameter (main parameter: denoted as "M-wp" in FIGS. 5A and 5B). called. In this specification, "M-wp" may be referred to as "first value of correction parameter".

Further, when the passenger (passenger) who is the main viewer is absent, the correction parameter specified on the assumption that the driver as the secondary viewer sees the image is set as the sub-parameter (sub-parameter: FIG. 5 ( B), referred to as “S-wp”). In this specification, "S-wp" may be referred to as "second value of correction parameter".

In FIG. 5(B), the passenger (fellow passenger) 17 is absent from the center front passenger seat. Along with this, in the example of FIG. 5B, in the display device (HUD device) 110b arranged in the center of the vehicle 1, when correcting image distortion, the sub-parameter (sub-parameter: S-wp) are using.

Next, refer to FIG. FIGS. 6A and 6B are diagrams showing an example of display control when the display device starts operating.

In FIG. 6A, when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c is started, the display of each display device 110a to 110c (the display control device) The unit 160) hides the image before the values of the correction parameters are determined.

In FIG. 6B, the presence or absence of passengers, the arrangement of passengers, and the like are detected, and after the values of the correction parameters are determined based on the results, each of the display devices 110a to 110c (the control unit 160 thereof) displays the image. is displayed. The correction parameters used to display each image in FIG. 6B are the same as in the example of FIG. 5B.

By displaying the image after the correction parameters to be applied have been determined in this way, compared to the case where the correction parameters are changed after the image is displayed, for example, it is possible to reduce the viewer's sense of discomfort due to display distortion. can.

Next, refer to FIG. FIGS. 7A and 7B are diagrams showing another example of display control when the display device starts operating.

In FIG. 7A, when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c is started, the display device (HUD) with the driver 15 as the main viewer is displayed. The control unit 160 of the device 110a corrects the image data using a correction parameter (main parameter M-wp) suitable for the driver 15 and starts image display.

On the other hand, the control units 160 of the display devices 110b and 110c whose main viewers are the passengers (passengers) 17 and 19 do not display the images until the correction parameters are determined.

In FIG. 7B, the control units 160 of the display devices 110b and 110c detect the presence or absence and arrangement of passengers, determine correction parameters, and then display images corrected using the determined correction parameters. it's shown. The correction parameters used to display each image in FIG. 7(B) are the same as the examples in FIGS. 5(B) and 6(B).

In this way, for the display device for which the driver is the main viewer, it is possible to present, for example, useful information for driving the vehicle without delay by immediately starting to display the image. On the other hand, for a display device in which a passenger is the main viewer, it is possible to display an image with little distortion by opening the display after the presence or absence of the passenger is confirmed and the value of the correction parameter is determined.

Reference is now made to FIG. FIGS. 8A and 8B are diagrams showing still another example of display control when the display device starts operating.

In FIG. 8A, when the ignition switch (IGN) of the vehicle 1 is turned on and the operation of each display device (HUD device) 110a to 110c is started, the display device 110a whose main viewer is the driver 15 is displayed. The control unit 160 corrects the image data using a correction parameter suitable for the driver 15 (main parameter M-wp: first value of the correction parameter) and starts image display.

In addition, the control unit 160 of the display devices 110b and 110c whose main viewers are the passengers 17 and 19 also first sets a correction parameter suitable for the passengers (passengers) 17 and 19 (primary parameter M-wp: the first correction parameter). value) is used to correct the image data and image display is started.

In FIG. 8B, the control unit 160 of the display devices 110b and 110c whose main viewers are the occupants 17 and 19 performs the above-described display, and the correction parameter If necessary, the value of the correction parameter is changed, the image data is corrected using the changed correction parameter, and the image is displayed.

In FIG. 8B, the occupant (passenger) 17 in the middle front passenger seat is absent, so for the display device 110b, the correction parameter is changed from M-wp (the first value of the correction parameter) to S- wp (the second value of the correction parameter). The correction parameters used to display each image in FIG. 8(B) are the same as the examples in FIGS. 5(B), 6(B), and 7(B).

When the display control of FIG. 8 is adopted, even when the passenger is the main viewer, the image can be displayed quickly without waiting for the determination of the value of the correction parameter, and the delay in image display can be suppressed. be able to.

(Third Embodiment)
See FIG. FIG. 9(A) is a diagram showing an example in which one of three display devices is used to display an image to an occupant seated in the rear seat, and FIG. FIG. 4 is a diagram showing an example in which two display devices located close to the driver are both used to display an image to the driver. In FIGS. 9A and 9B, parts common to those in FIG. 1 are given the same reference numerals.

In FIG. 1, two display devices (here, HUD devices) 100a and 100b are arranged in a passenger car with two seats in the front row, but in the example of FIG. In between, a display device 100d is added.

In the above-described embodiment, for convenience of explanation, the main viewers are the driver and passengers (passengers) seated in the front row of the vehicle 1, but in the example of FIG. 11, for example, an image is displayed with the passenger (fellow passenger) 22 seated at, for example, the center position (in a broader sense, the position where the display by the display device 100d can be seen) as the main viewer.

In other words, when an occupant is the main viewer, the occupant may be an occupant sitting in the front passenger seat of the vehicle 1 or an occupant sitting in the rear row seat (rear seat) of the vehicle 1. may

The reason why the occupants who are the main viewers includes the occupants sitting in the rear seats is that the occupants may also be provided with, for example, navigation information, equipment operation information, entertainment information, and the like.

In front of the occupant (occupant 22 in FIG. 9A), preferably at a position directly facing the occupant (however, the term "directly facing" is interpreted broadly and includes positions other than strictly in front), the dashboard A display device 100d is installed inside, on the surface of the dashboard, or the like.

In FIG. 9B, there is no passenger 22 seated in the rear seat. In this case, the correction parameters of the image displayed by the display device 100d are changed so that the image is suitable for the driver 7 to see. As a result, the distortion of the image is reduced, and the feeling of strangeness of the driver 7 is suppressed.

In FIG. 9B, when the passenger 10 in the passenger seat 9 in the front row is also absent, the driver 7 can see the image displayed by the display device 100b. Therefore, the correction parameters of the image displayed by the display device 100b are changed so that the image is suitable for the driver 7 to see. As a result, the distortion of the image is reduced, and the feeling of strangeness of the driver 7 is suppressed.

(Fourth embodiment)
Please refer to FIG. FIG. 10A shows an example of a display system using a display device (such as a liquid crystal display device), and FIG. 10B shows an example of a display system in which a HUD device and a display device are mixed. FIG. 4 is a diagram showing;

As described above, as the display device, in addition to the HUD device, a display device that displays a real image on a display unit as a display panel and allows at least one of the driver and the passenger to visually recognize the real image. can be used. As the display device, a liquid crystal device (for example, a TFT liquid crystal device) or a micro LED display device, which is a self-luminous display device, may be used. By applying the present invention to a display device, it is possible to realize a high-performance display device capable of displaying an image with little distortion to both the driver and the passengers.

In FIG. 10A, the HUD devices 100a, 100d, and 100b in the configuration of FIG. 9A described above are replaced with display devices 300a, 300b, and 300c.

In FIG. 10B, by replacing the display device 100a in FIG. 10A with a HUD device 100a, a display system in which a head-up display (HUD) device and a display device are mixed is constructed. .

Thus, according to the present invention, a display system can be freely constructed using only the HUD device, only the display device, or a combination of the HUD device and the display device. Therefore, a user-friendly display system can be constructed.

(Fifth embodiment)
Reference is now made to FIG. FIG. 11 is a diagram showing an example of gradually changing the correction parameter value when changing the correction parameter.

When changing the value of the correction parameter during image display, the control unit 160 of the display device may gradually change the value before the change to the value after the change. When it becomes necessary to change the correction parameter, by gradually changing the value, it is possible to suppress abrupt changes in the appearance of the image, thereby reducing discomfort given to the driver and passengers. can.

In FIG. 11, the value of the correction parameter gradually changes from the first value N1 to the second value N2 over the period from time t1 to time t2. A characteristic line Q1 indicates a linear change.

The characteristic line Q2 changes gradually around the time t1, so that abrupt changes in the appearance of the image can be suppressed. Since the change becomes abrupt as time t2 approaches, delay in switching the correction parameter value can be suppressed.

In the characteristic line Q3, the change is gradual near time t1 and near time t2, and sharp changes in the appearance of the image are suppressed. On the other hand, the change is abrupt except around time t1 and around time t2, thereby suppressing the delay in switching the correction parameter value.

(Sixth embodiment)
Reference is now made to FIG. FIG. 12 is a diagram illustrating an example of the main configuration of a display device and an example of a display system. Although the main structure of the display device was shown in FIGS. 2 and 3, FIG. 12 shows a more detailed structure. Also, in the example of FIG. 12, three HUD devices are used as display devices. Also, the example of FIG. 12 shows a configuration that can handle both warping correction and deformation suppression correction.

12, the display system includes a communication unit 110, an ECU (electronic control unit) 120, an in-vehicle imaging camera 200, and an occupant (passenger) detection unit 130 for detecting an occupant (or a passenger including a driver). , first to third HUD devices 100a, 100b, and 100c. Each HUD device has the same configuration. The main configuration of the third HUD device 100c will be described below.

The third HUD device 100c includes a display unit 101, a display control unit 145, an image generation unit 150 (including an image data correction unit 155), and at least one processor (display control device) 159 (control unit 160). , the control unit 160 has a projective transformation correction control unit 161 and a warping correction control unit 163), a database 170 storing navigation data, display image data, etc., and a ROM 210 as a storage device. .

ROM 210 has an image conversion table 212 . The image transformation table 212 has a projective transformation matrix JM. The projective transformation matrix JM includes a primary transformation parameter M-jm and a secondary transformation parameter S-jm.

The image conversion table 212 also has a warping parameter WP. The warping parameters WP include a primary warping parameter (first value) M-wp and a secondary warping parameter (second value) S-wp.

The processor (display control device) 159 (the control unit 160 thereof) has a function of appropriately changing the transformation parameters and warping parameters of the projective transformation. The image data correction unit 155 corrects the image data of the display image using a correction parameter (at least one of the transformation parameter and the warping parameter) so as to suppress the distortion of the image seen by the driver or passenger. . A display unit 101 displays an image based on the corrected image data.

According to this configuration, when the passenger (fellow passenger) is not present, the image corrected for distortion suitable for the driver viewing the image from an oblique direction can be displayed on the display unit 101, thereby reducing discomfort. can be done. As a result, it is possible to realize a high-performance display device and a display system capable of displaying an image with little distortion to both the driver and the passengers.

(Seventh embodiment)
Reference is now made to FIG. FIG. 13 is a flowchart illustrating an example of a display control procedure.

In step S1, the presence or absence and arrangement of passengers (or passengers including the driver) are detected.

In step S2, image correction parameters (projective transformation parameters, warping parameters, etc.) for each display content are determined based on the detection result (at least one of the presence or absence of passengers and their arrangement). Change parameters (parameter determination, change processing).

In step S3, the image data of each display content is corrected using the determined (or changed) image correction parameters (image data correction processing).

In step S4, at least one of display of a virtual image by the HUD device and display of a real image by a display (TFT liquid crystal, micro LED, etc.) is executed (image display processing).

As described above, according to the present invention, even when the passenger who is the main viewer of the display device is absent, the distortion of the image is reduced, and for example, the driver's sense of discomfort when viewing the image is suppressed. .

The present invention can be used in either a monocular HUD device in which display light of the same image is incident on the left and right eyes, or a parallax HUD device in which an image with parallax is incident on the left and right eyes.

As the display device, a liquid crystal display device, a micro LED device, and other various display devices can be used.

In this specification, the term vehicle can also be interpreted broadly as a vehicle. Also, terms related to navigation (such as signs) should be interpreted in a broad sense, taking into consideration the viewpoint of broad-sense navigation information that is useful for driving a vehicle, for example. HUD devices also include those used as simulators (for example, aircraft simulators, game device simulators, etc.).

The invention is not limited to the exemplary embodiments described above, and the person skilled in the art will easily be able to modify the exemplary embodiments described above to the extent that they fall within the scope of the claims. .

1... vehicle (self-vehicle), 2... projected member (reflecting translucent member, windshield, etc.),
6 Driver's seat 7 Driver 8 Steering wheel 9 Passenger seat
DESCRIPTION OF SYMBOLS 10... Passenger, 11... Rear seat, 100... HUD device (display device in a broad sense), 101... Display part (liquid crystal panel etc.), 102... Image display surface of a display part, 110... communication unit, 120... ECU (electronic control unit), 130... passenger (passenger) detection unit, 145... display control unit, 150... image generation unit, 155... image data corrector,
159... Processor (display control device), 160... Control unit, 161... Projective transformation correction control unit, 163... Warping correction control unit, 170... Database, 210... Storage device ( ROM), 200... In-vehicle imaging camera, 212... Image conversion table, 300... Display device, JM... Projective conversion matrix, M-jm... Main conversion parameter (first value) , S-jm... secondary transformation parameter (second value), WP... warping parameter, M-wp... main warping parameter (first value), S-wp... secondary warping parameter ( second value)

Claims (12)

A display control device that controls image display of a display device that is mounted in a vehicle and causes an image to be viewed by a driver of the vehicle or an occupant other than the driver,
a control unit having a function of changing a correction parameter used when correcting image data of the image so as to suppress distortion of the image seen by the driver or the passenger;
The control unit
When the display device has the occupant as a primary viewer and the driver as a secondary viewer,
If the occupant is present as a result of determination based on at least one of the occupant presence information and the occupant arrangement information, the value of the correction parameter is set to a first value suitable for the occupant. ,
setting the value of the correction parameter to a second value suitable for the driver and different from the first value when the occupant is absent;
Display controller. The occupant as the primary viewer,
An occupant sitting in the front passenger seat of the vehicle,
or
An occupant sitting in a seat in the back row of the vehicle,
The display control device according to claim 1. The correction parameter is
a warping parameter for pre-distortion correction that preliminarily gives the image data a distortion having characteristics opposite to the distortion that occurs when the image is displayed;
as well as,
When the outline of the display area of the image is a rectangle including rectangles and squares, and the rectangle appears to be deformed into a trapezoid or a parallelogram according to the angle at which the viewer views the image, the transformation parameters for deformation suppression correction, which transforms a trapezoid or parallelogram into a rectangle;
as well as,
3D image correction parameters for correcting three-dimensional image distortion in the case of three-dimensional (3D) display;
including at least one of
The display control device according to claim 1 or 2. The control unit
When the operation of the display device is started, the image is hidden before the value of the correction parameter is determined, and the image is displayed after the value of the correction parameter is determined.
The display control device according to any one of claims 1 to 3. The control unit
When the operation of the display device is started,
When the display device has the driver as the main viewer, correcting the image data using a correction parameter suitable for the driver and starting image display,
When the display device makes the occupant the main viewer, the image is displayed after the value of the correction parameter is determined.
The display control device according to any one of claims 1 to 3. The control unit
When the operation of the display device is started,
When the display device has the driver as the main viewer, correcting the image data using a correction parameter suitable for the driver and starting image display,
When the occupant is the main viewer, the display device sets the correction parameter to the first value and performs display while determining whether or not the correction parameter needs to be changed. Change the value of the correction parameter if
The display control device according to any one of claims 1 to 3. The control unit
When changing the value of the correction parameter while displaying the image, gradually change the value before the change to the value after the change;
The display control device according to any one of claims 1 to 6. A display control device according to any one of claims 1 to 7;
an image data correction unit that corrects the image data of the image using the correction parameter so as to suppress distortion of the image as viewed by the driver or the passenger;
a display that displays an image based on the corrected image data;
display device. The display device
The head-up device further includes an optical system for projecting display light of an image displayed on the display unit onto a projection target member provided in the vehicle, and has a function of allowing at least one of the driver and the passenger to visually recognize a virtual image. is a display device;
or
A display device that displays a real image on the display unit as a display panel and allows at least one of the driver and the passenger to visually recognize the real image.
The display device according to claim 8. A plurality of display devices according to claim 8 are arranged along the width direction of the vehicle in the front portion, which is the portion on the windshield side of the vehicle,
one of the plurality of display devices is a first display device with the driver as a viewer;
At least one of the other display devices is a second display device in which the occupant in the front passenger seat or the occupant in the back row is the main viewer and the driver is the secondary viewer.
display system. The plurality of display devices are head-up display devices,
or
wherein the plurality of display devices are display devices;
or
A head-up display device and a display device are mixed in the plurality of display devices,
11. A display system according to claim 10. An image display control method for a display device mounted on a vehicle, comprising:
a first step of detecting at least one of the presence or absence and arrangement of passengers, including passengers or drivers;
a second step of determining an image correction parameter for the displayed content based on the detection result, and changing the image correction parameter if change is necessary;
a third step of correcting the image data of the display content using the determined or changed image correction parameters;
a fourth step of performing at least one of displaying a virtual image by a head-up display device and displaying a real image by a display device;
An image display control method, comprising:

Images