JP2021118372A - Display system - Google Patents

Abstract

To provide a display system that can match the impression of the color of a subject in a displayed image with the impression of the color of a subject visually recognized.SOLUTION: A white balance adjustment unit 12 of a camera 1 adjusts the white balance of an image output from an imaging unit 11 such that a white portion of a subject becomes white, and outputs the image to a monitor device 2. An image processing unit 21 of the monitor device 2 performs a predetermined color adjustment process on the image input from the camera 1 and outputs the image to a display white balance correction unit 212. The display white balance correction unit 212 generates an image in which the input image is adjusted in the reverse direction of the white balance adjustment performed by the camera 1 and is to be displayed on a display 22, and displays the image on the display 22.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display system that displays an image captured by a camera.

As a display system for displaying images taken by a camera, an electronic mirror system is known that displays an image of the rear of an automobile taken by a camera on a display arranged above a front windshield in an automobile (for example). , Patent Document 1).

Further, as a technique related to a camera that can be used in such a display system, a video output from an image sensor so that the pixel value of the pixel representing the white part of the subject correctly becomes the pixel value representing white. A technique for adjusting the white balance that outputs after adjusting the gain of each color is known (for example, Patent Document 2).

In addition, the image sensor can be used to obtain an image that reflects the color of the subject under the color environment (ambient light color, etc.) at the time of shooting, that is, an image of the subject that has the same color as the visually recognized color. A technique for adjusting the tint of an output image after adjusting the gain of each color is also known (for example, Patent Document 3).

Japanese Unexamined Patent Publication No. 2016-22934 Japanese Unexamined Patent Publication No. 2002-185977 International Publication No. 2005/04158

For example, in the above-mentioned electronic mirror system, when an image whose white balance has been adjusted by the camera is displayed on the display, the driver visually recognizes the color of the subject in the image displayed on the display depending on the color environment at that time. The color of the subject may be different.

On the other hand, in the above-mentioned electronic mirror system, when the color tone is adjusted by the camera, the white part of the subject is chromatic in the image output from the camera, so the color that adjusts the chromatic color set by the user. When adjusting the image output from the camera and displaying it on the display, the achromatic part such as white of the subject displayed on the display becomes a color that cannot be recognized as achromatic. There is.

In this way, the impression of the color of the subject in the image displayed on the display may be different from the impression of the color of the subject recognized by the driver directly or visually through the side mirror. , It is a negative factor for the driver's intuitive understanding of the situation around the car.

Therefore, according to the present invention, even when the image captured by the camera is displayed with the chromatic color adjusted, the impression of the color of the subject in the displayed image is visually recognized as the color of the subject. The challenge is to provide a display system that matches the impression.

In order to achieve the above object, the present invention comprises a camera that captures an image, an image processing unit that performs image processing of the output image of the camera, and a display that displays an image that the image processing unit has performed image processing. Provide a complete display system. Here, the camera has an imaging unit that captures and outputs an image, a white balance adjusting unit that adjusts the white balance of the image output by the imaging unit and outputs the output image of the camera, and the imaging unit outputs the image. It is provided with a color information output means for outputting information that can identify the color of the image as color information. Further, the image processing unit includes a color adjustment processing unit that performs a predetermined color adjustment process for adjusting the color of the output image of the camera, and the color adjustment processing unit based on the tint information output from the camera. A color correction means for generating an image displayed on the display as an image having a color similar to the color of the image output by the image pickup unit of the camera from the image subjected to the color adjustment processing and outputting the image to the display. And have.

Here, in such a display system, in the color correction means, the color of the image subjected to the color adjustment processing by the color adjustment processing unit is similar to the color of the image output by the imaging unit of the camera. An image may be configured to generate an image that has been converted into an image and a display white balance adjustment for displaying a white portion of the image in white on the display and output to the display.

The display system as described above may be configured so that the color information output means of the camera outputs information representing the content of the white balance adjustment performed by the white balance adjusting unit as the color information.

Alternatively, in the above display system, the camera is provided with a color tint calculating means for calculating the tint of the image output by the imaging unit, and the tint information output means of the camera is calculated by the tint calculating means. Information representing the color may be output as the color information.

Further, the above display system may be mounted on an automobile. In this case, the camera captures an image of the rear of the automobile, and the display is a position visible to a seated person in the driver's seat of the automobile. It may be arranged in.

According to the display system as described above, the image pickup unit of the camera outputs the image obtained by performing the color adjustment process on the image whose white balance has been adjusted in the camera by the image processing unit and then performing the color adjustment process. Since the image is converted into an image displayed on the display as an image having a color similar to the color of the image and displayed on the display, the white part of the subject is displayed even when the chromatic color is adjusted by the color adjustment process. It is possible to match the impression of the color of the subject in the displayed image with the impression of the color of the subject visually recognized while suppressing the unnatural coloring of the camera.

As described above, according to the present invention, even when the image captured by the camera is displayed with the chromatic color adjusted, the impression of the color of the subject in the displayed image is visually recognized. It can be matched to the impression of the color of the subject.

It is a block diagram which shows the structure of the electronic mirror system which concerns on embodiment of this invention. It is a figure which shows the arrangement of the electronic mirror system which concerns on embodiment of this invention. It is a figure which shows the operation of the electronic mirror system which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described by taking application to an electronic mirror system as an example.
FIG. 1 shows the configuration of the electronic mirror system according to the present embodiment.
The electronic mirror system is a system mounted on an automobile, and includes a camera 1 and a monitoring device 2 as shown in the figure.
Here, for example, as shown in FIG. 2a, the camera 1 is arranged at the rear of the automobile and photographs the rear view of the automobile.
Further, for example, as shown in FIG. 2b, the monitoring device 2 is arranged at a position above the front windshield in the automobile vehicle, that is, at a position where a rearview mirror is traditionally arranged, and an image taken by the camera 1 is provided. Is displayed.

Returning to FIG. 1, the camera 1 has an imaging unit 11 that captures and outputs an image of a CCD image sensor or the like, a white balance adjusting unit 12 that adjusts and outputs the gain of each color of the image output by the imaging unit 11, and a white balance adjustment. The white color of the subject in the image output from the white balance adjustment unit 12 based on the color temperature estimated by the color temperature estimation unit 13 and the color temperature estimation unit 13 that estimate the color temperature of the image output from the unit 12. It has a white balance control unit 14 that adjusts the white balance by controlling the gain of each color in the white balance adjustment unit 12 so that the pixel value of the pixel representing the portion is the pixel value that correctly represents white. Here, in the control of the gain of each color in the white balance adjustment unit 12, for example, in the color temperature estimation unit 13, the average level of the colors in the image output from the white balance adjustment unit 12 is calculated. Then, the white balance control unit 14 controls the gain of each color so that the average level calculated by the color temperature estimation unit 13 becomes a predetermined value (for example, 0). The white balance adjustment by controlling the gain of each color in the white balance adjusting unit 12 estimates a region corresponding to the white portion of the subject in the image captured by the imaging unit 11, and the pixel value of the region represents white. It can also be performed by controlling the gain of each color so that it becomes a value.

Then, the image output from the white balance adjusting unit 12 is sent to the monitor device 2 as a camera output image V. Further, the control content of the gain of each color in the white balance adjusting unit 12 by the white balance control unit 14 is sent to the monitoring device 2 as gain control information I.

Next, the monitor device 2 includes a video processing unit 21, a display 22, a control unit 23, and an input device 24.
The display 22 is a display device such as an LCD, and is arranged so that the display surface faces the rear of the automobile. Further, the input device 24 is an input device 24 such as a touch panel provided on the display surface of the switch or the display 22, and accepts a user operation.

The image processing unit 21 includes a color adjustment unit 211, a display white balance correction unit 212, and a display white balance control unit 213.
The color adjustment unit 211 applies color adjustment processing of the contents set in the color adjustment unit 211 according to a user operation or the like to the input device 24 by the control unit 23 to the camera output image V input from the camera 1 to correct the display white balance. Output to unit 212.

The display white balance correction unit 212 adjusts the gain of each color of the camera output image V after the color adjustment process input from the color adjustment unit 211 to the gain set from the display white balance control unit 213, and displays the gain on the display 22. ..

The display white balance control unit 213 is applied to the display white balance correction unit 212 according to the gain control information I input from the camera 1 and the display characteristics of the display 22, for each color of the camera output image V after the color adjustment process. Set the gain.

That is, the display white balance control unit 213 adjusts to the color of the image captured by the image pickup unit 11 obtained from the content of the white balance adjustment applied to the image captured by the image pickup unit 11 in the camera 1 represented by the gain control information I. , The gain of each color of the camera output image V after the color adjustment process applied by the display white balance correction unit 212 is set so that the color tone of the image displayed on the display 22 is reproduced.

More specifically, the display white balance control unit 213 cancels all the gains of the color i of the white balance adjustment unit 12 of the camera 1 represented by the gain control information I for each color i, or cancels to a predetermined degree. The gain, that is, the gain obtained by multiplying the inverse of the gain of the color i of the white balance adjusting unit 12 by a predetermined coefficient of 1 or less and more than 0 is calculated as the gain Gci for adjusting the tint of the color i.

As a result, when the gain control information I indicates that the white balance adjustment unit 12 of the camera 1 is performing white balance adjustment with the gain of each color that reduces the yellowness, it corresponds to the amount of the reduction of the yellowness. The gain Gci for adjusting the tint of each color i that increases the yellow tint is calculated.

Further, the display white balance control unit 213 sets Gci × Gwi as the display white balance correction unit, with the gain of the color i for adjusting the white balance of the display 22 determined in advance from the display characteristics of the display 22 as Gwi for each color i. It is set as the gain of the color i of the camera output image V after the color adjustment process applied in 212.

The gain of each color for adjusting the white balance of the display 22 is a pixel value representing white in the image A when the image B obtained by applying the gain Gwi of the color i to each color of the image A is output to the display 22. The pixels of the image B that the image B has are the gains that are displayed in white on the display 22.

FIG. 3 shows the operation of the electronic mirror system as described above.
Now, as shown in FIG. 3a, when the camera 1 captures a landscape in which the white part of the subject in the evening is visually recognized as yellowish, the white part of the subject is yellowish from the image pickup unit 11 of the camera 1. A yellowish image is output. Then, as shown in FIG. 3b1, the white balance adjusting unit 12 of the camera 1 generates an image obtained by adjusting the white balance of the image output from the imaging unit 11 so that the white portion of the subject becomes white, and the monitoring device. It is output to 2.

When the color adjustment unit 211 of the image processing unit 21 of the monitor device 2 is set to the color adjustment process for darkening the color, the image shown in FIG. 3b2 in which the chromatic color of the image of FIG. 3b1 is darkened by the color adjustment unit 211. Is generated and output to the display white balance correction unit 212. Here, since the white portion of the subject is white, which is an achromatic color even in the image of FIG. 3b1 input from the camera 1, the image shown in FIG. 3b2 after the color adjustment processing remains white.

Then, in the display white balance correction unit 212, the image in which the white balance adjustment performed by the camera 1 is canceled is displayed on the display 22, that is, the yellowish white color performed by the camera 1 is displayed. Contrary to the white balance adjustment of white, an image in which the white portion is yellowish is generated and displayed on the display 22 as shown in FIG. 3b3.

Therefore, the image displayed on the display 22 is an image in which the white portion of the subject appears yellowish, as in the case of visual inspection.
Further, even if the color adjustment unit 211 performs the color adjustment process for darkening the chromatic color, the white portion of the subject in the displayed image is not colored to the extent that it is not recognized as white. Further, even when the color adjustment process for converting the hue is performed by the color adjustment unit 211, the hue of the white portion of the subject in the displayed image does not deviate from the visually recognized hue.

Here, as a comparative example, a case where only the gain Gwi of each color for adjusting the white balance of the display 22 is applied to the display white balance correction unit 212 of the image processing unit 21 of the monitor device 2 will be shown.

In this case, as shown in FIG. 3c, the image displayed on the display 22 is an image in which the white portion of the subject appears in white, and the displayed image and the white portion of the subject visually as shown in FIG. 3a. However, there will be a difference in the impression of color from the image that is recognized as yellowish.

Next, as a comparative example, in the camera 1, color adjustment is performed to adjust the image output from the image pickup unit 11 so that an image of the subject having the same color as visually recognized is obtained, and the monitor is used. The case where only the gain Gwi of each color for adjusting the white balance of the display 22 is applied in the display white balance correction unit 212 of the image processing unit 21 of the apparatus 2 is shown.

In this case, as shown in FIG. 3d1, the camera 1 outputs an image in which the white portion of the subject is yellowish as in the case of visual recognition.
When the color adjustment unit 211 of the image processing unit 21 of the monitor device 2 is set to the color adjustment process for darkening the color, the image shown in FIG. 3d2 in which the chromatic color of the image of FIG. 3d1 is darkened by the color adjustment unit 211. Is generated and output to the display white balance correction unit 212. Here, the white part of the subject is yellowish white even in the image of FIG. 3d1 input from the camera 1, so that the yellowness is further enhanced in the image shown in FIG. 3d2 after the color adjustment processing. It may be a color that is perceived as yellow.

Then, in this case, as shown in FIG. 3d3, the image displayed on the display 22 is an image in which the white portion of the subject appears in a color recognized as yellow, and the displayed image and the image as shown in FIG. 3a. There will be a difference in the impression of color from the image in which the white part of the subject is visually recognized as yellowish white.

Further, in this case, when the color adjustment processing unit performs the color adjustment processing including the hue conversion, the white part of the subject is changed to a hue different from the visually recognized hue, and further, the display is performed. There may be a discrepancy between the displayed image and the visually recognized image.

The embodiment of the present invention has been described above.
In the above embodiment, the camera 1 estimates the color temperature of the image captured by the imaging unit 11 and notifies the monitoring device 2, and the display white balance control unit 213 of the monitoring device 2 notifies each color i. The gain for adjusting the image to the color temperature may be calculated as the gain Gci of the color i.

Further, the above embodiment has shown an example of application to an electronic mirror system in which the monitor device 2 is arranged at a position above the front windshield, but in the present embodiment, two monitor devices 2 arranged on the left and right are used. It can be similarly applied to an electronic mirror system that displays images taken by the camera 1 behind the left and right sides of the vehicle, and an arbitrary display system that displays images / images taken by other cameras.

1 ... Camera, 2 ... Monitor device, 11 ... Imaging unit, 12 ... White balance adjustment unit, 13 ... Color temperature estimation unit, 14 ... White balance control unit, 21 ... Video processing unit, 22 ... Display, 23 ... Control unit, 24 ... Input device, 211 ... Color adjustment unit, 212 ... Display white balance correction unit, 213 ... Display white balance control unit.

Claims (5)

A display system including a camera that captures an image, an image processing unit that performs image processing of the output image of the camera, and a display that displays the image that the image processing unit has processed the image.
The camera
An imaging unit that captures and outputs video,
A white balance adjustment unit that adjusts the white balance of the image output by the imaging unit and outputs it as the output image of the camera.
It has a color information output means for outputting information that can identify the color of the image output by the imaging unit as color information.
The video processing unit
A color adjustment processing unit that performs a predetermined color adjustment process for adjusting the color of the output image of the camera, and a color adjustment processing unit.
Based on the color tint information output from the camera, the display is used as an image having a tint that is close to the tint of the image output by the image pickup unit of the camera from the image obtained by the color adjustment processing unit for color adjustment processing. A display system characterized by having a color correction means for generating an image to be displayed on the display and outputting the image to the display. The display system according to claim 1.
The color correction means converts an image that has undergone color adjustment processing by the color adjustment processing unit into an image having a color that is close to the color of the image output by the image pickup unit of the camera, and a white portion of the image. A display system characterized by generating an image with white balance adjustment for display for displaying white on the display and outputting the image to the display. The display system according to claim 1 or 2.
The color information output means of the camera is a display system characterized by outputting information representing the content of white balance adjustment performed by the white balance adjustment unit as the color information. The display system according to claim 1 or 2.
The camera has a color tint calculation means for calculating the tint of the image output by the image pickup unit.
The color information output means of the camera is a display system characterized by outputting information representing a color calculated by the color calculation means as the color information. The display system according to claim 1, 2, 3 or 4.
The display system is installed in automobiles
The camera captures an image of the rear of the vehicle.
The display is a display system characterized in that the display is arranged at a position visible to a seated person in the driver's seat of the automobile.

Images