JP6805968B2 - Image processing device, image projection device, and image processing method - Google Patents

Description

The present invention relates to an image processing device, an image projection device, and an image processing method.

As shown in FIG. 1, an image (display, projected image, etc.) that can be displayed by an image display device or an image projection device has a large difference from human vision in the dynamic range of brightness. For example, human vision is on the order of 10 ¹⁰ , but a general LDR (Low Dynamic Range) display is about 10 ³ to 10 ⁵ , so that the contrast of brightness is insufficient.

Therefore, the high dynamic range (hereinafter abbreviated as HDR), which greatly expands the dynamic range of the brightness of the display, is attracting a great deal of attention.

There are two main ways to realize HDR. One is an approach to increase the brightness of the display itself to several thousand bits (LDR is several hundreds of bits), and the other is an image that is tailored to HDR-like images on the LDR display. It is an approach from processing.

When the display itself is greatly increased in brightness, the power consumption is greatly increased due to the increase in brightness, and the cost increase is inevitable.

Therefore, as another means for realizing HDR, there are a plurality of methods of image processing, and one example thereof is a histogram smoothing technique as shown in FIG.

By broadening the histogram (FIG. 2 (a)) of the image by histogram smoothing, the part where the distribution of the histogram is concentrated is stretched (FIG. 2 (b)), and the display of the display is expanded. The dynamic range can be expanded in the display reproduction range to increase the contrast (for example, Patent Document 1).

However, in a color image, when the histograms for each color of RGB are different, if the RGB signal is smoothed as it is, the histogram is widened to a different degree for each RGB, and the color balance is lost.

Therefore, Non-Patent Document 1 proposes applying different processing to the luminance image (Y) and the color difference image (CrCb) to retouch the preferred image.

However, the effect of increasing the brightness of the histogram smoothing is content-dependent, for example, the effect of the histogram smoothing is small when the image content (content) of the original image is originally wide. For example, in the case of a continuous histogram such as a natural image, it is possible to improve the contrast by smoothing the histogram, but in the case of content close to a binary image such as a sentence or a chart, the histogram becomes local, so the histogram Smoothing is not suitable.

Further, Non-Patent Document 1 is premised on application to still images, and moving images are not considered.

Therefore, in view of the above circumstances, the present invention provides an image processing device capable of adjusting the correction intensity by histogram smoothing for each frame of a moving image according to the content of the content without giving a sense of discomfort to the viewer. The purpose.

In order to solve the above problems, in one aspect of the present invention,
An input image color space converter that converts an input image signal for each frame of a moving image signal into a luminance image signal and a color difference image signal, and
A luminance image correction unit that creates a luminance correction image signal by using a histogram smoothing process in which the correction intensity is adjusted for the luminance image signal for each frame.
A color difference image correction unit that creates a color difference correction image signal with emphasized saturation for the color difference image signal for each frame, and a color difference image correction unit.
An image processing device including an output image color space conversion unit that converts the created luminance-corrected image signal and the color difference-corrected image signal into an output image signal in the same format as the input moving image signal.
The processing by the luminance image correction unit and the processing by the color difference image correction unit are executed in parallel for each frame.
The luminance image correction unit adjusts the correction intensity of the histogram smoothing in the histogram smoothing process for each frame .
The luminance image correction unit
A smoothing presence / absence determination unit that determines whether or not to execute the histogram smoothing correction based on the content content of the luminance image signal of one frame, and
The determination value is obtained by adding or subtracting a predetermined number of determination values associated with the correction intensity of histogram smoothing in the created luminance-corrected image signal with respect to a predetermined range according to the determination. A determination value calculation unit that gradually changes between consecutive frames of the moving image signal,
A past frame determination value storage unit that accumulates and stores the determination value for each frame within the predetermined range, and
When adjusting the correction strength of the histogram smoothing for each frame according to the determination value, the amount of change in the correction strength of the histogram smoothing with respect to the frame immediately before the one frame is gradually changed to a plurality of frames. Provided is an image processing apparatus including a weighted composite image creating unit for switching the state of presence / absence of correction of the histogram smoothing .

According to one aspect, in the image processing device, the correction strength by histogram smoothing can be adjusted for each frame of the moving image according to the content of the content without giving a sense of discomfort to the viewer.

Schematic of compressing dynamic range for a typical display (LDR). Conceptual diagram of histogram smoothing. The explanatory view which shows the schematic structure of the image processing apparatus which concerns on one Embodiment of this invention. An example of image correction in which an S-shaped tone curve is applied to enhance saturation. An example of creating a brightness histogram and a cumulative histogram from an image. An example of tone curve conversion using a cumulative histogram for a lookup table, and an example of image correction with histogram smoothing. The figure explaining the case classification determination of the histogram smoothing ON / OFF. As the first control example of the present invention, a transition example of the weighting coefficient of the histogram smoothed image and the original image when creating a corrected image of a moving image. The schematic diagram which shows the weighting composition procedure at the time of creating a composition image. An example of tone curve conversion and an example of image correction using a cumulative histogram with offset added to the lookup table. The detailed function block diagram of the image processing apparatus which concerns on the 1st control example of this invention. The flowchart which shows the image processing procedure which concerns on 1st Embodiment of this invention. A detailed flowchart showing the calculation procedure in the frame counter. An explanatory diagram for clipping the smoothing intensity of a histogram as a second control example of the present invention. The detailed function block diagram of the image processing apparatus which concerns on the 2nd control example of this invention. Color wheel for DLP projectors. The block diagram which shows an example of the configuration example of a DLP projector. The block diagram of the DLP controller including the chip which performs the image processing control of this invention.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In each drawing, the same components may be designated by the same reference numerals and duplicate description may be omitted.

〔overall structure〕
FIG. 3 is an explanatory diagram showing a schematic configuration of an image processing device 10 according to an embodiment of the present invention. The image processing device 10 according to the embodiment of the present invention includes an input image (RGB) storage unit 1, an RGB / YCbCr conversion unit 2, a luminance image correction unit 3, a color difference image correction unit 4, a YCbCr / RGB conversion unit 5, and an output. An image (RGB) storage unit 6 is provided. As will be described later in FIG. 11, the image processing device 10 may further include a storage color reflecting unit 7 and a setting information acquisition unit 8.

As shown in FIG. 3, after converting the RGB input signal (each frame of the moving image signal) into a YCbCr signal (luminance color difference signal), Y (luminance: Luminance) and CbCr (color difference Blue, Color Difference Red). Perform different conversion processes with.

Then, correction by histogram smoothing (also referred to as flattening or averaging) is applied to the luminance image (Y), and saturation enhancement is applied to the chromaticity image (CbCr).

Here, by setting the correction target of the image to the converted YCbCr signal, the processing load is reduced because the histogram is smoothed only for the luminance (Y), and the smoothing is executed only for the luminance signal, so that the color balance is also improved. It is preferably held.

At the same time, by applying saturation enhancement to CbCr, the vividness of color is increased and the HDR tone can be made more suitable.

Specifically, the RGB / YCbCr conversion unit (input image color space reflection unit) 2 executes the following conversion algorithm.

First, the RGB input signals (Rin, Gin, Bin) are converted into YCbCr signals (Yin, Cbin, Crin) using the following (Equation 1) to (Equation 3).
For example, the RGB signals are described herein on a 10-bit scale (0-123).

Here, ">>" is a shift operation, "<<" shifts to the left side, and ">>" shifts to the right side by the specified number of bits.

As described above, YCbCr converted into an integer type is used as the luminance color difference signal. Further, an integer operation and a shift operation are used for mutual conversion between RGB and YCbCr.

The above equations (1) to (3) mean shifting to the right side of 8 bits, that is, dividing by 256 (28). Since division is often wasted in the hardware implementation, it is very advantageous to replace division with shift operation.

After converting the input image signal in RGB into a YCbCr signal as described above, saturation enhancement (see FIG. 4) for CbCr (blue difference image, red difference image) and histogram smoothing for Y (luminance image) (see FIG. 4). (See FIGS. 5 to 9).

Further, the YCbCr / RGB conversion unit (output image color space conversion unit) 5 has YCbCr values (Yout, Cb_out, Cr_out) in which saturation is emphasized for CbCr and histogram smoothing processing is applied to Y. Is returned to RGB values (Rout, Gout, Bout) in the same format as the input image signal using the following (Equation 4) to (Equation 6) to obtain an output image signal.

The above equation is for the case where each RGB image is 10 bits, and if the conversion result is out of the range of 10 bits (0 to 1023), clipping is performed in that range.

<Saturation correction>
Next, saturation enhancement for CbCr (color difference image signal) converted from RGB will be described.

FIG. 4 shows an example of image correction in which an S-shaped tone curve is applied to enhance saturation. It is suitable for high-speed processing to use (reference) the S-shaped tone curve as a look-up table for saturation enhancement.

Assuming that the lookup table is Satu_Curve (gradation conversion function), Cb (blue difference image signal) and Cr (red difference image signal) are described as shown in (Equation 7) and (Equation 8) below. Multiply by the gradation conversion function to convert to the output value.

When the relationship between the input pixel value and the output pixel value in the lookup table, that is, the gradation conversion function is graphed, an S-shaped tone curve (S-shaped tone curve) is obtained.

When the S-shaped tone curve is used, in a low-contrast image in which the distribution of the histogram is biased to the center, the halftone is stretched and the contrast is increased. That is, the high frequency portion of the color difference image can be made higher and the low frequency portion can be made lower.

As a result, the saturation is preferably emphasized without changing the color.

As shown in Table 1, the output image is output by selecting and using one of the gradation conversion functions constituting the S-shaped tone curves having different correction intensities so that the correction intensity level of saturation enhancement can be set. (Output image signal) may be corrected.

By performing such saturation enhancement, a visual effect called the Helmholtz-Korlausch effect (HK effect) can be obtained. The HK effect is a visual effect that makes a color with higher saturation feel brighter even if the brightness is the same, and is an effective means for producing an HDR tone.

In particular, since a DLP (Digital Light Processing) projector tends to have dull colors as will be described later in FIG. 16, saturation enhancement that makes the colors vivid is effective in improving image quality.

The gradation conversion function used for this saturation enhancement can be greatly simplified.

For example, even if the input value of CbCr is 10 bits, the CbCr value converted by the gradation conversion function does not correspond to 10 bits (0 to 1023), and even if it is about 1/10 of 7 bits (0 to 127), the image quality deterioration does not occur. It turned out to be almost imperceptible.

Since the color difference (CbCr) is less sensitive to human vision than the luminance (Y), even if the CbCr processing is simplified, deterioration is less likely to be felt.

Therefore, the number of bits of the created color difference corrected image signal may be processed so as to be smaller than the number of bits of the input color difference image. As a result, the computational load of processing the color difference image can be reduced, the scale of hardware logic can be reduced, and the processing speed can be increased.

<Brightness correction>
A method of creating a luminance-corrected image signal for the luminance (Y) by using the histogram smoothing process will be described in detail below.

FIG. 5 shows an example of creating a luminance histogram and a cumulative histogram from the luminance image (original image). FIG. 6 shows a conversion example of a tone curve (referenced) using a cumulative histogram as a lookup table, and an image correction example of histogram smoothing.

First, a histogram of the luminance image (Y) converted by the above (Equation 1) is created.
It is preferable to spatially thin out the target pixels when obtaining the histogram because the processing load is lightened.

For example, sampling is performed every 16 pixels in both the horizontal and vertical directions, and a histogram is formed from them. Even if thinned out, the characteristics of the histogram, that is, the image quality, does not change so much, and only the processing load can be greatly reduced.

Assuming that the YCbCr image is compatible with 10 bits, the matrix for storing the histogram has 1024 steps (Org_Hist [0] to Org_Hist [1023]), so the histogram is for one frame of the target video (moving image) in (Equation 9). Is calculated as follows. "++" in the formula means that the frequency of the histogram is added one by one. An example of the created histogram is shown in FIG. 5 (b). In FIG. 5B, it is assumed that the horizontal axis shows the number of pixels (luminance) and the vertical axis shows the frequency (frequency), and the histogram is schematically shown.

Next, a cumulative histogram is formed from the obtained histogram. Specifically, as shown in (Equation 10), the cumulative histogram is set to Sum_Hist [0] to Sum_Hist [1023], and the cumulative histogram is formed by sequentially accumulating the frequencies in that range. An example of the created histogram is shown in FIG. 5 (c). In FIG. 5B, the horizontal axis shows the number of pixels (luminance) and the vertical axis shows the cumulative power (standardized power, relative ratio). When FIG. 5B is used as a tone curve, The horizontal axis may correspond to the input pixel value, and the vertical axis may correspond to the output pixel value.

In order to use the frequency of the cumulative histogram as a lookup table for brightness correction, the cumulative histogram is normalized to the range of 10 bits (0 to 1023), and it is designated as Equ_Table [0] to Equ_Table [1023].

The following (Equation 11) normalizes the cumulative histogram shown in FIG. 5 (c) to the range 0-1023 (see, for example, FIG. 2 (b)).

The Equ_Table [0] to Equ_Table [1023] generated here are used as a lookup table for the brightness correction of the output image as shown in the following (Equation 12).

A specific example of the luminance correction using the cumulative histogram as described above will be described with reference to FIG. As shown in the upper part of FIG. 6, in the case of dark content, the smoothing tone curve (correction tone curve, see FIG. 6B) based on the cumulative histogram is a high key that becomes convex, so it is corrected so that the brightness increases. Will be done.

On the contrary, as shown in the lower part of FIG. 6, in the case of bright content, the smoothing tone curve based on the cumulative histogram (see FIG. 6B) is a low key having a concave shape, and is corrected so as to suppress the brightness.

However, if the cumulative histogram is used as it is as a look-up table (see) in this way, it may be overcorrected depending on the content of the image included in the video, so the effect can be weakened in some cases. Is desirable.

<Adjustment of histogram smoothing>
FIG. 7 is a diagram for explaining the ON / OFF case classification determination of histogram smoothing. FIG. 8 shows a transition example of the mixing ratio of the histogram smoothed image and the original image when creating a corrected image (signal) of a moving image as a control example of the present invention.

Regarding the histogram smoothing process as described above, there is no problem if it is a continuous histogram such as the natural image shown in FIG. 7A, but in the case of content close to a binary image such as a sentence or a chart, the histogram is Since it is local, unnecessary gradation conversion may result in undesired luminance conversion. That is, the quality of correction by histogram smoothing depends on the content.

As an example, in the case of solid black in which all pixels are black (0), the frequency of the histogram is concentrated in Org_Hist [0], so that the cumulative histogram also has a constant value in the range of Sum_Hist [0] --Sum_Hist [1023]. Therefore, applying histogram smoothing to solid black results in inappropriate conversion results that are not black.

In addition to the black solid example, if the content is close to a binary image such as a sentence or a chart as shown in FIG. 7 (d), there is a possibility that an erroneous conversion close to a black change in the black solid image may occur. There is.

In order to prevent such improper conversion, it is easily determined whether the histogram of the image content is continuous or discrete. Then, when it is determined to be continuous, the histogram smoothing is performed, and when it is determined to be discrete, the histogram smoothing is not performed, and the Yin value (luminance image) is unprocessed and passed through. That is, the input / output tone curve uses a non-processed tone curve having a diagonal linear shape.

In order to determine whether the histogram is continuous or discrete based on the content content, methods such as using the variance of the histogram and the standard deviation can be considered, but these methods increase the processing load.

Therefore, the peak value of the histogram Org_Hist is used as an example of the determination method with light processing. Since the total frequency of the histogram is the number of pixels of the content, if the peak value which is the maximum value of the histogram is less than the peak predetermined value of the total frequency, it is regarded as continuous, and if it is equal to or more than the peak predetermined value, it is regarded as discrete. For example, FIG. 7B shows a peak value. It is assumed that the peak value in FIG. 7 (e) reaches the upper limit at the right end.

Although FIG. 7 is schematically shown, the “predetermined peak value” that is the threshold value is preferably around 1/16 of the maximum frequency, and the denominator is a power of 2 is suitable for hardware mounting. ..

Alternatively, whether or not the ratio of the frequency exceeding a certain predetermined value, which is smaller than the peak predetermined value, has a frequency of whether or not the ratio is equal to or more than the predetermined ratio with respect to the total luminance value (pixel value). If it is continuous or less than a ratio, it may be regarded as discrete. For example, predetermined values are also shown in FIGS. 7 (b) and 7 (e).

As described above, the histogram smoothing is switched ON / OFF according to the content by the determination using the peak value (or the ratio exceeding the predetermined value) of the histogram.

However, when the ON / OFF switching using the threshold value of the peak value and the predetermined peak value as described above is applied to the moving image, when the ON / OFF switching fluctuates above and below the ON / OFF threshold value depending on the frame, the histogram smoothing ON / A phenomenon occurs in which the OFF is also finely switched, and an inappropriate moving image may occur.

<First control example>
In order to avoid this, as a first control example, it is preferable to gradually change the intensity of histogram smoothing.

Therefore, in the case of a moving image in which it is determined that the ON / OFF of the histogram smoothing is switched infrequently with time, the count value (hdr_frame_cnt) for counting the number of consecutive ON and OFF frames is used as the weighting coefficient. It is used to gradually change the correction intensity of histogram smoothing. As a result, as shown in FIG. 8, it is possible to gently shift ON / OFF of histogram smoothing according to the content of the content, and it is possible to reduce the discomfort of the viewer.

<Adjustment of correction strength for histogram smoothing by adjusting mixing ratio>
For the transition of the above histogram smoothing (HDR), a smoothing tone curve (see FIG. 6B) created by using the cumulative histogram as a lookup table and an unprocessed tone curve (diagonal straight line rising to the right, so-called input). Create a correction tone curve by combining with (the pixel value as it is). Then, a luminance correction image (signal) is created based on the correction tone curve.

By weighting and setting the mixing ratio of the smoothed tone curve and the unprocessed tone curve when synthesizing the correction tone curve according to the judgment value (count value), the histogram smoothing of the brightness correction image is performed. The correction strength of the curve can be adjusted.

The correction by the mixed adjustment of the tone curve is executed for each frame described later.

For example, when it is determined that the histogram smoothing is OFF by comparing the peak value of the histogram with the predetermined peak value or the ratio of the pixel value (luminance value) having a frequency equal to or higher than the predetermined value as described above, the count value (cumulative). Judgment value) hdr_frame_cnt is decremented.

If the histogram smoothing is an OFF condition, the count value of the frame counter is decremented, that is, "-1 subtraction".

However, the lower limit of the count value hdr_frame_cnt is 0.

On the other hand, if the histogram smoothing is an ON condition, the count value of the frame counter is incremented, that is, "+1 addition" is performed.

However, the upper limit of the count value hdr_frame_cnt is an arbitrary value such as 32 frames or 64 frames, but the larger the upper limit, the more gradually the ON / OFF of the histogram smoothing is changed.

By weighting ON / OFF of histogram smoothing with this count value as a judgment value accumulated from the past frame, it is possible to switch wrinkles without suddenly switching ON / OFF of histogram smoothing depending on the content. ..

The following (Equation 15) shows an example in which the correction intensity of histogram smoothing is switched over 32 frames in 32 steps.

Since the correction strength is gradually changed in this way, it is possible to reduce the visual discomfort when switching the histogram smoothing ON / OFF according to the content.

In this example, an example in which the count value is changed by ± 1 is shown, but if the number is sufficiently smaller than the count range of the count value (for example, 0 to 31, 0 to 63, a predetermined range). , A predetermined number (for example, 1 to 5) may be changed.

<Creation of composite image>
FIG. 9 shows a schematic diagram showing a weighted composition procedure when creating a composite image. This schematic diagram corresponds to step S8 shown in the control flow of FIG. 12 described later.

The count value is input in step S801. Here, the count value is an example of a weighting coefficient, and as described in (Equation 13) to (Equation 15) above, for example, assuming that the total is 64 and one of the weighting coefficients hdr_frame_cnt is 50. , The other count value (weighting coefficient) is 14.

In step S802, a weighting coefficient (for example, 50) is applied to the image A to which the histogram smoothing is applied. Specifically, a weighting coefficient is applied to the smoothed tone curve for the histogram smoothed image.

In step S803, the original image B, which is an image to which histogram smoothing is not applied, is multiplied by a number (for example, 14) obtained by subtracting the weighting coefficient from the total number. Specifically, a weighting coefficient is applied to the unprocessed tone curve (diagonal straight line) for the original image.

Then, in step S804, the smoothed image created in S802 multiplied by the count value and the original image multiplied by the number obtained by subtracting the count value from the total value in S803 are mixed and combined, and divided by the total value (64). To create a composite image. As a specific example, the smoothed tone curve multiplied by the weighting coefficient (50) and the unprocessed tone curve (diagonal straight line) multiplied by the number obtained by subtracting the weighting coefficient from the total number (for example, 14) are added to 64. Divide by to create a correction tone curve.

Then, the image created in step S805 is output. Specifically, the brightness correction image (signal) is output by using the correction tone curve as a look-up table (multiplying the brightness input image by a gradation conversion coefficient that becomes a correction tone curve when graphed).

Therefore, as shown in FIG. 8, in the case of an image in which the above ON / OFF is switched in small steps in time, as shown in FIG. 9, the count value (hdr_frame_cnt) for counting the number of consecutive numbers is used as the weighting coefficient. The correction of histogram smoothing is switched ON / OFF (presence / absence) while gradually changing the mixing ratio of the image. Therefore, it is possible to reduce the discomfort of the viewer switching ON / OFF of the histogram smoothing.

<Adjusting the strength of the cumulative histogram (adjusting the setting level)>
Next, a method of setting the strength adjustment to be specified in advance for the range of brightness correction using the cumulative histogram will be shown. It is assumed that this setting is an arbitrary selection setting that is a prerequisite for the correction strength of the histogram smoothing of the luminance correction, regardless of the content.

FIG. 10 shows an example of tone curve conversion and an example of image correction using the cumulative histogram with offset added to the look-up table.

Allows the user to arbitrarily set the level of correction intensity for histogram smoothing of the output image.

When a uniform offset is added to the histogram of the content, the degree of concentration (bias) of the histogram is dispersed, so that the unevenness of the cumulative histogram becomes small and approaches a straight line.

Depending on the magnitude of the offset to be added, the strength of the brightness correction can be controlled by the user.

In the luminance image correction unit 3, a predetermined amount of offset is added to the histogram except for the level at which the correction intensity set by the user is the strongest, and the weaker the correction intensity level is, the more offset is added to the histogram.

For example, as shown in Table 2, the strength of histogram smoothing (maximum strength when mixed) is set by dividing into levels.

Specifically, the brightness image correction unit 3 adds a predetermined amount of offset to the histogram, creates an offset cumulative histogram from the histogram to which the offset is added, and creates an offset cumulative histogram normalized to the number of bits of the brightness signal. Use (see) as a lookup table to perform histogram smoothing.

Even when the offset is added to the histogram in this way, as shown in FIG. 9, the mixing ratio of the smoothed tone curve and the unprocessed tone curve is weighted according to the judgment value (count value) for correction. By synthesizing the tone curves, the correction intensity of histogram smoothing of the brightness correction image can be adjusted within the set correction intensity level.

<Structure of image processing unit>
FIG. 11 shows a functional block diagram of the image processing apparatus capable of executing the control shown in FIG.

The image processing device 10 according to the embodiment of the present invention includes an input image (RGB) storage unit 1, an RGB / YCbCr conversion unit 2, a luminance image correction unit 3, a color difference image correction unit 4, a YCbCr / RGB conversion unit 5, and an output image. (RGB) A storage unit 6, a storage color reflection unit 7, and a setting information acquisition unit 8 are provided.

The luminance image correction unit 3 includes a Y image storage unit 301, a histogram creation unit 302, an offset amount storage unit 303, an offset addition unit 304, a histogram accumulation unit 305, and a histogram standardization, which are used for histogram smoothing. A unit 306 and a smoothing tone curve creating unit 307 are provided.

Further, the luminance image correction unit 3 includes a peak value / predetermined value comparison unit 308, a past frame count value storage unit (past frame determination value storage unit) 309, and a count value calculation unit 310, which are used for creating a corrected image. , A weighted composite image creating unit 311 is provided. The weighted composite image creation unit 311 includes a correction tone curve synthesis unit 312 and a composite image creation unit 313.

The color difference image correction unit 4 includes a CbCr image storage unit 41, a tone curve storage unit 42, and a saturation enhancement processing unit 43.

The storage color reflection unit 7 includes a memory color content determination unit 71, a skin color storage unit 72, a sky blue storage unit 73, a gray storage unit 74, and a storage color ratio counter unit 75.

In the luminance image correction unit 3, the histogram creation unit 302 creates the histogram (luminance histogram) shown in FIG. 5B from the Y image stored in the Y image storage unit 301.

The offset amount is determined and the offset is added according to the correction intensity level set by the setting information acquisition unit 8.

The histogram accumulation unit 305 accumulates the histograms to create the cumulative histogram shown in FIG. 5 (c).

The histogram standardization unit 306 standardizes the histogram using the cumulative histogram (FIG. 5 (c)) as a lookup table, and the smoothing tone curve creation unit 307 uses the smoothing tone curve (FIG. 6 (b)) for correction. ) Is created.

Alternatively, the histogram standardization unit 306 and the smoothing tone curve creation unit 307 offset the offset cumulative histogram as an uptable and the offset adjustment cumulative histogram as a lookup table when the smoothing correction strength is set to be weak. An adjusted smoothing tone curve (FIG. 10 (b), FIG. 10 (c)) is created.

In addition, the peak value / predetermined value comparison unit 308 determines whether the peak value of the histogram of the image at that time (frame) exceeds the peak predetermined value, or the ratio of the frequency exceeding the predetermined value (total luminance value (pixels). The ratio of the luminance values having a frequency equal to or higher than the predetermined value (concentrated, frequently) to the value) is calculated (FIGS. 7 (b) and 7 (e)), and the histogram is smoothed. Determine whether to strengthen or weaken. That is, the peak value / predetermined value comparison unit 308 functions as a smoothing presence / absence determination unit.

The count value calculation unit (frame counter) 310 reads the count value immediately before being stored in the past frame count value storage unit 309, and determines the peak value / predetermined value comparison unit 308 to strengthen the smoothing process. If you increase or weaken, decrease the count value. Since the count value becomes the determination value, the count value calculation unit 310 functions as the determination value calculation unit.

The weighted composite image creation unit 311 is a correction tone curve in which an unprocessed tone curve (diagonal straight line) and a smoothing curve created by the smoothing tone curve creation unit 307 are mixed according to the weighting coefficient indicated by the count value. Create a composite image based on.

The processing in the storage color reflecting unit 7 that performs the processing of the storage color will be described in detail with reference to FIG.

In the color difference image correction unit 4, the saturation enhancement processing unit 43 refers to the correction tone curve stored in the tone curve storage unit 42 with respect to the CbCr image (FIG. 4A) stored in the CbCr image storage unit 41. (S-shaped curve) (see FIG. 4 (b)) is applied to create a saturated image (FIG. 4 (c)).

<Control flow>
FIG. 12 shows a flowchart showing the image processing procedure of the present invention.

In step S1, the image processing apparatus 10 starts scanning.

In step S2, the RGB pixel value is input as the input image and stored in the input image storage unit 1.

In step S3, the RGB / YCbCr conversion unit 2 converts the RGB image into a YCbCr image. After step S3, the luminance image (original image) B and the color difference image are separately processed in parallel.

The luminance image B created in step S3 is used in the histogram creation step of S4, the count value calculation step of S7, and the composite image creation step of S8.

In step S4, the histogram creation unit 302 thins out the histogram creation targets in the pixels to create a histogram.

In step S5, the histogram accumulation unit 305 creates a cumulative histogram based on the histogram or the histogram to which the offset is added.

In step S6, the smoothing tone curve creation unit 307 creates a smoothing tone curve B using the cumulative histogram or the offset cumulative histogram as a lookup table.

In step S7, the count value calculation unit 310 calculates the count value of the frame counter. Details of the counter will be described later together with FIG.

In step S8, the weighted composite image creation unit 311 applies the count value as a weighting coefficient, weights and mixes the luminance image (original image) and the histogram smoothed image using the tone curve, and the composite image (Y image). ) Is created.

Further, in parallel with steps S4 to S8, in step S9, a saturation-weighted image is created with respect to the color difference (saturation).

In step S10, the YCbCr / RGB conversion unit 5 combines the luminance correction image (Y), which is the composite image created in S8, with the color difference correction image (CbCr) with enhanced saturation, and converts it into an RGB image for output. To do.

In step S11, the output image (RGB) storage unit 6 outputs an RGB image (output image signal) for output.

The image processing of the present invention shown in FIG. 12 can be used in combination with other image processing. For example, the projector may be used in combination with pixel shift (wobbling) as a powerful means for achieving high resolution such as 4K / 8K.

For example, when outputting a 4K image with pixel shifts in two directions (when shift positions α and β exist), 4K can be obtained by applying the image processing algorithm of the present invention to shift positions α and shift positions β, respectively. It saves memory usage etc. compared to applying it to the video itself.

When the above image processing is controlled in combination with this pixel shift, both high resolution and HDR tone (easiness to see) can be realized.

<Counting method>
FIG. 13 shows a detailed flowchart showing the calculation procedure in the frame counter. This detailed flow corresponds to step S7 shown in FIG.

First, a standard flow that can realize a superposition transition by gradually changing the correction intensity of histogram smoothing shown in FIG. 8 will be described.

In step S701, the count value calculation unit 310 calls the immediately preceding count value stored in the past frame count value storage unit 309.

In step S702, the peak value / predetermined value comparison unit 308 assumes that the largest peak value of the target frame is continuous if it is less than the peak predetermined value, and discrete if it is greater than or equal to the peak predetermined value. .. Alternatively, the rate at which the frequency exceeds a predetermined value is calculated and compared to see if it is larger than the threshold value (FIGS. 8 (b) and 8 (e)). It is assumed that the peak predetermined value> the predetermined value indicating the ratio.

When the peak value is smaller than the peak predetermined value in S702, or when the frequency exceeds a certain predetermined value is large, the original image is regarded as a continuous histogram, and in step S703, the histogram smoothing is strengthened. The frame counter is added, and the previous count value + 1 is output in step S704.

When the peak value is larger than the peak predetermined value in S702, the original image is assumed to be a discrete histogram, a frame counter is added so as to strengthen the histogram smoothing in step S705, and the previous count value in step S706. Output +1.

When the calculation of the count value is completed, the process proceeds to the composition step by the weighted composition image creation unit 311 (FIGS. 11, S8).

The above S701 to S706 are normal counts that realize the gradual change of the correction intensity shown in FIG. 8, but the setting of the count value may be further changed in consideration of the memory color.

<Adjustment by memory color>
Skin color, sky, and green are called memory colors, and are known as colors that are particularly memorable. While the above-mentioned histogram smoothing greatly improves the clarity of the image, if there is noise in the image, the noise may also be emphasized.

Noise often contained in video includes block noise and mosquito noise when compressed to MPEG (Moving Picture Experts Group).

When histogram smoothing is applied to an image, the block noise and mosquito noise may also be emphasized, and the impression of the image is likely to be impaired especially when the skin color or the sky is emphasized.

Therefore, the flesh color and sky color, which are memory colors, and the content of gray, which is visually sensitive, are used for the ON / OFF determination of the histogram smoothing, and when the content is above a certain level, the above S702 is used. By reducing the count value and gradually shifting the histogram smoothing to OFF regardless of the determination result, it is possible to suppress unpleasant noise enhancement of skin color and sky.

In S707, the storage color content determination unit 71 records the RGB pixel value for determining whether or not the skin color is from the RGB image which is the input image for each frame in the skin color storage unit 72.

Then, the RGB pixel value recorded in the skin color storage unit 72 is called up in S708. Whether or not it is skin color (whether or not it corresponds to skin color) is determined from the RGB pixel value, and if it is determined that it is not skin color (No), the skin color counter of S709 is not incremented.

If it is determined that the skin color is skin color (Yes in S708), the process proceeds to S709 and the skin color counter is incremented.

Specifically, the skin color count value skin_counter that counts the number of skin color pixels, the sky blue count value sky_counter that counts the number of sky blue pixels, and the gray count value gray_counter are used for determination.

For example, in the case of skin color, for RGB pixel values (10 bits), the pixel value of R (red) is 1.2 times or more that of G (green), 1.4 times or more of B (blue), and further R (red). ) Is considered to be skin color when the pixel value is 255 (/ 1023) or more, and the skin color count value skin_counter is incremented.

In S710, when the count is equal to or greater than the threshold value, that is, the proportion of the skin color in the input image is equal to or greater than the threshold value, the process proceeds to S705 regardless of the presence or absence of the result of S702, and the frame counter is forcibly subtracted.

The determination for the sky color of S711 to S713 is the same as the determination for the skin color of S708 to S710, and is executed in parallel with the determination for the skin color.

For example, in the case of sky color, for RGB pixel value (10 bits), the pixel value of B (blue) is 1.2 times or more of G (green), 1.2 times or more of R (red), and further R. When the pixel value of is 511 (/ 1023), it is regarded as the sky color, and the count value sky_counter of the sky color is incremented.

Further, the determination for gray of S714 to S716 is the same as the determination for the skin color of S708 to S710 and the determination for the sky color of S711 to S713, and is parallel to the determination for the skin color and the sky color. Will be executed.

For example, in the case of gray, for RGB pixel values (10 bits), the pixel value of R (red) is ± 10 of B (blue), the pixel value of G (green) is ± 10 of B (blue), and even B. When the pixel value of is 511 (/ 1023) or more, it is regarded as gray, and the gray count value gray_counter is incremented.

When the cumulative values of the above memory color count values skin_counter, sky_counter, and gray_counter each exceed a predetermined value, histogram smoothing is subtracted so that noise is not emphasized.

By controlling in this way, the image processing device does not give a sense of discomfort to the viewer even when the natural image and the binary image are switched according to the content of the content, for example, in frame units of the moving image. The correction intensity by histogram smoothing can be adjusted.

<Second control example>
In the first control example described above, when an extreme change occurs, such as switching between the content of a natural image as shown in FIG. 7 (a) and the content of a binary image as shown in FIG. 7 (d). Therefore, it was particularly effective when the switched image was maintained for a certain period of time.

However, if the change in the video is rapid (for example, when the scene suddenly changes to a significantly different scene), the correction strength of the smoothing of the histogram for each frame as described above should be gradually changed before the sudden change. There may be cases where the optimum correction intensity is dragged and it is not optimal in the scene after a sudden change (for example, when the image correction cannot catch up).

Further, even when similar scenes continue continuously, visually unpleasant flicker may occur due to fine fluctuations in the correction intensity of histogram smoothing.

Therefore, the luminance correction control may be switched as in the second control example below, depending on the type of moving image.

In this control example, the correction strength of histogram smoothing is clipped. That is, in this control example, the luminance image correction unit adjusts for each frame so that the correction intensity for histogram smoothing calculated based on the content content of the luminance image for each frame is within a predetermined range. ..

The outline is shown in FIG. In FIG. 14, (a) shows the correspondence between the histogram and the smoothing intensity, (b) shows an explanatory diagram of the clipping of the smoothing intensity, and (c) is the darkness to which the clipping of (b) is applied. An example of brightness adjustment in a scene is shown.

In this control example, when the smoothing intensity calculated from the peak value of the histogram calculated based on the luminance image for each frame exceeds the bright direction threshold value or the dark direction threshold value, the bright direction threshold value or the dark direction threshold value is exceeded. The smoothing intensity is clipped at the light or dark threshold so that it does not. That is, the correction intensity is set so that the correction intensity calculated from the darkly biased or brightly biased frame is rounded to the bright direction threshold value or the dark direction threshold value. The main body of this control example will be described with reference to FIG. 15 below.

Here, in the graph of FIG. 14B, when the smoothing intensity calculated from the peak value of the histogram exceeds the bright direction threshold value, that is, in a darkly biased frame, the smoothing intensity is the limit value of the bright direction correction. Correct so that it falls within the bright direction threshold.

Further, in the graph of FIG. 14B, when the smoothing intensity calculated from the peak value of the histogram exceeds the dark threshold value (below the darkness threshold value in FIG. 14B), that is, it is brightly biased. In the frame, the smoothing intensity is corrected so as to fall within the dark threshold value which is the limit value of the dark correction.

By this control, when the brightness and darkness of the original image, that is, the peak value in the histogram of the luminance image signal fluctuates excessively, the smoothing intensity is set according to the bright direction threshold value and the dark direction threshold value. The correction strength falls within a certain range. Therefore, the stability of the intensity of histogram smoothing is improved, and while it is possible to cope with a sudden ON / OFF transition, it is possible to reduce the flicker that appears when the brightness of the image changes suddenly, and the viewer feels uncomfortable. Can be reduced.

Here, as in the dark scene illustrated in FIG. 14C, the brightness value is obtained by clipping the correction intensity of the smoothing process of the histogram of the luminance image signal of the frame biased to the dark side to the bright direction threshold value. The upper and lower limit values of are also clipped, which is effective in reducing flicker.

In this control example, the correction intensity of smoothing is not cumulatively changed as in the above control example 1 in order to correspond to the quick operation for each frame, and as shown in FIG. 14B, the bright direction threshold value and the dark direction The smoothing correction intensity is limited (clipped) so that the intensity is below the threshold value, and is set for each frame.

FIG. 15 shows a detailed functional block diagram of the image processing apparatus 1A according to the second control example of the present invention. Hereinafter, the points different from the image processing apparatus 1 of FIG. 11 described above will be described.

In FIG. 15, the luminance image correction unit 3A includes a Y image storage unit 301, a histogram creation unit 302, a histogram accumulation unit 321, a peak value calculation unit 322, a correction intensity calculation unit 323, a correction intensity setting unit 324, and a correction intensity threshold storage unit. It has 325, a histogram standardization unit 326, a smoothing tone curve creation unit 327, and a brightness correction image creation unit 328.

The histogram accumulation unit 321 accumulates a histogram of the luminance image (signal) for each frame. Based on this cumulative result, it is determined whether to correct the luminance image in the bright direction or in the dark direction.

The peak value calculation unit 322 detects the peak of the histogram and calculates (calculates) the peak value.

The correction intensity calculation unit 323 calculates the smoothing intensity based on the relational expression shown in FIG. 14A based on the calculated peak value of the histogram. FIG. 14A shows a correspondence formula (relational formula) between the histogram and the smoothing intensity. It is assumed that the relational expression shown in FIG. 14A is preliminarily mounted inside the image processing apparatus (circuit) 10, for example.

With reference to the relational expression of FIG. 14A, the smaller the peak value of the histogram, the stronger the smoothing intensity. Further, when the peak value of the histogram exceeds a predetermined value, the smoothing intensity of the histogram becomes 0.

For example, in images such as photographs and illustrations, the peak value of the histogram is often low, and as shown in FIG. 14A, a relatively strong smoothing intensity is calculated.

On the other hand, in the case of a text image such as Word or Excel as shown in FIG. 7 (d), the peak value is high, so that the smoothing intensity is 0 as shown in FIG. 14 (a), that is, smoothing is performed. Not performed.

The correction intensity calculation unit 323 also acquires information on the correction direction (bright direction or dark direction) of the luminance image determined from the cumulative histogram by the histogram accumulation unit 321, and together with the above-calculated smoothing intensity, the correction intensity. Output to the setting unit 324.

The correction intensity threshold storage unit 325 sets a bright direction threshold value when correcting in the bright direction and a dark direction threshold value when correcting in the dark direction as the correction intensity threshold value shown in FIG. 14B. This threshold value may be arbitrarily set by the user by the setting information acquisition unit 8A, or may be preset at the time of manufacturing. Further, the intensities of the bright threshold and the dark threshold may be the same or different.

The correction intensity setting unit 324 compares the correction intensity calculated by the correction intensity calculation unit 323 with the bright direction threshold value and the dark direction threshold value of the correction intensity stored in the correction intensity threshold value storage unit 325, and compares the bright direction threshold value. If it is within the range that does not exceed the dark threshold, the calculated correction strength is set as it is as the correction strength.

On the other hand, when the smoothing intensity associated with the peak value calculated based on the luminance image signal of the dark frame (one claim) exceeds the bright direction threshold value, the correction intensity setting unit 324 smoothes the smoothing. The conversion intensity is set according to the brightness threshold.

Further, when the smoothing intensity associated with the peak value calculated based on the luminance image signal of the bright frame (other claim) exceeds the dark threshold value, the correction intensity setting unit 324 smoothes the smoothing. The chemical intensity is set according to the dark threshold.

Then, the histogram standardization unit 326 creates a cumulative histogram normalized to the number of bits of the luminance image signal.

The smoothing tone curve creation unit 327 uses the normalized cumulative histogram as a look-up table (see), and uses the smoothing intensity set by the correction intensity setting unit 324 as a smoothing tone for histogram smoothing. Create a curve.

The luminance-corrected image creation unit 328 performs histogram smoothing processing on the luminance image using the smoothing tone curve to create a luminance-corrected image.

Then, as in FIG. 11, the YCbCr / RGB conversion unit 5 combines the luminance-corrected image (Y), which is the created composite image, with the color difference-corrected image (CbCr) with enhanced saturation, and outputs an RGB image for output. Convert to.

After that, an RGB image for output is output from the output image (RGB) storage unit 6.

By controlling in this way, in the image processing device, for example, even when a natural image is included and the change in the moving image is rapid, the flicker of the screen is prevented without giving a sense of discomfort to the viewer, and the moving image is displayed. The correction strength by histogram smoothing can be adjusted for each frame.

Although the first control example and the second control example have been described separately, the above two control examples may be selectively selected or may be executed in combination. ..

Specifically, the luminance image correction unit determines, depending on the type of video content to be projected.
(1) The judgment value associated with the correction strength of histogram smoothing in the created luminance correction image is gradually changed between consecutive frames of the moving image signal, and the correction strength of histogram smoothing is set in frames according to the judgment value. Brightness correction adjusted for each,
(2) The luminance correction of the luminance image signal for each frame to be created is calculated by calculating the correction intensity of histogram smoothing based on the peak value of the histogram and clipping the luminance correction.
It is selectable or can be combined.

For example, as described with reference to FIG. 13, each count value of the skin color count value (skin_counter), the sky blue count value (sky_counter), and the gray count value (gray_counter) of the storage color ratio counter unit 75 that subtracts the correction intensity of the histogram smoothing. Set a threshold for the upper limit of. Then, when the counter value exceeds the upper limit, clipping to the threshold value also leads to stabilization of the correction strength of histogram smoothing, and it is possible to prevent flicker on the screen.

The image processing control according to the embodiment of the present invention as described above is effective for all video devices, and in particular, an integrated circuit in which an FPGA (Field-programmable gate array) is formed into an array in which a logic circuit can be programmed in the field. ) And ASIC (application specific integrated circuit), it is suitable for application to a DLP projector, which is an example of an image projection device, which has restrictions on the implementation and modification of hardware.

<Application to color wheels>
FIG. 16 shows the color wheel of the DLP projector. In FIG. 16, (a) is a configuration example in which fan-shaped color separation filters of each RGB color are sequentially arranged on a disk, and (b) is each RGB color + C (cyan), Y (yellow), W (white). ) Is shown in a configuration example in which the color separation filters are sequentially arranged.

A DLP projector as shown in FIG. 17, which will be described later, displays by time-division color mixing in which a color wheel in which primary colors are divided and arranged is rotated at high speed as shown in FIG. 16 (a). Therefore, the utilization efficiency of the primary colors is lower than that of a general display or a liquid crystal projector, and the colors tend to be dull when projected in a raw state. Therefore, it is preferable to carry out the above-mentioned image processing control of the present invention.

Further, since the color wheel shown in FIG. 16B is provided with a cyan or yellow color separation filter, it is easier to reproduce a yellow or light blue color reproduction region than in FIG. 16A due to the structure.

Therefore, when the color wheel of the filter color shown in FIG. 16A is used in the setting of the saturation enhancement of the color difference image as shown in FIGS. 4 and 1 described above, the saturation enhancement is strengthened and FIG. 16B is shown. When using a color wheel with a filter color as shown in, it is preferable to weaken the saturation enhancement.

<Application to DLP projectors>
FIG. 17 is a block diagram showing an example of a configuration example of a DLP projector (image projection device) to which the image processing method of the present invention can be applied.

For example, as shown in FIG. 17, the projector 20 includes an input I / F 201, an operation panel 213, a DLP controller 210, a ROM 215, a RAM 216, a speaker 221 and a light emitting device 231 and a color wheel 233, and a DMD ( A Digital Mirror Device) 235 is provided.

The DLP controller 210 is an example of a control unit, and the light emitting device 231, the color wheel 233, and the DMD 235 are examples of a projection unit.

The input I / F 201 has an input interface such as a LAN controller 202, an AFE (Analog Front End) 203, an HDMI (registered trademark) 204, and a USB 205.

The remote controller 214 capable of communicating with the operation panel 213 and the projector 20 notifies the DLP controller 210 of various operation inputs from the user.

The communication unit 223 wirelessly communicates with an external device such as a PC or an electronic pen, and can be realized by the wireless communication device.

The DLP controller 210 executes a program stored in the ROM 215 and controls each part of the projector 20 by using the RAM 216 as a work area.

The DLP controller 210 performs various image processing such as gradation conversion processing, scaling, and trapezoidal correction on the projected image input from the input I / F 201, and displays the partially projected image after the image processing on the DMD 235. Further, the DLP controller 210 interlocks with light emission control for the light emitting device 231, rotation control for the color wheel 233, display control for the DMD 235, and audio output control for the speaker 221.

Examples of the light emitting device 231 include a lamp and a ballast, an LED (Light Emitting Diode) light source, and the like.

Then, when the DLP controller 210 rotates the color wheel 233, the color wheel 233 separates the light source light from the light emitting device 231 into each RGB color by time division, and the monochromatic light of each RGB is incident on the entire display area of the DMD 235. Let me.

FIG. 18 shows an example of a DLP controller.

An FPGA or ASIC that includes a functional configuration capable of executing the above-mentioned image processing method can be easily implemented.

Therefore, as shown in FIG. 18, by retrofitting the FPGA or ASIC inside the DLP controller and in front of the CPU, the effect of improving the image quality by HDR by the image processing according to the content by the gradation conversion processing is brought about.

In this way, an image processing device that has a large HDR-like effect and can switch the correction intensity by histogram smoothing for each frame of a moving image according to the content of the content without giving a sense of discomfort to the viewer is relatively available. Hardware implementation is possible with inexpensive FPGAs and the like.

Although FIGS. 17 and 18 have described an example of implementing the image processing of the present invention on a DLP projector, the application of the image processing of the present invention is not limited to the DLP projector. For example, it can be applied to a video projector that reproduces a moving image, a projector, a video equipped with a liquid crystal panel, an in-vehicle camera, a video, a television, a personal computer, a tablet, a smartphone, a mobile phone, and the like. An example in which YCbCr is used as a signal has been described, but a YUV ((Y) Lumina, U Axis blue component, V axis red component) or the like, which is a luminance color difference signal similar to that, can be suitably converted in the same manner as YCbCr.

Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and is within the scope of the gist of the embodiment of the present invention described in the claims. Various modifications and changes are possible.

1 Input image (RGB) storage unit 2 RGB / YCbCr conversion unit (input image color space conversion unit)
3 Luminance image correction unit 4 Saturation image correction unit 5 YCbCr / RGB conversion unit (output image color space conversion unit)
7 Output image (RGB) storage unit 8 Storage color reflection unit 9 Setting information acquisition unit 10 Image processing device (FPGA, ASIC)
20 Image projection device 41 CbCr image storage unit 42 S-shaped tone curve storage unit 43 Saturation enhancement processing unit 71 Memory color content determination unit 72 Skin color storage unit 73 Sky blue storage unit 74 Gray storage unit 75 Storage color ratio counter unit 301 Y Image storage Part 302 Schematic creation part 303 Offset amount storage part 304 Offset addition part 305 histogram accumulation part 306 histogram standardization part 307 Smoothing tone curve creation part 308 Peak value / predetermined value comparison part (smoothing presence / absence judgment part)
309 Past frame count value storage unit (past frame judgment value storage unit)
310 Count value calculation unit (judgment value calculation unit)
311 Weighted composite image creation unit 312 Correction tone curve composition unit 313 Composite image creation unit 321 Histogram accumulation unit 322 Peak value calculation unit 323 Correction strength calculation unit 324 Correction strength setting unit 325 Correction intensity threshold storage unit 326 Histogram standardization unit 327 Smoothing Tone curve creation unit 328 Brightness correction image creation unit

Japanese Unexamined Patent Publication No. 2012-049636

Standard retouching technique using YCrCb color (written by Toru Kasai)

Claims (14)

An input image color space converter that converts an input image signal for each frame of a moving image signal into a luminance image signal and a color difference image signal, and
A luminance image correction unit that creates a luminance correction image signal by using a histogram smoothing process in which the correction intensity is adjusted for the luminance image signal for each frame.
A color difference image correction unit that creates a color difference correction image signal with emphasized saturation for the color difference image signal for each frame, and a color difference image correction unit.
An image processing device including an output image color space conversion unit that converts the created luminance-corrected image signal and the color difference-corrected image signal into an output image signal in the same format as the input moving image signal.
The processing by the luminance image correction unit and the processing by the color difference image correction unit are executed in parallel for each frame.
The luminance image correction unit adjusts the correction intensity of the histogram smoothing in the histogram smoothing process for each frame .
The luminance image correction unit
A smoothing presence / absence determination unit that determines whether or not to execute the histogram smoothing correction based on the content content of the luminance image signal of one frame, and
The determination value is obtained by adding or subtracting a predetermined number of determination values associated with the correction intensity of histogram smoothing in the created luminance-corrected image signal with respect to a predetermined range according to the determination. A determination value calculation unit that gradually changes between consecutive frames of the moving image signal,
A past frame determination value storage unit that accumulates and stores the determination value for each frame within the predetermined range, and
When adjusting the correction strength of the histogram smoothing for each frame according to the determination value, the amount of change in the correction strength of the histogram smoothing with respect to the frame immediately before the one frame is gradually changed to a plurality of frames. An image processing apparatus comprising: a weighted composite image creating unit for switching a state of presence / absence of correction of histogram smoothing . The luminance image correction unit
A histogram creation unit that creates a histogram of the luminance image signal for each frame,
A histogram accumulation part that creates a cumulative histogram from the histogram and
A smoothing tone curve creation unit that creates a smoothing tone curve with reference to the cumulative histogram is provided.
The weighted composite image creation unit combines the smoothed tone curve and the unprocessed tone curve to create a correction tone curve, and the brightness correction based on the correction tone curve synthesis unit and the correction tone curve. It is equipped with a composite image creation unit that creates an image signal.
In the correction tone curve synthesizing unit, the mixing ratio of the smoothed tone curve and the unprocessed tone curve when synthesizing the correction tone curve is weighted and set according to the determination value. The image processing apparatus according to claim 1 , wherein the correction intensity of the histogram smoothing of the luminance-corrected image signal can be adjusted. The level of the correction intensity of the output image signal can be arbitrarily set by the user.
Except for the level where the correction intensity set by the user is the strongest, the luminance image correction unit adds a predetermined amount of offset to the histogram of the created luminance image signal for each frame.
The image processing apparatus according to claim 2 , wherein the weaker the level of the correction intensity, the larger the amount of offset added to the histogram. The luminance image correction unit further includes an offset addition unit that adds the predetermined amount of offset to the histogram.
The histogram accumulation unit creates an offset adjustment accumulation histogram from the histogram to which the offset is added.
The smoothing tone curve creation unit creates an offset adjustment smoothing tone curve with reference to the offset adjustment cumulative histogram normalized to the number of bits of the luminance image signal.
The correction tone curve synthesizing unit creates a correction tone curve by synthesizing the offset adjustment smoothing tone curve and the unprocessed tone curve.
The composite image creation unit creates a luminance correction image signal based on the correction tone curve, and creates a luminance correction image signal.
In the correction tone curve synthesizing unit, the mixing ratio of the offset adjustment smoothing tone curve and the unprocessed tone curve when synthesizing the correction tone curve is weighted according to the determination value. The image processing apparatus according to claim 3 , wherein the correction intensity of the histogram smoothing of the brightness-corrected image signal can be adjusted within the set correction intensity level. When the determination value calculation unit of the luminance image correction unit determines to execute the correction of the histogram smoothing, the determination value is added, and when it is determined not to execute the correction of the histogram smoothing, the determination value is described. The image processing apparatus according to any one of claims 1 to 4 , wherein the determination value is subtracted. The content rate of the stored color of the input image signal for each frame is detected, and the content rate of the stored color is also reflected in the determination of whether to strengthen or weaken the histogram smoothing for the luminance image correction unit. The image processing apparatus according to any one of claims 1 to 4 , further comprising a storage color reflecting unit. When the storage color reflecting unit determines that the content rate of the stored color in one frame is higher than a predetermined threshold value, the luminance image correction unit invalidates the determination value and disables the determination value of the one frame. The image processing apparatus according to claim 6 , wherein the correction intensity due to the histogram smoothing is changed so as to be weakened with respect to the frame immediately before the one frame. An input image color space converter that converts an input image signal for each frame of a moving image signal into a luminance image signal and a color difference image signal, and
A luminance image correction unit that creates a luminance correction image signal by using a histogram smoothing process in which the correction intensity is adjusted for the luminance image signal for each frame.
A color difference image correction unit that creates a color difference correction image signal with emphasized saturation for the color difference image signal for each frame, and a color difference image correction unit.
An image processing device including an output image color space conversion unit that converts the created luminance-corrected image signal and the color difference-corrected image signal into an output image signal in the same format as the input moving image signal.
The processing by the luminance image correction unit and the processing by the color difference image correction unit are executed in parallel for each frame.
The luminance image correction unit adjusts the correction intensity of the histogram smoothing in the histogram smoothing process for each frame.
The luminance image correction unit
A histogram creation unit that creates a histogram of the luminance image signal for each frame,
A peak value calculation unit that detects a peak from the histogram and calculates the peak value,
A correction intensity calculation unit that calculates the correction intensity of histogram smoothing that converts the luminance image signal for each frame for each frame based on the peak value.
A correction intensity threshold storage unit that stores in advance a bright threshold when correcting the correction intensity of the histogram smoothing in the bright direction and a dark threshold when correcting in the dark direction.
When the correction strength of the histogram smoothing associated with the peak value calculated based on the brightness image signal of the dark frame exceeds the bright direction threshold value, the correction strength of the histogram smoothing in the created brightness correction image signal Is set according to the bright-direction threshold value, and when the correction intensity of histogram smoothing associated with the peak value calculated based on the brightness image signal of the bright frame exceeds the dark-direction threshold value, the brightness to be created is created. An image processing apparatus including a correction intensity setting unit that sets the correction intensity of the histogram smoothing in the corrected image signal in accordance with the dark threshold value. The luminance image correction unit depends on the type of content of the moving image to be projected.
The judgment value associated with the correction intensity of the histogram smoothing in the brightness-corrected image signal for each frame to be created is gradually changed between consecutive frames of the moving image signal, and the histogram smoothing is performed according to the judgment value. Brightness correction that adjusts the correction strength for each frame,
A histogram of the luminance image signal for each frame is created, and based on the peak value calculated from the histogram, the correction intensity of the histogram smoothing of the luminance image signal for each frame to be created is adjusted for each frame. The image processing apparatus according to any one of claims 1 to 8 , wherein the luminance correction, which is set by limiting the correction intensity to the light direction threshold value and the dark direction threshold value or less, can be selected. The image processing apparatus according to any one of claims 1 to 9 , wherein the luminance image correction unit spatially thins out the pixels of the luminance image signal for each frame to form a histogram. The color difference image correction unit is characterized in that the saturation is emphasized by using an S-shaped tone curve that makes the high frequency portion of the pixel value of the color difference image signal higher and the low frequency portion lower. The image processing apparatus according to any one of claims 1 to 10 . The image processing apparatus according to claim 11 , wherein the number of bits of the created color difference corrected image signal is smaller than the number of bits of the input color difference image signal. An image projection apparatus according to any one of claims 1 to 12 , wherein the image processing apparatus according to any one of claims 1 to 12 is executed by FPGA or ASIC. An input image color space conversion step for converting an input image signal in frame units of a moving image signal into a luminance image signal and a color difference image signal, and
A luminance image correction step that creates a luminance image signal for the luminance image signal of each frame by using a histogram smoothing process in which the correction intensity is adjusted, and
A color difference image correction step that creates a color difference correction image signal with emphasized saturation for the color difference image signal of each frame, and
An image processing method comprising the output image color space conversion step of converting the created luminance-corrected image signal and the color difference-corrected image signal into an output image signal of the same format as the input moving image signal.
The luminance image correction step and the color difference image correction step are executed in parallel for each frame.
In the luminance image correction step, the correction intensity of the histogram smoothing in the histogram smoothing process is adjusted for each frame .
The luminance image correction step
A smoothing presence / absence determination step for determining whether or not to execute the histogram smoothing correction based on the content content of the luminance image signal of one frame, and
The determination value is obtained by adding or subtracting a predetermined number of determination values associated with the correction intensity of histogram smoothing in the created luminance-corrected image signal with respect to a predetermined range according to the determination. A determination value calculation process that gradually changes between consecutive frames of the moving image signal, and
A past frame determination value storage step of accumulating and storing the determination value for each frame within the predetermined range, and
When adjusting the correction strength of the histogram smoothing for each frame according to the determination value, the amount of change in the correction strength of the histogram smoothing with respect to the frame immediately before the one frame is gradually changed to a plurality of frames. An image processing method comprising a weighted composite image creating step of switching the state of presence / absence of correction of the histogram smoothing .