CN100477721C - Visual processing device, visual processing method, visual processing program, integrated circuit, display device, photographing device, and portable information terminal - Google Patents

Abstract

The present invention provides a visual processing device having a hardware configuration independent of the implemented visual processing. The visual processing device (1) includes a spatial processing unit (2) and a visual processing unit (3). A spatial processing unit (2) performs predetermined processing on an input signal (IS) and outputs an unsharp signal (US). The visual processing part (3) is based on the relationship between the input signal (IS) and the unsharp signal (US) given as input, and the output signal (OS) which is the input signal (IS) after visual processing 2D (LUT4), output the output signal (OS).

Description

Visual processing device, visual processing method, visual processing program, integrated circuit, display device, photographing device, and portable information terminal

technical field

The present invention relates to a visual processing device, in particular to a visual processing device for visual processing such as spatial processing or grayscale processing of image signals. Furthermore, another present invention relates to a visual processing method, a visual processing program, an integrated circuit, a display device, a photographing device, and a portable information terminal.

Background technique

As visual processing of an image signal of an original image, spatial processing and gradation processing are known.

The so-called spatial processing is a process of performing a pixel of interest using pixels around the pixel of interest to which filtering is applied. In addition, there is known a technique for performing contrast enhancement, dynamic range (DR) compression, and the like of an original image using a spatially processed image signal. In contrast enhancement, the difference between the original image and the blurred signal (the clear component of the image) is added to the original image to perform image sharpening processing. In DR compression, a part of the blurred signal is subtracted from the original image to compress the dynamic range.

The gradation processing is a process of converting pixel values using a look-up table (LUT) for each pixel of interest regardless of surrounding pixels of the pixel of interest, and is also called gamma correction. For example, in the case of contrast enhancement, pixel value conversion is performed using a LUT that creates a gradation of a gradation level that appears frequently (larger in area) in the original image. As gradation processing using LUT, there are known gradation processing (histogram equalization method) in which one LUT is determined for the entire original image, and determination of LUT for each image region obtained by dividing the original image into a plurality of image regions. The gradation processing to be used (local histogram equalization method) (see, for example, JP-A-2000-57335 (page 3, FIGS. 13 to 16 )).

104 to 107 , the gradation processing used to determine the LUT for each of the divisions of the original image into a plurality of image areas will be described.

FIG. 104 shows the visual processing device 300 used to determine the LUT for each of the original image divided into a plurality of image areas. The visual processing device 300 includes: an image segmentation unit 301, which divides the original image input as the input signal IS into a plurality of image regions Sm (1≤m≤: n is the number of divisions of the original image); The part 310 derives the gradation transformation curve Cm for each image area Sm; and the gradation processing part 304 loads the gradation transformation curve Cm, and outputs the output signal OS after gradation processing for each image region Sm. The gradation transformation curve derivation unit 310 is composed of: the histogram creation unit 302, which creates a brightness histogram Hm in each image region Sm; The gradation of the region Sm is configured by converting the region Cm.

The operation of each part will be described using FIGS. 105 to 107 . The image dividing unit 301 divides the original image input as the input signal IS into a plurality (n) of image regions (see FIG. 105( a )). The histogram creation unit 302 creates a brightness histogram Hm (see FIG. 106 ) for each image region Sm. Each brightness histogram Hm shows the distribution state of brightness values of all pixels in the image region Sm. That is, in the brightness histogram Hm shown in FIGS. 106( a ) to ( d ), the horizontal axis represents the brightness level of the input signal IS, and the vertical axis represents the number of pixels. The gradation curve creation unit 303 accumulates the "number of pixels" of the lightness histogram Hm in order of lightness, and sets the accumulated curve as the gradation transformation region Cm (see FIG. 107 ). In the gradation transformation curve Cm shown in FIG. 107 , the horizontal axis represents the brightness value of the pixels in the image region Sm in the input signal IS, and the vertical axis represents the brightness values of the pixels in the image region Sm in the output signal OS. The gradation processing unit 304 loads the gradation transformation curve, and converts the lightness values of the pixels in the image region Sm in the input signal IS based on the gradation transformation curve Cm. In this way, the slope of the gray scale with higher frequency of occurrence is established in each block, and the contrast feeling of each block is improved.

On the other hand, visual processing combining spatial processing and gradation processing is also known. Conventional visual processing that combines spatial processing and gradation processing will be described using FIGS. 108 to 109 .

FIG. 108 shows a visual processing device 400 for edge enhancement and contrast enhancement using unsharp masking. The visual processing device 400 shown in FIG. 108 includes: a spatial processing unit 401, which performs spatial processing on the input signal IS, and outputs an unsharp signal US; a subtraction unit 402, which subtracts the unsharp signal from the input signal IS. US, which outputs the difference signal DS; the enhancement processing unit 403, which performs enhancement processing on the difference signal DS, and outputs the enhancement processing signal TS; the addition unit 404, which adds the input signal IS to the enhancement processing signal TS, and outputs the signal OS output.

Here, the enhancement processing is performed on the difference signal DS using a linear or nonlinear enhancement function. Fig. 109 shows reinforcement functions R1 to R3. 109, the horizontal axis represents the difference signal DS; the vertical axis represents the enhanced signal TS. The enhancement function R1 is a linear enhancement function for the difference signal DS. The enhancement function R1 is a gain adjustment function represented by R1(x)=0.5x(x, is the value of the difference signal DS). The enhancement function R2 is a nonlinear enhancement function for the difference signal DS, and is a function for suppressing excessive contrast. That is, for an input x with a large absolute value (x is the value of the difference signal DS), a larger suppression effect (suppression effect due to a larger suppression rate) is exhibited. For example, the reinforcement function R2 is represented by a curve with a smaller slope for an input x with a larger absolute value. The enhancement function R3 is a non-linear enhancement function for the difference signal DS, and suppresses noise components with small amplitudes. That is, for an input x with a small absolute value (x is the value of the difference signal DS), a larger suppression effect (suppression effect due to a larger suppression ratio) is exhibited. For example, the reinforcement function R3 is represented by a curve having a larger slope for an input x with a larger absolute value. Any one of these enhancement functions R1 to R3 is used in the enhancement processing unit 403 .

The difference signal, DS, is a distinct component of the input signal IS. In the visual processing device 400, the intensity of the difference signal DS is converted and added to the input signal IS. Therefore, in the output signal OS, the edge and contrast of the input signal are enhanced.

FIG. 110 shows a visual processing device 406 for improving local contrast (intensity: lightness and darkness) (see, for example, Japanese Patent Laid-Open No. 2832954 (page 2, FIG. 5 )). The visual processing device 406 shown in FIG. 110 includes a spatial processing unit 407 , a subtraction unit 408 , a first conversion unit 409 , a multiplication unit 410 , a second conversion unit 411 , and an addition unit 412 . The spatial processing unit 407 performs spatial processing on the input signal IS, and outputs an unsharp signal US. The subtraction unit 408 subtracts the unsharp signal US from the input signal IS to output a difference signal DS. The first conversion unit 409 outputs an amplification factor signal GS that partially amplifies the difference signal DS based on the intensity of the unsharp signal US. The multiplication unit 410 multiplies the amplification factor signal GS by the difference signal DS, and outputs the contrast enhanced signal HS after partially amplifying the difference signal DS. The second conversion unit 411 locally corrects the intensity of the unsharp signal US, and outputs the corrected unsharp signal AS. The adding unit 412 adds the contrast enhanced signal HS and the modified unsharp signal AS, and outputs an output signal OS.

The amplification factor signal GS is a non-linear weighting factor for locally optimizing the contrast for a part of the input signal IS where the contrast is inappropriate. Therefore, in the input signal IS, a portion with an appropriate contrast is output as it is, and an unsuitable portion is output after being optimized.

FIG. 111 shows a visual processing device 416 that compresses the dynamic range (for example, refer to Japanese Unexamined Patent Publication No. 2001-298619 (page 3, FIG. 9 )). The visual processing device 416 shown in FIG. 111 includes: a spatial processing unit 417, which performs spatial processing on the input signal IS, and outputs an unsharp signal US; a LUT calculation unit 418, which reverses the unsharp signal US using a LUT. The converted LUT processed signal LS is output; and the addition unit 419 adds the input signal IS and the LUT processed signal LS, and outputs the output signal OS.

The LUT processes the signal LS and adds it to the input signal IS to compress the dynamic range of the low frequency components (frequency components lower than the cutoff frequency of the spatial processing unit 417 ) of the input signal IS. Therefore, the dynamic range of the input signal IS is compressed while maintaining high frequency components.

Conventionally, when visual processing combining gradation processing and spatial processing is performed, in order to realize visual processing with different effects, for example, separate circuit configurations as shown in FIG. 108 , FIG. 110 , and FIG. 111 are required. Therefore, it is necessary to install a dedicated LSI in accordance with the visual processing to be realized, and there is a disadvantage in that the circuit scale is large.

Contents of the invention

Therefore, an object of the present invention is to provide a visual processing device having a hardware configuration independent of the visual processing to be realized.

The visual processing device according to claim 1 is provided with: an input signal processing unit that performs pixel value conversion processing of adding conversion to the pixel value of the image signal input to the input image signal, and outputs the processed signal; a processing means for processing the input image based on a conversion means that provides a conversion relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal; The signal is transformed, and the output signal is output. The transformation relationship given by the transformation mechanism is determined based on the transformation that changes the brightness. The transformation that changes the brightness will monotonically decrease with respect to the processed signal. Transformation of the output signal output.

Here, the term "predetermined processing" means, for example, direct or indirect processing of an image signal, including processing of adding transformation to pixel values of an image signal, such as spatial processing or gradation processing.

In the visual processing device of the present invention, visual processing is performed using a conversion mechanism that provides a conversion relationship from an image signal and a processed signal to a visually processed output signal. Here, the so-called conversion mechanism refers to a look-up table (LUT) including, for example, storing the value of the output signal corresponding to the value between the image signal and the processed signal; and for converting the value between the image signal and the processed signal An arithmetic mechanism, etc. that output signal output matrix data, etc. Therefore, a hardware configuration that does not depend on the functions realized by the conversion mechanism can be realized. That is, it is possible to implement a hardware configuration that does not depend on the visual processing realized as a whole device.

The visual processing device according to claim 2 is the visual processing device according to claim 1, wherein the processed signal is a signal for performing predetermined processing on a pixel of interest and pixels surrounding the pixel of interest included in the image signal.

Here, the predetermined processing refers to, for example, spatial processing using surrounding pixels for a pixel of interest, and is processing for deriving an average value, a maximum value, or a minimum, etc., of the pixel of interest and surrounding pixels.

In the visual processing device of the present invention, for example, even if the visual processing corresponds to the pixel of interest with the same value, different visual processing can be realized due to the influence of surrounding pixels.

The visual processing device according to technical solution 3 is the visual processing device according to technical solution 1, wherein the conversion relationship given by the conversion mechanism is at least a part of the image signal or at least a part of the processed signal, and at least a part of the output signal There is a non-linear relationship between them.

Here, the so-called nonlinear relationship means, for example, that the value of at least a part of the output signal is represented by a nonlinear function that uses at least a part of the value of the image signal or at least a part of the value of the processed signal as a variable, or it is difficult to express it by a function Formulation etc.

In the visual processing device of the present invention, for example, visual processing based on visual characteristics of an image signal, or visual processing based on nonlinear characteristics of a device that outputs an output signal can be realized.

The visual processing device according to claim 4 is the visual processing device according to claim 3 , wherein the conversion relationship given by the conversion mechanism has a non-linear relationship with both the image signal and the processed signal and the output signal.

Here, the expression that there is a nonlinear relationship between both the image signal and the processed signal and the output signal means, for example, that the value of the output signal is determined by a nonlinear function that makes the value of the image signal and the value of the processed signal two variables. represent, or are difficult to formulate from a function.

In the visual processing device of the present invention, for example, even when the value of the image signal is the same, when the value of the processed signal is different, different visual processing can be realized according to the value of the processed signal.

The visual processing device described in technical solution 5 is the visual processing device according to any one of technical solutions 1 to 4, and the conversion relationship given by the conversion mechanism is based on the calculation based on the image signal and the processed signal. The value is determined by intensive calculation.

Here, the value calculated from the image signal and the processed signal refers to, for example, a value obtained by four arithmetic operations between the image signal and the processed signal, or a value obtained by converting the image signal or the processed signal through a certain function operation. The value obtained by the value and so on. The enhancement calculation refers to, for example, a calculation for adjusting gain, a calculation for suppressing excessive contrast, and a calculation for suppressing small-amplitude noise components.

In the visual processing device of the present invention, the value calculated from the image signal and the processed signal can be enhanced.

The visual processing device described in technical solution 6 is the visual processing device described in technical solution 5, and the calculation is strengthened, which is a nonlinear function.

In the visual processing device of the present invention, for example, enhancement of visual characteristics of an image signal or enhancement of non-linear characteristics of a device that outputs an output signal can be realized.

The visual processing device according to claim 7 is the visual processing device according to claim 5 or 6, wherein the enhancement operation is a conversion using a value obtained by converting an image signal or a processed signal.

The visual processing device according to claim 8 is the visual processing device according to any one of technical solutions 5 to 7, wherein the enhancement operation is the difference between the transformed values after transforming the image signal and the processed signal Enhancement function for values to be enhanced.

Here, the enhancement function refers to, for example, a function for adjusting gain, a function for suppressing excessive contrast, a function for suppressing small-amplitude noise components, and the like.

In the visual processing device of the present invention, each difference can be enhanced after transforming the image signal and the processed signal into another space. In this way, for example, reinforcement with visual characteristics can be realized.

The visual processing device according to Claim 9 is the visual processing device according to any one of Claims 5 to 8, wherein the enhancement operation is an enhancement function that enhances a ratio between the image signal and the processed signal.

In the visual processing device of the present invention, for example, the ratio between the image signal and the processed signal indicates the sharp component of the image signal. Thus, for example, visual manipulations that intensify clear components can be performed.

The visual processing device according to claim 10 is the visual processing device according to claim 1 or 2, wherein the conversion relationship given by the conversion mechanism is determined based on the conversion that changes brightness.

In the visual processing device of the present invention, visual processing for changing the brightness of an image signal can be realized.

The visual processing device according to Claim 11 is the visual processing device according to Remark 10, wherein the conversion for changing brightness is a conversion for changing the level or gain of the image signal.

Here, changing the level of the image signal means changing the value of the image signal by, for example, applying a bias to the image signal, changing the gain of the image signal, or performing calculations using other image signals as variables. Changing the gain of the image signal means changing a coefficient for multiplying the image signal.

The visual processing device described in claim 12 is the visual processing device described in Remark 10, wherein the transformation for changing brightness is determined based on the processing signal.

In the visual processing device of the present invention, for example, even if the value of the image signal is the same, when the value of the processed signal is different, different conversions can be realized according to the value of the processed signal.

The visual processing device according to claim 13 is the visual processing device according to claim 10 , wherein the conversion for changing brightness is a conversion for outputting an output signal which is monotonically decreased relative to the processed signal.

In the visual processing device of the present invention, for example, when the processed signal is an image signal after spatial processing, etc., the darker and larger-area part of the image is transformed into brighter; the brighter and larger-area part of the image parts, are darkened. Therefore, for example, backlight correction or whitening correction can be performed.

The visual processing device described in technical solution 14 is the visual processing device according to any one of technical solutions 1 to 13, and the transformation mechanism uses the relationship between the image signal and the output signal as a combination of a plurality of grayscale transformation curves. The composed grayscale transformation curve group is saved.

Here, the gradation transformation curves are a collection of gradation transformation curves in which gradation processing is performed on pixel values such as lightness and lightness of an image signal.

In the visual processing device of the present invention, gradation processing of an image signal can be performed using a gradation transformation curve selected from a plurality of gradation transformation curves. Therefore, more appropriate gradation processing can be performed.

The visual processing device according to technical solution 15 is the visual processing device according to technical solution 14, and the processing signal is a signal for selecting a corresponding gray-scale transformation curve from a plurality of gray-scale transformation curves.

Here, the processed signal is a signal for selecting a grayscale transformation curve, such as a spatially processed image signal or the like.

In the visual processing device of the present invention, the gradation processing of the image signal can be performed using the gradation transformation curve selected by processing the signal.

The visual processing device according to technical solution 16 is the visual processing device according to technical solution 15, wherein the value of the processed signal is associated with at least one gray-scale transformation curve included in the plurality of gray-scale transformation curve groups.

Here, at least one gradation transformation curve used for gradation processing is selected based on the value of the processed signal.

In the visual processing device of the present invention, at least one gradation transformation curve is selected by processing the value of the signal. Furthermore, using the selected gradation transformation curve, gradation processing of the image signal can be performed.

The visual processing device according to Claim 17 is the visual processing device according to any one of Claims 1 to 16, wherein the conversion mechanism is composed of a look-up table (hereinafter referred to as LUT), and in the LUT, registration is made by specifying Operations on pre-made profile data.

In the visual processing device of the present invention, visual processing is performed using a LUT in which pre-created profile data is registered. In the case of visual processing, processing such as creating profile data is unnecessary, and the execution speed of visual processing can be increased.

The visual processing device according to claim 18 is the visual processing device according to claim 17, wherein the LUT can be changed by registering profile data.

Here, the so-called profile data is the data of the LUT that realizes different visual processing.

In the visual processing device of the present invention, various changes can be made to the visual processing realized by registering the profile data. That is, various visual processing can be realized without changing the hardware configuration of the visual processing device.

The visual processing device according to claim 19 is the visual processing device according to claim 17 or 18, further comprising profile data registration means for registering profile data in the visual processing means.

Here, the profile data registration means registers the profile data calculated in advance according to the visual processing in the visual processing means.

In the visual processing device of the present invention, various changes can be made to the visual processing realized by registering the profile data. That is, various visual processing can be realized without changing the hardware configuration of the visual processing device.

The visual processing device described in technical solution 20 is the visual processing device according to technical solution 19, and the visual processing mechanism obtains the prepared description file data through an external device.

The profile data is created in advance by an external device. The external device is, for example, a computer having a program capable of creating profile data and a CPU. The visual processing mechanism obtains the description file data. Acquired via, for example, a network or a recording medium. The visual processing means executes visual processing using the acquired profile data.

In the visual processing device of the present invention, visual processing can be executed using profile data created by an external device.

The visual processing device according to claim 21 is the visual processing device according to claim 20, and the LUT can be changed by the acquired profile data.

In the visual processing device of the present invention, the obtained profile data is newly registered as a LUT. In this way, the LUT can be changed to achieve different visual processing.

The visual processing device described in technical solution 22 is the visual processing device according to technical solution 20 or 21, and the visual processing mechanism obtains the description file data via a communication network.

Here, the term "communication network" means, for example, a communication connection mechanism such as a dedicated line, a public line, the Internet, or a LAN, and may be wired or wireless.

In the visual processing device of the present invention, visual processing can be realized using profile data obtained through a communication network.

The visual processing device according to claim 23 is the visual processing device according to claim 17, and further includes a profile data creating means for creating profile data.

The profile data creating means creates profile data using, for example, the characteristics of an image signal or a processed signal.

In the visual processing device of the present invention, visual processing can be realized using the profile data created by the profile data creating means.

The visual processing device according to claim 24 is the visual processing device according to claim 23 , wherein the profile data creating means creates profile data based on a histogram of grayscale characteristics of the image signal.

In the visual processing device of the present invention, visual processing is realized using profile data regarded as a histogram based on the gradation characteristic of an image signal. Therefore, appropriate visual processing can be realized according to the characteristics of the image signal.

The visual processing device according to claim 25 is the visual processing device according to claim 17, wherein the profile data registered in the LUT is switched according to predetermined conditions.

In the visual processing device of the present invention, visual processing is realized using profile data switched according to predetermined conditions. Therefore, more suitable visual processing can be realized.

The visual processing device according to claim 26 is the visual processing device according to claim 25, and the predetermined condition refers to a condition related to brightness.

In the visual processing device of the present invention, more appropriate visual processing can be realized based on conditions related to shading.

The visual processing device according to technical solution 27 is the visual processing device according to technical solution 26, and the brightness is the brightness of the image signal.

In the visual processing device of the present invention, more appropriate visual processing can be realized based on the conditions related to the brightness of the image signal.

The visual processing device according to claim 28 is the visual processing device according to claim 27 , further comprising a brightness determination means for determining brightness and darkness of the image signal. The profile data registered in the LUT is switched according to the judgment result of the lightness judging means.

Here, the lightness judging means judges the lightness and darkness of the image signal based on, for example, pixel values such as brightness and lightness of the image signal. Furthermore, according to the determination result, the profile data is switched.

In the visual processing device of the present invention, more appropriate visual processing can be realized according to the brightness and darkness of the image signal.

The visual processing device according to claim 29 is the visual processing device according to claim 26 , further comprising brightness input means for inputting conditions related to signal brightness. The profile data registered in the LUT is switched according to the input result of the brightness input mechanism.

Here, the brightness input means is, for example, a wired or wirelessly connected switch or the like for the user to input conditions related to brightness.

In the visual processing device of the present invention, the user judges the conditions related to the brightness, and can switch the profile data through the brightness input mechanism. Therefore, more suitable visual processing for the user can be realized.

The visual processing device according to claim 30 is the visual processing device according to claim 29 , wherein the lightness input means inputs the lightness of the output environment of the output signal or the lightness of the input environment of the input signal.

Here, the lightness and darkness of the output environment refers to the lightness and darkness of the environment around the medium that outputs the output signal, such as a computer, television, digital camera, mobile phone, and PDA, or the medium that outputs the output signal, such as printing paper. its own brightness etc. The brightness of the input environment refers to, for example, the brightness of the medium itself from which the input signal is output, such as scanning paper.

In the visual processing device of the present invention, for example, the user judges the conditions related to the lightness and darkness of the house, and can switch the description file data through the lightness input mechanism. Therefore, visual processing suitable for the user can be realized.

The visual processing device according to claim 31 is the visual processing device according to claim 26 , further comprising lightness detection means for detecting at least two types of lightness and darkness. The profile data registered in the LUT is switched according to the detection result of the lightness detection mechanism.

Here, the so-called lightness detection mechanism is, for example, a mechanism that detects the lightness and darkness of an image signal based on pixel values such as brightness and lightness of the image signal, or a mechanism that detects the lightness and darkness of an output environment or an input environment such as a photosensitive element. , or a mechanism that detects a condition related to the brightness input by the user, or the like. In addition, the lightness and darkness of the output environment refers to the lightness and darkness of ambient light around media that output output signals, such as computers, televisions, digital cameras, mobile phones, and PDAs, or media that output output signals, such as printing paper. its own brightness etc. The brightness of the input environment refers to, for example, the brightness of the medium itself into which the input signal is input, such as scanning paper.

In the visual processing device of the present invention, at least two types of brightness and darkness are detected, and profile data is switched based on the detection. Therefore, more suitable visual processing can be realized.

The visual processing device described in technical solution 32 is the visual processing device according to technical solution 31, the brightness detected by the brightness detection mechanism includes: the brightness of the image signal, the brightness of the output environment of the output signal, or The lightness and darkness of the input environment for the input signal.

In the visual processing device of the present invention, more appropriate visual processing can be realized according to the brightness of the image signal, the brightness of the output environment of the output signal, or the brightness of the input environment of the input signal.

The visual processing device according to claim 33 is the visual processing device according to claim 25 , further comprising profile data selection means for selecting profile data registered in the LUT. The profile data registered in the LUT is switched according to the selection result of the profile data selection mechanism.

The profile data selection mechanism enables the user to select profile data. Furthermore, in the visual processing device, visual processing is realized using the selected profile data.

In the visual processing device of the present invention, the user can select description file data according to preference to realize visual processing.

The visual processing device according to claim 34 is the visual processing device according to claim 33, wherein the profile data selection mechanism is an input device for selecting profile data.

Here, the input device is, for example, a switch built in or connected to the visual processing device via a wire or wirelessly.

In the visual processing device of the present invention, the user can select favorite profile data using the input device.

The visual processing device according to claim 35 is the visual processing device according to claim 25 , and further includes an image characteristic judgment means for judging the image characteristic of the image signal. The profile data registered in the LUT is switched according to the judgment result of the image characteristic judging means.

The image characteristic judging means judges image characteristics such as luminance, lightness, or spatial frequency of the image signal. The visual processing device implements visual processing using the profile data switched according to the judgment result of the image characteristic judging means.

In the visual processing device of the present invention, the image characteristic judging mechanism automatically selects the description file data corresponding to the image characteristic. Therefore, visual processing can be realized using profile data more suitable for image signals.

The visual processing device according to claim 36 is the visual processing device according to claim 25 , further comprising user identification means for identifying a user. The profiler data registered in the LUT is switched according to the identification result of the user identification mechanism.

The user identification means is, for example, an input device or a camera for identifying a user.

In the visual processing device of the present invention, visual processing suitable for a user identified by the user identification means can be realized.

The visual processing device according to claim 37 is the visual processing device according to claim 17, wherein the visual processing mechanism performs interpolation calculation on the value stored in the LUT, and outputs an output signal.

The LUT stores values of image signals or processed signals at predetermined intervals. By interpolating the value of the LUT corresponding to the interval including the value of the input image signal or the value of the processed signal, the value of the output signal corresponding to the value of the input image signal or the value of the processed signal is obtained. output.

In the visual processing device of the present invention, there is no need to store LUT values for all possible values of the image signal or the processed signal, and the storage capacity for the LUT can be reduced.

The visual processing device according to claim 38 is the visual processing device according to claim 37, wherein the interpolation operation is based on the value of at least one of the low-order bits of the image signal represented by a binary number or the processed signal. Linear interpolation.

The LUT stores a value corresponding to a value of an upper bit of an image signal or a processed signal. The visual processing mechanism performs linear interpolation on the value of the LUT corresponding to the interval including the value of the input image signal or processing signal by using the value of the lower bit of the image signal or processing signal, thereby outputting the output signal .

In the visual processing device of the present invention, more accurate visual processing can be realized while storing LUTs with less storage capacity.

The visual processing device according to Claim 39 is the visual processing device according to any one of Claims 1 to 38, wherein a signal processing mechanism is input to perform spatial processing on an image signal.

In the visual processing device of the present invention, the visual processing can be realized by using the image signal and the spatially processed image signal through the LUT.

The visual processing device according to Claim 40 is the visual processing device according to Claim 39, wherein a signal processing mechanism is input to generate a passivation signal based on an image signal.

Here, the so-called unsharp signal refers to a signal directly or indirectly subjected to spatial processing on an image signal.

In the visual processing device of the present invention, visual processing can be realized by the LUT using the image signal and the unsharp signal.

The visual processing device according to claim 41 is the visual processing device according to claim 39 or 40, which derives an average value, a maximum value, or a minimum value of an image signal during spatial processing.

Here, the average value may be, for example, a simple average of image signals or a weighted average.

In the visual processing device of the present invention, visual processing can be realized by using the image signal, the average value, the maximum value, or the minimum value of the image signal by using the LUT.

The visual processing device according to claim 42 is the visual processing device according to any one of claims 1 to 41, wherein the visual processing mechanism uses the input image signal and processing signal to perform spatial processing and grayscale processing. degree processing.

In the visual processing device of the present invention, spatial processing and gray scale processing can be realized simultaneously by using LUT.

The visual processing method described in claim 43 includes: an input signal processing step of performing pixel value conversion processing of adding conversion to pixel values of the image signal on the input image signal, and outputting the processed signal; a processing step of converting the input image based on a conversion mechanism that provides a conversion relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal; The signal is transformed, and the output signal is output. The transformation relationship given by the transformation mechanism is determined based on the transformation that changes the brightness. The transformation that changes the brightness will monotonically decrease with respect to the processed signal. Transformation of the output signal output.

Here, the term "predetermined processing" refers to, for example, direct or indirect processing of image signals, including processing of converting pixel values of image signals such as spatial processing or gradation processing.

In the visual processing method of the present invention, visual processing is performed using a conversion mechanism that provides a relationship between an image signal, a processed signal, and a visually processed output signal. Therefore, hardware or software other than the conversion mechanism can be commonly used. That is, a hardware configuration or a software configuration that does not depend on the function of the visual processing can be used. Therefore, as additional effects, cost reduction of hardware, generalization of software, and the like can be realized.

The visual processing program according to claim 44 is a visual processing program for performing a visual processing method with a computer, and causes the computer to execute the visual processing method including an input signal processing step and a visual processing step. In the input signal processing step, predetermined processing is performed on the input image signal, and the processed signal is output. The visual processing step converts the input image signal based on a conversion mechanism that provides a relationship between the input image signal and the processed signal, and an output signal that is a visually processed image signal, and converts the input image signal to The above output signal output.

Here, the term "predetermined processing" refers to, for example, direct or indirect processing of an image signal, including processing of adding transformation to pixel values of an image signal such as spatial processing or gradation processing.

In the visual processing program of the present invention, visual processing is performed using a conversion mechanism that provides a conversion relationship between an image signal, a processed signal, and a visually processed output signal. Therefore, software other than general-purpose conversion mechanisms can be used. That is, a software configuration that does not depend on functions of visual processing can be used. Therefore, generalization of software and the like can be realized as an additional effect.

The integrated circuit according to Claim 45 is the visual processing device according to any one of Claims 1 to 42.

In the integrated circuit of the present invention, the same effect as that of the visual processing device according to any one of claims 1 to 42 can be obtained.

The display device according to claim 46 is the visual processing device according to any one of claims 1 to 42, and includes display means for displaying an output signal output from the visual processing device.

In the display device of the present invention, the visual processing device described in any one of claims 1 to 42 can be obtained.

The photographing device described in technical claim 47 includes: a photographing mechanism for photographing images; Visual processing device.

In the imaging device of the present invention, the same effect as that of the visual processing device according to any one of claims 1 to 42 can be obtained.

The portable information terminal described in technical claim 48 includes: a data receiving mechanism that receives transmitted or broadcast image data; and performs visual processing on the received image data as an image signal in technical claims 1 to 42. The visual processing device according to any one of the preceding claims; and a display mechanism for displaying the image signal visually processed by the visual processing device.

In the portable information terminal of the present invention, the same effect as that of the visual processing device according to any one of claims 1 to 42 can be obtained.

The portable information terminal described in claim 49 includes: a photographing mechanism that captures an image; a visual processing device; and a data transmission mechanism for transmitting the visually processed image signal.

In the portable information terminal of the present invention, the same effect as that of the visual processing device according to any one of claims 1 to 42 can be obtained.

According to the visual processing device of the present invention, it is possible to provide a visual processing device having a hardware configuration independent of the visual processing to be realized.

Description of drawings

FIG. 1 is a block diagram illustrating the configuration of a visual processing device 1 (first embodiment).

Fig. 2 is an example of profile data (first embodiment).

Fig. 3 is a flowchart illustrating a visual processing method (first embodiment).

FIG. 4 is a block diagram illustrating the configuration of the visual processing unit 500 (first embodiment).

Fig. 5 is an example of profile data (first embodiment).

FIG. 6 is a block diagram illustrating the configuration of a visual processing device 520 (first embodiment).

FIG. 7 is a block diagram illustrating the configuration of the visual processing device 525 (first embodiment).

FIG. 8 is a block diagram illustrating the configuration of the visual processing device 530 (first embodiment).

FIG. 9 is a block diagram illustrating the configuration of a profile data registration device 701 (first embodiment).

Fig. 10 is a flowchart illustrating a method for creating a visual processing description file (first embodiment).

FIG. 11 is a block diagram illustrating the configuration of a visual processing device 901 (first embodiment).

Fig. 12 is a graph showing the relationship between the input signal IS' and the output signal OS' when the change degree function fk(z) is changed (first embodiment).

Fig. 13 is a graph showing change degree functions f1(z) and f2(z) (first embodiment).

FIG. 14 is a block diagram illustrating the configuration of the visual processing device 905 (first embodiment).

FIG. 15 is a block diagram illustrating the configuration of the visual processing device 11 (first embodiment).

FIG. 16 is a block diagram illustrating the configuration of the visual processing device 21 (first embodiment).

FIG. 17 is a block diagram illustrating the dynamic range compression function F4 (first embodiment).

FIG. 18 is an explanatory diagram for explaining the enhancement function F5 (first embodiment).

FIG. 19 is a block diagram illustrating the configuration of the visual processing device 31 (first embodiment).

FIG. 20 is a block diagram illustrating the configuration of the visual processing device 41 (first embodiment).

FIG. 21 is a block diagram illustrating the configuration of the visual processing device 51 (first embodiment).

FIG. 22 is a block diagram illustrating the configuration of the visual processing device 61 (first embodiment).

FIG. 23 is a block diagram illustrating the configuration of the visual processing device 71 (first embodiment).

FIG. 24 is a block diagram illustrating the configuration of a visual processing device 600 (second embodiment).

Fig. 25 is a graph illustrating conversion based on Expression M20 (second embodiment).

Fig. 26 is a graph illustrating conversion based on Expression M2 (second embodiment).

Fig. 27 is a graph illustrating conversion based on the expression M21 (second embodiment).

Fig. 28 is a flowchart illustrating a visual processing method (second embodiment).

Fig. 29 is a graph showing the slope of the function α1(A) (second embodiment).

Fig. 30 is a graph showing the slope of the function α2(A) (second embodiment).

Fig. 31 is a graph showing the slope of the function α3(A) (second embodiment).

Fig. 32 is a graph showing the slope of the function α4(A, B) (second embodiment).

FIG. 33 is a block diagram illustrating a configuration of an actual contrast setting unit 605 as a modified example (second embodiment).

FIG. 34 is a block diagram illustrating a configuration of an actual contrast setting unit 605 as a modified example (second embodiment).

Fig. 35 is a flowchart illustrating the operation of the control unit 605e (second embodiment).

FIG. 36 is a block diagram illustrating the configuration of a visual processing device 600 including a chromatic aberration correction processing unit 608 (second embodiment).

FIG. 37 is an explanatory diagram illustrating the outline of chromatic aberration correction processing (second embodiment).

FIG. 38 is a flowchart for explaining the estimation calculation in the chromatic aberration correction processing unit 608 (second embodiment).

FIG. 39 is a block diagram illustrating the configuration of a visual processing device 600 as a modified example (second embodiment).

FIG. 40 is a block diagram illustrating the configuration of a visual processing device 910 (third embodiment).

FIG. 41 is a block diagram illustrating a configuration of a visual processing device 920 (third embodiment).

Fig. 42 is a block diagram illustrating the configuration of a visual processing device 920' (third embodiment).

Fig. 43 is a block diagram illustrating the configuration of a visual processing device 920' (third embodiment).

FIG. 44 is a block diagram illustrating the configuration of the visual processing device 101 (fourth embodiment).

FIG. 45 is an explanatory diagram for explaining the image region Pm (fourth embodiment).

FIG. 46 is an explanatory diagram for explaining the luminance histogram Hm (fourth embodiment).

Fig. 47 is an explanatory diagram for explaining the gradation transformation curve Cm (fourth embodiment).

Fig. 48 is a flowchart illustrating a visual processing method (fourth embodiment).

FIG. 49 is a block diagram illustrating the configuration of the visual processing device 111 (fifth embodiment).

Fig. 50 is an explanatory diagram for explaining the gradation transformation curve candidates G1 to Gp (fifth embodiment).

FIG. 51 is an explanatory diagram for explaining the two-dimensional LUT 141 (fifth embodiment).

FIG. 52 is an explanatory diagram for explaining the operation of the gradation correction unit 115 (fifth embodiment).

Fig. 53 is a flowchart illustrating a visual processing method (fifth embodiment).

FIG. 54 is an explanatory diagram for explaining a modified example of selection of the gradation transformation curve Cm (fifth embodiment).

FIG. 55 is an explanatory diagram for explaining gradation processing as a modified example (fifth embodiment).

FIG. 56 is a block diagram illustrating the configuration of the gradation processing execution unit 144 (fifth embodiment).

Fig. 57 is an explanatory diagram illustrating the relationship between the curve parameters P1 and P2 and the gradation conversion curve candidates G1 to Gp (fifth embodiment).

FIG. 58 is an explanatory diagram illustrating the relationship between the curve parameters P1 and P2 and the selection signal Sm (fifth embodiment).

FIG. 59 is an explanatory diagram illustrating the relationship between the curve parameters P1 and P2 and the selection signal Sm (fifth embodiment).

FIG. 60 is an explanatory diagram illustrating the relationship between the curve parameters P1 and P2 and the gradation conversion curve candidates G1 to Gp (fifth embodiment).

FIG. 61 is an explanatory diagram illustrating the relationship between the curve parameters P1 and P2 and the selection signal Sm (fifth embodiment).

FIG. 62 is a block diagram illustrating the configuration of the visual processing device 121 (sixth embodiment).

FIG. 63 is an explanatory diagram for explaining the operation of the selection signal correction unit 124 (sixth embodiment).

Fig. 64 is a flowchart illustrating a visual processing method (sixth embodiment).

FIG. 65 is a block diagram illustrating the configuration of the visual processing device 161 (seventh embodiment).

FIG. 66 is an explanatory diagram for explaining spatial processing by the spatial processing unit 162 (seventh embodiment).

Fig. 67 is a table explaining the weighting coefficient [Wij] (seventh embodiment).

FIG. 68 is an explanatory diagram for explaining the effect of visual processing by the visual processing device 161 (seventh embodiment).

FIG. 69 is a block diagram illustrating the configuration of a visual processing device 961 (seventh embodiment).

FIG. 70 is an explanatory diagram for explaining spatial processing by the spatial processing unit 962 .

Fig. 71 is a table explaining the weighting coefficient [Wij] (seventh embodiment).

Fig. 72 is a block diagram illustrating the overall configuration of a content supply system (ninth embodiment).

Fig. 73 shows an example of a mobile phone equipped with the visual processing device of the present invention (ninth embodiment).

Fig. 74 is a block diagram illustrating the configuration of a mobile phone (ninth embodiment).

Fig. 75 shows an example of a system for digital broadcasting (ninth embodiment).

FIG. 76 is a block diagram illustrating the configuration of a display device 720 (tenth embodiment).

FIG. 77 is a block diagram illustrating the configuration of an image processing device 723 (tenth embodiment).

FIG. 78 is a block diagram illustrating the configuration of the profile information output unit 747 (tenth embodiment).

FIG. 79 is a block diagram illustrating the configuration of a color visual processing device 745 (tenth embodiment).

FIG. 80 is a block diagram illustrating a configuration of a visual processing device 753 (tenth embodiment).

FIG. 81 is an explanatory diagram for explaining the operation of a visual processing device 753 as a modified example (tenth embodiment).

Fig. 82 is a block diagram illustrating the configuration of a visual processing device 753a (tenth embodiment).

Fig. 83 is a block diagram illustrating the configuration of a visual processing device 753b (tenth embodiment).

Fig. 84 is a block diagram illustrating the configuration of a visual processing device 753c (tenth embodiment).

FIG. 85 is a block diagram illustrating the configuration of an image processing device 770 (tenth embodiment).

FIG. 86 is a block diagram illustrating the configuration of the user input unit 772 (tenth embodiment).

FIG. 87 is a block diagram illustrating the configuration of an image processing device 800 (tenth embodiment).

Fig. 88 shows an example of the format of the input image signal d362 (tenth embodiment).

FIG. 89 is a block diagram illustrating the configuration of the attribute determination unit 802 (tenth embodiment).

FIG. 90 shows an example of the format of the input image signal d362 (tenth embodiment).

FIG. 91 shows an example of the format of the input image signal d362 (tenth embodiment).

FIG. 92 shows an example of the format of the input image signal d362 (tenth embodiment).

FIG. 93 shows an example of the format of the input image signal d362 (tenth embodiment).

FIG. 94 shows an example of the format of the input video signal d362 (tenth embodiment).

FIG. 95 is a block diagram illustrating the configuration of the imaging device 820 (eleventh embodiment).

FIG. 96 is a block diagram illustrating the configuration of an image processing device 832 (eleventh embodiment).

FIG. 97 is a block diagram illustrating the configuration of an image processing device 886 (eleventh embodiment).

Fig. 98 shows an example of the format of the output image signal d361 (eleventh embodiment).

FIG. 99 is a block diagram illustrating the configuration of an image processing device 894 (eleventh embodiment).

FIG. 100 is a block diagram illustrating the configuration of an image processing device 896 (eleventh embodiment).

FIG. 101 is a block diagram illustrating the configuration of an image processing device 898 (eleventh embodiment).

FIG. 102 is a block diagram illustrating the configuration of an image processing device 870 (eleventh embodiment).

FIG. 103 is an explanatory diagram for explaining the operation of the image processing device 870 (eleventh embodiment).

FIG. 104 is a block diagram illustrating the configuration of the visual processing device 300 (background art).

FIG. 105 is an explanatory diagram for explaining the image region Sm (background art).

FIG. 106 is an explanatory diagram for explaining the lightness histogram Hm (background art).

FIG. 107 is an explanatory diagram for explaining the gradation conversion region Cm (background art).

FIG. 108 is a block diagram illustrating the configuration of the visual processing device 500 after the sharpening masking process (background art).

FIG. 109 is an explanatory diagram for explaining enhancement functions R1 to R3 (background art).

FIG. 110 is a block diagram illustrating the configuration of a visual processing device 406 that improves local contrast (background art).

FIG. 111 is a block diagram illustrating the configuration of a visual processing device 416 that performs dynamic range compression (background art).

Detailed ways

Hereinafter, the first to eleventh embodiments which are best embodiments of the present invention will be described.

In the first embodiment, a visual processing device using a two-dimensional LUT will be described.

In the second embodiment, a visual processing device that performs ambient light correction when ambient light exists in an environment where an image is displayed will be described.

In the third embodiment, application examples of the first embodiment and the second embodiment will be described.

In the fourth to sixth embodiments, a visual processing device that realizes gradation processing that improves the visual effect will be described.

In the seventh embodiment, a visual processing device that performs visual processing using an appropriate blur signal will be described.

In the eighth embodiment, application examples of the fourth to seventh embodiments will be described.

In the ninth embodiment, application examples of the first to eighth embodiments will be described.

In the tenth embodiment, an application example in which the visual processing device of the above-mentioned embodiment is applied to a display device will be described.

In an eleventh embodiment, an application example in which the visual processing device of the above-mentioned embodiment is applied to a camera will be described.

(first embodiment)

A visual processing device 1 using a two-dimensional LUT as a first embodiment of the present invention will be described using FIGS. 1 to 10 . Furthermore, modifications of the visual processing device will be described using FIGS. 11 to 14 . In addition, a visual processing device that realizes visual processing equivalent to the visual processing device 1 will be described using FIGS. 15 to 23 .

The visual processing device 1 is a device that performs visual processing such as spatial processing and gradation processing of image signals. The visual processing device 1 constitutes a device that performs color processing of image signals and an image processing device in devices that process images such as computers, televisions, digital cameras, mobile phones, PDAs, printers, and scanners.

(visual processing device 1)

FIG. 1 shows a basic configuration of a visual processing device 1 that outputs a visually processed image (output signal OS) obtained by visually processing an image signal (input signal IS). The visual processing device 1 includes: a spatial processing unit 2 that performs spatial processing on the luminance value of each pixel of an original image obtained as an input signal IS, and outputs an unsharp signal US; and a visual processing unit 3, It performs visual processing of the original image using the input signal IS and the unsharp signal US for the same pixel, and outputs an output signal OS.

The spatial processing unit 2 obtains the unsharp signal US through, for example, a low-frequency spatial filter that passes only the low-frequency space of the input signal IS. As a low-frequency spatial filter, a FIR (Finite Impuls Respons, finite impulse response) type low-frequency spatial filter or an IIR (Infinite Impulse Response, infinite impulse response) type low-frequency spatial filter that is generally used in the generation of a passivation signal can be used. device etc.

The visual processing unit 3 has a two-dimensional LUT4 that provides a relationship between the input signal IS, the unsharp signal US, and the output signal OS, and outputs the output signal OS with reference to the two-dimensional LUT4 for the input signal IS and the unsharp signal US.

(2D LUT4)

In 2D LUT4, matrix data called profile data is registered. The profile data has a row (or column) corresponding to each pixel value of the input signal IS and a column (or row) corresponding to each pixel value of the unsharp signal US. The combination of profile signals US corresponds to the pixel value of the output signal OS. The profile data is registered in the two-dimensional LUT through the profile data registration device 8 built in or connected to the visual processing device 1 . The profile data registration device 8 stores a plurality of profile data created in advance by a personal computer (PC) or the like. For example, a plurality of profile data for realizing contrast enhancement, D-range compression processing, gradation correction, etc. (refer to the "Profile data" column described below for details) are stored. In this way, in the visual processing device 1 , various visual processing can be realized by changing the registered content of the profile data of the two-dimensional LUT 4 using the profile data registration device 8 .

FIG. 2 shows an example of profile data. The profile data shown in FIG. 2 is profile data for realizing processing equivalent to that of the visual processing device 400 shown in FIG. 108 in the visual processing device 1 . In Fig. 2, the description file data is expressed in the form of a 64×64 matrix, and the high-order 6-bit value of the luminance value of the input signal IS expressed in 8 bits is represented in the column direction (vertical direction), and the high-order 6-bit value in the row direction (horizontal direction) Indicates the value of the upper 6 bits of the luminance value of the unsharp signal US represented by 8 bits. In addition, as an element of a matrix corresponding to two luminance values, the value representing the output signal OS is 8 bits.

The value C (value of the output signal OS) of each element of the profile data shown in Fig. 2 uses the value A of the input signal IS (for example, the value obtained by subtracting the lower 2 bits of the input signal IS represented by 8 bits) The sum value B of the unsharp signal US (for example, a value obtained by subtracting the lower 2 bits of the unsharp signal US represented by 8 bits) is represented by C=A+0.5×(A-B) (hereinafter referred to as formula M11). That is, in the visual processing device 1 , it is shown that processing equivalent to that of the visual processing device 400 (see FIG. 108 ) using the enhancement function R1 (see FIG. 109 ) is performed.

In addition, the value C obtained by the expression M11 becomes a negative value by a combination of the value A of the input signal IS and the value B of the unsharp signal US. In this case, the element of the profile data corresponding between the value A of the input signal IS and the value B of the unsharp signal US may have a value of zero. In addition, the value C obtained by the expression M11 is saturated by a combination of values between the value A of the input signal IS and the value B of the unsharp signal US. That is, the maximum value 255 that can be represented by 8 bits is exceeded. In this case, the element of the profile data corresponding between the value A of the input signal IS and the value B of the unsharp signal US may be the value 255 . In FIG. 2, each element of the profile data obtained in this way is shown by a contour line.

And, for example, if the value C of each element uses the profile data represented by C=R6(B)+R5(B)×(A-B) (hereinafter referred to as M12), then the same as shown in FIG. 110 can be realized. Visual processing device 406 performs equivalent processing. Here, the function R5 is a function for outputting the amplification factor signal GS from the unsharp signal US in the first conversion unit 409, and the function R6 is a function for outputting the corrected unsharp signal from the unsharp signal US in the second conversion unit 411. Function of signal AS.

Furthermore, if the value C of each element uses the profile data expressed by C=A+R8(B) (hereinafter referred to as the expression M13), it is possible to realize the equivalent of the visual processing device 416 shown in FIG. 111 . deal with. Here, the function R8 is a function for outputting the processing signal LS from the passivation signal US.

In addition, when the value C of a certain element of the profile data calculated by Equation M12 and Equation M13 exceeds the range of 0≤C≤255, the value of the element can be 0 or 255.

(visual processing method and visual processing program)

FIG. 3 shows a flowchart illustrating a visual processing method in the visual processing device 1 . The visual processing method shown in FIG. 3 is a method for performing visual processing of an input signal IS (see FIG. 1 ) implemented by hardware in the visual processing device 1 .

In the visual processing method shown in FIG. 3 , the input signal IS is subjected to spatial processing through a low-frequency spatial filter (step S11 ) to obtain an unsharp signal US. Furthermore, the output signal OS is output with reference to the value of the two-dimensional LUT4 corresponding between the input signal IS and the unsharp signal US (step S12). The above processing is performed for each pixel input as the input signal IS.

In addition, each step of the visual processing method shown in FIG. 3 is realized by a computer or the like as a visual processing program.

(Effect)

(1)

In the case of performing visual processing based only on the value A of the input signal IS (for example, in the case of performing conversion based on a one-dimensional gradation conversion curve, etc.), if there are pixels of the same density in different places in the image, the same brightness is performed. transform. More specifically, if the darker background of the person in the image is brightened, the hair of the person with the same density will also be brightened.

In contrast, in the visual processing device 1 , visual processing is performed using profile data created based on a two-dimensional function corresponding between the value A of the input signal IS and the value B of the unsharp signal US. Therefore, pixels of the same density that exist in different places in the image are not uniformly converted, but are either brightened or darkened including surrounding information, so that each area in the image can be optimally brightened. adjustment. More specifically, it is possible to brighten the background of the same density without changing the hair density of the person in the image.

(2)

In the visual processing device 1 , visual processing of the input signal IS is performed using a two-dimensional LUT 4 . The visual processing device 1 has a hardware configuration that does not depend on the realized visual processing effects. That is, the visual processing device 1 can be configured with general-purpose hardware, which is effective in reducing hardware costs and the like.

(3)

The profile data registered in the two-dimensional LUT 4 can be changed by the profile data registration device 8 . Therefore, in the visual processing device 1, since the profile data can be changed without changing the hardware configuration of the visual processing device 1, various visual processing can be realized. More specifically, in the visual processing device 1, spatial processing and gradation processing can be realized simultaneously.

(4)

Profile data registered in 2D LUT4 can be calculated in advance. The time required for visual processing using this method is constant no matter how complex processing is performed on profile data that has already been created. Therefore, in the case of hardware or software, even if it is visual processing with a complicated structure, in the case of using the visual processing device 1, the processing time does not depend on the complexity of the visual processing, and it can be achieved. Speed up video processing.

(Modification)

In FIG. 2 , the profile data in the form of a 64×64 matrix is described. Here, the effect of the present invention does not depend on the size of the profile data. For example, the 2D LUT4 may have unsharp data corresponding to combinations of all values of the input signal IS and the unsharp signal US. For example, when the input signal and the passivation signal US are represented by 8 bits, the profile data may be in the form of a 256×256 matrix.

In this case, although the memory capacity required for 2D LUT4 increases, more accurate visual processing can be realized.

(2)

In FIG. 2 , the description file data is described, and the upper 6-bit value of the luminance value of the input signal IS expressed in 8 bits and the upper 6-bit value of the luminance value of the unsharp signal US expressed in 8 bits are stored. , the value of the output signal OS. Here, the visual processing device 1 further includes an interpolation unit that linearly interpolates the value of the output signal OS based on the size of the lower 2 bits of the elements of the adjacent profile data and the input signal IS and the unsharp signal US. repair.

In this case, more accurate visual processing can be realized without increasing the memory capacity required for 2D LUT4.

Furthermore, the visual processing unit 3 includes an interpolation unit that outputs a value obtained by linearly interpolating the value stored in the two-dimensional LUT 4 as an output signal OS.

FIG. 4 shows a visual processing unit 500 including an interpolation unit 501 as a modified example of the visual processing unit 3 . The visual processing unit 500 includes: a two-dimensional LUT 4 that assigns the relationship between the input signal IS, the unsharp signal US, and the output signal NS before interpolation; and an interpolation unit 501 that inputs the output signal NS before interpolation, inputs The signal IS and the passivation signal US output the output signal OS.

The 2-dimensional LUT4 stores the value of the upper 6 bits of the luminance value of the input signal IS represented by 8 bits and the value of the upper 6 bits of the luminance value of the unsharp signal US represented by 8 bits, and is output before interpolation The value of the signal NS. The value of the output signal NS before interpolation is stored as, for example, an 8-bit value. In the two-dimensional LUT4, when the 8-bit value of the input signal IS and the 8-bit value of the unsharp signal US are input, four values of the output signal NS before interpolation corresponding to the intervals including the respective values are output. The so-called interval including each value refers to (the value of the upper 6 bits of the input signal IS, the value of the upper 6 bits of the passivation signal US), (the minimum 6 bits that exceed the value of the upper 6 bits of the input signal IS value, the upper 6-bit value of the passivation signal US), (the value exceeding the upper 6-bit value of the input signal IS, the minimum 6-bit value of the upper 6-bit value of the passivation signal US), (more than the upper 6-bit value of the input signal IS Each combination of the minimum 6-bit value of the 6-bit value and the minimum 6-bit value of the high-order 6-bit value of the unsharp signal US) stores the interval surrounded by the four pre-interpolation output signals NS.

The value of the lower 2 bits of the input signal IS and the value of the lower 2 bits of the unsharp signal US are input to the interpolation unit 501, and using these values, the four pre-interpolation output signals NS after the two-dimensional LUT4 are output Values are linearly interpolated. More specifically, the weighted average of four pre-interpolation output signal NS values is calculated using the lower 2-bit values of the input signal IS and the lower 2-bit values of the unsharp signal US, and the output signal OS is output.

Through the above, more accurate visual processing can be realized without increasing the memory capacity required for 2D LUT4.

In addition, in the interpolation unit 501, it is only necessary to perform linear interpolation on either the input signal IS or the unsharp signal US.

(3)

In the spatial processing performed by the spatial processing unit 2, for the input signal IS of the pixel of interest, the average value (simple average or weighted average), the maximum value of the input signal IS between the pixel of interest and the surrounding pixels of the pixel of interest, , the minimum value, or the middle value, as the passivation signal US output. In addition, the average value, maximum value, minimum value, or median value of only surrounding pixels of the pixel of interest may be output as the unsharp signal US.

(4)

In FIG. 2 , the value C of each element of the profile data is created based on the linear function M11 for each of the value A of the input signal IS and the value B of the unsharp signal US. On the other hand, the value C of each element of the profile data is created based on a nonlinear function with respect to the value A of the input signal IS.

In this case, for example, it is possible to implement visual processing corresponding to visual characteristics, or to implement a process suitable for processing images of computers, televisions, digital cameras, mobile phones, PDAs, printers, scanners, etc. that output the output signal OS. , Visual processing of non-linear properties of machines.

Furthermore, the value C of each element of the profile data can be created based on a nonlinear function, that is, a two-dimensional nonlinear function, for each of the value A of the input signal IS and the value B of the profile signal US.

For example, in the case of performing visual processing based only on the value A of the input signal IS (for example, in the case of performing conversion based on a 1-dimensional grayscale transformation curve, etc.), if there are pixels of the same density in different places in the image, the same Brightness transformation. More specifically, if the darker background of the person in the image is brightened, the hair of the person with the same density will also be brightened.

On the other hand, when visual processing is performed using profile data created based on a 2-dimensional nonlinear function, pixels of the same density that exist in different places in the image are not converted in the same way, but surrounding information is included. Brighten or darken within, enabling optimal brightness adjustments for each area in the image. More specifically, the background of the same density can be brightened without changing the density of the hair of the person in the image. Furthermore, in the visual processing based on the linear function, it is possible to perform visual processing that maintains gradation even with respect to a pixel region where the processed pixel value is saturated.

FIG. 5 shows an example of such profile data. The profile data shown in FIG. 5 is profile data for realizing contrast enhancement in visual characteristics in the visual processing device 1 . In Fig. 5, the description file data is realized in the form of a 64×64 matrix, and the value of the upper 6 bits of the luminance value of the input signal IS represented by 8 bits is represented in the column direction (vertical direction), and the value of the upper 6 bits of the luminance value of the input signal IS expressed in 8 bits, and is represented in the row direction (horizontal direction). The value of the upper 6 bits of the luminance value of the unsharp signal US expressed in 8 bits. In addition, as a matrix element corresponding to two luminance values, the value indicating the output signal OS is 8 bits.

The value C of each element of the profile data (the value of the output signal OS) shown in FIG. 5 uses the value A of the input signal IS (for example, the value obtained by subtracting the lower 2 bits of the input signal IS represented by 8 bits) , the value B of the passivation signal US (such as the value after discarding the lower 2 bits of the passivation signal US represented by 8 bits), the transformation function F1, the inverse transformation function F2 of the transformation function, and the strengthening function F3, by C=F2 (F1(A)+F3(F1(A)-F1(B))) (hereinafter referred to as formula M14) is represented. Here, the transformation function F1 is a common logarithmic function. The inverse transformation function F2 is an exponential function (antilogarithmic function) that is an inverse function of a common logarithmic function. The enhancement function F3 is any one of the enhancement functions R1 to R3 described using FIG. 109 .

In the description file data, the visual processing of the input signal IS and the unsharp signal US transformed into the logarithmic space is realized through the transformation function F1. Human visual characteristics are logarithmic, and by transforming into logarithmic space and processing, visual processing suitable for visual characteristics can be realized. In this way, in the visual processing device 1, a contrast intensity in logarithmic space is realized.

In addition, the value C obtained by the expression M14 becomes a negative value by a combination of the value A of the input signal IS and the value B of the unsharp signal US. In this case, the elements of the profile data corresponding between the value A of the input signal IS and the value B of the unsharp signal US may be zero. Then, the value C obtained by the expression M14 becomes saturated by a combination of values between the value A of the input signal IS and the value B of the unsharp signal US. That is, it exceeds the maximum value of 256 represented by 8 bits. In this case, the element of the profile data corresponding between the value A of the input signal IS and the value B of the unsharp signal US may be the value 255 . In FIG. 5, each element of the profile data obtained in this way is represented by a contour line.

A more detailed description of nonlinear profile data is given below (Profile Data).

(5)

The description file data included in the 2D LUT4 may include a plurality of grayscale transformation curves (gamma curves) for realizing grayscale correction of the input signal IS.

Each grayscale transformation curve, for example, is a monotonically increasing function such as a gamma function with different gamma coefficients, and is associated with the value of the unsharp signal US. The correlation is performed, for example, by selecting a gamma function with a larger gamma coefficient for a smaller value of the unsharp signal US. In this way, the unsharp signal US functions as a selection signal for selecting at least one gradation transformation curve from the group of gradation transformation curves included in the profile data.

With the above configuration, the gradation conversion of the value A of the input signal IS is performed using the gradation conversion curve selected by the value B of the unsharp signal US.

In addition, the output of the two-dimensional LUT 4 can also be interpolated as described by the above-mentioned formula (2).

(6)

The profile data registration device 8 is built in or connected to the visual processing device 1, stores a plurality of profile data prepared in advance by a PC, etc., and changes the registered content of the 2D LUT4.

Here, the profile data stored in the profile data registration device 8 is created by a PC installed outside the visual processing device 1 . The profile data registration means 8 obtains profile data from a PC via a network or via a recording medium.

The profile data registration means 8 registers a plurality of stored profile data in the two-dimensional LUT 4 according to predetermined conditions. The details will be described using FIGS. 6 to 8 . In addition, the parts having basically the same functions as those of the visual processing device 1 described with reference to FIG. 1 are denoted by the same reference numerals and their descriptions are omitted.

【1】

FIG. 6 is a block diagram of a visual processing device 520 that judges the image of the input signal IS and switches the profile data registered in the two-dimensional LUT 4 based on the judgment result.

Visual processing device 520 has the same configuration as visual processing device 1 shown in FIG. 1 , and further includes profile data registration unit 521 having the same function as profile data registration device 8 . Furthermore, the visual processing device 520 includes an image determination unit 522 .

The image determination unit 522 receives an input signal IS and outputs a determination result SA of the input signal IS. The profile data registration unit 521 inputs the judgment result SA, and outputs the profile data PD selected based on the judgment result S.

The image judging unit 522 judges the image of the input signal IS. In the determination of the image, the brightness of the input signal IS is determined by obtaining pixel values such as brightness and lightness of the input signal IS.

The profile data registration unit 521 obtains the judgment result SA, and switches and outputs the profile data PS based on the judgment result SA. More specifically, for example, when it is determined that the input signal IS is bright, a profile that compresses the dynamic range is selected. In this way, contrast is also maintained for an overall bright image. Then, a profile that outputs an output signal OS in an appropriate dynamic range is selected in consideration of the characteristics of the device that outputs the signal OS.

Through the above, in the visual processing device 520, appropriate visual processing can be realized according to the input signal IS.

In addition, the image determination unit 522 determines not only pixel values such as luminance and lightness of the input signal IS but also image characteristics such as spatial frequency.

In this case, for example, for an input signal IS with a low spatial frequency, it is possible to realize more appropriate visual processing such as selecting a profile that enhances sharpness more.

【2】

FIG. 7 is a block diagram of a visual processing device 525 that switches profile data registered in the two-dimensional LUT 4 based on an input result from an input device for inputting conditions related to brightness.

The visual processing device 525 has the same configuration as the visual processing device 1 shown in FIG. 1 , and has a profile data registration unit 526 having the same function as the profile data registration device 8 . Furthermore, the visual processing device 525 includes an input device 527 connected by wire or wirelessly. More specifically, the input device 527 is an input key provided in a computer, a television, a digital camera, a mobile phone, a PDA, a printer, a scanner, etc. that output an output signal OS, or a device that processes an image, or each device Realized by remote control, etc.

The input device 527 is an input device for inputting conditions related to brightness, and includes switches such as "bright" and "dark", for example. The input device 527 outputs the input result SB by the user's operation.

The profile data registration unit 526 obtains the input result SB, and switches and outputs the profile data PD based on the input result SB. More specifically, for example, when the user inputs "bright", profile data that compresses the dynamic range of the input signal IS is selected and output as profile data PS. In this way, the contrast can be maintained even when the environment in which the device for displaying the output signal OS is placed is in a "bright" state.

As described above, in the visual processing device 525 , appropriate visual processing can be realized based on the input from the input device 527 .

In addition, the so-called conditions related to brightness include not only conditions related to the brightness of ambient light around the media that output the output signals of computers, televisions, digital cameras, mobile phones, PDAs, etc., but also conditions related to the use of paper for printers, for example etc. The output signal is output in terms of brightness related to the media itself. In addition, for example, conditions related to the brightness of the medium itself into which an input signal such as a scanner paper is input may be used.

Moreover, these conditions can be input not only through switches or the like, but also through the uterus such as photosensitive elements.

In addition, the input unit 527 is not only used for inputting conditions related to brightness, but also for directly operating the profile data registration unit 526 to switch profiles. In this case, the input device 527 displays a list of profile data in addition to the brightness-related conditions for the user to select.

Through this, the user can perform visual processing that requires the corresponding.

In addition, the input device 527 may be a device for identifying a user. In this case, the input device 527 may be a camera for identifying a user, or a device for inputting a user name.

For example, when the input user is a child through the input device 527, profile data that suppresses excessive luminance changes, etc. are selected.

In this way, visual processing according to the user can be realized.

【3】

FIG. 8 is a block diagram of a visual processing device 530 that switches profile data registered in the two-dimensional LUT 4 based on detection results from a brightness detection unit for detecting two types of brightness.

The visual processing device 530 has the same configuration as the visual processing device 1 shown in FIG. 1 , and has a profile data registration unit 531 having the same function as the profile data registration device 8 . Furthermore, the visual processing device 530 includes a brightness detection unit 532 .

The brightness detection unit 532 is composed of the image determination unit 522 and the input device 527 . The image determination unit 522 and the input device 527 are the same as those described using FIGS. 6 and 7 . In this way, the brightness detection unit 522 receives the input signal IS, and outputs the determination result SA from the image determination unit 522 and the input result SA from the input device 527 as detection results.

The profile data registration unit 531 inputs the judgment result SA and the input result SB, and switches and outputs the profile data PD based on the judgment result SA and the input result SB. More specifically, for example, when the ambient light is "bright" and the input signal IS is also judged to be bright. A profile or the like that compresses the dynamic range of the input signal IS is selected and output as profile data PS. In this way, the contrast can be maintained when the output signal OS is displayed.

Through the above, in the visual processing device 530, appropriate visual processing can be realized.

【4】

In the visual processing devices of FIGS. 6 to 8 , each profile data registration unit may not be integrated with the visual processing device. Specifically, the profile data registration unit may be connected to the visual processing device via a network as a server including a plurality of profile data, or as a plurality of servers including individual profile data. Here, the network is, for example, a communication connection mechanism such as a dedicated line, a public line, the Internet, or a LAN, and may be wired or wireless. And in this case, both the judgment result SA and the input result SB are transmitted from the visual processing device side to the profile data registration unit side via the same Internet.

(7)

In the above-mentioned embodiment, it has been explained that the profile data registration device 8 includes a plurality of profile data, and realizes different visual processing by switching the registration to the two-dimensional LUT 4 .

Here, the visual processing device 1 may include a plurality of two-dimensional LUTs for registering profile data for realizing different visual processing. In this case, in the visual processing device 1 , different visual processing is realized by switching the input to each 2-dimensional LUT or switching the output to each 2-dimensional LUT.

In this case, the storage capacity to be secured for the two-dimensional LUT increases, but the time required for switching the visual processing can be shortened.

Furthermore, the profile data registration device 8 may be a device that generates new profile data based on a plurality of profile data, and registers the generated profile data in the two-dimensional LUT 4 .

These will be described using FIGS. 9 to 10 .

FIG. 9 is a block diagram mainly explaining a profile data registration device 701 as a modified example of the profile data registration device 8 . The profile data registration device 701 is a device for switching the profile data registered in the 2-bit LUT 4 of the visual processing device 1 .

The profile data registration device 701 is composed of: a profile data registration unit 702 for registering a plurality of profile data, a profile creation execution unit 703 for generating new profile data based on a plurality of profile data, inputting a A parameter input unit 706 for parameters of file data and a control unit 705 for controlling each unit are configured.

In the profile data registration unit 702, similar to the profile data registration device 8 or each profile data registration unit shown in FIGS. C10 reads the selection profile data selected by C10. Here, two pieces of selected profile data are read out from the profile data registration unit 702 as first selected profile data d10 and second selected profile data d11 .

The profile data read from the profile data registration unit 702 is determined by the output of the parameter input unit 706. For example, in the parameter input unit 706, the desired visual processing effect, its processing degree, and Information on the visual environment of the image, etc., is input as parameters manually or automatically from a sensor or the like. The control unit 705 designates the profile data to be read out through the control signal C10 based on the parameters input through the parameter input unit 706, and at the same time specifies the value of the synthesis degree of each profile data through the control signal c12,

The profile creating execution unit 703 includes a profile generating unit 704 for creating generated profile data d6 as new profile data based on the first selected profile data d10 and the second selected profile data d11.

The profile generation unit 704 obtains the first selected profile data d10 and the second selected profile data d11 from the profile data registration unit 702 . Furthermore, from the control unit 705, a control signal c12 designating the synthesis degree of each selected profile data is obtained.

Furthermore, the profile generating unit 704 creates Generate description file data d6 for value [l]. Here, the value [l] is calculated by [l]=(1-k)×[m]+k×[n]. In addition, when the value [k] satisfies 0≤k≤1, the first selection profile data d10 and the second selection profile data d11 are internally divided; when [k] satisfies k<0 or k>1 Next, the first selection profile data d10 and the second selection profile data d11 are divided.

The two-dimensional LUT 4 obtains the profile data d6 generated by the profile generating unit 704 , and stores the obtained value in an address specified by the count signal c11 of the control unit 705 . Here, the generated profile data d6 is associated with the same image signal value as associated with each selected profile data used to create the generated profile data d6.

As described above, new profile data for realizing different visual processing can further be created based on, for example, profile data for realizing different visual processing.

Using FIG. 10 , a description will be given of a visual processing profile creation method executed in a visual processing device including the profile data registration device 701 .

The address of the profile data registration unit 702 is designated at a constant calculation cycle by the count signal c10 from the control unit 705, and the image signal value stored in the designated address is read (step S701). Specifically, the control unit 705 outputs the count signal c10 based on the parameters input through the parameter input unit 706 . The count signal c10 designates the addresses of two profile data for realizing different visual processing in the profile data registration unit 702 . In this way, the first selected profile data d10 and the second selected profile data d11 are read from the profile data registration unit 702 .

The profile generation unit 704 obtains the control signal c12 specifying the degree of synthesis from the control unit 705 (step S702).

The profile generating unit 704 creates a value [ l] to generate description file data d6 (step S703). Here, the value [l] is calculated by [l]=(1-k)×[m]+k×[n].

The generation profile data d6 is written for the 2-dimensional LUT4 (step S704). Here, the address of the writing destination is designated by the count signal c11 from the control unit 705 provided for the two-dimensional LUT4.

The control unit 705 judges whether the processing of all data of the selected profile data has been completed (step S705), and repeats the processing of steps S70 to S705 until the processing is completed.

And, in this way, the new profile data stored in the two-dimensional LUT4 is used for execution of the visual processing.

【Effect of (7)】

In the visual processing device provided with the profile data registration device 701, based on the profile data for realizing different visual processing, new profile data for realizing different visual processing can be created to perform visual processing. That is, in the profile data registration unit 702, as long as a small amount of profile data is provided, visual processing of any degree of processing can be realized. The storage capacity of the profile data registration unit 702 can be reduced.

Note that the profile data registration device 701 may be provided not only in the visual processing device 1 shown in FIG. 1 but also in the visual processing devices shown in FIGS. 6 to 8 . In this case, the profile data registration unit 702 and the profile creation execution unit 703 can be used instead of the profile data registration units 521, 526, and 531 shown in FIGS. 6 to 8; the parameter input unit 706 and the control unit 705 may be used instead of the image determination unit 522 in FIG. 6 , the input device 527 in FIG. 7 , and the brightness detection unit 532 in FIG. 8 .

(8)

The visual processing device may also be a device that converts the brightness of the input signal IS. Using FIG. 1 , a visual processing device 901 that converts luminance will be described.

【constitute】

The visual processing device 901 is a device that converts the brightness of an input signal IS', and includes a processing unit 902 that outputs a processed signal US' that performs predetermined processing on the input signal IS', and uses the input signal IS' and the processed signal US', The conversion unit 903 for converting the input signal IS' is constituted.

processing unit 902 . It operates in the same manner as the spatial processing unit 2 (see FIG. 1 ), and performs spatial processing of the input signal IS'. In addition, the spatial processing described in the above (modified example) may also be performed.

The conversion unit 903 includes a two-dimensional LUT similarly to the visual processing unit 3, and converts the output signal OS' (value [y]) based on the input signal IS' (value [x]) and the processed signal US' (value [z]). ) output.

Here, the value of each element of the two-dimensional LUT included in the conversion unit 903 is the value of the input signal IS′ with respect to the gain or offset determined by the value of the function fk(z) related to the degree of change in luminance. [x] works and decides. Hereinafter, the function fk(z) related to the degree of change in luminance is referred to as a "degree of change function".

The value of each element of the two-dimensional LUT (=value [y] of the output signal OS') is determined based on a function of the value [x] of the input signal IS' and the value [z] of the processed signal US'. Hereinafter, this function is referred to as a "transformation function", and the transformation functions (a) to (d) are shown as examples. 12(a) to (d) show the relationship between the input signal IS' and the output signal OS' when the change degree function fk(z) is changed.

【About transformation function (a)】

The transformation function (a) represents [y]=f1(z)×[x].

Here, the change degree function f1(z) functions as a gain of the input signal IS'. Therefore, by changing the value of the degree function f1(z), the gain of the input signal IS' is changed, and the value [y] of the output signal OS' is changed.

Fig. 12(a) shows changes in the relationship between the input signal IS' and the output signal OS' when the value of the degree of change function f1(z) changes.

As the change degree function f1(z) becomes larger (f1(z)>1), the value [y] of the output signal becomes larger. That is, the changed image becomes brighter. On the other hand, as the change degree function f1(z) becomes smaller (f1(z)<1), the value [y] of the output signal becomes smaller. That is, the changed image becomes darker.

Here, the change degree function f1(z) is a function in which the minimum value in the domain of the value [z] is not smaller than the value [0].

Furthermore, when the value [y] of the output signal exceeds the range of values taken by the calculation of the transformation function (a), it may be clipped within the range of values taken. For example, when the value [l] is exceeded, the value [y] of the output signal can be limited by the value [l], and when it is less than the value [0], the value [y] of the output signal can be limited by the value [0]. This is the same as the conversion functions (b) to (d) below.

【About transformation function (b)】

Here, the degree of change function f2(z) acts as an offset of the input signal IS'. Therefore, by changing the value of the degree function f2(z), the offset of the input signal IS' is changed, and the value [y] of the output signal OS' is changed.

Fig. 12(b) shows changes in the relationship between the input signal IS' and the output signal OS' when the value of the degree of change function f2(z) changes.

As the change degree function f2(z) becomes larger (f2(z)>0), the value [y] of the output signal becomes larger. That is, the changed image is bright. On the other hand, as the change degree function f2(z) becomes smaller (f2(z)<0), the value [y] of the output signal becomes smaller. That is, the changed image becomes darker.

【About transformation function (c)】

The transformation function (c) represents [y]=f1(z)×[x]+f2(z).

Here, the degree of change function f1(z) acts as a gain of the input signal IS'. Furthermore, the degree of change function f2(z) acts as an offset of the input signal IS'. Therefore, by changing the value of the degree function f1(z), the gain of the input signal IS' is changed, and at the same time, by changing the value of the degree function f2(z), the offset of the input signal IS is changed, and the value of the output signal OS'[ y] change.

Fig. 12(c) shows changes in the relationship between the input signal IS' and the output signal OS' when the values of the change degree function f1(z) and the change degree function f2(z) are changed.

As the degree-of-change function f1(z) and the degree-of-change function f2(z) become larger, the value [y] of the output signal becomes larger. That is, the transformed image becomes brighter. On the other hand, as the degree-of-change function f1(z) and the degree-of-change function f2(z) become smaller, the value [y] of the output signal becomes smaller. That is, the transformed image, darkened.

【About transformation function (d)】

The transformation function (d) represents [y]=[x]^(1-f2(z)).

Here, the "power" of the "power function" is determined by changing the degree function f2(z). Therefore, by changing the value of the degree function f2(x), the input signal IS' is changed, and the value [y] of the output signal OS' is changed.

Fig. 12(d) shows changes in the relationship between the input signal IS' and the output signal OS' when the value of the degree of change function f2(z) changes.

As the change degree function f2(z) becomes larger (f2(z)>0), the value [y] of the output signal becomes larger. That is, the changed image becomes brighter. On the other hand, as the change degree function f2(z) becomes smaller (f2(z)<1), the value [y] of the output signal becomes smaller. That is, the changed image becomes darker. Also, when the value of the change degree function f2(z) is [0], conversion of the low-frequency input signal IS' is not performed.

In addition, the value [x] is a value obtained by normalizing the value of the input signal IS' within the range of [0] to [1].

【Effect】

(1)

In the visual processing device 901, visual processing of the input signal IS' is performed by having a two-dimensional LUT having elements determined using any one of the transformation functions (a) to (d) described above. Each element of the 2D LUT stores the value [y] corresponding to the value [x] and the value [z]. Therefore, based on the input signal IS' and the processed signal US', viewing angle processing for converting the brightness of the input signal IS' is realized.

(2)

Here, when both the degree of change function f1(z) and the degree of change function f2(z) are monotonically decreasing functions, further effects such as backlight correction or anti-whitening can be obtained. These will be described.

Fig. 13(a)~(b) shows an example of monotonously decreasing degree of change functions f1(z) and f2(z). Although they show three curves (a1~a3, b1~b3) respectively, they are all monotonically decreasing example of.

The degree of change function f1(z) is a function having a range spanning the value [l], and is an area where the domain of the minimum value for the value [z] is not smaller than the value [0]. The degree of change region, f2(z), is a function with a region spanning the value [0].

For example, the value [z] of the processed signal US' is smaller in a darker and larger area of the image. The value of the degree of change function becomes larger with respect to a smaller value [z]. That is, if the two-dimensional LUT created based on the transformation functions (a) to (d) is used, darker and larger-area parts of the image become brighter. Therefore, for example, in an image captured by backlight, for a darker and larger area, the dark part is improved to improve the visibility.

Also, for example, the value [z] of the processed signal US' increases in bright and large areas of the image. The value of the degree of change function corresponding to a larger value [z] becomes smaller. That is, if the two-dimensional LUT created based on the change functions (a) to (d) is used, bright and large areas in the image will be darkened. Therefore, for example, in an image having a bright part such as a blank, the bright and large-area part is highlighted, and the visual effect is improved.

【Modification】

(1)

The transformation function described above is an example, and any function may be used as long as it is a transformation having the same properties.

(2)

The value of each element of the 2D LUT can be determined strictly by the above-mentioned transformation function.

For example, when the value of the conversion function described above exceeds the range of values that can be handled as the output signal OS', the two-dimensional LUT can store values limited by the range of values that can be handled as the output signal OS'.

(3)

The same processing as above can be performed without using the 2D LUT. For example, the conversion unit 903 can output the output signal OS' by calculating the conversion functions (a) to (d) on the input signal IS' and the processed signal US'.

(9)

The visual processing device includes a plurality of spatial processing units, and can perform visual processing using a plurality of deactivated signals with different degrees of spatial processing.

【constitute】

FIG. 14 shows the configuration of the visual processing device 905 . The visual processing device 905 is a device that performs visual processing of the input signal IS", and its configuration includes: a first processing unit 906a, which performs a first predetermined processing on the input signal IS", and outputs the first processed signal U1; A processing unit 906b, which performs a second predetermined process on the input signal IS′, and outputs the second processed signal U2; and a conversion unit 908, which uses the input signal IS″ and the first processed signal U1 and the second processed signal U2 , to transform the input signal IS".

The first processing unit 906a and the second processing unit 906b operate in the same manner as the spatial processing unit 2 (refer to FIG. 1 ), and perform spatial processing of the input signal IS". In addition, the above-mentioned (modified example) (3) can also be performed. The documented spatial processing.

Here, the first processing unit 906a and the second processing unit 906b differ in size from the area of surrounding pixels used in spatial processing.

Specifically, in contrast to the use of peripheral pixels (small unsharp signals) included in an area of 30 vertical pixels and 30 horizontal pixels centering on the pixel of interest in the first processing unit 906a, the In the second processing unit 906a, peripheral pixels (large unsharp signals) included in an area of 90 pixels in length and 90 pixels in width around the pixel of interest are used. In addition, the area of the surrounding pixels described here is just an example and is not limited thereto. In order to fully exploit the visual processing effect, it is preferable to generate the unsharp signal from a relatively large area.

The conversion unit 908 is equipped with a LUT, and converts the output signal OS" (value [y]) output.

Here, the LUT included in the conversion unit 903 is a three-dimensional LUT, and stores the output corresponding to the value [x] of the input signal IS″, the value [z1] of the first processed signal U1, and the value [z2] of the second processed signal U2. The value of the signal OS''[y]. The value of each element of this 3D LUT (=value [y] of the output signal OS'') is based on the value [x] of the input signal IS', the value [z1] of the first processed signal U1, and the value [z1] of the second processed signal U2 It is determined by the function between the value [z2].

This 3D LUT can realize the processing described in the above-mentioned embodiments and the following embodiments, but here, for the 3D LUT, [the case of converting the brightness of the input signal IS"] and [the input signal IS] The case of strengthening transformation of IS"] will be explained.

[When changing the brightness and darkness of the input signal IS"]

The conversion unit 908 performs conversion so that the input signal IS becomes brighter when the value “z1” of the first processed signal U1 becomes smaller. However, if the value [z2] of the second processed signal U2 also becomes smaller, it suppresses Brightness.

As an example of such conversion, the value of each element of the three-dimensional LUT included in the conversion unit 903 is determined based on the following conversion function (e) or (f).

(Regarding the transformation function (e))

The transformation function (e) represents [y]=[f11(z1)/f12(z2)]×[x].

Here, the degree-of-change functions f11(z1) and f12(z2) are the same functions as the degree-of-change function f1(z) described above (Modification) (8). Furthermore, the change degree function f11(z1) and the change degree function f12(z2) are different functions.

In this way, [f11(z1)/f12(z2)] plays the role of the gain of the input signal IS", by changing the gain of the input signal IS" through the value of the first processed signal U1 and the value of the second processed signal U2, changing Value of output signal OS"[y]

(Regarding the transformation function (f))

The transformation function (f) is expressed as [y]=[x]+f21(z1)-f22(z2).

Here, the degree-of-change functions f21(z1) and f22(z2) are the same functions as the degree-of-change function f2(z) described in the above (modified example) (8). Furthermore, the change degree function f21(z1) and the change degree function f22(z2) are different functions.

In this way, [f21(z1)-f22(z2)] acts as the offset of the input signal IS", and the offset of the input signal IS" can be changed by the value of the first processed signal U1 and the value of the second processed signal U2. Shift, change the value "y" of the output signal OS".

(Effect)

By using such transformation functions (e) to (f), it is possible to achieve, for example, the effect of brightening a small dark portion of a backlit portion and brightening only a large dark portion of a night scene image.

(Modification)

In addition, the processing in the conversion unit 908 is not limited to processing using a three-dimensional LUT, and the same calculation as the conversion function (e) or (f) may be performed.

Also, each element of the 3D LUT can be determined strictly based on the transformation function (e) or (f).

[The case of enhancing the input signal IS"]

When the conversion in the conversion unit 908 is a conversion for enhancing the input signal IS", a plurality of frequency components may be enhanced individually.

For example, if the first processed signal U1 is converted to enhance, the high-frequency shading part can be enhanced, and if the second processed signal U2 is transformed to be enhanced, the low-frequency shading part can be enhanced. strengthening.

(Profile data)

The visual processing device 1 may include profile data for realizing various visual processing in addition to those described above. Hereinafter, for the first to seventh profile data that realize various visual processing, expressions characterized by the profile data, and visual processing equivalent to that of the visual processing device 1 equipped with the profile data are shown. The composition of the visual processing device.

Each profile data is determined based on a mathematical formula including an intensive operation of a value calculated from the input signal IS and the unsharp signal US. Here, the calculation of enhancement refers to calculation performed by, for example, a non-linear enhancement function.

By doing so, it is possible to enhance the visual characteristics of the input signal IS or enhance the nonlinear characteristics of the device that outputs the output signal OS in each profile data.

(1)

【1st description file data】

The first profile data is determined based on calculations including a function that enhances the difference between converted values after a predetermined conversion with respect to the input signal IS and the unsharp signal US. In this way, after transforming the input signal IS and the unsharp signal US into different spaces, the difference of each can be enhanced. In this way, it is possible to achieve, for example, the enhancement of existing visual properties, and the like.

It will be described in detail below.

The value C of each element of the first profile data (the value of the output signal OS), using the value A of the input signal IS, the value B of the unsharp signal US, the transformation function F1, the inverse transformation function F2 of the transformation function, and the enhancement function F3 , expressed as C=F2(F1(A)+F3(F1(A)-F1(B)) (hereinafter referred to as formula M1).

Here, the transformation function F1 is a common logarithmic function. The inverse transformation function F2 is an exponential function (antilog) that is an inverse function of a common logarithmic function. The enhancement function F3 is any one of the enhancement functions R1 to R3 described using FIG. 109 .

[Equivalent visual processing device 11]

FIG. 15 shows a visual processing device 1 equivalent to the visual processing device 1 in which the first profile data is registered in the two-dimensional LUT4.

The visual processing device 11 is a device that outputs an output signal OS based on an enhanced calculation of the difference between the conversion values obtained by performing a predetermined conversion on the input signal IS and the unsharp signal US. In this way, after transforming the input signal IS and the unsharp signal US into different spaces, the difference of each can be enhanced, for example, the enhancement of presence visual characteristics can be realized.

The visual processing device 11 shown in FIG. 15 includes: a spatial processing unit 12 that performs spatial processing according to the luminance value of each pixel of the original image obtained as the input signal IS, and outputs an unsharp signal US; The processing unit 13 performs visual processing of the original image using the input signal IS and the unsharp signal US, and outputs an output signal OS.

Since the spatial processing unit 12 performs the same operation as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 13 includes: a signal space conversion unit 14, which converts the signal space between the input signal IS and the unsharp signal US, and outputs the converted input signal TIS and the converted unsharp signal TUS; a subtraction unit 17, which The transformed input signal TIS is used as a first input, the transformed unsharp signal TUS is used as a second input, and a difference signal DS as a difference between each is output; the enhancement processing part 18, whose output takes the difference signal DS as an input, performs an enhanced signal TS after the enhanced processing; an adder 19 that takes the transformed input signal TIS as a first input and the enhanced signal TS as a second input, and outputs an added signal PS obtained by adding the two; and an inverse transform unit 20, which outputs an output signal OS that takes the addition signal PS as an input.

The signal space conversion unit 14 further includes: a first conversion unit 15, which takes the input signal IS as an input, and outputs a converted input signal TIS; Signal TUS.

[The role of the equivalent visual processing device 11]

The first conversion unit 15 converts the input signal of value A into a converted input signal TIS of value F1(A) using a conversion function F1. The second conversion unit 16 converts the unsharp signal US of the value B into the converted unsharp signal TUS of the value F1 (B) using the conversion function F1. The subtraction unit 17 calculates the difference between the transformed input signal TIS of the value F1(A) and the transformed unsharp signal TUS of the value F1(B), and converts the difference signal of the value F1(A)-F1(B) DS output. The enhancement processing unit 18 outputs the enhancement processing signal TS of the value F3 (F1(A)-F1(B)) based on the difference signal DS of the value F1(A)-F1(B) using the enhancement function F3. The addition unit 19 adds the transformed input signal TIS of value F1(A) and the enhanced signal TS of value F3(F1(A)−F1(B)), and outputs the value F1(A)+F3(F1( Addition signal PS of A)-F1(B)). The inverse transform unit 20 uses the inverse transform function F2 to inversely transform the addition signal PS of the value F1(A)+F3(F1(A)-F1(B)), and converts the value F2(F1(A)+F3(F1 (A)-F1(B))) output signal OS output.

In addition, calculations using the transformation function F1, the inverse transformation function F2, and the enhancement function F3 may or may not be performed using a one-dimensional LUT for each function.

【Effect】

The same effect of visual processing is realized as the visual processing device 1 and the visual processing device 11 provided with the first profile data.

Via the transformation function F1, visual processing using the transformed input signal TIS transformed into a logarithmic space and the transformed unsharp signal TUS is realized. Human visual characteristics are logarithmic, and by transforming into logarithmic space and processing, visual processing suitable for visual characteristics can be realized.

(ii)

Implements contrast enhancement in logarithmic space in various visual processing devices.

The conventional visual processing device 400 shown in FIG. 108 is generally used for profile (edge) enhancement using an unsharp signal US with a small degree of blur. However, when the visual processing device 400 performs contrast enhancement using the unsharp signal US with a large degree of blur, the bright part of the original image is under-enhanced, and the dark part is over-enhanced, which is not suitable for visual characteristics. That is, corrections in the direction of brightening tend to under-emphasize; corrections in the direction of darkening tend to over-enhance.

On the other hand, when the visual processing is performed using the visual processing device 1 or the visual processing device 11, visual processing suitable for visual characteristics from dark parts to bright parts can be performed, and brightening can be performed with a good balance. Intensify in direction and intensify in darken direction.

(iii)

In the conventional visual processing device 400, the output signal OS after the visual processing may become negative and malfunction.

On the other hand, when the value C of a certain element of the profile data obtained by M1 exceeds the range of 0≤C≤255, by setting the value of the element to 0 or 255, it is possible to prevent the corrected pixel signal from becoming Negative, a fault occurs, or saturation produces a fault. This is achieved regardless of the bit length of the elements used to represent the profile data.

【Modification】

The transformation function F1 is not limited to a logarithmic function. It can also be, for example, let the transformation function F1 be the transformation of gamma correction (for example, remove the gamma coefficient (0.45)) for the input signal IS, and let the inverse transformation function F2 be the gamma correction for the input signal IS to produce role transformation.

In this way, the gamma correction for the input signal IS is removed, and the processing can be performed under the linear characteristic. Therefore, correction of optical blur can be performed.

(ii)

In the visual processing device 11, the visual processing unit 13 can calculate the above expression M1 based on the input signal IS and the unsharp signal US without using the two-dimensional LUT4. In this case, a one-dimensional LUT may be used for each of the functions F1 to F3.

(2)

【Second description file data】

The second profile data is determined based on calculations including a function that enhances the ratio of the input signal IS to the unsharp signal US. In this way, it is possible, for example, to achieve enhanced visual processing of passivating components.

Furthermore, the second profile data is determined based on the calculation of dynamic range compression for the ratio of the enhanced input signal IS and the unsharp signal US. In this way, for example, visual processing such as compression of the dynamic range while strengthening the passivation component can be realized.

It will be described in detail below.

The value C (value of the output signal OS) of each element of the second unsharp data is expressed as C=F4 using the value A of the input signal IS, the value B of the unsharp signal US, the dynamic range compression function F4, and the enhancement function F5. (A)×F5(A/B) (hereinafter referred to as formula M2).

Here, the dynamic range compression function F4 is, for example, a monotonically increasing function such as an upwardly convex power function. For example, it is expressed as F4(x)=x^γ (0<γ<1). The strengthening function F5 is a power function. For example, it is expressed as F5(x)=x^α (0<α≤1).

[Equivalent visual processing device 21]

FIG. 16 shows a visual processing device 21 equivalent to the visual processing device 1 in which the second profile data is registered in the two-dimensional LUT4.

The visual processing device 21 is a device that outputs an output signal OS based on a calculation that enhances the ratio between the input signal IS and the unsharp signal US. In this way, it is possible, for example, to implement enhanced visual treatments of passivating components.

In addition, the visual processing device 21 outputs the output signal OS based on the dynamic range compression calculation of the ratio between the enhanced input signal IS and the unsharp signal US. In this way, for example, visual processing such as compression of the dynamic range while strengthening the passivation component can be realized.

The visual processing device 21 shown in FIG. 16 includes: performing spatial processing according to the luminance value of each pixel of the original image obtained as the input signal IS, and outputting the unsharp signal US; Using the input signal IS and the passivation signal US, the visual processing of the original image is performed, and the output signal OS is output.

Since the spatial processing unit 23 performs the same operation as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 23 includes a dividing unit 25 that receives the input signal IS as a first input, takes the unsharp signal US as a second input, and outputs a division signal RS obtained by dividing the input signal IS by the unsharp signal US; The enhancement processing part 26, which takes the division signal RS as input, and the enhancement processing signal TS as output; the output processing part 27, which takes the input signal IS as the first input, takes the enhancement processing signal TS as the second input, and outputs the signal OS output. The output processing unit 27 includes: a DR compression unit 28, which takes the input signal IS as input, and outputs a DR compressed signal DRS after dynamic range (DR) compression; and a multiplication unit 29, which takes the DR compressed signal DRS as a first input , takes the enhanced signal TS as the second input, and outputs the output signal OS.

[The role of the equivalent visual processing device 21]

The operation of the visual processing unit 23 will be further described.

The dividing unit 25 divides the input signal IS of the value A by the unsharp signal US of the value B, and outputs the divided signal RS of the value A/B. The enhancement processing unit 26 outputs the enhancement processing signal TS of the value F5 (A/B) based on the division signal RS of the value A/B using the enhancement function F5. The DR compression unit 28 outputs a DR compressed signal DRS of a value F4 (A) based on an input signal IS of a value A using a dynamic range compression function F4. The multiplier 29 multiplies the DR compressed signal DRS of value F4(A) by the enhanced signal TS of value F5(A/B), and outputs an output signal OS of value F4(A)×F5(A/B).

In addition, the calculation using the dynamic range compression function F4 and the enhancement function F5 may be performed using a one-dimensional LUT for each function, or may be performed without using a LUT.

【Effect】

The visual processing device 1 and the visual processing device 21 provided with the second profile data realize the same visual processing effect.

(i)

Conventionally, when compressing the dynamic range of the entire image, the dynamic range compression function F4 shown in FIG. 17 is used to compress the grayscale level so as not to saturate from dark parts to highlights. That is, assuming that the reproduction target black level in the uncompressed image signal is L0 and the maximum white level is L1, the uncompressed dynamic range L1:L0 is compressed into the compressed dynamic range Q1:Q0. However, the contrast ratio, which is the ratio of the image signal levels, decreases to (Q1/Q0)×(L0/L1) times due to the compression of the dynamic range. Here, the dynamic range compression function F4 is an upwardly convex power function.

On the other hand, in the visual processing device 1 and the visual processing device 21 equipped with the second profile data, the division signal RS of the value A/B, that is, the sharp signal is enhanced by the enhancement function F5, and the DR compressed signal DRS are multiplied. Therefore, local contrast is enhanced. Here, the enhancement function F5 is a power function (F5(x)=x^α) as shown in Figure 18. When the value of the division signal RS is greater than 1, the bright side is strengthened; Fortify.

In general, human vision has the property that if the local contrast is maintained, even if the overall contrast is lowered, the same contrast can be seen. In this way, in the visual processing device 1 and the visual processing device 21 provided with the second profile data, it is possible to realize visual processing that does not visually lower the contrast while compressing the dynamic range.

The effects of the present invention will be described more specifically.

The dynamic range compression function F4 is F4(x)=x^γ (for example, γ=0.6). Also, the enhancement function F5 is F5(x)=x^α (for example, α=0.4). Furthermore, the black level of the playback target when the maximum white level of the input signal IS is normalized to a value of 1 is assumed to be a value of 1/300. That is, the dynamic range of the input signal IS is 300:1.

When the dynamic range of the input signal IS is compressed using the dynamic range compression function F4, the compressed dynamic range becomes F4(1):F4(1/300)=30:1. That is, the dynamic range is compressed to 1/10 by the dynamic range compression function F4.

On the other hand, the value C of the output signal OS is expressed by the above formula 2, C=(A^0.6)×((A/B^0.4)), that is, C=A/(B^0.4). Here, C is proportional to A because the value of B is seen to be constant in a local range. That is, the ratio of the amount of change in the value C to the amount of change in the value A becomes 1, and there is no change in the local contrast between the input signal IS and the output signal OS.

Similar to the above, human vision has the property that if the local contrast is maintained, the same contrast can be seen even if the overall contrast is lowered. In this way, in the visual processing device 1 and the visual processing device 21 provided with the second profile data, it is possible to realize visual processing that does not visually lower the contrast while compressing the dynamic range.

In addition, if the power multiplier α of the enhancement function F5 shown in FIG. 18 is set to be greater than 0.4, the appearance contrast of the output signal OS can be improved more than that of the input signal IS while compressing the dynamic range.

(iii)

In the present invention, in order to achieve the above effects, it is particularly effective in the next situation. That is, a display with a small physical dynamic range can reproduce a high-contrast image with good dark areas and bright and dark areas. In addition, for example, a TV projector in a bright environment can display a high-contrast image, and a high-contrast printed matter can be obtained with a low-density ink (light-colored ink).

【Modification】

(i)

In the visual processing device 21, the visual processing unit 23 can calculate the above-mentioned expression M2 based on the input signal IS and the unsharp signal US without using the two-dimensional LUT4. In this case, it is not necessary to use a one-dimensional LUT in the calculation of each of the functions F4 and F5.

(ii)

In addition, when the value C of a certain element of the profile data obtained by the formula M2 is C>255, the value of the element can be set to 255.

(3)

【3rd description file data】

The third profile data is determined based on calculations including a function that enhances the ratio of the input signal IS to the unsharp signal US. In this way, it is possible, for example, to implement enhanced visual treatments of passivating components.

It will be described in detail below.

In the expression M2 of the above-mentioned second profile data, the dynamic range compression function F4 may be a proportional function with a proportionality factor of 1. In this case, the value C (value of the output signal OS) of each element of the third profile data is expressed as C=A× F5(A/B) (hereinafter referred to as formula M3).

[Equivalent visual processing device 31]

FIG. 19 shows a visual processing device 31 equivalent to the visual processing device 1 that registers the third profile data in the two-dimensional LUT.

The visual processing device 31 is a device that outputs an output signal OS based on a calculation that enhances the ratio between the input signal IS and the unsharp signal US. In this way, it is possible, for example, to implement enhanced visual treatments of passivating components.

The visual processing device 31 shown in FIG. 19 is different from the visual processing device 21 shown in FIG. 16 in that it does not include the DR compression unit 28 . Hereinafter, in the visual processing device 31 shown in FIG. 19 , the parts that operate similarly to those of the visual processing device 21 shown in FIG. 16 are assigned the same reference numerals, and detailed description thereof will be omitted.

The visual processing device 31 includes: a spatial processing unit 22 that performs spatial processing according to the luminance value of each pixel of the original image obtained as the input signal IS, and outputs an unsharp signal US; and a visual processing unit 32 that Using the input signal IS and the passivation signal US, the visual processing of the original image is performed, and the output signal OS is output.

Since the spatial processing unit 22 performs the same operations as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 32 includes: a dividing unit 25, which takes the input signal IS as the first input, takes the unsharp signal US as the second input, and outputs the division signal RS obtained by removing the unsharp signal US from the input signal IS; A processing section 26, which takes the division signal RS as input, and outputs the enhanced processing signal TS; and a multiplication section 33, which takes the input signal IS as a first input, takes the enhanced processing signal TS as a second input, and outputs an output signal OS .

[The role of the equivalent visual processing device 31]

The division unit 25 and the enhancement processing unit 26 perform the same operations as those described for the visual processing device 21 shown in FIG. 16 .

The multiplication unit 33 multiplies the input signal IS of value A by the enhanced signal TS of value F5 (A/B), and outputs an output signal OS of value A×F5 (A/B). Here, the enhancement function F5 is the same as that shown in FIG. 18 .

In addition, the calculation using the enhancement function F5 may be performed using a one-dimensional LUT for each function, as described for the visual processing device 21 shown in FIG. 16 , or may not use a LUT.

【Effect】

The visual processing device 1 and the visual processing device 31 provided with the third profile data realize the same visual processing effect.

(i)

In the enhancement processing unit 26, enhancement processing is performed on the unsharp signal (division signal RS) expressed as a ratio of the input signal IS to the unsharp signal US, and the enhanced sharp signal is multiplied by the input signal IS. The enhancement of the unsharp signal expressed as the ratio of the input signal IS to the unsharp signal US corresponds to calculating the difference between the input signal IS and the unsharp signal US in logarithmic space. That is, visual processing suitable for logarithmic human visual characteristics is realized.

(ii)

The enhancement amount by the enhancement function F5 becomes larger when the input signal IS is large (bright), and becomes small when the input signal IS is small (very dark). In addition, the amount of enhancement in the direction of brightening is larger than the amount of enhancement in the direction of darkening. Therefore, visual processing suitable for visual characteristics and well-balanced visual processing can be realized.

(iii)

In addition, when the value C of a certain element of the profile data obtained by Equation M3 is C>255, the value C of the element can be set to 255.

(iv)

In the processing using Equation M3, the dynamic range of the input signal IS is not compressed, and the local contrast can be enhanced, but the local contrast can be enhanced, and the dynamic range can be visually compressed and expanded.

(4)

(4th profile data)

The fourth profile data is determined based on calculations including a function for emphasizing the difference between the input signal IS and the unsharp signal US according to the value of the input signal IS. In this way, for example, the blunt component of the input signal IS can be enhanced according to the value of the input signal IS. Therefore, appropriate enhancement can be performed from the dark part to the bright part of the input signal IS.

Furthermore, the fourth profile data is determined based on the calculation of adding the value obtained by compressing the dynamic range of the input signal IS to the enhanced value. In this way, the dynamic range can be compressed while enhancing the sharp components of the input signal IS according to the value of the input signal IS.

It will be described in detail below.

The value C of each element of the fourth profile data (the value of the output signal OS), the value A of the input signal IS, the value B of the unsharp signal US, the enhancement amount adjustment function F6, the enhancement function F7, and the dynamic range compression function F8 , expressed as C=F8(A)+F6(A)×F7(A-B) (hereinafter referred to as formula M4).

Here, the enhancement amount adjustment function F6 is a function that monotonically increases with respect to the value of the input signal IS. That is, when the value A of the input signal IS is small, the value of the enhancement amount adjustment function F6 also becomes smaller; when the value A of the input signal IS becomes larger, the value of the enhancement amount adjustment function F6 also becomes larger. The enhancement function F7 is any one of the enhancement functions R1 to R3 described using FIG. 109 . The dynamic range compression function F8 is a power function illustrated in FIG. 17, expressed as F8(x)=x^γ(0<γ<1).

[Equivalent visual processing device 41]

FIG. 20 shows a visual processing device 41 equivalent to the visual processing device 1 in which the fourth profile data is registered in the two-dimensional LUT4.

The visual processing device 41 is a device that outputs an output signal OS based on a calculation that enhances the difference between the input signal IS and the unsharp signal US based on the value of the input signal IS. In this way, for example, the clear components of the input signal IS can be enhanced according to the value of the input signal IS. Therefore, appropriate enhancement can be performed from the dark part to the bright part of the input signal IS.

Furthermore, the visual processing device 41 outputs the output signal OS based on the calculation of adding the value after dynamic range compression to the input signal IS to the value after enhancement. In this way, it is possible to compress the dynamic range while enhancing the unsharp component of the input signal IS according to the value of the input signal IS.

The visual processing device 41 shown in FIG. 20 includes: a spatial processing unit 42, which performs spatial processing according to the luminance value of each pixel of the original image obtained as the input signal IS, and outputs the unsharp signal US; and The visual processing unit 43 performs visual processing of the original image using the input signal IS and the unsharp signal US, and outputs an output signal US.

Since the spatial processing unit 42 performs the same operations as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 43 includes: a subtraction unit 44 that takes the input signal IS as a first input, takes the unsharp signal US as a second input, and outputs a difference signal DS as each difference; an enhancement processing unit 45 that The difference signal DS is used as input, and the enhanced processing signal TS is output; the enhanced amount adjustment unit 46, which takes the input signal IS as input, outputs the enhanced amount adjusted signal IS; the multiplication unit 47, which uses the enhanced amount adjusted signal IC as the first 1 input, taking the enhanced processing signal TS as the second input, and outputting the multiplication signal MS obtained by multiplying the enhancement amount adjustment signal IC and the enhanced processing signal TS; and an output processing unit 48, which takes the input signal IS as the first input, and outputs The multiplication signal MS is used as the second input, and the output signal OS is output. The output processing unit 48 includes: a DR compression unit 49, which receives the input signal IS as an input, and outputs a DR compressed signal DRS after dynamic range (DR) compression; and an addition unit 50, which takes the DR compressed signal DRS as a first input , taking the multiplication signal MS as the second input, and outputting the output signal OS.

[The role of the equivalent visual processing device 41]

The operation of the visual processing unit 43 will be further described.

The subtraction unit 44 calculates the difference between the input signal IS of value A and the unsharp signal US of value B, and outputs a difference signal DS of value A-B. The enhancement processing unit 45 outputs the enhancement processing signal TS of the value F7 (A-B) based on the difference signal DS of the value A-B using the enhancement function F7. The enhancement amount adjustment unit 46 outputs the enhancement amount adjustment signal IC of the value F6 (A) based on the input signal IS of the value A using the enhancement amount adjustment function F6. The multiplication unit 47 multiplies the enhancement amount adjustment signal IC of the value F6(A) by the enhancement signal TS of the value F7(A-B), and outputs the multiplication signal MS of the value F6(A)×F7(A-B). The DR compression unit 49 outputs a DR compressed signal DRS of a value F8 (A) based on an input signal IS of a value A using a dynamic range compression function F8. The addition unit 50 adds the DR compressed signal DRS to the multiplication signal MS of the value F6(A)×F7(A-B), and outputs the output signal OS of the value F8(A)+F6(A)×F7(A-B) .

In addition, the calculation using the enhancement amount adjustment function F6, the enhancement function F7, and the dynamic range compression function F8 is performed using a one-dimensional LUT for each function, or may not be performed using a LUT.

【Effect】

The visual processing device 1 and the visual processing device 41 provided with the fourth profile data realize the same visual processing effect.

(i)

According to the value A of the input signal IS, the enhancement amount of the difference signal DS is adjusted. Therefore, it is possible to maintain local contrast from dark parts to bright parts while performing dynamic range compression.

(ii)

Although the enhancement amount adjustment function F6 is a monotonically increasing function, it is a function in which the increase amount of the value of the function decreases as the value A of the input signal IS increases. In this case, the value of the output signal OS is prevented from being saturated.

(iii)

The enhancement function F7 is an enhancement amount that suppresses the increase in the absolute value of the difference signal DS when the enhancement function R2 described in FIG. 109 is used. Therefore, saturation of the enhancement amount of a high-resolution portion is prevented, and natural visual processing can also be performed visually.

【Modification】

(i)

In the visual processing device 41, the visual processing unit 43 can calculate the above expression M4 based on the input signal IS and the unsharp signal US without using the two-dimensional LUT4. In this case, a one-dimensional LUT can be used for calculation of each of the functions F6 to F8.

(ii)

When the enhancement function F7 is a proportional function with a proportional coefficient of 1, it is not necessary to provide the enhancement processing unit 45 in particular.

(iii)

In addition, when the value C of a certain element of the profile data obtained by the formula M4 exceeds the range of 0≤C≤255, the value C of the element can be set to 0 or 255.

(5)

【The fifth description file data】

The fifth profile data is determined based on calculations including a function that enhances the difference between the input signal IS and the unsharp signal US according to the value of the input signal IS. In this way, for example, the sharp components of the input signal IS can be enhanced according to the value of the input signal IS, and therefore, appropriate enhancement can be performed from dark parts to bright parts of the input signal IS.

It will be described in detail below.

In the above formula M4 of the fourth profile data, the dynamic range compression function F8 may be a proportional function with a proportional coefficient of 1. In this case, the value C (value of the output signal OS) of each element of the fifth profile data uses the value A of the input signal IS, the value B of the unsharp signal US, the enhancement amount adjustment function F6, and the enhancement function F7, It is expressed as C=A+F6(A)×F7(A-B) (hereinafter referred to as formula M5).

[Equivalent visual processing device 51]

FIG. 21 shows a visual processing device 51 equivalent to the visual processing device 1 in which the fifth profile data is registered in the two-dimensional LUT4.

The visual processing device 51 is a device that outputs an output signal OS based on a calculation that enhances the difference between the input signal IS and the unsharp signal US based on the value of the input signal IS. In this way, for example, the sharp components of the input signal IS can be enhanced according to the value of the input signal IS. Therefore, appropriate enhancement can be performed from the dark part to the bright part of the input signal IS.

The visual processing device 51 shown in FIG. 21 is different from the visual processing device 41 shown in FIG. 20 in that it does not include the DR compression unit 49 . Hereinafter, in the visual processing device 51 shown in FIG. 21 , parts that perform the same operations as those of the visual processing device 41 shown in FIG. 20 are assigned the same reference numerals, and detailed description thereof will be omitted.

The visual processing device 51 includes: a spatial processing unit 42 that performs spatial processing on the luminance value of each pixel of the original image obtained as the input signal IS, and outputs an unsharp signal US; and a visual processing unit 52 that It uses the input signal IS and the passivation signal US to perform visual processing of the original image, and outputs the output signal OS.

Since the spatial processing unit 42 performs the same processing as that of the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 52 includes: a subtraction unit 44 that takes the input signal IS as a first input and the unsharp signal US as a second input, and outputs a difference signal DS as a difference between them; an enhancement processing unit 45, It takes the difference signal DS as input, and outputs the enhanced processing signal TS; the enhancement amount adjustment unit 46, which takes the input signal IS as input, and outputs the enhancement amount adjustment signal IC; the multiplication unit 47, which takes the enhancement amount adjustment signal IC as The first input takes the enhanced processing signal TS as the second input, and outputs the multiplication signal MS obtained by multiplying the enhancement amount adjustment signal IC and the enhanced processing signal TS; The signal MS is used as the second input, and the output signal OS is output.

[The role of the equivalent visual processing device 51]

The operation of the visual processing unit 52 will be further described.

The subtraction unit 44 , the enhancement processing unit 45 , the enhancement amount adjustment unit 46 , and the multiplication unit 47 perform operations similar to those described for the visual processing device 41 shown in FIG. 20 .

Adder 53 adds input signal IS of value A to multiplication signal MS of value F6(A)×F7(A-B), and outputs output signal OS of value A+F6(A)×F7(A-B).

In addition, the calculation using the enhancement amount adjustment function F6 and the enhancement function F7 may be performed using a one-dimensional LUT for each function as described for the visual processing device 41 shown in FIG. 20 , or may be performed without using a LUT.

【Effect】

The visual processing device 1 and the visual processing device 51 provided with the fifth profile data realize the same visual processing effect. Furthermore, the effects achieved by the visual processing device 1 and the visual processing device 41 having the fourth profile data are basically the same visual effects.

(i)

The adjustment of the enhancement amount of the difference signal DS is performed by the value A of the input signal IS. Therefore, the enhancement amount of the contrast from the dark part to the bright part can be made uniform.

【Modification】

(i)

When making the enhancement function F7 a proportional function with a proportional coefficient of 1, it is not necessary to provide an enhancement processing unit 45 in particular.

(ii)

In addition, when the value C of a certain element of the profile data obtained by Equation M5 exceeds the range of 0≤C≤255, the value C of the element can be set to 0 or 255.

(6)

【6th description file data】

The sixth profile data is determined based on the calculation of gradation correction to the value obtained by adding the value of the input signal ID to the value obtained by adding the value of the difference between the enhanced input signal IS and the unsharp signal US. In this way, for example, visual processing such as performing gradation correction on the input signal IS after enhancing the sharp components can be realized.

It will be described in detail below.

The value C (the value of the output signal OS) of each element of the sixth profile data is expressed as C= F10(A+F9(A-B))) (hereinafter referred to as formula M6).

Here, the enhancement function F9 is any one of the enhancement functions R1 to R3 described using FIG. 109 . The gradation correction function F10 is, for example, a gamma correction function, an S-shaped gradation correction function, an inverted S-shaped gradation correction function, etc., which are used for general gradation correction.

[Equivalent visual processing device 61]

FIG. 22 shows a visual processing device 61 equivalent to the visual processing device 1 in which the sixth profile data is registered in the two-dimensional LUT4.

The visual processing device 61 is based on the calculation of grayscale correction for the value obtained by adding the value of the difference between the enhanced input signal IS and the unsharp signal US to the value of the input signal IS, and outputs Signal OS output device. In this way, for example, visual processing such as performing gradation correction on the input signal IS after enhancing clear components can be realized.

The visual processing device 61 shown in FIG. 22 includes: a spatial processing unit 62, which performs spatial processing according to the luminance value of each pixel of the original image obtained as the input signal IS, and outputs the unsharp signal US; and The visual processing unit 63 performs visual processing of the original image using the input signal IS and the unsharp signal US, and outputs an output signal OS.

Since the spatial processing unit 62 performs the same operation as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 63 includes: a subtraction unit 64 that takes the input signal IS as a first input, takes the unsharp signal US as a second input, and outputs a difference signal DS as a difference between them; an enhancement processing unit 65, which outputs the enhanced processing signal TS after taking the difference signal DS as input and performing the enhanced processing; the adding unit 66, which takes the input signal IS as the first input, takes the enhanced processing signal TS as the second input, and outputs the two phases the added addition signal PS; and a gradation correction unit 67 which outputs the addition signal PS as an input and outputs an output signal OS.

[The role of the equivalent visual processing device 61]

The operation of the visual processing unit 63 will be further described.

The subtraction unit 64 calculates the difference between the input signal IS of value A and the unsharp signal US of value B, and outputs a difference signal DS of value A-B. The enhancement processing unit 65 outputs the enhancement processing signal TS of the value F9 (A-B) from the difference signal DS of the value A-B using the enhancement function F9. The adding unit 66 adds the input signal IS of the value A and the enhanced signal TS of the value F9 (A-B), and outputs the added signal PS of the value A+F9 (A-B). The gradation correction unit 67 outputs the output signal OS of the value F10 (A+F9(A-B)) based on the addition signal of the value A+F9(A-B) using the gradation correction function F10.

In addition, the calculation using the enhancement function F9 and the gradation correction function F10 may be performed using a one-dimensional LUT for each function, or may be performed without using a LUT.

【Effect】

The visual processing device 1 and the visual processing device 61 provided with the sixth profile data can achieve the same visual processing effect.

(i)

The difference signal DS is enhanced by the enhancement function F9 and added to the input signal IS. Therefore, the contrast of the input signal IS can be enhanced. Furthermore, the gradation correction unit 67 executes gradation correction processing of the added signal PS. Therefore, for example, halftones that appear frequently in the original image can further enhance the contrast. Also, for example, the addition signal PS can be brightened as a whole. Through the above, it is possible to simultaneously combine the spatial processing and the gradation processing.

【Modification】

(i)

In the visual processing device 61, the visual processing unit 63 can calculate the above-mentioned expression M6 based on the input signal IS and the unsharp signal US without using the two-dimensional LUT4. In this case, a one-dimensional LUT can be used in the calculation of the respective functions F9 and F10.

(ii)

In addition, when the value C of a certain element of the profile data obtained by Equation M6 falls within the range of 0≤C≤255, the value of the element may be 0 or 255.

(7)

【The 7th description file data】

The seventh profile data is determined based on the calculation of adding the value after gradation correction of the input signal IS to the value obtained by enhancing the difference between the input signal IS and the unsharp signal US. Here, the enhancement of the sharp components and the gradation correction of the input signal IS are performed separately. Therefore, regardless of the gradation correction amount of the input signal IS, constant sharp component enhancement can be performed.

It will be described in detail below.

The value C of each element of the seventh profile data (the value of OS of the output signal) is the value A of the input signal IS, the value B of the unsharp signal US, the enhancement function F11, and the gradation correction function F12, expressed as C=F12(A)+F11(A-B) (hereinafter referred to as formula M7).

Here, the enhancement function F11 is any one of the enhancement functions R1 to R3 described using FIG. 109 . The grayscale correction function F12 is, for example, a gamma correction function, an S-shaped grayscale correction function, or an inverted S-shaped grayscale correction function.

[Equivalent visual processing device 71]

FIG. 23 shows a visual processing device 71 equivalent to the visual processing device 1 in which the seventh profile data is registered in the two-dimensional LUT.

The visual processing device 71 is a device that outputs the output signal OS by adding the value obtained by adding the gray scale correction value of the input signal IS to the value after strengthening the difference between the input signal IS and the unsharp signal US. . Here, the enhancement of the sharp components and the gradation correction of the input signal IS are performed separately. Therefore, regardless of the gradation correction amount of the input signal IS, constant sharp component enhancement can be performed.

The visual processing device 71 shown in FIG. 23 is provided with: a spatial processing unit 72, which performs spatial processing to output the unsharp signal US according to the luminance value of each pixel of the original image obtained as the input signal IS; The visual processing unit 73 performs visual processing of the original image using the input signal IS and the unsharp signal US, and outputs an output signal OS.

Since the spatial processing unit 72 performs the same operation as the spatial processing unit 2 included in the visual processing device 1 , description thereof will be omitted.

The visual processing unit 73 includes: a subtraction unit 74 that takes the input signal IS as a first input, takes the unsharp signal US as a second input, and outputs a difference signal DS as a difference between them; an enhancement processing unit 75 , which outputs the enhanced processing signal TS that takes the difference signal DS as an input and performs enhanced processing; the gray scale correction unit 76, which takes the input signal IS as an input, and outputs the gray scale corrected signal GC after gray scale correction; and the addition unit 77, which takes the grayscale correction signal GC as the first input, takes the enhanced processing signal TS as the second input, and outputs the output signal OS.

[The role of the equivalent visual processing device 71]

The operation of the visual processing unit 73 will be further described.

The subtraction unit 74 calculates the difference between the input signal IS of value A and the unsharp signal US of value B, and outputs a difference signal DS of value A-B. The enhancement processing unit 75 outputs the enhancement processing signal TS of the value F11(A-B) based on the difference signal DS of the value A-B using the enhancement function F11. The gradation correction unit 76 outputs the gradation correction signal GC of the value F12 (A) based on the input signal IS of the value A using the gradation correction function F12. The adding unit 77 adds the gradation correction signal GC of the value F12(A) and the enhanced signal TS of the value F11(A-B), and outputs the output signal OS of the value F12(A)+F11(A-B).

In addition, the calculation using the enhancement function F11 and the gradation correction function F12 may be performed using a one-dimensional LUT for each function, or may be performed without using a LUT.

【Effect】

The visual processing device 1 and the visual processing device 71 provided with the seventh profile data realize the same visual processing effect.

(i)

The input signal IS is added to the enhanced signal TS after performing gradation correction by the gradation correction unit 76 . Therefore, even in a region where the gradation of the gradation correction function F12 changes little, that is, a region where the contrast is lowered, the local contrast can be enhanced by the subsequent addition of the enhancement processing signal TS.

【Modification】

(i)

In the visual processing device 71, the visual processing unit 73 can calculate the above expression M7 based on the input signal IS and the unsharp signal US without using the two-dimensional LUT. In this case, it is not necessary to use a one-dimensional LUT in the calculation of each of the functions F11 and F12.

(ii)

In addition, when the value C of a certain element of the profile data obtained by Equation M7 exceeds the range of 0≤C≤255, the value C of the element may be 0 or 255.

(8)

[Modification of the 1st to 7th profile data]

In the above (1) to (7), each element of the first to seventh profile data is described, and the values calculated based on the formulas M1 to M7 are stored. Furthermore, it has been explained that in each profile data, when the value calculated by the formulas M1 to M7 exceeds the range in which the profile data can be stored, the value of the element can be limited.

Furthermore, in the profile data, some values may be arbitrary. For example, in a very small bright part in a very dark night scene (neon lights in a night scene, etc.), although the value of the input signal IS is large, but the value of the passivation signal US is small, the input signal after visual processing The value of IS has little effect on image quality. In this way, in the part where the visually processed value has little influence on the image quality, the value stored in the profile data may be an approximate value of the value calculated by the formulas M1-M7, or an arbitrary value.

When the value stored in the profile data is an approximate value of the value calculated by the formulas M1 to M7, or an arbitrary value, it is preferable to maintain the input signal with respect to the value stored in the input signal IS and the unsharp signal US of the same value. There is a monotonically increasing or monotonically decreasing relationship between IS and the value of the passivation signal US. Among the profile data created based on the formulas M1 to M7, etc., the values stored in the profile data corresponding to the input signal IS and the unsharp signal US of the same value represent the summary of the characteristics of the profile data. Therefore, in order to maintain the characteristics of the two-dimensional LUT, it is preferable to perform tuning of the profile data while maintaining the relationship described above.

[Second Embodiment]

A visual processing device 600 as a second embodiment of the present invention will be described using FIGS. 24 to 39 .

The visual processing device 600 is a visual processing device that performs visual processing on an image signal (input signal IS) and outputs a visually processed image (output signal OS), and performs and sets a display device (not shown) that displays the output signal OS. ) environment (hereinafter referred to as the display environment) corresponding to the visual processing device.

Specifically, the visual processing device 600 is a device that improves the decrease in "visible contrast" of a display image due to the influence of ambient light in the display environment through visual processing utilizing human visual characteristics.

Visual processing device 600 is a device that processes images such as computers, televisions, digital cameras, mobile phones, PDAs, printers, scanners, etc., and constitutes a device that performs color processing of image signals and an image processing device.

【Visual processing device 600】

FIG. 24 shows the basic configuration of the visual processing device 600 .

The visual processing device 600 is composed of a target contrast conversion unit 601 , a converted signal processing unit 602 , an actual contrast conversion unit 603 , a target contrast setting unit 604 , and an actual contrast setting unit 605 .

The target contrast conversion unit 601 receives the input signal IS as a first input, the target contrast C1 set in the target contrast setting unit 604 as a second input, and outputs a target contrast signal JS. In addition, the definition of the target contrast C1 will be described later.

The converted signal processing unit 602 takes the target contrast signal JS as the first input, the target contrast C1 as the second input, and the actual contrast C2 set in the actual contrast setting unit 605 as the third input, and uses The visually processed signal KS is output according to the processed target contrast signal JS. In addition, the definition of the actual contrast C2 will be described later.

The actual contrast conversion unit 603 receives the visually processed signal KS as a first input, the actual contrast C2 as a second input, and outputs an output signal O.

The target contrast setting unit 604 and the actual contrast setting unit 605 allow the user to set the values of the target contrast C1 and the actual contrast C2 through an input interface or the like.

Hereinafter, details of each part will be described.

[Target contrast conversion unit 601]

The target contrast conversion unit 601 converts the input signal IS input to the visual processing device 600 into a target contrast signal JS suitable for contrast expression. Here, in the input signal IS, the luminance value of the image input to the visual processing device 600 is represented by a gradation value of [0.0 to 1.0].

The target contrast conversion unit 601 converts the input signal IS (value "P") using the target contrast C1 (value [m]) by "Equation M20", and outputs the target contrast signal JS (value [A]). Here, the formula M20 is A={(m-1)/m}×P+1/m.

The value [m] of the target contrast C1 is set to a contrast value with the best contrast seen from a display image displayed on the display device.

Here, the so-called contrast value is a value expressed as the brightness of the white level relative to the black level of the image, and represents the brightness value of the white level when the black level is 1 (black level: white level = 1: m).

Although the value [m] of the target contrast ratio C1 is suitable to be set at about 100 to 1000 (black level: white level = 1:100 to 1:1000), it is also possible for the display device to display based on the white level. Determined by the light-to-dark ratio of the black level.

Using FIG. 25, the conversion by the expression M20 will be described in more detail. FIG. 25 shows the relationship between the value of the input signal IS (horizontal axis) and the value of the target contrast signal JS (vertical axis). As shown in FIG. 25 , the target contrast changing unit 601 converts the input signal IS having a value in the range of [0.0 to 1.0] into a target contrast signal JS having a value in the range of “1/m to 1.0”.

[Converted signal processing unit 602]

The details of the converted signal processing unit 602 will be described with reference to FIG. 24 .

The converted signal processing unit 602 compresses the dynamic range while maintaining the local contrast of the input target contrast signal JS, and outputs the visually processed signal KS. Specifically, the converted signal processing unit 602 regards the input signal IS (see FIG. 16 ) in the visual processing device 21 shown in the first embodiment as the target contrast signal JS, and has an output signal OS (see FIG. 16) Consider the same composition, action, and effect as the visually processed signal KS.

The converted signal processing unit 602 outputs the visually processed signal KS based on the calculation of enhancing the ratio between the target contrast signal JS and the unsharp signal US. In this way, for example, visual manipulations such as enhancing clear components can be realized.

Furthermore, the converted signal processing unit 602 performs dynamic range compression calculation based on the ratio between the enhanced target contrast signal JS and the unsharp signal US, and outputs the visually processed signal KS. In this way, visual processing such as compression of the dynamic range while enhancing sharp components can be realized.

[Structure of the converted signal processing unit 602]

The converted signal processing unit 602 includes: a spatial processing unit 622, which performs spatial processing on the luminance value of each pixel in the target contrast signal JS, and outputs the unsharp signal US; and a visual processing unit 622, which uses the target contrast The signal JS and the passivation signal US perform visual processing on the target contrast JS, and output the visual processing signal KS.

Since the spatial processing unit 622 performs the same operation as the spatial processing unit 2 included in the visual processing device 1 (see FIG. 1 ), detailed description thereof will be omitted.

The visual processing unit 623 includes a division unit 625 , an enhancement processing unit 626 , and an output processing unit 627 including a DR compression unit 628 and a multiplication unit 629 .

The division unit 625 takes the target contrast signal JS as a first input, takes the unsharp signal US as a second input, and outputs a division signal RS obtained by dividing the target contrast signal JS by the unsharp signal US. The enhancement processing unit 626 receives the division signal RS as a first input, the target contrast C1 as a second input, and the actual contrast C2 as a third input, and outputs an enhancement processing signal TS.

The output processing unit 627 takes the target contrast signal JS as the first input, the enhanced processing signal TS as the second input, the target contrast C1 as the third input, and the actual contrast C2 as the fourth input, and outputs the visually processed signal KS . The DR compression unit 628 receives the target contrast signal JS as a first input, the target contrast C1 as a second input, and the actual contrast C2 as a third input, and outputs a DR compressed signal DRS after dynamic range (DR) compression. The multiplication unit 629 receives the DR compressed signal DRS as a first input, the enhanced processed signal TS as a second input, and outputs the visually processed signal KS.

[Function of the conversion signal processing unit 602]

The converted signal processing unit 602 uses the target contrast C1 (value [m]) and the actual contrast C2 (value [m]) to convert the target contrast signal JS (value [A]) by [Formula M2], and converts the visual processing The signal KS (value [C]) is output. Here, Equation M2 is expressed as C=F4(A)×F5(A/B) using the dynamic range compression function F4 and the enhancement function F5. In addition, the value [B] is the value of the unsharp signal US obtained by spatially processing the target contrast JS.

The dynamic range compression function F4 is a [power function] that is an upwardly convex monotonically increasing function, expressed as F4(x)=x^γ. The exponent γ of the dynamic range compression function F4 is expressed as γ=log(n)/log(m) using common logarithms. The strengthening function F5 is a [power function] expressed as F5(x)=x^(1-γ).

Hereinafter, the relationship between Equation M2 and the operations of the respective units of the converted signal processing unit 602 will be described.

The spatial processing unit 622 performs spatial processing on the target contrast signal JS of the value [A], and outputs the unsharp signal US of the value [B].

The dividing unit 625 divides the target contrast signal JS of the value [A] by the unsharp signal US of the value [B], and outputs the divided signal RS of the value [A/B]. The enhancement processing unit 626 outputs the enhancement processing signal TS of the value [F5(A/B)] from the division signal RS of the value [A/B] using the enhancement function F5. The DR compression unit 628 outputs the DR compressed signal DRS of the value [F4(A)] based on the target contrast signal JS of the value [A] using the dynamic range compression function F4. The multiplication unit 629 multiplies the DR compressed signal DRS with a value of [F4(A)] and the enhanced signal TS with a value of [F5(A/B)] to obtain a value of [F4(A)×F5(A/B)] The visual processing signal KS output.

In addition, the calculation using the dynamic range compression function F4 and the enhancement function F5 may be performed using a one-dimensional LUT for each function, or may be performed without using a LUT.

[Effect of the conversion signal processing unit 602]

The visual dynamic range in the visually processed signal KS is determined by the value of the dynamic range compression function F4.

The conversion by the expression M2 is further described in detail using FIG. 26 . FIG. 26 is a graph showing the relationship between the value of the target contrast signal JS (horizontal axis) and the value of the target contrast signal JS after applying the dynamic range compression function F4 (vertical axis). As shown in FIG. 26, the dynamic range of the target contrast signal JS is compressed by the dynamic range compression function F4. More specifically, the target contrast signal JS having a value in the range of [1/m to 1.0] is converted to a value in the range of [1/n to 1.0] by the dynamic range compression function F4. As a result, the visible dynamic range in the visually processed signal KS is compressed to 1/n (minimum value: maximum value = 1:n).

Here, the actual contrast C2 will be described. The value [n] of the actual contrast C2 is set as the visible contrast value of the displayed image under the ambient light of the display environment. That is, the value [n] of the actual contrast C2 can be determined as a value obtained by reducing the value [m] of the target contrast C1 by only the amount of influence caused by the brightness of the ambient light of the display environment.

Since the value [n] of the actual contrast C2 thus set is used, the dynamic range of the target contrast signal JS is compressed from 1:m to 1:n by the expression M2. In addition, the term "dynamic range" here refers to the ratio of the maximum value to the minimum value of a signal.

On the other hand, the change in local contrast in the visually processed signal KS represents the ratio of the amount of change before and after conversion between the value [A] of the target contrast signal JS and the value [C] of the visually processed signal KS. In this case, the value [B] of the unsharp signal US is considered constant in a local, ie narrow, range. Therefore, the ratio between the change amount of the value C and the change amount of the value A in the expression M2 becomes 1, and the local contrast between the target contrast signal JS and the visually processed signal KS does not change.

Human vision has the property that if the local contrast is maintained, even if the overall contrast is lowered, the same contrast can be seen. Therefore, in the converted signal processing unit 602 , it is possible to implement visual processing that does not lower the visible contrast while compressing the dynamic range of the target contrast signal JS.

[Actual contrast conversion unit 603]

The details of the actual contrast conversion unit 603 will be described using FIG. 24 .

The actual contrast conversion unit 603 converts the visually processed signal KS into image data within a range that can be input to a display device (not shown). The image data within the range that can be input to the display device is, for example, image data in which the luminance value of the image is expressed in a gradation value of [0.0 to 1.0].

The actual contrast conversion unit 603 converts the visually processed signal KS (value [C]) using the actual contrast C2 (value [n]) by the expression "M21", and outputs the output signal OS (value "Q"). Here, the formula M21 is Q={a/(n-1)}×C-{1/(n-1)}.

Using FIG. 27, the conversion by the expression M21 will be further described. FIG. 27 is a graph showing the relationship between the value (horizontal axis) of the visually processed signal KS and the value (horizontal axis) of the output signal OS. As shown in FIG. 27 , the actual contrast conversion unit 603 converts the visually processed signal KS in the range of [1/n to 1.0] into the output signal OS in the range of [0.0 to 1.0]. Here, the value of the output signal OS decreases relative to the value of each visually processed signal KS. This amount of reduction corresponds to the influence of ambient light on each brightness of the displayed image.

In addition, in the actual contrast conversion unit 603, when the visually processed signal KS having an input value [1/n] or less is input, the output signal OS is converted to a value [0]. Then, in the actual contrast conversion unit 603, when the visually processed signal KS having a value of [1] or more is input, the output signal OS is converted to a value of [1].

[Effects of the visual processing device 600]

The visual processing device 600 achieves the same effect as the visual processing device 21 described in the first embodiment. Hereinafter, characteristic effects in the visual processing device 600 will be described.

(i)

When there is ambient light in the display environment where the output signal OS of the visual processing device 600 is displayed, the output signal OS is visualized under the influence of the ambient light. However, the output signal OS is a signal subjected to processing to correct the influence of ambient light by the actual contrast conversion unit 603 . That is, in a display environment where ambient light exists, the output signal OS displayed on the display device is seen as a display image having the characteristics of the visually processed signal KS.

The characteristic of the visually processed signal KS is the same as the output signal OS (see FIG. 16 ) of the visual processing device 21 described in the first embodiment, and compresses the dynamic range of the entire image while maintaining local contrast. That is, the visually processed signal KS is a signal compressed to a displayable dynamic range (corresponding to the actual contrast C2 ) under the influence of ambient light while maintaining the target contrast C1 for locally displaying an optimal display image.

Therefore, in the visual processing device 600 , it is possible to maintain the visual contrast by processing using the visual characteristics while correcting the contrast lowered by the presence of ambient light.

【Visual processing method】

Using FIG. 28 , a visual processing method for achieving the same effects as those of the visual processing device 600 described above will be described. In addition, since the specific processing of each step is the same as the processing in the visual processing device 600 described above, description thereof will be omitted.

In the visual processing method shown in FIG. 28 , first, the set target contrast C1 and the actual contrast C2 are obtained (step S601 ). Next, the obtained target contrast C1 is used to perform conversion corresponding to the input signal IS (step S602 ), and output the target contrast signal JS. Next, perform spatial processing on the target contrast signal JS (step S603 ), and output the unsharp signal US. Next, the target contrast signal JS is divided by the unsharp signal US (step S604 ), and the divided signal RS is output. The divided signal RS is enhanced by the enhancement function F5 having an exponent determined by the target contrast C1 and the actual contrast C2 (step S605 ), and an enhanced signal TS is output. On the other hand, the target contrast signal JS is dynamically range compressed by the dynamic range compression function F4 which is a [power function] having an exponent determined by the target contrast C1 and the actual contrast C2 (step S606), and the DR compressed signal DRS output. Next, the enhanced signal TS output in step 695 is multiplied by the DR compressed signal DRS output in step S606 (step S607 ), and the visually processed signal KS is output. Next, the actual contrast C2 is used to perform conversion corresponding to the visually processed signal KS (step S608 ), and output the output signal OS. The processing of steps S602 to S608 is repeated for all pixels of the input signal IS (step S609 ).

Each step of the visual processing method shown in FIG. 28 can be implemented as a visual processing program in the visual processing device 600 or other computers. In addition, the processing of steps S604 to S607 can be performed once by the calculation formula M2.

【Modification】

The present invention is not limited to the above-described embodiments, and various modifications and corrections are possible without departing from the scope of the present invention.

(i) When there is no formula M2-enhancement function F5

In the above-mentioned embodiment, it was described that the converted signal processing unit 602 outputs the visually processed signal KS based on the expression M2. Here, the converted signal processing unit 602 may output the visually processed signal KS based only on the dynamic range enhancement function F4. In this case, the transformed signal processing unit 602 as a modified example does not need to include the spatial processing unit 622 , the dividing unit 625 , the enhancement processing unit 626 , and the multiplication unit 629 , and only needs to include the DR compression unit 628 .

In the converted signal processing unit 602 as a modified example, the visually processed signal KS compressed to a displayable dynamic range under the influence of ambient light may be output.

(ii) Strengthening function F5-exponent, other modified examples

In the above-mentioned embodiment, it is described that the enhancement function F5 is a [power function] expressed as F5(x)=x^(1-γ). Here, the index of the enhancement function F5 may be a function of the value [A] of the target contrast signal KS or the value [B] of the unsharp signal US.

Specific examples (1) to (6) are shown below.

(1)

The exponent of the enhancement function F5 is a function of the value [A] of the target contrast signal JS, and is a monotonically decreasing function when the value [A] of the target contrast signal JS is larger than the value [B] of the passivation signal US. More specifically, the exponent of the enhancement function F5, expressed as α1(A)×(1-γ), the function α1(A), is a function that monotonically decreases with respect to the value [A] of the target contrast signal JS as shown in FIG. 29 . In addition, the maximum value of the function α1(A) is [1.0].

At this time, the enhancement amount of the high-brightness local contrast is reduced by the enhancement function F5. Therefore, when the luminance of the pixel of interest is higher than the luminance of surrounding pixels, excessive enhancement of the local contrast of the high-luminance portion is suppressed. That is, it is suppressed that the luminance value of the pixel of interest saturates toward a high luminance and becomes a so-called reversed state.

(2)

The index of the enhancement function F5 is the index of the value [A] of the target contrast signal JS. When the value [A] of the target contrast signal JS is smaller than the value [B] of the passivation signal US, it is a monotonically increasing function. More specifically, the exponent of the enhancement function F5 is expressed as α2(A)×(1-γ), and the function α2(A) is monotonically increasing for the value (A) of the target contrast signal JS as shown in Figure 30 function. In addition, the maximum value of the function α2(A) is [1.0].

In this case, the enhancement function F5 reduces the enhancement amount of the local contrast of the low-brightness area. Therefore, when the luminance of the pixel of interest is lower than the luminance of surrounding pixels, excessive enhancement of the local contrast in the low-luminance portion is suppressed. That is, the luminance value of the pixel of interest is suppressed from being saturated to a low luminance, a so-called reversed black state.

(3)

The index of the enhancement function F5 is a function of the value [A] of the target contrast signal JS, and is a monotonically increasing function when the value [A] of the target contrast JS is larger than the value [B] of the passivation signal US. More specifically, the exponent of the enhancement function F5, expressed as α3(A)×(1-γ), the function α3(A), is monotonically increasing with respect to the value [A] of the target contrast signal JS as shown in FIG. 31 function. In addition, the maximum value of function α3(A) is [1.0].

In this case, the enhancement function F5 reduces the enhancement amount of the local contrast of the low-brightness portion. Therefore, when the luminance of the pixel of interest is higher than the luminance of surrounding pixels, excessive enhancement of the local contrast in the low luminance portion is suppressed. The low-luminance part in the image has a relatively high noise ratio because the signal level is low, but by performing such processing, the deterioration of the SN ratio can be suppressed.

(4)

The exponent of the enhancement function F5, which is a function between the value [A] of the target contrast signal JS and the value [B] of the passivation signal US, is monotonously decreasing in the absolute value of the difference between the relative value [A] and the value [B] function. In other words, the exponent of the strengthening function F5 can also be called a function in which the ratio of the value [A] to the value [B] increases to approximately 1. More specifically, the exponent of the strengthening function F5, expressed as α4(A, B) × (1-γ), the function α4(A, B), is monotonic with respect to the absolute value of the value "A-B" as shown in Figure 32 Reduced function.

In this case, it is possible to suppress enhancement of the local contrast of a pixel of interest having a large brightness difference from surrounding pixels by particularly enhancing the local contrast of the pixel of interest having a small brightness difference from surrounding pixels.

An upper limit or a lower limit may also be set for the calculation results of the enhancement function F5 in (1) to (4) above. Specifically, when the value ( F5 (A/B)) exceeds a predetermined upper limit, the predetermined upper limit is used as the calculation result of the reinforcement function F5 . And, when the value (F5(A/B)) exceeds a predetermined lower limit, the predetermined lower limit is used as the calculation result of the enhancement function F5.

In this case, the enhancement function F5 can be used to limit the enhancement amount of the local contrast within an appropriate range, and suppress too much or a little contrast enhancement.

(6)

In addition, the above (1) to (5) are similarly applied to the case of performing calculations using the enhancement function F5 in the above-mentioned first embodiment (for example, the first embodiment [profile data] (2) or (3) wait). In addition, in the first embodiment, the value [A] is the value of the input signal IS, and the value [B] is the value of the unsharp signal US obtained by spatially processing the input signal IS.

(iii) When Equation 2-Dynamic Range Compression is not performed

In the above-mentioned embodiment, it was described that the converted signal processing unit 602 has the same configuration as that of the visual processing device 21 shown in the first embodiment. The visual processing device 31 (see FIG. 19 ) shown in the first embodiment has the same configuration. Specifically, the converted signal processing unit 602 as a modified example is realized by regarding the input signal IS in the visual processing device 31 as the target contrast signal JS and the output signal OS as the visually processed signal KS.

In this case, in the converted signal processing unit 602 as a modified example, for the target contrast signal JS (value (A)) and the unsharp signal US (value (B)), based on "Formula M3", the visually processed signal KS(value[C]) output. Here, the expression M3 is expressed as C=A×F5(A/B) using the enhancement function F5.

In the processing using Equation M3, although dynamic range compression is applied to the input signal IS, local contrast can also be enhanced. This partial contrast enhancement effect can give various impressions of "visually" dynamic range compression or expansion.

In addition, the above-mentioned [Modifications] (ii) (1) to (5) can also be similarly applied to this embodiment. That is, in this modified example, the strengthening function F5 is a [power function], and its exponent may have the functions α1(A), α2( A), α3(A), α4(A, B) functions with the same slope. In addition, as described in [Modification] (ii) (5) above, an upper limit or a lower limit may be set in the calculation result of the enhancement function F5.

(iv) Parameter automatic setting

In the above embodiment, it has been described that the target contrast setting unit 604 and the actual contrast setting unit 605 allow the user to set the values of the target contrast C1 and the actual contrast C2 through an input interface or the like. Here, the target contrast setting unit 604 and the actual contrast setting unit 605 can automatically set the values of the target contrast C1 and the actual contrast C2.

(1) Display

The display device that displays the output signal OS is a display such as a PDP, LCD, or CRT. For the known white brightness (white level) and black brightness (black level) that can be displayed in a state without ambient light, The actual contrast setting unit 605 that automatically sets the value of the actual contrast C2 will be described.

FIG. 33 shows an actual contrast setting unit 605 that automatically sets the value of the actual contrast C2. The actual contrast setting unit 605 includes a brightness measurement unit 605a, a storage unit 605b, and a calculation unit 605c.

The luminance measurement unit 605a is a luminance sensor that measures the luminance value of the ambient light in the display environment of the display that displays the output signal OS. The storage unit 605b stores white luminance (white level) and black luminance (black level) that can be displayed by the display that displays the output signal OS in a state where there is no ambient light. The calculation unit 605c obtains each value from the luminance measurement unit 605a and the storage unit 605b, and calculates the value of the actual contrast C2.

An example of calculation performed by the calculation unit 605c will be described. The calculation unit 605c adds the luminance value of the ambient light obtained from the luminance measurement unit 605a to the luminance value of the black level and the luminance value of the white level stored in the storage unit 605b. Furthermore, the calculation unit 605c uses the addition result to the brightness value of the black level and removes the addition result to the brightness value of the white level as the value [n] of the actual contrast C2. In this way, the value [n] of the actual contrast C2 represents the value of the contrast displayed by the display in a display environment where ambient light exists.

Moreover, the storage unit 605b shown in FIG. 33 can store the ratio between the white brightness (white level) and the black brightness (black level) that the display can display in the state of no ambient light as the target contrast C1 The value of [m]. In this case, the actual contrast setting unit 605 simultaneously realizes the function of the target contrast setting unit 604 for automatically setting the target contrast C1. In addition, the storage unit 605b may not store the ratio, and the ratio may be calculated by the calculation unit 605c.

(2) Projector

The display device that displays the output signal OS is a projector or the like, and when the white luminance (white level) and black luminance (black level) that can be displayed in a state where there is no ambient light depend on the distance to the screen, The actual contrast setting unit 605 that automatically sets the value of the actual contrast C2 will be described.

FIG. 34 shows an actual contrast setting unit 605 that automatically sets the value of the actual contrast C2. The actual contrast setting unit 605 includes a brightness measuring unit 605d and a control unit 605e.

The luminance measuring unit 605d is a luminance sensor that measures the luminance value of the output signal OS displayed by the projector in the display environment. The control unit 605e performs display between the white level and the black level for the projector. Furthermore, the brightness value at the time of displaying each level is obtained from the brightness measurement part 605d, and the value of actual contrast C2 is calculated.

An example of the operation of the control unit 605e will be described using FIG. 35 . First, the control unit 605e operates the projector to display a white level in a display environment where ambient light exists (step S620). The control unit 605e obtains the measured brightness of the white level from the brightness measurement unit 605d (step S621). Next, the control unit 605e operates the projector to display a black level in a display environment where ambient light exists (step S622). The control unit 605e obtains the luminance of the measured black level from the luminance measuring unit 605d (step S623). The control unit 605e calculates the obtained ratio between the brightness value of the white level and the brightness value of the black level, and outputs it as the value of the actual contrast C2. In this way, the value [n] of the actual contrast C2 represents the value of the contrast displayed by the projector in a display environment where ambient light exists.

In addition, the value [m] of the target contrast C1 can also be derived by calculating the ratio between the white level and the black level in a display environment where no ambient light exists, as described above. In this case, the actual contrast setting unit 605 simultaneously realizes the function of the target contrast setting unit 604 for automatically setting the target contrast C1.

(v) Other signal spaces

In the above-mentioned embodiments, it has been described that the processing in the visual processing device 600 is performed on the brightness of the input signal IS. Here, the present invention is not effective only when the input signal IS is represented by the YCbCr color space. The input signal IS can also be displayed by YUV color space, Lab color space, Luv color space, YIQ color space, XYZ color space, YPbPr color space, etc. In these cases, the processing described in the above-mentioned embodiment can be executed for brightness and brightness of each color space.

Furthermore, when the input signal IS is represented by the RGB color space, the processing in the visual processing device 600 can be performed independently for each component of RGB. That is, the RGB components of the input signal IS are individually processed by the target contrast conversion unit 601, and the RGB components of the target contrast signal JS are output. Furthermore, the RGB components of the target contrast signal JS are individually processed by the converted signal processing unit 602, and the RGB components of the visually processed signal KS are output. Furthermore, the RGB components of the output signal OS are output by independently performing the process of the actual contrast conversion unit 603 on the RGB components of the visually processed signal KS. Here, the target contrast C1 and the actual contrast C2 use a common value in each processing of the RGB components.

(vi) Chromatic aberration correction processing

The visual processing device 600 may further include a chromatic aberration correction processing unit in order to suppress the color tone difference between the output signal OS and the input signal IS due to the influence of the luminance component processed by the converted signal processing unit 602 .

FIG. 36 shows a visual processing device 600 including a chromatic aberration correction processing unit 608 . In addition, the same code|symbol is attached|subjected to the structure similar to the visual processing apparatus 600 shown in FIG. 24. In addition, the input signal IS has a color space of YCbCr, and the same processing as that described in the above-mentioned embodiment is performed on the Y component. Hereinafter, the chromatic aberration correction processing unit 608 will be described.

The chromatic aberration correction unit 608 takes the target contrast signal JS as the first input (value [Yin]), the visually processed signal KS as the second input (value [Yout]), and the Cb component of the input signal IS as the third input ( value[CBin]), the Cr component of the input signal IS is taken as the fourth input (value[CRin]), the Cb component after the chromatic aberration correction is taken as the first output (value[CBout]), and the Cr component after the chromatic aberration correction is As the 2nd output (value [CRout]).

FIG. 37 shows the outline of the chromatic aberration correction process. The chromatic aberration correction processing unit 608 has four inputs of [Yin], [Yout], [CBin], and [CRin], and obtains two outputs of [CBout] and [CRout] by computing these four inputs.

[CBout] and [CRout] are derived from the difference and ratio between [Yin] and [Yout] based on the following equation corrected for [CBin] and [CRin].

[CBout] is based on a1×([Yout]-[Yin])×[CBin]+a2×(1-[Yout])/[Yin])×[CBin]+a3×([Yout]-[Yin ])×[CRin]+a4×(1−[Yout]/[Yin])×[CRin]+[CBin] (hereinafter referred to as formula CB).

[CRout] is based on a5×([Yout]-[Yin])×(Cbin)+a6×(1-[Yout]/[Yin])×(CBin)+a7×([Yout]-[Yin] )×[CRin]+a8×(1−[Yout]/[Yin])×(CRin)+[CRin] (hereinafter referred to as formula CR).

In the coefficients a1 to a5 in the formulas CB and CR, values determined in advance by an external calculation device of the visual processing device 600 are used in the estimation calculation described below.

Using FIG. 38 , the estimation operation of the coefficients a1 to a8 in a computing device or the like will be described.

First, four inputs of [Yin], [Yout], [CBin], [CRin] are obtained (step S630). Each input value is data prepared in advance for determining the coefficients a1 to a8. For example, as [Yin], [CBin], and [CRin], values obtained by subtracting the middle part from all possible values at regular intervals are used. Furthermore, as [Yout], a value that can be output when the value of [Yin] is input to the converted signal processing unit 602 is used, and the value obtained by removing the middle part at constant intervals. The data thus prepared are obtained as 4 inputs.

The obtained [Yin], [CBin], [CRin] are transformed into Lab color space, and the chromaticity values [Ain] and [Bin] in the transformed Lab color space are calculated (step S631).

Next, the default coefficients a1 to a8 are used to calculate [Formula CB[ and [Formula CR] to obtain the values of [CBout] and [CRout] (step S632). The obtained value and [Yout] are converted into the Lab color space, and the chromaticity values [Aout] and [Bout] of the converted Lab color space are calculated (step S633).

Next, an evaluation function is calculated using the calculated chromaticity values [Ain], [Bin], [Aout], and [Bout] (step S634), and it is judged whether or not the value of the evaluation function is equal to or less than a predetermined threshold. Here, the evaluation function is a function that takes a smaller value when the hue change between [Ain] and [Bin] and [Aout] and [Bout] is small, and is, for example, the sum of the squares of deviations of the respective components The function. More specifically, the evaluation function is ([Ain]-[Aout])^2+([Bin]-[Bout])^2 and so on.

When the value of the evaluation function is larger than the predetermined threshold (step S635), the coefficients a1 to a8 are corrected (step S636), and the calculations of steps S632 to S635 are repeated using new coefficients.

When the value of the evaluation function is smaller than the predetermined threshold (step S635), the coefficients a1 to a8 used for the calculation of the evaluation function are output as the result of the estimation calculation (step S637).

In addition, in the estimation calculation, one of the four input combinations of [Yin], [Yout], [CBin] and [CRin] prepared in advance can be used to estimate the operation coefficients a1 to a8, but the combination can also be used The above processing is performed for a plurality of groups, and the coefficients a1 to a8 that minimize the evaluation function are output as the result of the estimation calculation.

(Modification in chromatic aberration correction processing)

(1)

In the above-mentioned chromatic aberration correction processing unit 608, the value of the target contrast signal JS is [Yin], the value of the visually processed signal KS is [Yout], the value of the Cb component of the input signal IS is [CBin], and the value of the input The value of the Cr component of the signal IS is [CRin], the value of the Cb component of the output signal OS is [CBout], and the value of the Cr component of the output signal OS is [CRout]. Here, [Yin], [Yout], [CBin], [CRin] [CBout], [CRout] can also be the value of the signal.

For example, when the input signal IS is a signal in the RGB color space, the target contrast conversion unit 601 (see FIG. 24 ) processes each component of the input signal IS. In this case, the processed RGB color space signal can be transformed into a YCbCr color space signal, and the value of the Y component is [Yin], the value of the Cb component is [CBin], and the value of the Cr component is [CRin]

Further, when the output signal OS is a signal of RGB color space, the derived [Yout], [CBout], [CRout] can be transformed into RGB color space, and each component is converted by the actual contrast conversion unit 603, as output signal OS.

(2)

The chromatic aberration correction processing unit 608 can perform correction processing on each RGB component input to the chromatic aberration correction processing unit 608 using the ratio of the signal values before and after the processing by the converted signal processing unit 602 .

The configuration of a visual processing device 600 as a modified example will be described using FIG. 39 . In addition, the same reference numerals are assigned to the parts that realize basically the same functions as those of the visual processing device 600 shown in FIG. 36 , and description thereof will be omitted. A visual processing device 600 as a modified example includes a luminance signal generation unit 610 as a characteristic configuration.

Each component of the input signal IS, which is a signal in the RGB color space, is converted into a target contrast signal JS, which is a signal in the RGB color space, in the target contrast conversion unit 601 . As for the detailed processing, the processing for the above is omitted. Here, let the values of the respective components of the target contrast signal JS be [Rin], [Gin], and [Bin].

The luminance signal generation unit 610 generates a luminance signal of value [Yin] based on each component of the target contrast signal JS. The luminance signal is obtained by adding the values of the RGB components at a certain ratio. For example, the value [Yin] is obtained by the following formula [Yin]=0.299×[Rin]+0.587×[Gin]+0.114×[Bin].

The converted signal processing unit 602 processes the luminance signal of the value [Yin], and outputs the visually processed signal KS of the value [Yout]. The detailed processing is the same as the processing in the converted signal processing unit 602 (see FIG. 36 ) that outputs the visually processed signal KS based on the target contrast signal JS, and thus description thereof will be omitted.

The chromatic aberration correcting unit 608 uses the luminance signal (value [Yin]), the visually processed signal KS (value [Yout]), and the target contrast signal JS (values [Rin], [Gin], [Bin]) to generate The chromatic aberration correction signal (value [Rout], [Gout], [Bout]) of the spatial signal is output.

Specifically, in the chromatic aberration correction processing unit 608 , the ratio (value ([Yout]/[Yin])) of the value [Yin] to the value [Yout] is calculated. The calculated ratio is multiplied by each component of the target contrast signal JS (values [Rin], [Gin], [Bin]) as a chromatic aberration correction coefficient. In this way, color difference correction signals (values [Rout], [Gout], [Bout]) are output.

The actual contrast conversion unit 603 converts each component of the color difference correction signal which is a signal in the RGB color space, and converts it into an output signal OS which is a signal in the RGB color space. Regarding the detailed processing, the description for the above is omitted.

In the visual processing device 600 which is a modified example, the processing in the converted signal processing unit 602 is only processing on the luminance signal, and processing does not need to be performed on each component of RGB. Therefore, the load of visual processing on the input signal IS of the RGB color space is lightened.

(3)

"Formula CB" and "Formula CR" are examples, and other formulas may be used.

(vii) Visual processing unit 623

The visual processing unit 623 shown in FIG. 24 may also be formed by a two-dimensional LUT.

In this case, the 2-dimensional LUT stores the value of the visually processed signal KS relative to the value of the target contrast signal JS and the value of the unsharp signal US. More specifically, the value of the visually processed signal KS is determined based on [Formula M2] described in [First Embodiment] (profile data) (2) (second profile data). In addition, in [Equation M2], as the value of the target contrast signal JS of the value A, the value of the unsharp signal US of the value B is used.

Visual processing device 600 includes such a two-dimensional LUT in a storage device (not shown). Here, the storage device may be built in the visual processing device 600, or may be connected to the outside via a wire or wirelessly. Each of the two-dimensional LUTs stored in the storage device is associated with the value of the target contrast C1 and the value of the actual contrast C2. That is, for each combination of the value of the target contrast C1 and the value of the actual contrast C2, the same calculation as that described in [Second Embodiment] (converted signal processing unit 602) (action of the converted signal processing unit 602) is performed. , is stored as a 2D LUT.

When the visual processing unit 623 obtains a value between the target contrast C1 and the actual contrast C2 , it reads a two-dimensional LUT associated with each obtained value among the two-dimensional LUTs stored in the storage device. Furthermore, the visual processing unit 623 performs visual processing using the read two-dimensional LUT. Specifically, the visual processing unit 623 obtains the value of the target contrast signal JS and the value of the unsharp signal US, reads out the value of the visual processing signal KS corresponding to the obtained values from the two-dimensional LUT, and outputs the visual Process signal KS.

(third embodiment)

As a third embodiment of the present invention, a visual processing device, a visual processing method, an application example of a visual processing program, and a system to which the visual processing device described in the first and second embodiments described above will be described.

A visual processing device is, for example, a device that is built into or connected to an image processing machine such as a computer, a television, a digital camera, a mobile phone, a PDA, a printer, a scanner, etc., and performs visual processing of an image, and is implemented as an LSI and other integrated circuits.

In more detail, each functional module of the above-mentioned embodiment may be integrated into a chip separately, or part or all of them may be integrated into a chip. In addition, although it is referred to as an LSI here, it is also called an IC, a system LSI, a very large-scale LSI, or an extremely large-scale LSI depending on the degree of integration.

Furthermore, the method of circuit integration is not limited to LSI, and it can also be realized by a dedicated circuit or a general-purpose processor. After the LSI is manufactured, a programmable FPGA (Field Pragrammable Gate Array, Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connection or setting of the circuit units inside the LSI can also be used.

Furthermore, if the development of semiconductor technology or another technology derived from it is used instead of the integrated circuit technology of LSI, it is of course also possible to use this technology to perform integration of functional modules. There can also be applications for biotechnology and the like.

The processing of each module of each visual processing device described in the above-mentioned first embodiment and second embodiment is performed by, for example, a central processing unit (CPU) included in the visual processing device. In addition, programs for performing respective processes are stored in storage devices such as hard disks and ROMs, and are read and executed in the ROM or RAM.

In addition, in the visual processing device 1 of FIG. 1 , the two-dimensional LUT 4 is stored in a storage device such as a hard disk or a ROM, and is referred to as necessary. Furthermore, the visual processing unit 3 receives the supplied profile data from the profile data registration device 8 connected directly to the visual processing device 1 or indirectly via a network, and registers it as a two-dimensional LUT 4 .

In addition, the visual processing device may be built in or connected to a device that processes moving images, and performs grayscale processing of each frame (each field) of the image.

Visual processing programs are built into devices that process images, such as computers, televisions, digital cameras, mobile phones, PDAs, printers, and scanners, or connected devices, and stored in storage devices such as hard disks and ROMs , a program for executing visual processing of an image is provided, for example, via a recording medium such as a CD-ROM or via a network.

(2)

The visual processing devices described in the above-mentioned first and second embodiments can also be represented by the configurations shown in FIGS. 40 to 41 .

(1)

(constitute)

FIG. 40 is a block diagram showing the functions of a visual processing device 910 that realizes the same functions as, for example, the visual processing device 625 shown in FIG. 7 .

In the visual processing device 910, the sensor 911 and the user input unit 912 have the same functions as the input device 527 (see FIG. 7). More specifically, the sensor 911 is a sensor that detects ambient light in the environment in which the visual processing device 910 is installed, or in the environment in which the output signal OS from the visual processing device 910 is displayed, and the detected value Output as parameter P1 representing ambient light. In addition, the user input unit 912 is a device that enables the user to set the intensity of the ambient light step by step or continuously (continuously) to, for example, "strong, medium, weak", and uses the set value as an indicator representing the ambient light. Parameter P1 output.

The output unit 914 has the same function as the profile data registration unit 526 (see FIG. 7 ). More specifically, the output unit 914 includes a plurality of profile data associated with the value of the parameter P1 representing ambient light. Here, the profile data refers to tabular data providing values of the output signal OS corresponding to the input signal IS and the signal obtained by spatially processing the input signal IS. Furthermore, the output unit 914 outputs the profile data corresponding to the obtained value of the parameter P1 representing the ambient light to the conversion unit 915 as the brightness adjustment parameter P2.

The conversion unit 915 has the same function as the spatial processing unit 2 and the visual processing unit 3 (see FIG. 7 ). The conversion unit 915 receives as input the luminance of the target pixel (pixel of interest) to be visually processed, the luminance of surrounding pixels located around the target pixel, and the luminance adjustment parameter P2, converts the luminance of the target pixel, and outputs a signal OS output.

More specifically, the conversion unit 915 performs spatial processing on the target pixel and surrounding pixels. Furthermore, the conversion unit 915 reads out the value of the output signal OS corresponding to the target pixel and the spatially processed result from the brightness adjustment parameter P2 in the form of a table, and outputs it as the output signal OS.

(Modification)

(1)

In the above configuration, the brightness adjustment parameter P2 is not limited to the above profile data. For example, the brightness adjustment parameter P2 may be coefficient matrix data used when calculating the value of the output signal OS based on the brightness of the target pixel and the brightness of surrounding pixels. Here, the coefficient matrix data is data storing coefficients of a function used when calculating the value of the output signal OS based on the luminance of the target pixel and the luminance of peripheral pixels.

(2)

The output unit 914 does not need to have profile data or coefficient matrix data corresponding to all values of the parameter P1 representing ambient light. In this case, according to the parameter P1 representing the obtained ambient light, appropriate description file data, etc. can be generated by internally or externally dividing the available profile data, etc.

(2)

(constitute)

FIG. 41 is a block diagram showing the configuration of a visual processing device 920 that realizes the same function as, for example, the visual processing device 600 shown in FIG. 24 .

In the visual processing device 920 , the output unit 921 further obtains an external parameter P3 in addition to the parameter P1 representing ambient light, and outputs a brightness adjustment parameter P2 based on the parameter P1 representing ambient light and the external parameter P3 .

Here, the parameter P1 indicating the ambient light is the same as described in (1) above.

Also, the external parameter P3 is a parameter indicating, for example, a visual effect required by the user of the visual output signal OS. More specifically, the value of the contrast or the like required by the user of the visible image (target contrast). Here, the external parameter P3 is set by the target contrast setting unit 604 (see FIG. 24 ). Alternatively, it is set using a default value stored in the output unit 921 in advance.

The output unit 921 calculates the actual contrast value based on the parameter P1 representing the ambient light according to the configuration shown in FIG. 33 or FIG. 34, and outputs it as the brightness adjustment parameter P2. In addition, the output unit 921 outputs the external parameter P3 (target contrast) as the brightness adjustment parameter P2. In addition, the output unit 921 stores a plurality of profile data stored in the two-dimensional LUT described in [Second Embodiment] (Modification) (vii), from the external parameter P3 and the parameter P1 indicating the ambient light. The description file data is selected in the calculated actual contrast, and the data in the table form is output as the brightness adjustment parameter P2.

The conversion unit 922 has the same functions as the target contrast conversion unit 601, the converted signal processing unit 602, and the actual contrast conversion unit 603 (refer to FIG. 24 above). More specifically, the conversion unit 922 receives the input signal IS (the brightness of the target pixel and the brightness of the surrounding pixels) and the brightness adjustment parameter P2, and outputs the output signal OS. For example, the input signal IS is converted into the target contrast signal JS using the target contrast obtained as the brightness adjustment parameter P2 (see FIG. 24 ). Furthermore, the target contrast signal JS is spatially processed to derive the unsharp signal US (see FIG. 24 ).

The conversion unit 922 includes the visual processing unit 623 as a modification described in [Second Embodiment] (Modification) (vii), and based on the profile data obtained as the brightness adjustment parameter P2, the target contrast signal JS, the blunt UL signal US to output a visually processed signal KS (see FIG. 24 ). Further, the visually processed signal KS is converted into an output signal OS using the actual contrast obtained as the brightness adjustment parameter P2.

In the visual processing device 920, based on the external parameter P3 and the parameter P1 representing the ambient light, the description file data used in the visual processing can be selected, and the influence caused by the ambient light can be corrected at the same time, even in an environment with ambient light It can also improve local contrast. A user-pleasing contrast ratio of the visual output signal OS may be approximated.

(Modification)

In addition, also in this structure, the same deformation|transformation as described in (1) can be performed.

In addition, the structure described in (1) and the structure described in (2) can be switched and used as needed. Switching can be performed using a switching signal from the outside. Also, it may be determined whether or not to use a certain configuration of the presence or absence of the external parameter P3.

In addition, although the actual contrast is described as being calculated by the output unit 921 , the value of the actual contrast may be directly input to the output unit 921 .

(3) In the configuration shown in FIG. 41 , a mechanism for keeping the input from the output unit 921 to the conversion unit 922 from changing rapidly may be further used.

The visual processing device 920' shown in FIG. 42 is different from the visual processing device 920 shown in FIG. 41 in that it includes an adjustment unit that slows down the temporal change of the parameter P1 representing ambient light. The adjustment unit 925 receives the parameter P1 indicating the ambient light as input, and outputs the adjusted output P4.

In this way, the output unit 921 can obtain the parameter P1 indicating the ambient light that does not change rapidly, and as a result, the temporal change of the output of the output unit 921 also becomes slow.

The adjustment unit 925 is realized by, for example, an IIR filter. Here, in the IIR filter, the value [P4] of the output P4 of the adjustment unit 925 is obtained by [P4]=k1×[P4]′+k2×[P1] operation. In addition, k1 and k2 are parameters that take positive values respectively, [P1] is the value of the parameter P1 representing ambient light, and [P4]' is the delayed output of the output P4 of the adjustment unit 925 (for example, the last output) value. In addition, the processing in the adjustment unit 925 may be performed using a configuration other than the IIR filter.

Furthermore, the adjustment unit 925 may be provided on the output side of the output unit 921 as in the visual processing device 920" shown in FIG.

Here, the operation of the adjustment unit 925 is the same as that described above. Specifically, the value [P4] of the output P4 of the adjustment unit 925 is calculated by [P4]=k3×[P4]'+k4×[P2]. In addition, in the formula, k3 and k4 are parameters that take positive values respectively, [P2] is the value of the brightness adjustment parameter P2, and [P4]' is the delayed output of the output P4 of the adjustment unit 925 (for example, the last output ) value. In addition, the processing in the adjustment unit 925 may be performed using a configuration other than the IIR filter.

With the configuration shown in FIG. 42, FIG. 43, etc., it is possible to control the temporal change of the parameter P1 representing the ambient light or the brightness adjustment parameter P2. Therefore, for example, the sensor 911 that detects ambient light responds to a person moving in front of the sensor, and even when the parameter changes greatly in a short period of time, rapid parameter fluctuations can be suppressed. As a result, flickering of the display screen can be suppressed.

(fourth embodiment)

In the fourth to sixth embodiments, the following problems corresponding to the conventional gradation processing described using FIGS. 104 to 107 can be solved.

(Problems with conventional grayscale processing)

In the histogram creating unit 302 (see FIG. 104 ), the gradation transformation curve Cm is created based on the pixel brightness histogram Hm in the image region Sm. In order to more properly create the gradation transformation curve Cm suitable for the image region Sm, it is not necessary to cover the dark part (shadow) to the brightness (highlight) of the image, and it is necessary to refer to more pixels. Therefore, each image region Sm cannot be made sufficiently small, that is, the number n of divisions of the original image cannot be made sufficiently large. The number n of divisions varies depending on the content of the image, but a number of divisions of 4 to 16 is used empirically.

In this way, since each image region Sm cannot be made sufficiently small, the following problems arise in the output signal OS after gradation processing. That is, since the gradation processing is performed using one gradation transformation curve Cm for each image region Sm, the joint at the boundary of each image region Sm may appear unnaturally conspicuous, or a blurred outline may be generated in the image region Sm. Furthermore, since the number of divisions is at most 4 to 16, and the image area Sm is relatively large at that time, there may be cases where there are extremely different images between image areas, and the variation in shading between image areas becomes large, making it difficult to prevent the occurrence of blurred outlines. For example, as shown in FIG. 105( b ) and FIG. 105( c ), extreme changes in shading are caused by the positional relationship between the image (such as an object in the image) and the image region Sm.

Hereinafter, in the fourth to sixth embodiments, a visual processing device that can solve the above-mentioned problems with respect to conventional gradation processing will be described using FIGS. 44 to 64 .

(Features of the visual processing device 101 as the fourth embodiment)

A visual processing device 101 as a fourth embodiment of the present invention will be described using FIGS. 44 to 48 . The visual processing device 101 is a device that is built in or connected to an image processing device such as a computer, a television, a digital camera, a mobile phone, or a PDA, and performs image grayscale processing. The visual processing device 101 is characterized by performing gradation processing on each of the finely divided image regions compared to conventional ones.

(constitute)

FIG. 44 is a block diagram illustrating the configuration of the visual processing device 101 . The visual processing device 101 includes: an image segmentation unit 102, which divides the original image input as the input signal IS into a plurality of image regions Pm (1≤m≤n: n is the number of divisions of the original image); The derivation unit 110 derives the gradation transformation curve Cm for each image region Pm; and the gradation processing unit 105 loads the gradation transformation curve Cm and outputs the output signal OS obtained by performing gradation processing on each image region Pm. The gradation transformation curve derivation unit 110 includes: a histogram creation unit 103 that creates a luminance histogram Hm of pixels in a wide-area image region Em composed of each image region Pm and image regions around the image region Pm; The curve creation unit 104 creates a grayscale transformation curve Cm corresponding to each image region Pm based on the created brightness histogram Hm.

(effect)

The operation of each part will be described using FIGS. 45 to 47 . The image dividing unit 102 divides the original image input as the input signal IS into a plurality (n) of image regions Pm (see FIG. 45 ). Here, the number of divisions of the original image is larger than that of the conventional visual processing device 300 shown in FIG. 104 (for example, 4 to 16 divisions). 4800 divisions divided into 60 divisions etc.

The histogram creation unit 103 creates a luminance histogram Hm of the wide-area image region Em for each image region Pm. Here, the so-called wide-area image area Em refers to a collection of multiple image areas including each image area Pm, for example, 25 image areas of 5 vertical blocks and 5 horizontal blocks centered on the image area Pm. gather. In addition, in some cases, the wide-area image area Em of 5 blocks vertically and 5 blocks horizontally around the image area Pm cannot be obtained from the position of the image area Pm. For example, for the image area PI located around the original image, it is impossible to obtain the wide-area image area EI with 5 blocks in the vertical direction and 5 blocks in the horizontal direction around the image area PI. In this case, an area in which five blocks in the vertical direction and five blocks in the horizontal direction centering on the image area PI overlaps with the original image is used as the wide-area image area EI. The luminance histogram Hm created by the histogram creation unit 103 shows the distribution state of the luminance values of all the pixels in the wide-area image region Em. That is, in the brightness histogram Hm shown in FIGS. 46( a ) to ( c ), the horizontal axis represents the brightness level of the input signal IS, and the vertical axis represents the number of pixels.

The gradation curve creating unit 104 accumulates the "number of pixels" of the luminance histogram Hm of the wide-area image region Em in order of luminance, and uses the accumulated curve as a gradation transformation curve Cm of the image region Pm (see FIG. 47 ). In the gradation transformation curve Cm shown in FIG. 47 , the horizontal axis represents the luminance values of the pixels in the image region Pm in the input signal IS, and the vertical axis represents the luminance values of the pixels in the image region Pm in the output signal OS. The gradation processing unit 105 loads the gradation transformation curve Cm, and converts the luminance values of the pixels in the image region Pm in the input signal IS based on the gradation transformation curve Cm.

(visual processing method and visual processing system)

FIG. 48 is a flowchart illustrating a visual processing method in the visual processing device 101. The visual processing method shown in FIG. 48 is implemented by hardware in the visual processing device 101, and is a method of performing gradation processing on the input signal IS (see FIG. 1). In the visual processing method shown in FIG. 48, the input signal IS is processed in image units (steps S110 to S116). The original image input as the input signal IS is divided into a plurality of image regions Pm (1≤m≤n: n is the number of divisions of the original image) (step S111), and grayscale processing is performed for each image region Pm (step S112 ~S115).

A brightness histogram Hm of pixels in the wide-area image region Em composed of each image region Pm and image regions around the image region Pm is created (step S112 ). Furthermore, based on the luminance histogram Hm, a gradation transformation curve Cm for each image region Pm is created (step S113). Here, descriptions of the luminance histogram Hm and the gradation transformation curve Cm are omitted (refer to the above (operation) column). Using the created gradation transformation curve Cm, gradation processing is performed on pixels in the image region Pm (step S114). Furthermore, it is determined whether or not the processing for all the image regions Pm has been completed (step S115 ), and the processing of steps S112 to S115 is repeated up to the number of divisions of the original image until it is determined that the processing has completed. Through the above, the processing of the image unit is ended (step S116).

In addition, each step of the visual processing method shown in FIG. 48 can be realized by a computer or the like as a visual processing program.

(Effect)

(1)

The gradation transformation curve Cm is created for each image region Pm. Therefore, compared with the case where the same gradation transformation is performed on the entire original image, more appropriate gradation processing can be performed.

(2)

The gradation transformation curve Cm created for each image area Pm is created based on the luminance histogram Hm of the wide-area image area Em. Therefore, even if the size of each image area Pm becomes small, sufficient sampling of luminance values can be performed. And, as a result, an appropriate gradation transformation curve Cm can be created even for a small image region Pm.

(3)

The wide-area image regions corresponding to adjacent image regions have overlap. Therefore, the gradation transformation curves corresponding to adjacent image regions tend to show similarities. Therefore, the effect of spatial processing can be added to the gradation processing of each image area, and it is possible to prevent the joint of the border between adjacent image areas from being conspicuous and unnatural.

(4)

The size of each image region Pm is smaller than conventional ones. Therefore, the generation of blurred outlines in the image area Pm can be suppressed.

(Modification)

The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist thereof.

(1)

In the above-described embodiment, 4800 divisions were used as an example of the number of divisions of the original image, but the effects of the present invention are not limited to these cases, and the same effect can be obtained with other divisions. In addition, there is a trade-off relationship between the amount of grayscale processing and the visual effect, and the number of divisions. That is, as the number of divisions increases, the processing amount of gradation processing increases, and better visual effects (for example, suppression of blurred outlines, etc.) can be obtained.

(2)

In the above embodiment, the number of image regions constituting the wide-area image region is 25 as an example, but the effect of the present invention is not limited to these cases, and the same effect can be obtained with other numbers.

(fifth embodiment)

(Features of the visual processing device 111 as the fifth embodiment)

A visual processing device 111 as a fifth embodiment of the present invention will be described using FIGS. 49 to 61 . The visual processing device 111 is a device that is built in or connected to an image processing device such as a computer, a television, a digital camera, a mobile phone, or a PDA, and performs image grayscale processing. The visual processing device 111 is characterized in that it switches and uses a plurality of gradation transformation curves stored as an LUT in advance.

(constitute)

FIG. 49 is a block diagram illustrating the configuration of the visual processing device 111 . The visual processing device 111 includes an image division unit 112 , a selection signal derivation unit 113 , and a gradation processing unit 120 . The image dividing unit 112 receives an input signal IS, and outputs a plurality of divided image regions Pm (1≤m≤n, where n is the number of divisions of the original image) of the original image input as the input signal IS. The selection signal derivation unit 113 outputs a selection signal Sm for selecting a gradation transformation curve Cm used for gradation processing of each image region Pm. The gradation processing unit 120 includes a gradation processing execution unit 114 and a gradation correction unit 115 . The gradation processing execution unit 114 is provided with a plurality of gradation transformation curve candidates G1 to Gp (p is the number of candidates) as two-dimensional LUTs, receives the input signal IS and the selection signal Sm as input, and converts the gradation curves for pixels in each image area Pm The grayscale processed signal CS after the grayscale processing is output. The gradation correction unit 115 receives the gradation processed signal CS as input, and outputs an output signal OS obtained by correcting the gradation of the gradation processed signal CS.

(About gradation conversion curve candidates)

The gradation transformation curve candidates G1 to Gp will be described using FIG. 50 . The gradation transformation curve candidates G1 to Gp are curves that provide the relationship between the luminance value of the pixel of the input signal IS and the shading value of the pixel of the gradation processed signal CS. In FIG. 50 , the horizontal axis represents the brightness value of the pixel in the input signal IS, and the vertical axis represents the brightness value of the pixel in the gradation processed signal CS. The gradation transformation curve candidates G1 to Gp have a relationship of monotonically decreasing subscripts, and satisfy the relationship of G1≥G2≥...≥Gp with respect to the lightness values of all pixels of the input signal IS. For example, the gradation transformation curves G1～G are [power functions] when the lightness value of the pixel representing the input signal IS is a variable, and when expressed as Gm=x^(δm) (1≤m≤p, x is Variable, δm is a constant), satisfying the relationship of δ1≤δ2≤...≤δp. Here, the luminance value of the input signal IS is within the range of values [0.0 to 1.0].

In addition, the relationship between the above-mentioned gradation transformation curve candidates G1 to Gp, for the gradation transformation curve candidates with large subscripts, when the input signal IS is small, or for the gradation transformation curve candidates with small subscripts, In the case where the input signal IS is large, the relationship between the above-mentioned gradation transformation curve candidates G1 to Gp may not hold. In such a case, there is almost no effect, because the image quality is less affected.

The gradation processing execution unit 114 includes gradation transformation curve candidates G1 to Gp as two-dimensional LUTs. That is, the two-dimensional LUT is a look-up table (LUT) that provides brightness values of pixels in the gradation processed signal CS with respect to brightness values of pixels in the input signal IS and selection signals Sm for selecting brightness conversion curve candidates G1 to Gp. FIG. 51 shows an example of this two-dimensional LUT. The two-dimensional LUT 141 shown in FIG. 51 is a matrix of 64 rows and 64 columns, and each gradation transformation curve candidate G1 to G64 is arranged in the row direction (horizontal direction). In the column direction (longitudinal direction) of the matrix, the value of the upper 6 bits of the pixel value of the input signal IS expressed by, for example, 10 bits, that is, the pixel value of the grayscale processing signal CS corresponding to the value of the input signal IS divided into 64 steps is arranged . The pixel value of the gradation processed signal CS has, for example, a value within a range of values [0.0 to 1.0] when the gradation transformation curve candidates G1 to Gp are [power functions].

(effect)

The operation of each part is explained. The image dividing unit 112 operates almost in the same manner as the image dividing unit 102 in FIG. 44, and divides the original image input as the input signal IS into a plurality (n) of image areas Pm (see FIG. 45). Here, the number of divisions of the original image is greater than that of the conventional visual processing device 300 shown in FIG. The 4800 splits etc.

The selection signal derivation unit 113 selects the gradation transformation curve Cm to be applied to each image region Pm from the gradation transformation curve candidates G1 to Gp. Specifically, the selection signal derivation unit 113 calculates the average luminance value of the wide-area image region Em of the image region Pm, and selects any one of the gradation transformation curve candidates G1 to Gp based on the calculated average luminance value. That is, the gradation transformation curve candidates G1 to Gp are associated with the average luminance value of the wide-area image region Em, and the gradation transformation curve candidates G1 to Gp with larger subscripts are selected as the average luminance value is larger.

Here, the wide-area image region Em is the same as that described in (fourth embodiment) using FIG. 45 . That is, the wide-area image area Em is a set of multiple image areas including each image area Pm, for example, a set of 25 image areas with 5 blocks vertically and 5 blocks horizontally centering on the image area Pm. In addition, in some cases, the wide-area image area Em of 5 blocks vertically and 5 blocks horizontally around the image area Pm cannot be obtained from the position of the image area Pm. For example, for the image area P1 located around the original image, it is impossible to obtain a wide-area image area E1 with 5 vertical blocks and 5 horizontal blocks around the image area P1. In this case, an area in which five blocks in the vertical direction and five blocks in the horizontal direction centered on the image area P1 overlaps with the original image is used as the wide-area image area E1.

The selection result by the selection signal derivation unit 113 is output as a selection signal Sm indicating any one of the gradation transformation curve candidates G1 to Gp. More specifically, the selection signal Sm is output as the value of the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp.

The gradation processing execution unit 114 receives the luminance values of the pixels of the image region Pm included in the input signal IS and the selection signal Sm, and outputs the luminance values of the gradation processed signal CS using, for example, the two-dimensional LUT 141 shown in FIG. 51 .

The gradation correcting unit 115 performs luminance values of pixels in the image region Pm included in the gradation processed signal CS based on the position of the pixel and the gradation conversion region selected for the image region Pm and image regions around the image region Pm. Correction. For example, the gradation transformation curve Cm applied to the pixels included in the pixel region Pm and the gradation transformation curve selected for the image region around the image region Pm are corrected by the internal division ratio of the pixel position to obtain The luminance value of the corrected pixel.

The operation of the gradation correction unit 115 will be described in more detail using FIG. 52 . Figure 52 shows the gradation transformation curves Co, Cp, Cq, Cr selected gradation transformation of the image areas Po, Pp, Pq, Pr (o, p, q, r are positive integers equal to or less than the number of divisions n (refer to Fig. 45)) Curve candidates Gs, Gt, Gu, and Gv (s, t, u, and v are positive integers equal to or less than the number p of candidates for gradation transformation curves).

Here, the position of the pixel x (become the lightness value [x]) of the image region Po to be the target of gradation correction is internally divided as follows: the center of the image region Po and the center of the image region Pp are divided into [i: 1-I], and the center of the image area Po and the center of the image area Pq are divided into [j:1-j[. In this case, the lightness value [x'] of pixel x after gray scale correction is calculated as [x']={(1-j)·(1-i)·[Gs[+(1-j) ·(i)·[Gt]+(j)·(1−i)·[Gu]+(j)·(i)·[Gv]]·{(x)/[Gs]}. Also, let [Gs], [Gt], [Gu], and [Gv] be lightness values when the gradation transformation curve candidates Gs, Gt, Gu, and Gv are applied to the lightness value [x].

(visual processing method and visual processing program)

FIG. 53 is a flowchart illustrating a visual processing method in the visual processing device 111. The visual processing method shown in FIG. 53 is realized by hardware in the visual processing device 111 and performs gradation processing of the input signal IS (see FIG. 49 ). In the visual processing method shown in FIG. 53, the input signal IS is processed in image units (steps S120 to S126). The original image input as the input signal IS is divided into a plurality of image regions Pm (1≤m≤n: n is the number of divisions of the original image) (step S121), and grayscale processing is performed for each image region Pm (step S121). S122～S124).

In the processing for each image region Pm, the gradation transformation curve Cm to be applied to each image region Pm is selected from the gradation transformation curve candidates G1 to Gp (step S122 ). Specifically, the average brightness value of the wide-area image region Em of the image region Pm is calculated, and any one of the gradation transformation curve candidates G1 to Gp is selected based on the calculated average brightness value. The grayscale transformation curve candidates G1 to Gp are associated with the average lightness value of the wide-area image region Em, and the larger the average lightness value, the larger the grayscale transformation curve candidates G1 to Gp are selected. Here, description of the wide-area image region Em is omitted (refer to the above-mentioned (function) column).

For the lightness values of pixels in the image region Pm included in the input signal IS and the selection signal Sm of the gradation transformation curve candidate selected in step S122 among the gradation transformation curve candidates G1 to Gp, for example, 2 values shown in FIG. 51 are used. dimension LUT to output the lightness value of the gray scale processed signal CS (step S123). Furthermore, it is determined whether or not the processing for all the image regions Pm has ended (step S124 ), and the processing of steps S122 to S124 is repeated up to the number of divisions of the original image until it is determined that the dimension processing has ended. Through the above, the processing in units of image regions ends.

The lightness values of pixels in the image region Pm included in the gradation processed signal CS are corrected based on the pixel position and the gradation transformation curve selected for the image region Pm and image regions around the image region Pm (step S125 ). For example, the gradation transformation curve Cm applied to the pixels included in the image region Pm and the gradation transformation curve selected for the image region around the image region Pm are corrected at the internal division ratio of the pixel position to obtain the correction The lightness value of the following pixel. The description of the details of the correction is omitted (refer to the above (action) column, FIG. 52 ).

Through the above, the processing of the image unit ends (step S126).

In addition, each step of the visual processing method shown in FIG. 53 can be realized by a computer or the like as a visual processing program.

(Effect)

According to the present invention, basically the same effect as (effect) of the above-mentioned (fourth embodiment) can be obtained. Hereinafter, effects specific to the fifth embodiment will be described.

(1)

The gradation transformation curve Cm selected for each image region Pm is created based on the average lightness value of the wide-area image region Em. Therefore, even if the size of the image region Pm is small, sufficient sampling of lightness values can be performed. And, as a result, even for a small image area Pm, an appropriate grayscale transformation curve Cm can be selected and applied.

(2)

The gradation processing execution unit 114 has a pre-created 2D LUT. Therefore, the processing load required for gradation processing, more specifically, the processing load required for creating the gradation transformation curve Cm can be reduced. As a result, the processing required for the gradation processing of the image region Pm can be accelerated.

(3)

The gradation processing execution unit 114 executes gradation processing using a two-dimensional LUT. The two-dimensional LUT is read from a storage device such as a hard disk or ROM included in the visual processing device 11, and used for gradation processing. By changing the contents of the read two-dimensional LUT, various gradation processing can be realized without changing the hardware configuration. That is, gradation processing more suitable for the characteristics of the original image can be realized.

(4)

The gradation correction unit 115 corrects the gradation of the pixels in the image region Pm subjected to gradation processing using one gradation transformation curve Cm. Therefore, it is possible to obtain an output signal OS that has been subjected to more appropriate gradation processing. For example, the generation of blurred outlines can be suppressed. In addition, in the output signal OS, it is possible to further prevent the joints at the boundaries of the respective image regions Pm from being conspicuous and unnatural.

(Modification)

The present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the gist.

(1)

In the above embodiment, 4800 divisions are used as an example of the number of divisions of the original image, but the effect of the present invention is not limited to these cases, and the same effect can be obtained with other divisions. In addition, there is a trade-off relationship between the amount of grayscale processing and the visual effect, and the number of divisions. That is, as the number of divisions increases, the processing amount of gradation processing increases, and a better visual effect (for example, an image in which blurred outlines are suppressed, etc.) can be obtained.

(2)

In the above embodiment, the number of image regions constituting the wide-area image region is 25 as an example, but the effect of the present invention is not limited to these cases, and the same effect can be obtained with other numbers.

(3)

In the above-described embodiment, the two-dimensional LUT 141 composed of a matrix of 64 columns and 64 rows is used as an example of a two-dimensional LUT. Here, the effects of the present invention are not limited to the two-dimensional LUT of this size. For example, it may further be a matrix in which many gradation transformation curve candidates are arranged in the row direction. In addition, the pixel values of the grayscale processed signal CS corresponding to the values divided by the step of further reducing the pixel value of the input signal IS may be arranged in the column direction of the matrix. Specifically, the pixel values of the gradation processed signal CS may be arranged corresponding to the respective pixel values of the input signal IS represented by, for example, 10 bits.

If the size of the 2D LUT is larger, more appropriate gradation processing can be performed, and if it is smaller, the memory for storing the 2D LUT can be reduced.

(4)

In the above-mentioned embodiment, it has been described that in the column direction of the matrix, for example, the value of the upper 6 bits of the pixel value of the input signal IS represented by 10 bits, that is, the grayscale processing signal corresponding to the value of the input signal IS divided into 64 steps The pixel value of CS. Here, the gradation processed signal CS can be output as a matrix component linearly interpolated from the lower 4-bit values of the pixel values of the input signal IS by the gradation processing execution unit 114 . That is, in the column direction of the matrix, matrix components corresponding to the upper 6-bit values of the pixel values of the input signal IS represented by, for example, 10 bits are arranged, and the input signal IS is expressed using the lower 4-bit values of the pixel values of the input signal IS The matrix component corresponding to the high-order 6-bit value of the pixel value of the input signal IS, and the matrix component corresponding to the value after adding [1] to the low-order 6-bit value of the pixel value of the input signal IS (for example, in the first row in Figure 51 Components), perform linear interpolation, and output it as a gray-scale processed signal OS.

In this way, more appropriate gradation processing can be performed even if the size of the two-dimensional LUT 141 (see FIG. 51 ) is small.

(5)

In the above-mentioned embodiments, it has been described that the gradation transformation curve Cm suitable for the image region Pm is selected based on the average lightness value of the wide-area image region Em. Here, the selection method of the gradation transformation curve Cm is not limited to the above method. For example, the grayscale transformation curve Cm suitable for the image region Pm may also be selected based on the maximum brightness value or the minimum brightness value of the wide-area image region Em. In addition, when selecting the gradation transformation curve Cm, the value [Sm] of the selection signal Sm may be the average brightness value, the maximum brightness value, or the minimum brightness value of the wide-area image region Em. In this case, each value obtained by dividing the value taken by the selection signal Sm into 64 steps is associated with the gradation transformation curve candidates G1 to G4.

Alternatively, for example, the gradation transformation curve Cm suitable for the image region Pm may be selected as follows. That is, the average brightness value for each image region Pm is obtained, and the temporary selection signal Sm' for each image region Pm is obtained based on each average brightness value. Here, the temporary selection signal Sm' has the subscript numbers of the gradation transformation curve candidates G1 to Gp as values. Furthermore, with regard to each image area included in the wide-area image area Em, the values of the temporary selection signal Sm' are averaged to obtain the value [Sm] of the selection signal Sm of the image area Pm, and the selection order and the gradation transformation curve candidates G1- The nearest integer to the value [Sm] in Gp is used as a subscript candidate, and is used as a gradation transformation curve Cm.

(6)

In the above-mentioned embodiments, it has been described that the gradation transformation curve Cm suitable for the image region Pm is selected based on the average lightness value of the wide-area image region Em. Here, instead of simply averaging the wide-area image region Em, the gradation transformation curve Cm suitable for the image region Pm may be selected based on a weighted average (weighted average). For example, as shown in FIG. 54, the average lightness value of each image area constituting the wide-area image area Em is obtained, and the image areas Ps1, Ps2, . . . , make the weight smaller, or exclude it, and find the average brightness value of the wide-area image region Em.

In this way, even when the wide-area image region Em contains a region with abnormal brightness (for example, when the wide-area image region Em contains two types of boundaries of objects with different brightness values), the brightness value of the abnormal region is different from the gray value applied to the image region Pm. The influence brought by the selection of the degree transformation curve Cm is also less, and suitable grayscale processing can be performed.

(7)

In the above-described embodiments, the presence of the gradation correction unit 115 is optional. That is, even in the case where the gradation processed signal CS is output, compared with the conventional visual processing device 300 (see FIG. 104 ), the same (effect) as described in the (fourth embodiment) can be obtained. Effects and the same effects as described in (Effect) (1) and (2) of (Fifth Embodiment).

(8)

In the above-mentioned embodiment, it was explained that the gradation transformation curve candidates G1 to Gp have a monotonously decreasing relationship with respect to subscripts, and satisfy the relationship of G1≥G2≥...≥Gp with respect to the lightness values of all pixels of the input signal. Here, the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT may not satisfy the relationship of G1≥G2≥...≥Gp with respect to some of the brightness values of the pixels of the input signal IS. That is, any of the gradation transformation curve candidates G1 to Gp may intersect each other.

For example, there are small and bright parts in a dark night scene (neon lights in a night scene, etc.), although the value of the input signal IS is large, but when the average lightness value of the wide-area image area Em is small, after performing grayscale processing The value of the image signal has less influence on the image quality. In this case, the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT may not satisfy the relationship of G1≥G2≥...≥GP with respect to some of the brightness values of the pixels of the input signal IS. In other words, the value stored in the 2D LUT may be arbitrary in a portion where the value after gradation processing has little influence on the image quality.

In addition, when the value stored in the two-dimensional LUT is an arbitrary value, it is preferable that the value stored in the input signal IS and the selection signal Sm of the same value also maintains the value of the input signal IS and the selection signal Sm and monotonically increases, or monotonically decreasing relationship.

Furthermore, in the above-mentioned embodiment, it has been described that the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT are [power functions]. Here, the gradation transformation curve candidates G1 to Gp may not be strictly formulated as [power functions]. In addition, functions in shapes such as an S-shape and an inverted S-shape may be used.

(9)

The visual processing device 111 may further include a profile data creation unit that creates profile data of values stored as two-dimensional LUTs. Specifically, the profile data creation unit is composed of the image segmentation unit 102 and the gradation transformation curve derivation unit 110 in the visual processing device 101 (see FIG. 44 ), and a set of created plural gradation transformation curves is used as Profile data, stored in a 2D LUT.

Furthermore, each gradation transformation curve stored in the two-dimensional LUT may be associated with the spatially processed input signal IS. In this case, in the visual processing device 111, the image division unit 112 and the selection signal derivation unit 113 may be replaced by a spatial processing unit that performs spatial processing on the input signal IS.

(10)

In the above-described embodiment, the brightness value of the pixel of the input signal IS may not be a value within the range of [0.0 to 1.0]. When the input signal IS is input as a value within another range, the value within this range may be normalized to a value [0.0 to 1.0] and used. In addition, even without performing normalization, it is possible to appropriately change the value to be processed in the above processing.

(11)

Each of the gradation transformation curve candidates G1 to Gp may be a gradation transformation in which the gradation processing is performed on the input signal IS having a dynamic range wider than the general dynamic range, and the gradation processing signal CS of the general dynamic range is output. curve.

In recent years, by using a CCD with a good S/N that shields the amount of light, a sensor that uses an electronic shutter that is opened twice, or a sensor that uses low-sensitivity or high-sensitivity pixels, it is possible to handle a wider dynamic range than the general one. The development of machines with a wider dynamic range of 1 to 3 orders of magnitude is progressing.

Accordingly, even when the input signal IS has a wider dynamic range than a general dynamic range (for example, a signal within a value range (1.0 to 1.0)), appropriate grayscale processing is pursued.

Here, as shown in FIG. 55 , a gradation transformation curve that outputs a gradation processed signal CS of a value [0.0 to 1.0] is used also for an input signal IS exceeding the range of values [0.0 to 1.0].

In this way, suitable grayscale processing can also be performed on the input signal IS with a wide dynamic range, and the grayscale processed signal CS with a general dynamic range can be output

In addition, in the present embodiment, it is described that "the pixel value of the gradation processing signal CS has a value within the range of, for example, [0.0 to 1.0] when the gradation transformation curve candidates G1 to Gp are [power functions]". Here, the pixel value of the gradation processed signal CS is not limited to this range. For example, dynamic range compression may be performed on the gradation transformation curve candidates G1 to Gp for the input signal IS with a value of [0.0 to 1.0].

(12)

In the above-mentioned embodiment, "the gradation processing execution unit 114 has the gradation transformation curve candidates G1 to Gp as two-dimensional LUTs" has been described. Here, the gradation processing execution unit 114 may have a one-dimensional LUT that stores the relationship between the curve parameters for specifying the gradation transformation curve candidates G1 to Gp and the selection signal Sm.

(constitute)

FIG. 56 is a block diagram showing the configuration of the gradation processing execution unit 114 as a modified example of the gradation processing execution unit 114 . The gradation processing execution unit 144 receives the input signal IS and the selection signal Sm as input, and outputs the gradation processed signal CS which is the input signal IS subjected to the gradation processing. The gradation processing execution unit 144 includes a curve parameter output unit 145 and a calculation unit 148 .

The curve parameter output unit 145 is composed of a first LUT 146 and a second LUT 147 . The first LUT 146 and the second LUT 147 receive the selection signal Sm as input, and output the curve parameters P1 and P2 of the gradation transformation curve candidate Gm specified by the selection signal Sm, respectively.

The calculation unit 148 receives the curve parameters P1 and P2 and the input signal IS as input, and outputs the gradation processed signal CS.

(About 1D LUT)

The first LUT 146 and the second LUT 147 are one-dimensional LUTs that store the values of the curve parameters P1 and P2 corresponding to the respective selection signals Sm. Before describing in detail about first LUT 146 and second LUT 147 , the contents of curve parameters P1 and P2 will be described.

The relationship between the curve parameters P1 and P2 and the gradation transformation curve candidates G1 to Gp will be described using FIG. 57 . FIG. 57 shows gradation transformation curve candidates G1 to Gp. Here, the gradation transformation curve candidates G1 to Gp have a monotonically decreasing relationship with respect to subscripts, and satisfy the relationship of G1≥G2≥...≥Gp with respect to the lightness values of all pixels of the input signal IS. Also, when the input signal IS is small for a gradation transformation curve candidate with a large subscript, or when the input signal IS is large for a gradation transformation curve candidate with a small subscript, the above gradation transformation curve candidates The relationship of G1 to Gp may not be established.

The curve parameters P1 and P2 are output as values of the gradation processed signal CS of predetermined values for the input signal IS. That is, when the grayscale transformation curve candidate Gm is specified by the selection signal Sm, the value of the curve parameter P1 is output as the value [R1m] of the grayscale transformation curve candidate Gm corresponding to the predetermined value [X1] of the input signal IS. , the value of the curve parameter P2 is output as the value [R2m] of the gradation transformation curve candidate Gm corresponding to the predetermined value [X2] of the input signal IS. Here, the value [X2] is a value larger than the value [X1].

Next, first LUT 146 and second LUT 147 will be described.

The first LUT 146 and the second LUT 147 respectively store the values of the curve parameters P1 and P2 corresponding to the selection signal Sm. More specifically, for each selection signal Sm provided as a 6-bit signal, the values of the curve parameters P1 and P2 are provided in 6 bits, respectively. Here, the number of bits secured for the selection signal Sm or the curve parameters P1 and P2 is not limited thereto.

The relationship between the curve parameters P1 and P2 and the selection signal Sm will be described using FIG. 58 . Fig. 58 shows the change of the values of the curve parameters P1 and P2 corresponding to the selection signal Sm. The first LUT 146 and the second LUT 147 store the values of the curve parameters P1 and P2 corresponding to the respective selection signals Sm. For example, the value [R1m] is stored as the value of the curve parameter P1 corresponding to the selection signal Sm, and the value [R2m] is stored as the value of the curve parameter P2.

The first LUT 146 and the second LUT 147 above output the curve parameters P1 and P2 with respect to the input selection signal Sm.

(About computing unit 148)

The computing unit 148 derives the gradation processed signal CS corresponding to the input signal IS based on the obtained curve parameters P1 and P2 (value [R1m] and value [R2m]). The specific procedure is described below. Here, the value of the input signal IS is provided by a range of values [0.0 to 1.0]. In addition, the gradation transformation curve candidates G1 to Gp are gradation conversions of the input signal IS given in the value range of [0.0 to 1.0] into the value range of [0.0 to 1.0]. In addition, the present invention is also applicable when the input signal IS is not limited to this range.

First, the computing unit 148 compares the value of the input signal IS with predetermined values [X1] and [X2].

When the value of the input signal IS (value [x]) is greater than or equal to [0.0] and less than [X1], the value [ x] corresponds to the value of the gray-scale processed signal CS (for value [y]). More specifically, the value [y] is obtained by the following formula [y]=([x]/[X1])×[R1m].

When the value of the input signal IS is [X]1 or more and less than [X2], on the straight line connecting the coordinates ([X1], [R1m]) and the coordinates ([X2], [R2m]) in Fig. 57, Finds the value [y] for the value [x]. More specifically, the value [y], by the following formula [y]=[R1m]+{([R2m]-[R1m])/([X2]-[X1])}×([x]-[X1 ]) Find out.

When the value of the input signal IS is between [X2] and [1.0], find Value[x] corresponds to value[y]. More specifically, the value [y], by the following formula [y]=[R2m]+{[1.0]-[R2m])/([1.0]-[X2])}×([x]-[X2] ) to find out.

Through the above calculations, the computing unit 148 derives the gray scale processed signal CS corresponding to the input signal IS.

(grayscale processing method, program)

The above processing is executed by a computer or the like as a gradation processing program. The gradation processing program is a program for causing a computer to execute the gradation processing method described below.

The gradation processing method is a method of obtaining the input signal IS and the selection signal Sm, and outputting the gradation processing signal Cs, which is characterized in that the gradation processing is performed on the input signal IS using a one-dimensional LUT.

First, when the selection signal Sm is obtained, curves P1 and P2 are output from the first LT 146 and the second LUT 147 . Detailed descriptions of the first LUT 146 , the second LUT 147 , and the curve parameters P1 and P2 are omitted.

Furthermore, based on the curve parameters P1 and P2, the gradation processing of the input signal IS is performed. The details of the gradation processing are described in the description of the calculation unit 148, so the description is omitted.

Through the above grayscale processing method, the grayscale processed signal CS corresponding to the input signal IS is derived.

(Effect)

The gradation processing execution unit 144 which is a modified example of the gradation processing execution unit 114 includes not a two-dimensional LUT but two one-dimensional LUTs. Therefore, the storage capacity for storing the lookup table can be reduced.

(Modification)

(1)

In the above modification, it was explained that "the values of the curve parameters P1 and P2 are the values of the gradation transformation curve candidate Gm corresponding to a predetermined value of the input signal IS". Here, the curve parameters P2 and P2 may be other curve parameters of the gradation transformation curve candidate Gm. Below, it demonstrates concretely.

(1-1)

The curve parameter may be the slope of the grayscale transformation curve candidate Gm. This will be specifically described using FIG. 57 . When the grayscale transformation curve candidate Gm is designated by the selection signal Sm, the value of the curve parameter P1 is the slope value [K1m] of the grayscale transformation curve candidate Gm in the predetermined range [0.0 to X1] of the input signal IS, and the curve parameter The value of P2 is the value [K2m] of the slope of the gradation transformation curve candidate Gm in the predetermined range [X1-X2] of the input signal IS.

The relationship between the curve parameters P1 and P2 and the selection signal Sm will be described using FIG. 59 . FIG. 59 shows changes in the values of the curve parameters P1 and P2 corresponding to the selection signal Sm. The first LUT 146 and the second LUT 147 store the values of the curve parameters P1 and P2 corresponding to the respective selection signals Sm. For example, the value [K1m] is stored as the value of the curve parameter P1 corresponding to the selection signal Sm, and the value [K2m] is stored as the value of the curve parameter P2.

The above-mentioned first LUT146 and second LUT147 output the curve parameters P1 and P2 with respect to the input selection signal Sm.

In the computing unit 148 , based on the obtained curve parameters P1 and P2 , the gray scale processed signal CS corresponding to the input signal IS is derived. The specific procedure is described below.

First, the computing unit 148 compares the value of the input signal IS with predetermined values [X1] and [X2].

When the value of the input signal IS (value [x]) is more than [0.0] and less than [X1], the connection origin and coordinates ([X1], [K1m]×[X1]) in Fig. 57 (written below On the straight line [Y1]), the value (value [y]) of the gradation processed signal CS corresponding to the value [x] is obtained. More specifically, the value [y] is obtained by the following formula [y]=[K1m]×[x].

When the value of the input signal IS is more than [X1] and less than [X2], the connection coordinates ([X1], [Y1]) and coordinates ([X2], [K1m]×[X1]+[ On the straight line K2m]×([X2]-[X1]) hereinafter referred to as [Y2]), the value [y] corresponding to the value [x] is obtained. More specifically, the value [y] is obtained by the following formula [y]=[Y1]+[K2m]×([x]-[X1]).

When the value of the input signal IS is between [X2] and [1.0], find the value [x] on the straight line connecting the coordinates ([X2], [Y2]) and the coordinates (1.0, 1.0) in Fig. 57 The corresponding value[y]. More specifically, the value [y], by the following formula [y]=[Y2]+{([1.0]-[Y2])/([1.0]-[X2])}×([x]-[X2 ]) Find out.

Through the above calculations, the calculation unit 148 derives the gradation processed signal CS corresponding to the input signal IS.

(1-2)

The curve parameters may be coordinates on the grayscale transformation curve candidate Gm. This will be specifically described using FIG. 60 . When the grayscale transformation curve candidate Gm is designated by the selection signal Sm, the value of the curve parameter P1 is the component value [Mm] of the coordinate side on the grayscale transformation curve candidate Gm, and the value of the curve parameter P2 is the grayscale transformation curve candidate The other component value [Nm] of the coordinate on Gn. Furthermore, the gradation transformation curve candidates G1 to Gp are all curves passing through the coordinates (X1, Y1).

The relationship between the curve parameters P1 and P2 and the selection signal Sm will be described using FIG. 61 . FIG. 61 shows changes in the values of the curve parameters P1 and P2 corresponding to the selection signal Sm. The first LUT 146 and the second LUT 147 store the values of the curve parameters P1 and P2 corresponding to the respective selection signals Sm. For example, the value [Mm] is stored as the value of the curve parameter P1 corresponding to the selection signal Sm, and the value [Nm] is stored as the value of the curve parameter P2.

The above-mentioned first LUT146 and second LUT147 output the curve parameters P1 and P2 with respect to the input selection signal Sm.

In the computing unit 148 , the gradation processed signal CS is derived from the input signal IS by the same processing as the modification example described with reference to FIG. 57 . Detailed description is omitted.

(1-3)

The above modified example is an example, and the curve parameters P1 and P2 may be other curve parameters of the gradation transformation curve Gm.

Moreover, the number of curve parameters is not limited to the above. It can be less, it can be more.

In the description of the calculation unit 148 , calculations in a case where the gradation transformation curve candidates G1 to Gp are curves composed of straight lines are described. Here, when the coordinates on the gradation transformation curve candidates G1 to Gp are provided as curve parameters, a smooth curve (curve fitting) passing through the provided coordinates is created, and the gradation transformation can be carried out using the created curve. Transform processing.

(2)

In the modification described above, "the curve parameter output unit 145 is composed of the first LUT 146 and the second LUT 147" has been described. Here, the curve parameter output unit 145 may not have an LUT for storing the values of the curve parameters P1 and P2 corresponding to the value of the selection signal Sm.

In this case, the curve parameter output unit 145 calculates the values of the curve parameters P1 and P2. More specifically, the curve parameter output unit 145 stores parameters representing the curves of the curve parameters P1 and P2 shown in FIGS. 58 , 59 , and 61 . The curve parameter output unit 145 determines the curves of the curve parameters P1 and P2 based on the stored parameters. Furthermore, using the curves of the curve parameters P1 and P2, the values of the curve parameters P1 and P2 corresponding to the selection signal Sm are output.

Here, the parameters for specifying the curves of the curve parameters P1 and P2 are coordinates on the curve, slope of the curve, curvature, and the like. For example, the curve parameter output unit 145 stores the coordinates of two points on the curve of the curve parameters P1 and P2 as shown in FIG. 58 , and uses a straight line connecting the coordinates of the two points as the curve of the curve parameters P1 and P2 .

Here, when determining the curves of the curve parameters P1 and P2 based on the parameters, not only approximate straight lines but also approximate polylines, approximate curves, and the like can be used.

In this way, the curve parameters can be output without using a memory for storing the LUT. Right now. The memory capacity of the device can be further reduced.

(sixth embodiment)

(Features of the visual processing device 121 as the sixth embodiment)

A visual processing device 121 as a sixth embodiment of the present invention will be described using FIGS. 62 to 64 . The visual processing device 121 is built in or connected to devices that process images, such as computers, televisions, digital cameras, mobile phones, and PDAs, and performs grayscale processing of images. The visual processing device 121 is characterized in that it switches and uses a plurality of gradation transformation curves stored in advance as an LUT for each pixel to be subjected to gradation processing.

(constitute)

FIG. 62 is a block diagram illustrating the configuration of the visual processing device 121 . The visual processing device 121 includes an image division unit 122 , a selection signal derivation unit 123 , and a gradation processing unit 130 . The image dividing unit 122 receives the input signal IS, and outputs the divided original image input as the input signal IS into a plurality of image regions Pm (1≤m≤n, where n is the number of divisions of the original image). The selection signal derivation unit 123 outputs a selection signal Sm for selecting the gradation transformation curve Cm for each image region Pm. The gradation processing unit 130 includes a selection signal correction unit 124 and a gradation processing unit 125 . The selection signal correction unit 124 receives the selection signal Sm as input, and outputs the selection signal SS for each pixel, which is a signal obtained by correcting the selection signal Sm for each image region Pm. The gradation processing execution unit 125 is equipped with a plurality of gradation transformation curves G1 to Gp (p is the number of candidates) as a two-dimensional LUT, takes the input signal IS and the selection signal SS of each pixel as input, and performs gradation processing for each pixel. The processed output signal OS is taken as output.

(About gradation conversion curve candidates)

The gradation transformation curve candidates G1 to Gp are the same as those described in (fifth embodiment) using FIG. 50 , and thus description thereof will be omitted. However, in the present embodiment, the gradation transformation curve candidates G1 to Gp are curves that give the relationship between the lightness value of the pixel of the input signal IS and the lightness value of the pixel of the gradation processed signal OS.

The gradation processing execution unit 125 includes gradation transformation curve candidates G1 to Gp as two-dimensional LUTs. That is, the two-dimensional LUT is a look-up table (LUT) that provides the brightness values of the pixels of the output signal OS with respect to the selection signal SS that selects the brightness values of the pixels of the input signal IS and the gradation transformation curve candidates G1 to Gp. A specific example is basically the same as that described using FIG. 51 in (fifth embodiment), and thus description thereof will be omitted here. However, in the present embodiment, pixel values of the output signal OS corresponding to upper 6-bit values of the pixel values of the input signal IS represented by, for example, 10 bits are arranged in the column direction of the matrix.

(effect)

The operation of each part is explained. The image dividing unit 122 basically operates in the same manner as the image dividing unit 102 in FIG. 44, and divides the original image input as the input signal IS into a plurality (n) of image regions Pm (see FIG. 45). Here, the number of divisions of the original image is larger than that of the conventional visual processing device 300 shown in FIG. 104 (for example, 4 to 16 divisions). 4800 divisions of each and so on.

The selection signal derivation unit 123 selects the gradation transformation curve Cm for each image region Pm from the gradation transformation curve candidates G1 to Gp. Specifically, the selection signal derivation unit 123 calculates the average brightness value of the wide-area image region Em of the image region Pm, and selects any one of the gradation transformation curve candidates G1 to Gp based on the calculated average brightness value. That is, the grayscale transformation curve candidates G1 to Gp are associated with the average lightness value of the wide-area image region Em, and the grayscale transformation curve candidates G1 to Gp with larger subscripts are selected as the average lightness value is larger.

Here, the wide-area image region Em is the same as that described using FIG. 45 in (Fourth Embodiment). That is, the wide-area image region Em is a collection of multiple image regions including each image region Pm, for example, a collection of 25 image regions with 5 blocks in the vertical direction and 5 blocks in the horizontal direction centered on the image region Pm. In addition, in some cases, the wide-area image area Em of 5 blocks vertically and 5 blocks horizontally around the image area Pm cannot be obtained from the position of the image area Pm. For example, with respect to the image area P1 located around the original image, it is not possible to obtain a wide-area image area E1 with 5 blocks in the vertical direction and 5 blocks in the horizontal direction around the image area P1. In this case, an area in which five blocks in the vertical direction and five blocks in the horizontal direction centered on the image area P1 overlaps with the original image is used as the wide-area image area E1.

The selection result by the selection signal derivation unit 123 is output as a selection signal Sm indicating any one of the gradation transformation curve candidates G1 to Gp. More specifically, the selection signal Sm is output as the value of the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp.

The selection signal correction unit 124 outputs a selection signal SS for selecting each pixel of the gradation transformation curve for each pixel constituting the input signal IS by using the correction of each selection signal Sm output for each image region Pm. . For example, the selection signal SS corresponding to the pixel included in the image area Pm is obtained by correcting the value of the selection signal output to the image area Pm and the image area surrounding the image area Pm by the internal division ratio of the pixel position.

The operation of the selection signal correction unit 124 will be described in more detail using FIG. 63 . FIG. 63 shows a state in which selection signals So, Sp, Sq, and Sr are output for image regions Po, Pp, Pq, and Pr (o, p, q, and r are positive integers equal to or less than the number of divisions n (see FIG. 45 )).

Here, the position of the pixel x to be gradation corrected is divided into [i:1-i] between the center of the image region Po and the center of the image region Pp, and the center of the image region Po is divided into And the center of the image area Pq is divided into [j:1-j]. In this case, the value [SS] of the selection signal SS corresponding to the pixel x is calculated as [SS]={(1-j)(1-i)[So]+(1-j)(i )·[Sp]+(j)·(1-i)·[Sq]+(j)·(i)·[Sr]}. In addition, let [So], [Sp], [Sq], and [Sr] be the values of the selection signals So, Sp, Sq, and Sr.

The gradation processing execution unit 125 receives the lightness value of the pixel included in the input signal IS and the selection signal SS, and outputs the lightness value of the output signal OS using, for example, the two-dimensional LUT 141 shown in FIG. 51 .

In addition, when the value [SS] of the selection signal SS is not equal to the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT 141, it is used in the gradation processing of the input signal IS. The nearest integer to the value [SS] is the gradation transformation curve G1 to Gp of the subscript.

(visual processing method and visual processing program)

FIG. 64 is a flowchart illustrating a visual processing method in the visual processing device 121. The visual processing method shown in FIG. 64 is realized by hardware in the visual processing device 121, and is a method of performing gradation processing on the input signal IS (see FIG. 62). In the visual processing method shown in FIG. 64, the input signal IS is processed in image units (steps S130 to S137). The original image input as the input signal IS is divided into a plurality of image regions Pm (1≤m≤n: n is the number of divisions of the original image) (step S131), and the gradation transformation curve Cm is selected for each image region Pm (Steps S132 to S133), based on the selection signal Sm for selecting the grayscale transformation curve Cm for each image region Pm, the grayscale transformation curve is selected for each pixel of the original image, and the grayscale processing is performed in units of pixels (step S134～S136).

Each step will be specifically described.

For each image region Pm, a gradation transformation curve Cm is selected from the gradation transformation curve candidates G1 to Gp (step S132). Specifically, the average lightness value of the wide-area image area Em of the image area Pm is calculated, and any one of the gradation transformation curve candidates G1 to Gp is selected based on the calculated average lightness value. The grayscale transformation curve candidates G1 to Gp are associated with the average lightness value of the wide-area image region Em, and the larger the average lightness value, the larger the grayscale transformation curve candidates G1 to Gp are selected. Here, description of the wide-area image region Em is omitted (refer to the above-mentioned (function) column). As a result of the selection, a selection signal Sm indicating any one of the gradation transformation curve candidates G1 to Gp is output. More specifically, the selection signal Sm is output as the value of the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp. Furthermore, it is determined whether or not the processing for all the image regions Pm has ended (step S133 ), and the processing of steps S132 to S133 is repeated up to the number of divisions of the original image until it is determined that the processing has ended. Through the above, the processing of the image area unit is completed.

By using the correction of each selection signal Sm output for each image area Pm, a selection signal SS for selecting each pixel of the gradation transformation curve for each pixel constituting the input signal IS is output (step S134). For example, the selection signal SS corresponding to the pixel included in the image area Pm is obtained by correcting the value of the selection signal output to the image area Pm and the surrounding image area of the image area Pm by the internal division ratio of the pixel position. The description of the details of the correction is omitted (refer to the above (action) column, and refer to FIG. 63 ).

The lightness value of the pixel included in the input signal IS and the selection signal SS are input, and the lightness value of the output signal OS is output using, for example, a two-dimensional LUT shown in FIG. 51 (step S135 ). Furthermore, it is determined whether or not the processing for all pixels has been completed (step S136 ), and the processing of steps S134 to S136 is repeated several times for pixels until it is determined that the processing is completed. Through the above, the pixel-by-pixel processing ends.

In addition, each step of the visual processing method shown in FIG. 64 can be realized as a visual processing program by a computer or the like.

(Effect)

According to the present invention, substantially the same effect as that of the above-mentioned "fourth embodiment" and "fifth embodiment" can be obtained. Hereinafter, effects specific to the sixth embodiment will be described.

(1)

The gradation transformation curve Cm selected for each image region Pm is created based on the average lightness value of the wide-area image region Em. Therefore, even if the size of the image region Pm is small, sufficient sampling of lightness values can be performed. And, as a result, an appropriate gradation transformation curve Cm can be selected even for a small image region Pm.

(2)

The selection signal correcting unit 124 outputs the selection signal SS for each pixel through correction based on the selection signal Sm output in units of image regions. The pixels constituting the original image of the input signal IS are subjected to gradation processing using the gradation transformation curve candidates G1 to Gp specified by the selection signal SS for each pixel. Therefore, it is possible to obtain an output signal OS in which gradation processing is performed more appropriately. For example, the generation of blurred outlines can be suppressed. In addition, in the output signal OS, it is possible to further prevent the joints at the boundaries of the respective image regions Pm from being conspicuous and unnatural.

(3)

The gradation processing execution unit 125 has a pre-created two-dimensional LUT. Therefore, the processing load required for gradation processing, more specifically, the processing load required for creating the gradation transformation curve Cm can be reduced. As a result, gradation processing can be accelerated.

(4)

The gradation processing execution unit 125 executes gradation processing using a two-dimensional LUT. Here, the content of the two-dimensional LUT is read from a storage device such as a hard disk or a ROM included in the visual processing device 121 and used for gradation processing. By changing the content of the read 2D LUT, various gradation processing can be realized without changing the hardware configuration. That is, gradation processing more suitable for the characteristics of the original image can be realized.

(Modification)

The present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the gist. For example, basically the same modification as the above (fifth embodiment) (modification) can be applied to the sixth embodiment. In particular, in (10) to (12) of (fifth embodiment) (modified example), by converting the selection signal Sm into the selection signal SS, and converting the gradation processing signal CS into the output signal OS, similarly applicable.

Hereinafter, modifications specific to the sixth embodiment will be described.

(1)

In the above-described embodiment, the two-dimensional LUT 141 composed of a matrix of 64 rows and 64 columns is taken as an example of the two-dimensional LUT. Here, the effects of the present invention are not limited to the two-dimensional LUT of this size. For example, it may be a matrix in which more gradation transformation curve candidates are arranged in the row direction. In addition, the pixel values of the output signal OS corresponding to the values divided by the step of further reducing the pixel values of the input signal IS may be arranged in the column direction of the matrix. Specifically, the pixel values of the output signal OS may be arranged corresponding to the respective pixel values of the input signal IS represented by, for example, 10 bits.

If the size of the 2D LUT is larger, more appropriate gradation processing can be performed, and if it is smaller, the memory for storing the 2D LUT can be reduced.

(2)

In the above embodiment, it has been described that when the value [SS] of the selection signal SS is not equal to the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT 141 (see FIG. 51 ), In the gradation processing of the input signal IS, the gradation transformation curve candidates G1 to Gp whose subscripts are the integers closest to the value [SS] are used. Here, when the value [SS] of the selection signal SS is not a value equal to the subscripts (1 to p) of the gradation transformation curve candidates G1 to Gp included in the two-dimensional LUT 141, a value exceeding the selection signal SS is used. The grayscale transformation curve candidate Gk (1≤k≤p-1) with the largest integer (k) of the value [SS] as the subscript, and the grayscale transformation curve with the smallest integer (k+1) exceeding [SS] as the subscript The pixel values of the input signal IS subjected to gradation processing on both candidates Gk+1 are weighted averaged (inner divided) using values below the decimal point of the value [SS] of the selection signal SS, and an output signal OS is output.

(3)

In the above-mentioned embodiment, the pixel values of the output signal OS corresponding to the upper 6 bits of the pixel values of the input signal IS represented by, for example, 10 bits are arranged in the column direction of the matrix. Here, the output signal OS can be output as a matrix component linearly interpolated with the lower 4-bit values of the pixel values of the input signal IS by the gradation processing execution unit 125 . That is, in the column direction of the matrix, matrix components corresponding to the upper 6-bit values of the pixel values of the input signal IS represented by, for example, 10 bits are arranged, and the input signal IS is expressed using the lower 4-bit values of the pixel values of the input signal IS The matrix component corresponding to the high-order 6-bit value of the pixel value of the input signal IS, and the matrix component corresponding to the value after adding [1] to the low-order 6-bit value of the pixel value of the input signal IS (for example, in the first row in Figure 51 Components), perform linear interpolation, and output it as a gray-scale processed signal OS.

In this way, more appropriate gradation processing can be performed even if the size of the two-dimensional LUT 141 (see FIG. 51 ) is small.

(5)

In the above embodiments, it is explained that the selection signal Sm corresponding to the image region Pm is output based on the average brightness value of the wide-area image region Em. Here, the output method of the selection signal Sm is not limited to this method. For example, the selection signal Sm corresponding to the image region Pm may also be output based on the maximum brightness value or the minimum brightness value of the wide-area image region Em. In addition, the value [Sm] of the selection signal Sm may be the average brightness value, the maximum brightness value, or the minimum brightness value of the wide-area image region Em.

Alternatively, for example, the selection signal Sm corresponding to the image region Pm may be output as follows. That is, the average brightness value for each image region Pm is obtained, and the temporary selection signal Sm' for each image region Pm is obtained based on each average brightness value. Here, the temporary selection signal Sm' has the subscript numbers of the gradation transformation curve candidates G1 to Gp as values. Furthermore, the values of the provisional selection signal Sm' are averaged for each image area included in the wide-area image area Em to obtain the selection signal Sm for the image area Pm.

(5)

In the above embodiments, it is explained that the selection signal Sm corresponding to the image region Pm is output based on the average brightness value of the wide-area image region Em. Here, instead of simply averaging the wide-area image region Em, the selection signal Sm corresponding to the image region Pm may be output based on a weighted average (weighted average). The details are the same as those described above (fifth embodiment) using FIG. For image areas Ps1 , Ps2 , .

In this way, even when the wide-area image region Em includes a region with abnormal brightness (for example, when the wide-area image region Em includes two types of boundaries of objects with different brightness values), the brightness value of the abnormal region is affected by the output of the selection signal Sm. There are also few influences from coming, and an appropriate output of the selection signal Sm can be performed.

(6)

The visual processing device 121 may further include a profile data creation unit that creates profile data of values stored as two-dimensional LUTs. Specifically, the profile data creation unit is composed of the image segmentation unit 102 and the gradation transformation curve derivation unit 110 in the visual processing device 101 (see FIG. 44 ), and takes the set of the created plurality of gradation transformation curves as Profile data, stored in a 2D LUT.

In addition, each gradation transformation curve stored in the two-dimensional LUT may be associated with the spatially processed input signal IS. In this case, in the visual processing device 121, the image division unit 122, the selection signal derivation unit 123, and the selection signal correction unit 124 may be replaced by a spatial processing unit that performs spatial processing on the input signal IS.

(seventh embodiment)

A visual processing device 161 as a seventh embodiment of the present invention will be described using FIGS. 65 to 71 .

The visual processing device 161 shown in FIG. 65 is a device that performs visual processing such as spatial processing and gradation processing of an image signal. The visual processing device 161 is a device that processes images such as computers, televisions, digital cameras, mobile phones, PDAs, printers, scanners, etc., and constitutes a device that performs color processing of image signals and an image processing device.

The visual processing device 161 is a device that performs visual processing using an image signal and a blurred signal obtained by subjecting the image signal to spatial processing (blurring filter processing), and is characterized by spatial processing.

Conventionally, when a blurred signal is derived using surrounding pixels of a target pixel, if the surrounding pixels include pixels whose density is significantly different from the target pixel, the blurred signal is affected by the pixels with different densities. That is, in the case of performing spatial processing on pixels near the edge of an object in an image, pixels that are not originally edges are affected by the density of the edge. Therefore, this spatial processing may cause, for example, the generation of blurred outlines.

Therefore, spatial processing is performed in pursuit of adapting to the content of the image. In this regard, for example, in JP-A-10-75395, a plurality of fuzzy signals having different degrees of fuzziness are produced, and by synthesizing or switching the respective fuzzy signals, an appropriate fuzzy signal is output. In this way, the purpose is to change the filter size of the spatial processing and suppress the influence of pixels with different densities.

On the other hand, in the above-mentioned publication, since a plurality of fuzzy signals are produced and the respective fuzzy signals are combined or switched, the circuit scale or processing load in the device becomes large.

Therefore, in the visual processing device 161 which is the seventh embodiment of the present invention, it is an object to output an appropriate blur signal and reduce the circuit scale or processing load in the device.

(visual processing device 161)

FIG. 65 shows a basic configuration of a visual processing device 161 that visually processes an image signal (input signal IS) and outputs a visually processed image (output signal OS). The visual processing device 161 includes: a spatial processing unit 162 that performs spatial processing according to the lightness value of each pixel of the original image obtained as the input signal IS, and outputs the unsharp signal US; and a visual processing unit 163 that uses Regarding the input signal IS and the unsharp signal US of the same pixel, visual processing of the original image is performed, and an output signal OS is output.

(Spatial processing unit 162)

The spatial processing by the spatial processing unit 162 will be described using FIG. 66 . The spatial processing unit 162 obtains pixel values of the target pixel 165 to be spatially processed and the pixels in the surrounding area of the target pixel 165 (hereinafter referred to as peripheral pixels 166 ) based on the input signal.

The surrounding pixels 166 are pixels located in the surrounding area of the target pixel 165 , and are pixels included in a surrounding area of 9 vertical pixels and 9 horizontal pixels spread around the target pixel 165 . In addition, the size of the peripheral area is not limited to this case, and may be smaller or larger. Further, the surrounding pixels 166 are divided into a first surrounding pixel 167 and a second surrounding pixel 168 from the near side according to the distance from the target pixel 165 . In FIG. 66 , the first peripheral pixels 167 are pixels included in an area of 5 vertical pixels and 5 horizontal pixels around the target pixel 165 . Furthermore, the second peripheral pixel 168 is a pixel positioned around the first peripheral pixel 167 .

The spatial processing unit 162 performs filter calculation on the target pixel 165 . In the filtering operation, the pixel values of the target pixel 165 and the surrounding pixels 166 are weighted and averaged based on the difference and distance between the pixel values of the target pixel 165 and the surrounding pixels 166 . The weighted average is calculated based on the following formula F=(Σ[Wij]×[Aij]/Σ[Wij]). Here, [Wij] is the weight coefficient of the pixel located in the i-th row and j-th column among the target pixel 165 and the surrounding pixels 166, and [Aij] is the weight coefficient of the pixel located in the i-th row among the target pixel 165 and the surrounding pixels 166 The pixel value of the pixel in column j. In addition, “Σ” refers to the calculation of the sum of the respective pixels of the target pixel 165 and the surrounding pixels 166 .

The weight coefficient [Wij] will be described using FIG. 67 . The weight coefficient [Wij] is a value determined based on the difference in pixel value and the distance between the target pixel 165 and the surrounding pixels 166 . More specifically, the larger the absolute value of the pixel value difference, the smaller the weighting factor is assigned. Alternatively, the larger the distance, the smaller the weight coefficient.

For example, for the target pixel 165, the weight coefficient [Wij] is the value [1].

Among the first peripheral pixels 167 , the weight coefficient [Wij] has the value [1] for a pixel having a pixel value whose absolute value of the difference with the pixel value of the target pixel 165 is smaller than a predetermined threshold value. Among the first peripheral pixels 167, the weight coefficient [Wij] has a value of [1/2] for a pixel having a pixel value whose absolute value of the difference is larger than a predetermined threshold value. That is, even the pixels included in the first peripheral pixels 167 are assigned different weights depending on the pixel values.

Among the second peripheral pixels 168 , the weight coefficient [Wij] is the value [1/2] for a pixel having a pixel value whose absolute value of the difference with the pixel value of the target pixel 165 is smaller than a predetermined threshold value. Among the second peripheral pixels 168 , the weight coefficient [Wij] is the value [1/4] for a pixel having a pixel value whose absolute value of the difference is larger than a predetermined threshold value. That is, even the pixels included in the second peripheral pixels 168 are given different weights depending on the pixel values. In addition, a smaller weight coefficient is given to the second peripheral pixel 168 whose distance from the target pixel 165 is greater than that of the first peripheral pixel 167 .

Here, the predetermined threshold is a value such as (20/256 to 60/256) for the pixel value of the target pixel 165 that takes a value within the range of [0.0 to 1.0].

The weighted average calculated above is output as the unsharp signal US.

(visual processing unit 163)

In the visual processing unit 163, visual processing is performed using the same values for the same input signal IS and unsharp signal US. The visual processing performed here is processing such as contrast enhancement of the input signal IS or dynamic range compression. In contrast enhancement, a signal enhanced by using a function that enhances the difference or ratio between the input signal IS and the unsharp signal US is added to the input signal IS to perform image sharpness. In dynamic range compression, the unsharp signal US is subtracted from the input signal IS.

The processing in the visual processing unit 163 can be performed using a two-dimensional LUT that receives the input signal IS and the unsharp signal US as inputs, and outputs the output signal OS.

(visual processing method, program)

The above processing can be executed by a computer or the like as a visual processing program. The visual processing program is a program for causing a computer to execute the visual processing method described below.

The visual processing method comprises: a spatial processing step of performing spatial processing according to the lightness value of each pixel of an original image obtained as an input signal IS, and outputting a passivation signal US; and a visual processing step of using the same The pixel input signal IS and passivation signal US perform visual processing of the original image and output the output signal OS.

In the spatial processing step, the weighted average described in the description of the spatial processing unit 162 is performed for each pixel of the input signal IS, and the unsharp signal US is output. Details are omitted because of the above.

In the visual processing step, the visual processing described in the description of the visual processing unit 163 is performed using the input signal IS and the unsharp signal US for the same pixel, and an output signal OS is output. The details are omitted because of the above.

(Effect)

The effect of the visual processing realized by the visual processing device 161 will be described using FIGS. 68( a ) to ( b ). Fig. 68(a) and Fig. 68(b) show processing performed by a conventional filter. Fig. 68(b) shows the processing performed by the filter of the present invention.

FIG. 68( a ) shows a case where an object 171 with peripheral pixels 166 having different densities is included. In the spatial processing of the target pixel 165, a smoothing filter having predetermined filter coefficients is used. Therefore, the target pixel 165 that is not part of the object 171 is affected by the density of the object 171 .

Fig. 68(b) shows the state of the spatial processing of the present invention. In the spatial processing of the present invention, for each of the part 166a of the surrounding pixel 166 including the object 171, the first surrounding pixel 167 not containing the object 171, the second surrounding pixel 168 not containing the object 171, and the object pixel 165, different values are used. The weight coefficients are used for spatial processing. Therefore, the target pixel 165 subjected to spatial processing can be suppressed from being affected by pixels with extremely different densities, and more appropriate spatial processing can be performed.

Furthermore, in the visual processing device 161, it is not necessary to create a plurality of blurred signals as in JP-A-10-75395. Therefore, the circuit scale and processing load in the device can be reduced.

Furthermore, in the visual processing device 161, the filter size of the spatial filter and the shape of the image referred to by the filter can be adaptively changed substantially according to the content of the image. Therefore, spatial processing suitable for image content can be performed.

(Modification)

(1)

The sizes of the aforementioned peripheral pixels 166 , first peripheral pixels 167 , and second peripheral pixels 168 are examples, and other sizes may be used.

The above-mentioned weight coefficient is an example, and other values may be used. For example, when the absolute value of the difference between pixel values exceeds a predetermined threshold, the value [0] may be assigned to the weight coefficient. In this way, the spatially processed target pixel 165 is not affected by pixels with extremely different densities. In the application for the purpose of contrast enhancement, this is called an effect of excessively enhancing the contrast of a part with a certain degree of contrast.

And, the weight coefficient may be a value provided as such a function as shown below.

(1-a)

The value of the weight coefficient may also be given by a function that makes the absolute value of the difference between pixel values a variable. For example, when the absolute value of the pixel value difference is small, the weight coefficient becomes larger (closer to 1); when the absolute value of the pixel value difference is larger, the weight coefficient becomes smaller (closer to 0). , a function that the absolute value of the difference of pixel values decreases monotonically.

(1-b)

The value of the weighting coefficient may also be given by a function in which the distance from the target pixel 165 is a variable. The function is relative to, for example, when the distance from the target pixel 165 is relatively short, the weight coefficient becomes larger (closer to 1); when the distance from the target pixel 165 is farther, the weight coefficient becomes smaller (closer to 0). A monotonically decreasing function of the distance of object pixels 165 .

In (1-a) and (1-b) above, weight coefficients can be assigned more continuously. Therefore, compared with the case of using a threshold value, it is possible to give more appropriate weight coefficients, suppress excessive contrast enhancement, suppress the occurrence of blurred outlines, etc., and perform processing with higher visual effects.

(2)

The above-mentioned processing of each pixel can be performed in units of a block including a plurality of pixels. Specifically, first, the average pixel value of the target block to be subjected to spatial processing and the average pixel value of surrounding blocks around the target block are calculated. Furthermore, each average pixel value is weighted and averaged using the same weight coefficient as above. In this way, the average pixel value of the target block is further subjected to spatial processing.

In such a case, the spatial processing unit 162 can be used as the selection signal derivation unit 113 (see FIG. 49 ) or as the selection signal derivation unit 123 (see FIG. 62 ). In this case, it is the same as described in (5th embodiment) (modification) (6) or (6th embodiment) (modification) (5).

This will be described using FIGS. 69 to 71 .

(constitute)

FIG. 69 is a block diagram showing the configuration of a visual processing device 961 that performs the processing described with reference to FIGS. 65 to 68 in units of blocks including a plurality of pixels.

The visual processing device 961 is configured to include: an image segmentation unit 964, which divides the image input as the input signal IS into a plurality of image blocks; a spatial processing unit 962, which performs spatial processing on each divided image block; and a visual processing unit 963 that performs visual processing using the input signal IS and the spatially processed signal US2 output from the spatial processing unit 962 .

The image dividing unit 964 divides the image input as the input signal IS into a plurality of image blocks. Furthermore, the processed signal US1 including the characteristic parameter of each divided image block is derived. The characteristic parameter is, for example, a parameter representing image characteristics of each divided image block, such as an average value (simple average, weighted average, etc.) or representative value (maximum value, minimum value, median value, etc.).

The spatial processing unit 962 obtains the processed signal US1 including the characteristic parameters of each image block, and performs spatial processing.

The spatial processing by the spatial processing unit 962 will be described using FIG. 70 . FIG. 70 shows an input signal IS divided into image blocks including a plurality of pixels. Here, each image block is divided into an area of 9 pixels including 3 pixels vertically and 3 pixels horizontally. In addition, this dividing method is an example, and is not limited to such a dividing method. Furthermore, in order to fully exhibit the effect of visual processing, it is preferable to generate the spatially processed signal US2 targeting a relatively large area.

The spatial processing unit 962 obtains characteristic parameters of the target image block 965 to be spatially processed and the surrounding image blocks included in the surrounding area 966 located around the target image block 965 based on the processing signal US1.

The peripheral area 966 is an area located around the target image block 965 , and is an area of 5 vertical blocks and 5 horizontal blocks spread around the target image block 965 . In addition, the size of the peripheral area 966 is not limited to this case, and may be smaller or larger. Furthermore, the peripheral area 966 is divided into a first peripheral area 967 and a second peripheral area 968 from the near side according to the distance from the target image block 966 .

In FIG. 70 , a first peripheral area 967 is an area of three vertical blocks and three horizontal blocks around the target image block 965 . Furthermore, the second peripheral area 968 is an area located around the first peripheral area 967 .

The spatial processing unit 962 performs filter calculation on the characteristic parameters of the target image block 965 .

In the filter calculation, the values of the feature parameters of the target image block 965 and surrounding image blocks in the surrounding area 966 are weighted and averaged. Here, the weight of the weighted average is determined based on the distance between the target image block 965 and the surrounding image blocks and the difference in the value of the characteristic parameter.

More specifically, the weighted average is calculated based on the following formula F=(Σ[Wij]×[Aij]/Σ[Wij]).

Here, [Wij] is the weight coefficient corresponding to the image block located in the i-th row and j-th column in the target image block 965 and the surrounding pixels 966, and [Aij] is in the target image block 965 and the surrounding pixels 966, located in the weight coefficient The value of the feature parameter of the image block at row i and column j. In addition, “Σ” refers to the calculation of the sum of the respective image blocks of the target image block 965 and the surrounding pixels 966 .

The weight coefficient [Wij] will be described using FIG. 71 .

The weight coefficient [Wij] is a value determined based on the distance between the target image block 965 and the surrounding pixel blocks of the surrounding pixel 966 and the value of the feature parameter. More specifically, the larger the absolute value of the difference between the values of the characteristic parameters, the smaller the weight coefficient. Alternatively, the larger the distance, the smaller the weight coefficient.

For example, for the target image block 965, the weight coefficient [Wij] is the value [1].

The weight coefficient [Wij] is the value [ 1]. The weight coefficient [Wij] is the value [1/2] for the surrounding image blocks in the first surrounding area 967 that have a characteristic parameter value whose absolute value of the difference is larger than a predetermined threshold. That is, even the surrounding image blocks included in the first surrounding area 967 are assigned different weights depending on the value of the feature parameter.

For the surrounding image blocks in the second peripheral region 968 that have a characteristic parameter value whose absolute value of the difference with the characteristic parameter value of the target image block 965 is smaller than a predetermined threshold value, the weight coefficient [Wij] is value[1/2]. In the second peripheral region 968 , the weight coefficient [Wij] has a value of [1/4] for peripheral image blocks having a characteristic parameter value whose absolute value of the difference is larger than a predetermined threshold value. That is, even the surrounding image blocks included in the second surrounding area 968 are given different weights depending on the value of the feature parameter. In addition, a smaller weight coefficient is assigned to the second peripheral area 968 that is farther away from the target image block 965 than the first peripheral area 967 .

Here, the predetermined threshold is a value such as (20/256 to 60/256) for the feature value of the target image block 965 that takes a value in the range of [0.0 to 1.0].

The weighted average calculated above is output as the spatially processed signal US2.

In the visual processing unit 963, the same visual processing as that of the visual processing unit 163 (see FIG. 65 ) is performed. However, the difference from the visual processing unit 163 is that, instead of the unsharp signal US, the spatially processed signal US2 of the target image block including the target pixel to be visually processed is used.

In addition, the processing in the visual processing unit 963 may be performed collectively in units of target image blocks including the target pixel, or may switch the spatially processed signal US2 in the order of pixels obtained from the input signal IS.

The above processing is performed on all pixels included in the input signal IS.

(Effect)

In the processing of the spatial processing unit 962, processing is performed in units of image blocks. Therefore, the amount of processing performed by the spatial processing unit 962 can be reduced, and higher-speed visual processing can be realized. Also, the scale of hardware can be reduced.

(Modification)

In the above, it is described that processing is performed in units of square blocks. Here, the shape of the block may be arbitrary.

In addition, the aforementioned weight coefficients, thresholds, and the like can be appropriately changed.

Here, the value of a part of the weight coefficient may be the value [0]. In this case, it is the same as making the shape of the peripheral region 966 an arbitrary shape.

Furthermore, although it has been described that the spatial processing unit 962 performs spatial processing using the feature parameters of the target image block 965 and the surrounding area 966 , the spatial processing may be performed using only the feature parameters of the surrounding area 966 . That is, the weight of the target image block 965 may be set to the value [0] in the weight of the weighted average of the spatial processing.

(3)

The processing in the visual processing unit 163 is not limited to the above. For example, the visual processing unit 163 may also use the value A of the input signal IS, the value B of the unsharp signal US, the dynamic range compression function F4, and the enhancement function F5, by the following formula C=F4(A)×F5( The calculated value C of A/B) is output as the value of the output signal OS. Here, the dynamic range compression function F4 is a monotonically increasing function such as an upward convex function. For example, it is expressed as F4(x)=x^γ (0<γ<1). The strengthening function F5 is a power function, for example expressed as F5(x)=x^α (0<α≤1).

In the case of performing such processing in the visual processing unit 163, if the appropriate unsharp signal US output by the spatial processing unit 162 of the present invention is used, the dynamic range of the input signal IS can be compressed while enhancing local contrast.

On the other hand, in the case where the unsharp signal US is not appropriate and the blur is too little, although there is edge enhancement, contrast enhancement is not appropriate. Also, in the case of excessive blur, although the contrast enhancement is performed, the dynamic range is not properly compressed.

(eighth embodiment)

As an eighth embodiment of the present invention, an application example of the visual processing device, the visual processing method, the visual processing program described in the above-mentioned fourth to seventh embodiments, and a system using these will be described.

A visual processing device is built in or connected to an image processing device such as a computer, television, digital camera, mobile phone, PDA, etc., and performs image grayscale processing, and is implemented as an integrated circuit such as an LSI.

In more detail, each functional block of the above-mentioned embodiment may be single-chip individually, or a part or all of them may be single-chip. In addition, although it is referred to as an LSI here, it is also called an IC, a system LSI, a very large-scale LSI, or an extremely large-scale LSI depending on the degree of integration.

Furthermore, the method of circuit integration is not limited to LSI, and it can also be realized by a dedicated circuit or a common processor. After the LSI is manufactured, a programmable FPGA (Field Pragrammable Gate Array, Field Programmable Gate Array), or a programmable device and processor that can be reconfigured to connect or set the circuit units inside the LSI can also be used.

Furthermore, if an integrated circuit technology that replaces LSIs with the development of semiconductor technology or other derived technologies comes out, it is of course possible to integrate functional modules using this technology. There can also be applications for biotechnology and the like.

The processing of each block in Fig. 44, Fig. 49, Fig. 62, Fig. 65, and Fig. 69 is performed by, for example, a central processing unit (CPU) included in the visual processing device. In addition, programs for performing respective processes are stored in a storage device such as a hard disk or a ROM, and the programs are read and executed in the ROM or in the RAM. In addition, the two-dimensional LUTs referred to by the gradation processing execution units 114 and 125 in FIG. 49 and FIG. 62 are stored in storage devices such as hard disks and ROMs, and are referred to as necessary. Furthermore, the 2D LUT may be provided by a 2D LUT provider directly connected to the visual processing device or connected via a network profile. The same applies to the one-dimensional LUT referred to by the gradation processing execution unit 144 of FIG. 56 .

In addition, the visual processing device may also be a device built in or connected to a machine that processes moving images, and processes the grayscale of each frame (each segment) of the image.

In addition, in each visual processing device, the visual processing methods described in the above-mentioned fourth to seventh embodiments are executed.

Visual processing programs are stored in storage devices such as hard disks and ROMs in devices that process images, such as computers, televisions, digital cameras, mobile phones, and PDAs, and are used to perform grayscale processing of images. program. It is provided via a recording medium such as a CD-ROM, or via a network.

In the above-mentioned embodiments, the processing performed on the brightness value of each pixel has been described. Here, the present invention does not depend on the color space of the input signal IS. That is, in the processing in the above-mentioned embodiment, when the input signal IS is represented by YCbCr color space, YUV color space, Lab color space, Luv color space, YIQ color space, XYZ color space, YPbPr color space, RGB color space, etc. The same applies to brightness and brightness of each color space.

In addition, when the input signal IS is represented by the RGB color space, the processing in the above-mentioned embodiment may be performed independently for each component of RGB.

(ninth embodiment)

As a ninth embodiment of the present invention, application examples of the above-described visual processing device, visual processing method, and visual processing program, and a system using these will be described with reference to FIGS. 72 to 75 .

Fig. 72 is a block diagram showing the overall configuration of a content supply system ex100 for realizing a content distribution service. The area for providing communication services is divided into desired sizes, and base stations ex107 to ex110 serving as fixed wireless stations are installed in each cell.

The content supply system ex100 is, for example, connected to the Internet ex101 via an Internet service provider ex102, a telephone network ex104, and base stations ex107 to ex110, and is connected to a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a Various devices such as camera, mobile phone ex115, etc.

However, the contents (contents) supply system ex100 is not limited to the combination shown in FIG. 72, and may be connected in any combination. Furthermore, each device may be directly connected to the telephone network ex104 without passing through the base stations ex107 to ex110 which are fixed wireless stations.

The camera ex113 is a device capable of shooting moving images such as digital video cameras. In addition, the mobile phone is a PDC (Personal Digital Communication) type, a CDMA (Code Division Multiple Access) type, a W-CDMA (Wideband-Code Division Multiple Access) type, or a GSM (Global System for Mobile Communication) type mobile phone, or Any one of PHS (Personal Handyphone System).

In addition, a streaming server ex103 is connected from the camera ex113 via the base station ex109 and the telephone network ex104, and the camera ex113 can be used to perform real-time distribution based on coded data sent by the user. The encoding processing of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs data transmission processing. In addition, video data captured by the camera ex116 can be sent to the streaming server 103 via the computer ex111. The digital camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, the encoding of the video may be performed by the camera ex116, or may be performed by the computer ex111. In addition, the encoding process is performed in the LSI ex117 included in the computer ex111 or the camera ex116. Also, software for image encoding and decoding may be embedded in a storage medium (CD-ROM, floppy disk, hard disk, etc.) that can be read by a computer ex111 or the like. Further, the dynamic data can be sent from the mobile phone ex115 with a camera. The dynamic data at this time is data coded by the LSI included in the mobile phone ex115.

In this content supply system ex100, the user encodes the content captured by the camera ex113, ex116, etc. (for example, video captured in real time with music) and transmits it to the streaming server ex103. On the other hand, the streaming server ex103, For the requesting client, traffic distribution is performed on the above content data. As clients, there are computers ex111, PDA ex112, cameras ex113, mobile phones ex114, etc., which can decode encoded data. By doing so, the content supply system ex100 can receive encoded data at the client side and reproduce it, and then receive, decode, and reproduce it in real time at the client side, thereby realizing stand-alone playback.

When displaying content, the visual processing device, visual processing method, and visual processing program described in the above embodiments can be used. For example, a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, etc., are equipped with the visual processing device described in the above-mentioned embodiments, and can implement a visual processing method and a visual processing program.

Also, the streaming server ex103 can provide profile data to the visual processing device via the network ex101. Furthermore, there are a plurality of streaming servers ex103, and each can provide different profile data. Also, the streaming server ex103 can create profile data. In this way, when the visual processing device can obtain the profile data via the network ex101, the visual processing device does not need to store the profile data used for visual processing in advance, and the storage capacity of the visual processing device can be reduced. In addition, since profile data can be obtained from multiple servers connected via the Internet ex101, various visualization processes can be realized. A mobile phone will be described as an example.

FIG. 73 shows a diagram of a mobile phone ex115 including the visual processing device of the above-mentioned embodiment. The mobile phone ex115 has a main body including an antenna ex201 for transmitting and receiving radio waves with the base station ex110, a camera unit ex203 capable of taking images and still images such as a CCD camera, and displaying images to be shot by the camera unit ex203, A display unit ex202 such as a liquid crystal display for receiving video decoded data received by the antenna ex201, and an operation key ex204 are composed of a sound output unit ex208 such as a speaker for outputting sound, and a sound input unit such as a microphone for sound input. ex205. Recording medium ex207 for storing encoded or decoded data such as captured moving image or still image data, received mail data, moving image data, or still image data, etc. . A slider ex206 for attaching and detaching the recording medium ex207 to the mobile phone ex115. The recording medium ex207 stores a flash memory element as a kind of EEPROM (Electrically Erasable and Programmbale Read Only Memory, electrically erasable and programmable read-only memory). Volatile memory.

Furthermore, the mobile phone ex115 will be described using FIG. 74 . The mobile phone ex115 includes a main control unit ex311 for collectively controlling each unit of the main body including the display unit ex202 and the operation keys ex204, the power supply circuit ex310, the operation input control unit ex304, the image encoding unit ex312, the camera interface unit ex303, The LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, multiplexing unit ex308, storage playback unit ex307, modem unit ex306, and audio processing unit ex305 are connected to each other via a synchronous bus ex313.

The power supply circuit unit ex310 activates the digital mobile phone with camera ex115 to be in an operable state by supplying power from the battery unit to each unit when the call end and the power key are turned on by the user's operation.

The mobile phone ex115 is based on the control of the main control unit ex311 composed of CPU, ROM, and RAM. In the voice call mode, the voice processing unit ex305 converts the voice signal collected by the voice input unit ex205 into a digital voice signal, and converts it into a digital voice signal. Spectrum spreading processing is performed by the modem circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing are performed by the transceiver circuit unit ex301, and then transmitted via the antenna ex201. In addition, the mobile phone ex115 amplifies the received signal received by the antenna ex201 in the voice communication mode, performs frequency conversion processing and analog-to-digital conversion processing, and performs spectrum despreading processing by the modulation and demodulation circuit part ex306 to pass the voice The processing unit ex305 converts the signal into an analog audio signal, and outputs it via the audio output unit ex208.

Furthermore, when sending an e-mail in the data communication mode, the text data of the e-mail input by operating the operation keys ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spectrum spreading processing on the text data by the modulation and demodulation circuit unit ex306, performs digital-to-analog conversion processing and frequency conversion processing by the transceiver signal circuit unit ex301, and then transmits the text data to the base station ex110 through the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. Furthermore, when the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The image encoding unit ex312 compresses and encodes the image data supplied from the camera unit ex203 to convert it into encoded image data, and sends it to the multiplexing unit ex308. At the same time, the mobile phone ex115 transmits the voice collected by the camera unit ex203 to the voice input unit ex205 when taking pictures, as digital voice data to the multiplexing unit ex308 via the voice output unit ex305.

The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 in a predetermined manner, and as a result, the obtained multiplexed data is demodulated The modulation circuit unit ex306 performs spectrum spreading processing, and the transmission and reception circuit unit ex301 performs digital-to-analog conversion processing and frequency conversion processing, and then transmits via the antenna ex201.

In the data communication mode, when receiving data of moving image files linked to homepages, etc., the received signal received from the base station ex110 via the antenna ex201 is subjected to inverse spectral spread processing by the modem unit ex306, and the obtained The multiplexed data is sent to the multiplex separation unit ex308.

In addition, in order to decode the multiplexed data received via the antenna ex201, the multiplexed data is separated by the demultiplexing unit ex308 into a coded bit stream of video data and a coded bit stream of audio data, and the multiplexed data is separated into a coded bit stream of audio data, and the multiplexed data is decoded via the synchronous bus ex313. The coded image data is supplied to the image decoding unit ex309, and the audio data is supplied to the audio output unit ex305.

Next, the image decoding unit ex309 decodes the decoded bit stream of the image data to generate reproduced video data, and supplies it to the display unit ex202 via the LCD control unit ex302, thereby displaying, for example, a video file linked to the home page. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies it to the audio output unit ex208. Data regeneration.

In the above configuration, the image decoding unit ex309 may include the visual processing device of the above-mentioned embodiment.

In addition, it is not limited to the example of the above-mentioned system. Recently, digital broadcasting by satellite and terrestrial waves has become a topic. In the system for digital broadcasting shown in FIG. Handling method, visual handler. Specifically, in the broadcast station ex409, the coded bit stream of video information is transmitted to the communication or broadcast satellite ex410 via radio waves. Receive its broadcasting satellite ex410, and send radio waves for broadcasting. The radio waves are received by the home antenna ex406 equipped with satellite broadcasting transceiver equipment, and decoded bits through devices such as TV (receiving equipment) ex401 or set-top box (ST B) ex407. The stream is decoded to regenerate it. Here, devices such as a television (receiving device) ex401 or a set-top box (STB) ex407 may include the visual processing device described in the above-mentioned embodiments. In addition, the visual processing method of the above-mentioned embodiment may also be used. Furthermore, a visual processing program may also be provided. In addition, in the playback device ex403 that reads and decodes the coded bit stream recorded on the storage medium ex402 such as CD or DVD as the recording medium, the visual processing device and the visual processing described in the above-mentioned embodiments can also be executed. method, visual handler. In this case, the reproduced video signal is displayed on the monitor ex404. In addition, it is also conceivable to install the visual processing device, visual processing method, and visual processing program described in the above embodiments in the set-top box ex407 connected to the optical cable ex405 for optical cable TV or the antenna ex406 of satellite/terrestrial broadcasting, It is a configuration to reproduce it with a TV monitor ex408. In this case, not only the set-top box but also the visual processing device described in the above embodiments may be embedded in the television. In addition, a car ex412 having an antenna ex411 may receive a signal from a satellite ex410 or a base station ex107, etc., and reproduce a moving image on a display device such as a car navigation system ex413 included in the car ex412.

Furthermore, an image signal may be encoded and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording image signals on a DVD disc ex421 or a disc recorder for recording them on a hard disk. Also, it can also be recorded in SD card ex422. If the record player ex420 is equipped with the decoding device of the above-mentioned embodiment, the video signal recorded on the DVD disk ex421 or the SD card ex422 is interpolated and reproduced, and displayed on the monitor ex408.

In addition, the configuration of the car navigation system ex413 is conceivable, for example, in the configuration shown in FIG. ex401 etc.

Furthermore, terminals such as the above-mentioned mobile terminal ex114 may be installed in three sets of a transmitting terminal having only an encoder and a receiving terminal only having a decoder, in addition to a transmitting and receiving terminal having both an encoder and a decoder.

In this way, the visual processing device, visual processing method, and visual processing program described in the above-mentioned embodiments can be used in any of the above-mentioned machines and systems, and the effects described in the above-mentioned embodiments can be obtained.

(tenth embodiment)

A display device 720 as a tenth embodiment of the present invention will be described using FIGS. 76 to 94 .

A display device 720 shown in FIG. 76 is a display device for displaying images such as a PDP, LCD, CRT, or projector. The display device 720 is characterized in that it includes the image processing device 723 including the visual processing device described in the above embodiment, and that the profile data used for the visual processing can be switched automatically or manually. In addition, the display device 720 may be an independent device, or may be a device included in a portable information terminal such as a mobile phone, a PDA, or a PC.

(display device 720)

The display device 720 includes: a display unit 721, a drive control unit 722, an image processing device 723, a CPU 724, an input unit 725, a tuner 726, an antenna 727, a codec 728, a memory controller 729, a memory 730, an external interface (1 /F) 731, and external device 740.

The display unit 721 is a display device that displays the image information d360 read from the drive control unit 722 . The drive control unit 722 reads out the output image signal d361 output from the image processing device 723 on the display unit 721 and drives the display unit 721 under the control of the CPU 724 . More specifically, the drive control unit 722 supplies a voltage value corresponding to the value of the output image signal d361 to the display unit 721 under the control of the CPU 724 to display an image.

The image processing device 723 receives control from the CP 724, performs image processing on the input image data d372 (refer to FIG. 77 ) contained in the input image signal d362, and outputs the output image signal d361 including the output image data d371 (refer to FIG. 77 ). installation. The image processing device 723 includes the visual processing device described in the above embodiments, and is characterized in that image processing is performed using profile data. For details, the file will be described later.

The CPU 724 is a device for performing calculations related to data processing of each unit of the display device 720 and simultaneously controlling each unit. The input unit 725 is a user interface for the user to operate the display device 720, and is composed of keys, buttons, a remote controller, and the like for controlling each unit.

The tuner 726 demodulates a signal received via radio or wire, and outputs it as digital data. Specifically, the tuner 726 receives terrestrial (digital/analog) broadcasts, BS (digital/analog) and CS broadcasts, etc. via an antenna 727 or a cable (not shown). The codec 728 demodulates the digital data demodulated by the tuner 726 , and outputs the input image signal d362 input to the image processing device 723 .

The memory controller 729 controls the address, access time, and the like of the working memory 730 of the CPU, which is constituted by DRAM or the like.

The external I/F 731 is an interface for obtaining image data or profile information from an external device 740 such as a memory card 733 or a PC 735 and outputting them as an input image signal d362. The profile information is information related to profile data used for image processing. Details will be described later. The external I/F731 is composed of, for example, a memory card I/F732, a PCI/F734, a network I/F736, a wireless I/F737, and the like. In addition, the external I/F 731 does not need to include all the devices exemplified here.

The memory card I/F 732 is an interface for connecting the memory card 733 on which image data, profile data information, etc. are recorded, to the display device 720 . The PCI/F 734 is an interface for connecting a PC 735 , which is an external device such as a personal computer on which image data, profile information, etc. are recorded, to the display device 720 . The network I/F 736 is an interface for connecting the display device 720 to a network to obtain image data, profile information, and the like. The wireless I/F 737 is an interface for connecting the display device 720 to an external device via a wireless LAN or the like, and obtaining image data, profile information, and the like. In addition, the external I/F 731 is not limited to the illustration, and may be an interface for connecting the display device 720 such as USB or light, for example.

The image data or profile information obtained via the external I/F 731 is decoded by the codec 728 as necessary, and then input to the image processing device 723 as an input image signal d362.

(image processing device 723)

(1) Configuration of the image processing device 723

The configuration of the image processing device 723 will be described using FIG. 77 . The image processing device 723 is a device that performs visual processing and color processing on the input image data d372 included in the input image signal d362, and outputs an output image signal d361 including the output image data d371. Here, the input image data d372 and the output image data d371 are image data having RGB components, and the input image data d372 has (IR, IG, IB) as the components of the RGB color space; the output image data d371 has (OtR, OtG, OtB) as a component of the RGB color space.

The image processing device 723 includes: a color visual processing device 745 for performing color visual processing on the input image data d372; processing; and a profile information output unit 747 that outputs profile information SSI, SCI for specifying profile data used in color visual processing and color processing. Here, the color visually processed signal d373 is image data having RGB components, and (OR, OG, OB) are components of the RGB color space.

Hereinafter, a detailed configuration will be described in order of the profile information output unit 747 , the color visual processing device 745 , and the color processing device 746 .

(2) Profile information output unit 747 and profile information SSI, SCI

(2-1) Outline of the profile information output unit 747

The profile information output unit 747 that outputs the profile information SSI and SCI will be described using FIG. 78 .

The description file information output unit 747 is a device for respectively outputting description file information SSI and SCI to the color visual processing device 745 and the color processing device 746 (refer to FIG. 77 ). Section 750 constitutes. The environment detection unit 749 automatically detects at least a part of environment information described later, and outputs it as a detection signal Sd1. The information input unit 748 obtains the detection signal Sd1, causes the user to input environmental information other than the environmental information included in the detection signal Sd1, and outputs it as an input signal Sd2. The output control unit 750 obtains the detection information Sd1 and the input information Sd2 , and outputs the profile information SSI and SCI to the color visual processing device 745 and the color processing device 746 .

Before describing each part in detail, the profile information SSI and SCI will be described first.

(2-1) Description file information SSI, SCI

The profile information SSI and SCI are information for specifying profile data used in the color visual processing device 745 and the color processing device 746 . Specifically, the profile information SSI and SIC include profile data, tag information such as a number specifying the profile data, parameter information indicating the characteristics of processing of the profile data, and a display on the display unit 721 (see FIG. 76 ). At least one of environmental information related to the environment or the visible environment of the image displayed on the display unit 721 .

The so-called description file data is the data used by the image processing in the color visual processing device 745 or the color processing device 746, which is to store the coefficient matrix data of the transformation coefficient corresponding to the processed image data or provide the processing corresponding to the processed image data Table data (for example, 2D LUT, etc.) of the image data to be displayed.

The tag information is identification information for identifying profile data from other profile data, such as numbers assigned to each of the plurality of profile data registered in the color visual processing device 745 and the color processing device 746, etc. .

The parameter information is information representing the characteristics of the processing of the profile data, and is information obtained by quantifying, for example, the degree of processing of the profile data such as contrast enhancement processing, dynamic range compression processing, and color change conversion processing.

The so-called environmental information is information related to the visible environment for displaying the image data after image processing, for example, ambient light information such as the brightness or color temperature of the ambient light at the installation place of the display device 720, the display unit 721 The product information (for example, product number, etc.), the image size information displayed by the display unit 721, the position information related to the distance between the displayed image and the user who can see the image, user information related to the user such as the user's age and gender, etc. Information.

In addition, below, the case where the profile information SSI, SCI contains tag information is demonstrated.

(2-2) Environment detection unit 749

The environment detection unit 749 is a device that detects environment information using a sensor or the like. The environment detection unit 749 is, for example, a light sensor that detects the brightness or color temperature of ambient light, or a device that reads product information mounted on the display unit 721 wirelessly or wired (for example, a wireless tag reading device, a barcode reader, etc.). reading device, a device that reads information from a database that manages information on each part of the display device 720, etc.), a wireless or infrared sensor that measures the distance to the user, or a camera that obtains information about the user, etc. .

(2-3) Information input unit 748

The information input unit 748 is an input device for the user to input environmental information, and outputs the input environmental information as input information Sd2. The information input unit 748 may be configured by, for example, a switch and a circuit for sensing input from the switch, or may be configured by the display unit 721 or software for operating an input user interface displayed on the information input unit 748 itself. In addition, the information input unit 748 may be built in the display device 720, or may be a device that inputs information via a network or the like.

Environmental information other than the environmental information included in the detection information Sd1 is input to the information input unit 748 . For example, in the information input unit 748, the environmental information that can be input by the user is controlled based on the environmental information included in the detection signal Sd1.

In addition, the information input unit 748 can input all environmental information irrespective of the detection information Sd1. In this case, the information input unit 748 may not obtain the detection information Sd1 , or the user may input more detailed information while obtaining the detection information Sd.

(2-4) Output control unit 750

The output control unit 750 obtains the detection information Sd1 and the input information Sd2, and outputs the profile information SS1 and SC1. Specifically, the output control unit 750 selects appropriate profile data based on the environment information obtained from the detection information Sd1 and the input signal Sd2, and outputs the tag information. More specifically, the output control unit 750 selects appropriate profile data for the obtained environment information by referring to a database of associations between the selected profile data candidates and each value of the environment information.

The association between the environment information and the description file data will be further explained.

For example, when the brightness of the ambient light of the display device 720 is high, it is preferable to perform visual processing for enhancing local contrast. Therefore, in the output control unit 750 , the flag information of the profile data that enhances the local contrast is output.

Also, for example, when the distance between the user and the display device 720 is long, the viewing angle of the image displayed on the display unit 721 becomes smaller, and the image appears smaller. If the size of the viewing angle is different, the perceived brightness of the image is different. Therefore, the output control unit 750 outputs the flag information of the profile data that changes the gradation and contrast based on the size of the viewing angle. In addition, the difference in the size of the display portion 721 of the display device 720 is also an element that should be prescribed in advance for the size of the viewing angle.

Furthermore, an example of the operation of the output control unit 750 will be described.

In human vision, when the size of a displayed image becomes larger, it tends to be perceived as brighter, and it is preferable to perceive that the improvement of the dark area is suppressed. Considering this point, for example, when it is judged that the size of the image displayed on the display unit 721 has become larger based on the obtained environmental information, the color visual processing device 745 will suppress the improvement of the dark area in the entire area of the image, and make the local area larger. The flag information of the profile data processed by the contrast improvement and increase is output as the profile information SSI. Furthermore, the color processing unit 746 outputs the flag information of the profile data subjected to color processing according to the profile information SSI and other environmental information as the profile information SCI. The so-called "color processing corresponding to the profile information SSI and other environmental information" refers to, for example, performing color reproduction appropriately under the influence of the environment on an image that has been visually processed using the profile data specified by the profile information SSI. color processing etc.

In addition, in the output control unit 750, when environmental information is repeatedly obtained from the detection signal Sd1 and the input information Sd2, either one of the detection signal Sd1 and the input signal Sd2 can be preferably used.

(3) Color visual processing device 745

(3-1) Configuration of the color visual processing device 745

Use Figure 79. The configuration using the visual processing device 745 will be described. The color visual processing station 745 is characterized in that it includes a visual processing device 753 capable of executing the visual processing described in the above-mentioned embodiment, performs visual processing on the luminance component of the input image data d372, and includes a color control unit 752, The visual processing performed on the brightness component is extended to the color component.

The color visual processing unit 745 includes a first color space conversion unit 751 , a visual processing unit 753 , a color control unit 752 , and a second color space conversion unit 754 .

The first color space conversion unit 751 converts the input image data d372 in the RGB color space into luminance components and color components. For example, the first color space conversion unit 751 converts the input image data d372 in the RGB color space into a signal in the YCbCr color space. The converted signal of the luminance component is used as the input signal IS, and the signal of the color component is used as the color signals ICb and ICr.

The visual processing device 753 is a device that performs visual processing of the input signal IS of the luminance component of the input image data d372, and outputs an output signal OS. Then, the profile information SSI is input to the visual processing device 753 from the profile information output unit 747 (see FIG. 77 ), and visual processing using the profile data specified by the input profile information SSI is performed. Details of the visual processing device 753 will be described later.

The chromaticity control unit 752 receives the input color signals ICb and ICr, the input signal IS, and the output signal OS, and outputs corrected color signals OCb and OCr which are corrected color signals. For example, in the color control section 752, correction using a ratio between the input signal IS and the output signal OS is performed. More specifically, the ratio of the signal value of the output signal OS corresponding to the signal value of the input signal IS is multiplied by the signal values of the color signals ICb and ICr, and the values are used as the corrected color signal and the corrected color signal OCb and OCr respectively. value.

The second color space conversion unit 754 converts the output signal OS and the corrected color signals OCb and OCr, which are signals in the YCbCr color space, into color visually processed signals d373 in the RGB color space.

(3-2) Configuration of the visual processing device 753

As the visual processing device 753 , a visual processing device similar to the visual processing device 1 (see FIG. 1 ) described in the above embodiment is used.

The configuration of the visual processing device 753 will be described using FIG. 80 .

A visual processing device 753 shown in FIG. 80 is a visual processing device having the same configuration as the visual processing device 1 shown in FIG. 1 . The same reference numerals are assigned to parts that realize almost the same functions as those of the visual processing device 1 . The visual processing device 753 shown in FIG. 80 is different from the visual processing device 1 shown in FIG. 1 in that the description file data registration device 8 registers the obtained description file data information in the two-dimensional LUT Profile data identified by SSI. The description of other parts is the same as that of the above-mentioned embodiment, so the description is omitted.

The visual processing device 753 shown in FIG. 80 performs visual processing of the input signal IS using the profile data registered in the two-dimensional LUT4, and outputs the output signal OS.

(4) Color processing device 746

The color processing unit 746 performs color processing on the color visual processing signal d373 output from the color visual processing unit 745 using the profile data specified by the obtained profile information SCI. The profile data used by the color processing device 746 is, for example, three three-dimensional queries that provide the components (OtR, OtG, OtB) of the output image data d371 with respect to the components (OR, OG, OB) of the color visually processed signal d373 Table or transformation coefficient matrix data with 3 rows and 3 columns.

(Effect of display device 720)

(1)

In the display device 720, image processing using profile data suitable for the obtained environmental information can be performed. In particular, since profile data can be selected based on not only automatically detected environmental information but also user-input environmental information, image processing with higher visibility can be performed by the user.

In the case of using a lookup table as profile data, since image processing can be performed by referring to the table, high-speed image processing can be realized.

In the display device 720, different image processing is realized by changing the profile data. That is, different image processing can be realized without changing the hardware configuration.

In image processing using profile data, complex image processing can be easily realized because profile files can be generated in advance.

(2)

The profile information output unit 747 can output different profile information for each of the color visual processing device 745 and the color processing device 746 . Therefore, it is possible to prevent duplication of image processing in the color visual processing device 745 and color processing device 746 , or processing in which effects are offset. That is, image processing can be appropriately performed by the image processing device 723 .

(Modification)

(1)

In the above-mentioned embodiment, although it was described that the input image data d372, the output image data d371, and the color visually processed signal d373 are signals in the RGB color space, they may be signals in other color spaces. For example, each signal may be a signal represented by a YCbCr color space, a YUV color space, a Lab color space, a Luv color space, a YIQ color space, an XYZ color space, a YPbPr color space, an RGB color space, or the like.

In addition, the same applies to the signals processed in the first color space conversion unit 751 and the second color space conversion unit 754 , and it is not limited to the description in the embodiment.

(2)

In the above-mentioned embodiment, the case where the profile information SSI, SCI includes tag information has been described. Here, the operation of each unit of the image processing device 723 when the profile information SSI, SCI includes other information (profile data, parameter information, environment information, etc.) will be described.

(2-1)

When the description file information SSI, SCI contains description file data, the output control unit 750 is a device that registers and stores the description file data, or can generate the description file data, and judges whether the color The profile data used in the visual processing device 745 and the color processing device 746 are output separately.

In the color visual processing unit 745 and the color processing unit 746, image processing is performed using the obtained profile data. For example, the profile data registration device 8 of the visual processing device 753 (see FIG. 80 ) registers the profile data included in the profile information SSI in the two-dimensional LUT 4 and performs visual processing. In addition, in this case, the visual processing device 753 may not include the profile description data registration device 8 .

In this image processing device 723, since the profile data is output from the profile information output unit 747 to the color visual processing device 745 and the color processing device 746, the profile data to be used can be specified accurately. Furthermore, the storage capacity for profile data in the color visual processing device 745 and the color processing device 746 can be reduced.

(2-2)

When the profile information SSI, SCI includes parameter information, the output control unit 750 is a device having a database or the like for outputting the parameter information based on the detection information Sd1 and the input signal Sd2. This database stores the relationship between the value of the environmental information and the image processing appropriate for the environment representing the value.

In the color visual processing unit 745 and the color processing unit 746, profile data that realizes image processing with a value close to the obtained parameter information is selected, and image processing is performed. For example, when the profile data registration means 8 of the visual processing device 753 (see FIG. 80 ) selects profile data using the parameter information included in the profile information SSI, and registers the selected profile data in the two-dimensional LUT 4 , for visual processing,

In this image processing device 723, the amount of data of the profile information SSI, SCI can be reduced.

(2-3)

When the profile information SSI, SCI includes environment information, the output control unit 750 is a device that outputs the detection information Sd1 and the input signal Sd2 as profile information. Here, the output control unit 750 may also output all the environmental information obtained from the detection information Sd1 and the input signal Sd2 as profile information SSI and SCI, and may also selectively divide them into profile information SSI and profile information SCI. and output.

In the color visual processing device 745 and the color processing device 746, the appropriate description file data is selected according to the environmental information to perform image processing. For example, the profile data registration means 8 of the visual processing device 753 (refer to FIG. 80 ) refers to the database for the correlation between each value of the environment information included in the profile information SSI and the candidate of the selected profile data. etc., for the obtained environmental information, select appropriate profile data, register the selected profile data in the 2D LUT 4, and perform visual processing.

When outputting all environmental information as profile information SSI and SCI, the processing in the output control unit 750 can be reduced. In the case of selectively outputting the environmental information as profile information SSI, SCI, since the processing in each of the color visual processing device 745 and the color processing device 746 can be considered, it is possible to prevent image processing that achieves overlapping effects. , or image processing to achieve offset effects. Furthermore, in the color visual processing device 745 and the color processing device 746, since only the selected environment information is properly obtained, more precise and simple selection of profile data is possible.

(2-4)

The profile information SSI and SCI only need to contain at least one of profile data, tag information, parameter information, and environment information, or may contain them at the same time.

Furthermore, the profile information SSI and the profile information SCI do not necessarily need to be different information, and may be the same information.

(3)

The visual processing device 753 is a device including the profile data registration device 701 (refer to FIG. 9 ) described in (First Embodiment) (Modification) (7), and may be configured based on the usage profile information SSI. A means for generating new profile data from the selected profile data and the synthesis degree obtained based on the profile information SSI.

The operation of the visual processing device 753 as a modified example will be described using FIG. 81 .

In the visual processing device 753 as a modified example, new profile data is generated using profile data selected based on the profile information SSI among the profile data registered in the profile data registration unit 702 .

The profile data registration unit 702 selects the profile data 761 and the profile data 762 based on flag information and the like included in the profile information SSI. Here, the profile data 761 is the profile data for dark part improvement processing, which is profile data selected when the ambient light is weak, etc., and the profile data 762 is the profile data for local contrast improvement processing. The data is the profile data selected when the ambient light is strong or the like.

The description file generating unit 704 obtains the intensity of the ambient light in the environmental information contained in the description file information SSI, and generates a description for performing appropriate image processing in the intensity of the ambient light according to the description file data 761 and the description file data 762 file data. More specifically, the values of profile data 761 and profile data 762 are internally divided using the value of the intensity of ambient light included in the environmental information.

As described above, the visual processing device 753 as a modified example can generate new profile data and perform visual processing. In the visual processing device 753 as a modified example, even if many profile data are not registered in advance, many different visual processing can be realized by generating profile data.

(4)

The visual processing device 753 is not limited to that shown in FIG. 80 . For example, any of the visual processing device 520 (see FIG. 6 ), the visual processing device 525 (see FIG. 7 ), and the visual processing device 530 (see FIG. 8 ) described in the above embodiments may be used.

Each structure will be described using FIGS. 82 to 84 .

(4-1)

The configuration of the visual processing device 753a will be described using FIG. 82 .

A visual processing device 753a shown in FIG. 82 is a visual processing device having the same configuration as the visual processing device 520 shown in FIG. 6 . The parts that realize the same functions as those of the visual processing device 520 are denoted by the same symbols. The difference between the visual processing device 753a shown in FIG. 82 and the visual processing device 520 shown in FIG. The profile data specified by the profile information SSI and the determination result SA from the image determination unit 522 . The description of other parts is omitted since it is the same as the above-mentioned embodiment.

In this visual processing device 753a, since not only profile information SSI but also profile data can be selected based on determination result SA, more appropriate visual processing can be performed.

(4-2)

The configuration of the visual processing device 753b will be described using FIG. 83 .

The visual processing device 753b shown in FIG. 83 is a visual processing device having the same configuration as the visual processing device 525 shown in FIG. 7 . The parts that realize the same functions as those of the visual processing device 525 are denoted by the same symbols. The difference between the visual processing device 753b shown in FIG. 83 and the visual processing device 525 shown in FIG. The profile information SSI and profile data determined by the input result SB from the input device 527 . Descriptions of other parts are omitted since they are the same as those in the above-mentioned embodiment.

In this visual processing device 753b, since not only the profile information SSI but also the profile data can be selected based on the input result SB, more appropriate visual processing can be performed.

(4-3)

The configuration of the visual processing device 753c will be described using FIG. 84 .

A visual processing device 753c shown in FIG. 84 is a visual processing device having the same configuration as the visual processing device 530 shown in FIG. 8 . The parts that realize the same functions as those of the visual processing device 530 are denoted by the same symbols. The difference between the visual processing device 753c shown in FIG. 84 and the visual processing device 530 shown in FIG. The profile data specified by the profile information SSI, the determination result SA from the image determination unit 522 , and the input result SB from the input device 527 . Descriptions of other parts are omitted since they are the same as those in the above-mentioned embodiment.

In this visual processing device 753b, since not only the profile information SSI but also profile data can be selected based on the judgment result SA and the input result SB, more appropriate visual processing can be performed.

(5)

Among the units of the display device 720 described in the above-mentioned embodiments, the parts that realize the same functions may be realized by common software.

For example, the input unit 725 (refer to FIG. 76 ) of the display device 720 may be the information input unit 748 of the profile information output unit 747, the input device 527 of the visual processing device 753b (refer to FIG. 83 ), or the visual processing device 753. (Refer to FIG. 84 ) and other devices that are also used as the input device 527.

Also, the description of the profile data registration unit 8 of the visual processing device 753 (see FIG. 80 ), the profile data registration unit 521 of the visual processing device 753a (see FIG. 82 ), and the visual processing device 753b (see FIG. 83 ) The document data registration unit 526, the profile data registration unit 531 of the visual processing device 753c (see FIG. 84 ), and the like may be devices provided outside the image processing device 723 (see FIG. The external device 740 implements.

Also, the profile data registered in each profile data registration unit or profile data registration device may be registered in each unit in advance, or may be obtained from the external device 740 or the tuner 726 .

In addition, each profile data registration unit or profile data registration device may also be used as a storage device for storing profile data in the color processing device 746 .

In addition, the profile information output unit 747 may be connected to the outside of the image processing device 723 or the outside of the display device 720 by wire or wirelessly.

(6)

The profile data registration unit 8 of the visual processing device 753 (see FIG. 80 ), the profile data registration unit 521 of the visual processing device 753a (see FIG. 82 ), and the profile data registration unit 521 of the visual processing device 753b (see FIG. 83 ) The unit 526, the profile data registration unit 531 of the visual processing device 753c (see FIG. 84 ), etc. may be devices capable of outputting profile information of profile data used for visual processing.

For example, the profile data registration unit 8 of the visual processing device 753 (see FIG. 80 ) outputs the profile information of the profile data registered in the two-dimensional LUT 4 . The outputted profile information is input to, for example, the color processing device 746 and is used by the color processing device 746 to select profile data.

In this way, even when profile data other than the profile data specified by the profile information SSI is used by the visual processing device 753 , the color processing device 746 can determine the profile data used by the visual processing device 753 . Therefore, it is further possible to prevent the image processing in the color visual processing device 745 and the color processing device 746 from being repeated processing or offsetting processing.

(7)

In the image processing device 723 , instead of the profile information output unit 747 , a user input unit for allowing the user to input may be provided.

FIG. 85 shows an image processing device 770 as a modified example of the image processing device 723 (see FIG. 77 ). The image processing device 770 is characterized by including a user input unit 772 for allowing a user to input. In the image processing device 770 , parts that realize substantially the same functions as those of the image processing device 723 are denoted by the same reference numerals and descriptions thereof are omitted.

The user input unit 772 outputs profile information SSI and SCI to the color visual processing device 745 and the color processing device 746 .

The user input unit 772 will be described using FIG. 86 .

The user input unit 772 is composed of a part for the user to input, and a part for outputting profile information SSI and SCI based on the input information.

The portion for the user to input is constituted by, for example, a brightness input unit 775 for inputting a user's preferred brightness and an image quality input unit 776 for inputting a user's preferred image quality.

The shading input unit 775 is composed of, for example, a switch for inputting the state of light in the displayed image, a switch for inputting the state of light in the environment where the image is displayed, and outputs the input result as the first input result Sd14. The switch for inputting the state of light in the displayed image is for inputting, for example, backlighting or frontlighting in the image, presence or absence of a flash during shooting, and the state of a macro program used during shooting. Here, the macro program refers to a program for controlling the imaging device according to the state of the subject. The switch for inputting the state of light in the environment where the image is displayed may be, for example, a switch for inputting the brightness, color temperature, etc. of the environment.

The image quality input unit 776 is a switch for inputting a user's favorite image quality, for example, a switch for inputting different visual effects such as default, dynamic, and classic. The image quality input unit 776 outputs the input result as the second input result Sd13.

The part that outputs the profile information SSI and SCI based on the input information is constituted by the output control unit 777 . The output control unit 777 obtains the first input result Sd14 and the second input result Sd13, and outputs the values of the profile information Sd14 and the second input result Sd13. More specifically, the profile information SSI and SCI of the profile data associated with the value of the 1st input result Sd14 and the 2nd input result Sd13 are output.

Furthermore, the operation of the output control unit 777 will be specifically described. For example, when "dynamic" "backlight mode" is input through the shading input unit 775 and the image quality input unit 776, the profile information SSI includes the profile data of the profile data for improving dark areas by backlighting. Information output. On the other hand, in the profile information SCI, the profile information of the profile data that is not subjected to the improved color processing of the backlight portion is output to optimize the image processing of the image processing device 770 as a whole.

Effects produced by the file through the image processing means 770 are described below.

In the image processing device 770 , image processing can be performed using appropriate profile data according to the user's wishes. Furthermore, since different profile information SSI and SCI can be output to the color visual processing device 745 and the color processing device 746 , it is possible to prevent each image processing from being duplicated or offsetting. Furthermore, since different profile information SSI and SCI are respectively output to the color visual processing device 745 and the color processing device 746, the amount of information of the profile information SSI and SCI that should be considered in each device can be reduced, making it easier to The selection of profile data can be carried out accordingly.

(8)

The image processing device 723 may also be a device that separates attribute information contained in the input image signal d362, selects profile data based on the separated attribute information, and performs image processing.

(8-1) Configuration of image processing device 800

FIG. 87 shows an image processing device 800 as a modified example of the image processing device 723 . The image processing device 800 is characterized by including a separation unit 801 for separating attribute information d380 from an input image signal d362, and outputs profile information SSI, SCI based on the separated attribute information d380.

The image processing device 800 shown in FIG. 87 includes: a separation unit 801 that separates image data d372 and attribute information d380 from an input image signal d362; and an attribute determination unit 802 that separates profile information SSI, SCI output; color visual processing means for visual processing based on input image data d372 and profile information SSI; and color processing means 746 for color processing based on color visual processing signal d373 and profile information SC1. In addition, about the part which has basically the same function as the above-mentioned embodiment, the same code|symbol is attached|subjected, and description is abbreviate|omitted.

The separation unit 801 separates the input image data d372 and the attribute information d380 from the input image signal d362. The attribute information d380 is information arranged in the header portion of the input image signal d362, etc., and is information related to the attribute of the input image signal d362. The separation unit 801 separates the attribute information d380 by reading the input image signal d362 of only a predetermined number of bits from the beginning. In addition, the attribute information d380 may be arranged at the end of the input image signal d362. Alternatively, it may be arranged in a separable state accompanying the flag information in the input image signal d362.

FIG. 88 shows an example of the format of the input image signal d362 including attribute information d380. In the input image signal d362 as shown in FIG. 88, at the beginning of the data, content information as attribute information d380 is arranged, while input image data d372 is arranged thereafter.

The content information is an attribute related to the overall content of the input image data d372, and includes the title, production company, director, production year, genre, and producer designation attributes of the input image data d372. Here, the genre refers to information related to the genre of the content, for example, information such as SF, action movie, drama movie, and horror movie. The so-called producer-specified attributes refer to information related to display characteristics specified by the content producer, such as information such as motion and horror.

The attribute judging unit 802 outputs the profile information SSI and SCI based on the separated attribute information d380.

Using FIG. 89 , the configuration of the attribute determination unit 802 will be described. The attribute determination unit 802 includes an attribute detection unit 806 , an attribute input unit 805 , and an output control unit 807 .

The attribute detection unit 806 detects content information included in the attribute information d380, and outputs it as detection information Sd3.

The attribute input unit 805 is means for the user to input content information. The attribute input unit 805 obtains the detection signal Sd3 , adds information that updates the information included in the detection signal Sd3 or that is not included in the detection information Sd3 , and outputs it as input information Sd4 .

Here, the attribute input unit 805 is an input device for the user to input content information, and outputs the input content information as input information Sd4. The attribute input unit 805 may be composed of, for example, a switch and a circuit for sensing input from the switch, or may be composed of software for operating an input interface displayed on the display unit 721 or the attribute input unit 805 itself. Furthermore, it may be built into the display device 720 or may be a device that inputs information via a network or the like.

In addition, in the attribute input unit 805, the content information that can be input by the user may be controlled based on the content information included in the detection information Sd3. For example, when the attribute detection unit 806 detects that the type of input image data d372 is “movie”, only items related to animation (for example, animation director, animation title, etc.) may be input in the attribute input unit 805 .

The output control unit 807 obtains the detection signal Sd3 and the input information Sd4, and outputs the profile information SSI and SCI.

The detailed operation of the output control unit 807 will be described. The output control unit 807 obtains the contents of the attribute information d380 based on the detection information Sd3 and the input information Sd4. Furthermore, profile data for performing appropriate image processing on an image having the attribute information d380 is determined. For example, the output control unit 807 determines the profile data by referring to a database storing the association between each item of the attribute information d380 and the profile data. Here, the output control unit 807 may give priority to either information when obtaining different values of content information on the same item from the detection information Sd3 and the input information Sd4. For example, the input signal Sd4 may always be preferentially utilized.

Furthermore, the output control unit 807 transmits at least one of the determined profile data, flag information such as a number specifying the determined profile data, and parameter information indicating a characteristic of processing of the determined profile data. The description file information SSI, SCI output.

A detailed description of the profile information SSI and SCI is omitted since it is the same as the above-mentioned embodiment.

In the color visual processing device 745 and the color processing device 746, the profile data used for image processing is judged based on the profile information SSI and SCI, and image processing is performed. For example, when the profile information SSI, SCI includes profile data, image processing is performed using the profile data. When the profile information SSI, SCI includes flag information and parameter information, image processing is performed using the profile data specified by each information.

In addition, the output control unit 807 may output each item of content information obtained from the detection information Sd3 and the input information Sd4 as profile information SSI, SCI. In this case, the color visual processing device 745 or the color processing device 746 specifies the profile data used for image processing based on the profile information SSI and SCI, and performs image processing.

(8-2) Effect

(1)

Image processing using appropriate profile data can be performed based on content information at the time of content creation. Therefore, image processing can be performed in consideration of the content producer's wishes.

More specifically, according to the title, production company, etc., it is possible to judge the tendency of the overall image brightness, color temperature, etc., and perform image processing to convert the overall image brightness, color temperature, etc. In addition, according to the attributes specified by the producer, etc., it is possible to display the image display desired by the producer.

(2)

The attribute determination unit 802 includes not only an attribute detection unit 806 for automatically detecting content information, but also an attribute input unit 805 for manually inputting content information. Therefore, even if there is a disadvantageous phenomenon in the detection of the content information, the content information can be appropriately input through the attribute input unit 805, and appropriate image processing can be performed. Furthermore, through the attribute input unit 805, the preferences of the user can be reflected in the image processing. For example, the preference of the user can be reflected by making animation an image that enhances the joyful atmosphere, and video image that is clear. Further, the content information of the corrected image can be corrected in a digital reprinting manner.

(3)

The profile information SSI, SIC can be indicated to the color visual processing device 745 and the color processing device 746, respectively. Therefore, even when multiple values such as "action movie and horror movie" are specified as the type of profile information, the color visual processing device 745 can properly perform a visual processing for a part with a lot of action like an action movie. For video processing, color processing is appropriately performed by the color processing device 746 on parts that give psychological impact like horror movies.

In addition, since different profile information SSI and SCI are output for each of the color visual processing device 745 and the color processing device 746, the information amount of the profile information SSI and SCI that should be considered in each device can be reduced, and the process can be simplified. The selection of profile data can be carried out accordingly.

(8-3) Variations

(1)

Content information that has already been obtained can be reused. In this case, even if all the information is not obtained again, the stored content information can be used to perform image processing to determine the profile data.

(2)

The image processing device 800 may not include either the attribute input unit 805 or the attribute detection unit 806 . Furthermore, the separation unit 801 does not necessarily need to be provided inside the image processing apparatus 800 .

(3)

The description file information SSI and the description file information SCI do not necessarily need to be different information, and may be the same information.

(4)

The attribute information d380 may include information other than content information. Specifically, as long as it includes scene attribute information that is an attribute related to a part of the input image data, shooting attribute information about the environment in which the input image signal d362 is generated, and information about the medium before the input image signal d362 is obtained on the display device 720 The playback attribute information, the recording attribute information related to the medium and the device for recording the input image signal d362, the profile attribute related to the profile data used in image processing, etc. may be used. Hereinafter, these will be specifically described.

In addition, in the following description, although the case where attribute information d380 includes each of scene attribute information, shooting attribute information, playback attribute information, recording attribute information, and profile attribute information will be described separately, these items including content information Information may be included in the attribute information d380 all at the same time or some combination thereof. In this case, the effects of these information can be further enhanced.

(4-1) Scene attribute information

(4-1-1)

FIG. 90 shows the format of the input image signal d362 including scene attribute information as attribute information d380. In the input image signal d362 as shown in FIG. 90, scene attribute information is arranged in units of scenes of the input image data d372. The scene attribute information is arranged in a separable state from the input image data d372, for example, along with flag information or the like.

The scene attribute information is information of the scene content of the input image data d372 following the description file. For example, the scene attribute information describes a file by a combination of items such as "lightness and darkness", "object", "action", "scene summary", etc. etc. Combination description files for such items. In addition, these are just examples of scene attribute information, and are not limited to these. For example, content such as news, sports, family drama, action movie, etc. may be specified as the "scene summary".

The image processing device that performs image processing on the input image signal d362 including the scene attribute information is the same as the device that makes the image processing device 800 correspond to the scene attribute information.

The separating unit 801 (see 87) separates the attribute information d380 based on the format shown in FIG. 90 .

The attribute detection unit 806 (see FIG. 89 ) detects the scene attribute information included in the attribute information d380, and outputs detection information Sd3. The attribute input unit 805 allows the user to input scene attribute information.

The output control unit 807 (see FIG. 89 ) obtains the detection information Sd3 and the input information Sd4, and outputs the profile information SSI and SCI. For example, the output control unit 807 determines the color visual processing device 745 and the color processing device 745 by referring to a database or the like storing the correlation between each item of the scene attribute information obtained from the detection information Sd3 and the input information Sd4 and the description file data. The profile data used by the 746.

The detailed description of the profile information SSI and SCI is the same as that of the above-mentioned embodiment, so the description is omitted. In addition, the profile information SSI and SCI may also include scene attribute information. In this case, the color visual processing unit 745 and the color processing unit 746 select profile data used for image processing from the acquired scene attribute information, and perform image processing.

In addition, since the operation of each part of the image processing apparatus 800 is the same as the case where the attribute information d380 includes content information, description thereof will be omitted.

(4-1-2)

According to the present invention, the same effects as those described in the above-mentioned embodiment are obtained. Hereinafter, characteristic effects of this modification will be described.

Image processing using appropriate profile data can be performed based on the scene attribute information. Therefore, image processing can be performed in consideration of the intention of the content producer.

Scene attribute information is arranged for each scene of the input image data d372 as needed. Therefore, in more detail, image processing can be switched, and image processing can be performed more appropriately.

For example, when the scene attribute information "darkness, forest, and landscape" is obtained through the detection information Sd3 and input information Sd4, the output control unit 807 outputs the profile information SSI specifying "profile data for improving the dark part of the shadow", and at the same time The profile information SCI specifying "the profile data of which the stored color correction of green is performed and the stored color correction of skin color is not performed" is output.

In addition, for example, when the scene attribute information "brightness, person and close-up" is obtained through the detection information Sd3 and input information Sd4, the output control unit 807 will designate the description file data of "strengthening the dark part of the person and suppressing the improvement of the dark part of the background". The description file information SSI of "is output, and at the same time, the description file information SCI outputting the description file information specifying "the description file data without white balance adjustment and stored color correction of skin color".

Also, for example, in the case where the scene attribute information "person and drama" is obtained from the detection information Sd3 and the input information Sd4, the main processing object in the image is a person. Therefore, the output control unit 807 designates, for the color visual processing device 745, that the contrast improvement of the skin-colored and low-brightness areas be performed, and that the contrast improvement of the other low-brightness areas be not performed, so that the profile data Profile information for SSI output. On the other hand, the color processing device 746 outputs profile information SCI that designates profile data that performs stored correction of skin color and weakens correction corresponding to stored colors such as green.

Profile data can be selected based on not only scene attribute information automatically detected by the attribute detection unit 806 but also scene attribute information input by the user. Therefore, it is possible to further improve the main image quality for the user.

In addition, in the moving scene of the person, if the background is a series of scenes in which the direction of the sun changes gradually, the scene attribute information may be added for each scene, and the scene attribute information may be added only for the first scene. . In addition, only the scene attribute information is first added to the first scene, and then only the change information of brightness compared with the first scene or object change information may be added as the scene attribute information to the consecutive scenes. In this way, it is possible to suppress flickering and sudden changes in image quality during image processing of moving images.

(4-2) Shooting attribute information

(4-2-1)

FIG. 91 shows the format of an input image signal d362 including shooting attribute information as attribute information d380. In the input image signal d361 shown in FIG. 91 , imaging attribute information is arranged in the header portion of the input image signal d362. In addition, the imaging attribute information is not limited to this, and may be arranged in a state where accompanying flag information can be separated from the input image data d372, for example.

The shooting attribute information is information describing the shooting state of the file input image data d372 thereafter. For example, shooting attribute information is a file described by a combination of items such as "position and direction", "date", "time", "shooting device information", and the like. "Position and direction" are information obtained by GPS or the like at the time of shooting. "Shooting device information" is the information of the device at the time of shooting, and stores information such as presence or absence of flash, aperture, shutter speed, presence or absence of macro photography (close-up photography). For example, it may be information for specifying a macro program (a program for executing a combination of controls such as flash presence or absence, aperture, door opening speed, etc.) used at the time of shooting.

The image processing device that performs image processing on the input image signal d362 including the shooting property information is the same as the device that makes the image processing device 800 correspond to the shooting property information.

The separating unit 801 (see FIG. 87 ) separates the attribute information d380 based on the format shown in FIG. 91 .

The attribute detection unit 806 (see FIG. 89 ) detects the imaging attribute information included in the attribute information d380, and outputs detection information Sd3. The attribute input unit 805 allows the user to input imaging attribute information.

The output control unit 807 (see FIG. 89 ) obtains the detection information Sd3 and the input information Sd4, and outputs the profile information SSI and SCI. For example, the output control unit 807 determines the color visual processing device 745 and the color processing device 746 by referring to a database storing the correlation between each item of imaging attribute information obtained from the detection information Sd3 and the input information Sd4 and the description file data. Profile data used in . The detailed description of the profile information SSI and SCI is the same as that of the above-mentioned embodiment, so the description is omitted.

In addition, the profile information SSI, SCI may also include imaging attribute information. In this case, the color visual processing unit 745 and the color processing unit 746 select profile data used for image processing from the obtained imaging attribute information, and perform image processing.

In addition, since the operation of each part of the image processing apparatus 800 is the same as the case where the attribute information d380 includes content information, description thereof will be omitted.

(4-2-2)

According to the present invention, the same effects as those described in the above-mentioned embodiment are obtained. Hereinafter, characteristic effects of this modification will be described.

Image processing using appropriate profile data can be performed based on the shooting attribute information. Therefore, image processing can be performed in consideration of the intention of the content producer.

For example, "direction of the sun", "season", and "weather" in the environment in which the input image data d372 is generated are obtained from items such as "position and direction", "date", "time", and "photographing device information". , "color of sunlight", "whether there is a flash" and other information can analyze the shooting status of the subject (such as whether it is facing the light or backlighting, etc.). Furthermore, for the analyzed shooting conditions, appropriate description file data can be used for image processing.

Profile data can be selected based on not only the imaging attribute information automatically detected by the attribute detection unit 806 but also the imaging attribute information input by the user. Therefore, it is possible to further improve the visual quality for the user.

(4-3) Play attribute information

(4-3-1)

FIG. 92 shows the format of the input video signal d362 including playback attribute information as attribute information d380. In the input image signal d362 shown in FIG. 92, playback attribute information is arranged in the header portion of the input image signal d362. In addition, the playback attribute information is not limited to this, and may be arranged in a state separable from the input image data d372, for example, with accompanying flag information and the like.

The playback attribute information is playback attribute information related to the medium before the input image signal d362 is obtained in the display device 720 , and in particular, information about the playback format used to obtain the input image signal d362 . For example, a value indicating any one of "digital terrestrial broadcasting", "analog terrestrial broadcasting", "digital satellite broadcasting", "analog satellite broadcasting", and "Internet broadcasting" is stored in broadcast attribute information.

The image processing device that performs image processing on the input image signal d362 including broadcast property information is the same as the device that makes the image processing device 800 correspond to the broadcast property information.

The separating unit 801 (see FIG. 87 ) separates the attribute information d380 based on the format shown in FIG. 92 .

The attribute detection unit 806 (see FIG. 89 ) detects the playback attribute information included in the attribute information d380, and outputs the detection information Sd3. The attribute input unit 805 allows the user to input playback attribute information.

The output control unit 807 (see FIG. 89 ) obtains the detection information Sd3 and the input information Sd4, and outputs the profile information SSI and SCI. For example, the output control unit 807 determines the color visual processing device 745 and the color processing device 746 to use by referring to a database that stores the association between the playback attribute information obtained from the detection information Sd3 and the input information Sd4 and the profile data. The profile data for . The detailed description of the profile information SSI and SCI is the same as that of the above-mentioned embodiment, so the description is omitted.

In addition, the profile information SSI, SCI may also include broadcast attribute information. In this case, the color visual processing device 745 and the color processing device 746 select the profile data used for image processing from the acquired playback attribute information, and perform image processing.

In addition, since the operation of each part of the image processing apparatus 800 is the same as the case where the attribute information d380 includes content information, description thereof will be omitted.

(4-3-2)

According to the present invention, the same effects as those described in the above-mentioned embodiment are obtained. Hereinafter, characteristic effects in this modified example will be described.

Image processing using appropriate profile data can be performed based on playback attribute information. For example, it is possible to correct the influence of the playback path on the image, and to perform image processing in consideration of the intention of the broadcasting station side.

More specifically, for example, with respect to images obtained by terrestrial analog broadcasting, satellite analog broadcasting, etc., selection of profile data is performed that does not excessively enhance noise during transmission. In this way, for an image in which a subject exists in a night scene, image processing and the like can be performed using profile data that brightens the subject while maintaining the brightness of the night scene area.

The profile data can be selected based on not only broadcast attribute information automatically detected by the attribute detection unit 806 but also broadcast attribute information input by the user. Therefore, it is possible to further improve the visual quality for the user.

(4-4) Record attribute information

(4-4-1)

Fig. 93 shows the format of an input image signal d362 including recording attribute information as attribute information d380. In the input image signal d362 shown in FIG. 93, recording attribute information is arranged in the header portion of the input image signal d362. In addition, the record attribute information is not limited to this, and for example, may be arranged in a state where the accompanying flag information and the input image data d372 are separable.

The recording attribute information is information related to the medium and the device on which the input image signal d362 is recorded. For example, the recording attribute information includes "year" in which the input image signal d362 is recorded, "supplier" of the recording medium and the device, "product information" for identifying the recording medium and the device, and the like.

The image processing device that performs image processing on the input image signal d362 including the recording attribute information is the same as the device that makes the image processing device 800 correspond to the recording attribute information.

The separating unit 801 (see FIG. 87 ) separates the attribute information d380 based on the format shown in FIG. 93 .

The attribute detection unit 806 (see FIG. 89 ) detects the record attribute information included in the attribute information d380, and outputs detection information Sd3. The attribute input unit 805 allows the user to input record attribute information.

The output control unit 807 (see FIG. 89 ) obtains the detection information Sd3 and the input information Sd4, and outputs the profile information SSI and SCI. For example, the output control unit 807 determines the color visual processing device 745 and the color processing device 746 to use by referring to a database or the like that stores the correlation between the record attribute information obtained from the detection information Sd3 and the input information Sd4 and the profile data. The profile data for . The detailed description of the profile information SSI and SCI is the same as that of the above-mentioned embodiment, so the description is omitted.

In addition, the profile information SSI, SCI may also include record attribute information. In this case, the color visual processing unit 745 and the color processing unit 746 select profile data used for image processing from the obtained record attribute information, and perform image processing.

In addition, since the operation of each part of the image processing apparatus 800 is the same as the case where the attribute information d380 includes content information, description thereof will be omitted.

(4-4-2)

According to the present invention, the same effects as those described in the above-mentioned embodiment are obtained. Hereinafter, characteristic effects of this modification will be described.

Image processing is performed using the appropriate profile data based on the record attribute information. For example, when the "provider" is a camera manufacturer that specializes in color processing, the profile information SCI is output so that the color processing device 746 does not perform much color processing. Also, for example, for input image data d372 recorded by a filter or the like, profile information SCI is output in such a manner that color processing is performed in consideration of the characteristics of the color expression area of the filter. In this way, the influence of the recording medium and the recording device on the image is corrected, and image processing can be carried out in consideration of the producer's intention.

The profile data can be selected based on not only the record attribute information automatically detected by the attribute detection unit 806 but also the record attribute information input by the user. Therefore, it is possible to further improve the visual quality for the user.

(4-5) Description file attribute information

(4-5-1)

Fig. 94 shows the format of an input image signal d362 including profile attribute information as attribute information d380. In the input image signal d362 shown in FIG. 94, the profile attribute information is arranged in the header part of the input image signal d362. In addition, the profile attribute information is not limited to this, and may be arranged in a state where accompanying flag information and the like are separable from the input image data d372, for example.

The profile attribute information is information for specifying profile data, for example, information for specifying profile data recommended by an imaging device or the like that generates the input image data d372. The profile attribute information includes at least one of profile data, tag information such as a number specifying the profile data, and parameter information indicating characteristics of processing of the profile data. The profile data, flag information, and parameter information are the same as those described in the description of the profile information SSI and SCI in the above-mentioned embodiment.

The profile data specified by the profile attribute information is profile data for performing any one of the following image processing (a) to image processing (c). The image processing (a) is image processing for judging that it is suitable for the input image data d372 in an imaging device or the like that generates the input image data d372. The image processing (b) is image processing for correcting a difference in characteristics between the display unit of the imaging device and the display device of the standard module, in addition to the image processing (a). The image processing (c) is image processing for correcting the difference in characteristics between the display unit of the imaging device and the display device 720 (see FIG. 76 ), in addition to the image processing (a).

Furthermore, the profile attribute information includes processing flag information related to whether or not the input image data d372 included in the input image signal d362 is data that has been image-processed by a camera or the like.

The image processing device that performs image processing on the input image signal d362 including profile attribute information is the same as the device that associates the image processing device 800 with the profile attribute information.

The separating unit 801 (see FIG. 87 ) separates the attribute information d380 based on the format shown in FIG. 94 .

The attribute detection unit 806 (see FIG. 89 ) detects the profile attribute information included in the attribute information d380, and outputs detection information Sd3. The attribute input unit 805 allows the user to input profile attribute information.

The output control unit 807 (see FIG. 89 ) obtains the detection information Sd3 and the input information Sd4, and outputs the profile information SSI and SCI. The description file information SSI and SCI, regardless of the form of the description file attribute information (any one of the description file data, tag information, and parameter information), all use any one of the description file data, tag information, and parameter information output.

Hereinafter, the operation of the output control unit 807 will be described in detail.

The output control unit 807 judges whether the information specifying the profile data in the profile attribute information obtained from the detection information Sd3 or the input information Sd4 is directly output as the profile information SSI, SCI.

For example, when the profile data is specified by the input information Sd4, it is judged as "output" regardless of the profile attribute information.

For example, when the profile attribute information includes information specifying profile data to be subjected to image processing (a) or image processing (c), and the processing flag information indicates "no processing", it is judged as "output".

In all other cases, it is judged as "not output".

For example, if the profile attribute information includes information specifying the profile data to be subjected to image processing (a), and the processing flag information indicates "processed", the output control unit 807 sets The image processing profile data of the device 745 and the color processing device 746 determine the information, which is output as profile information SSI and SCI.

For example, if the profile attribute information includes information for specifying the profile data to be subjected to image processing (b), and the processing flag information indicates "no processing", in addition to image processing (a), it will be used for Information for specifying the profile data subjected to image processing for correcting a difference in characteristics between the display device of the standard module and the display device 720 is output as profile information SSI, SCI.

For example, if the profile attribute information includes information to determine the profile data for image processing (b), and the processing flag information indicates "processing", it will be used to perform image processing on the profile data for image processing (b). The determined information is output as profile information SSI, SCI, and this image processing is for correcting the difference in characteristics between the display device of the standard module and the display device 720 .

For example, when the profile attribute information includes information specifying the profile data to be subjected to image processing (c), and the processing flag information indicates "processed", the output control unit 807, for the color visual processing device 745 and The color processing device 746 outputs, as profile information SSI and SCI, information for specifying the profile data for image processing, which is used for the communication between the display unit of the photographing device and the display device 720. Correct for differences in device characteristics.

In addition, these processes are just an example and are not limited to these.

In addition, since the operation of each part of the image processing apparatus 8000 is the same as the case where the attribute information d380 includes content information, description thereof will be omitted.

(4-5-2)

According to the present invention, the same effects as those described in the above-mentioned embodiment are obtained. Hereinafter, characteristic effects of the present invention will be described.

Based on the profile attribute information, image processing using appropriate profile data can be performed. For example, image processing using profile data recommended by the photographer can be performed. Furthermore, it is possible to perform a display similar to the image confirmed by the display unit on the shooting side. Therefore, image processing can be performed in consideration of the producer's intention.

Profile data can be selected based on not only the profile attribute information automatically detected by the attribute detection unit 806 but also the profile attribute information input by the user. Therefore, it is possible to further improve the visual quality for the user.

(the eleventh embodiment)

An imaging device 820 as an eleventh embodiment of the present invention will be described using FIGS. 95 to 103 .

The imaging device 820 shown in FIG. 95 is a still camera, a video camera, or the like that captures images, and is an imaging device that captures images. The imaging device 820 is characterized in that it includes the image processing device 832 including the visual processing device described in the above embodiment, and the profile data used for the visual processing is switched automatically or manually. In addition, the imaging device 820 may be an independent device, or may be a device included in a portable information terminal such as a mobile phone, a PDA, or a PC.

(camera 820)

The imaging device 820 includes: an imaging unit 821, an image processing device 832, a display unit 834, a CPU 846, an illumination unit 848, an input unit 850, a security determination unit 852, a codec 840, a storage controller 842, a memory 844, and an external interface ( I/F) 854, external device 856.

The imaging unit 821 is a unit that captures images and outputs the input image signal d362, and is composed of a lens 822, an aperture and shutter unit 824, a CCD 826, an amplifier 828, an A/D conversion unit 830, a CCD control unit 836, and an information detection unit 838. .

The lens 822 forms an image of the subject on the CCD 826 . The aperture and shutter unit 824 is a mechanism for controlling exposure by changing the passing range or passing time of the light beam passing through the lens 822 . The CCD826 is an image sensor that photoelectrically converts the image of the subject and outputs it as an image signal. Amplifier 828 is a device for amplifying the image signal output from CCD 826 . The A/D converter 830 is a device for converting the analog image signal amplified by the amplifier 828 into a digital image signal. The CCD control unit 836 is a device that controls the timing of driving the CCD 826 . The information detection unit 838 is a device that detects information such as auto-focus, aperture, and exposure based on the digital image signal, and outputs the information to the CPU 846 .

The image processing device 832 is the same image processing device as the image processing device 723 described using FIG. 77 in (tenth embodiment). The image processing device 832 receives the control from the CPU 846, performs image processing on the input image data d372 (see FIG. 96 ) included in the input image signal d362, and outputs the output image signal d361 including the output image data d371 (see FIG. 96 ). device. The image processing device 832 includes the visual processing device described in the above embodiments, and is characterized in that it performs image processing using profile data. The detailed configuration will be described later using FIG. 96 .

The display unit 834 is a device for displaying the output image signal d361 output by the image processing device 832 as, for example, a thumbnail. The display unit 834 is often composed of an LCD, but is not limited as long as it is a device that displays images such as a PDP, CRT, or projector. In addition, the display unit 834 is not only built into the imaging device 820, but may also be connected via a wired or wireless network or the like. Furthermore, the display unit 834 may be connected to the CPU 846 via the image processing device 832 .

The CPU 846 is connected via the image processing device 832, the codec 840, the memory controller 842, the external I/F 854, and the bus. The light emission information of the safety judgment unit 852 is received, and the lens 822, the aperture and shutter unit 824, the CCD control unit 836, the image processing device 832, the lighting unit 848, the input unit 850, the safety judgment unit 852 or the like are simultaneously executed. A device for controlling each part connected to the bus.

The illuminating unit 848 is a flash or the like that emits illuminating light to illuminate a subject.

The input unit 850 is a user interface for the user to operate the imaging device 820 , and is a key, a knob, a remote control, and the like for controlling each unit.

The security judging unit 852 is a part that judges security information obtained from the outside and controls the image processing device 832 via the CPU.

The codec 840 is a compression circuit that compresses the output image signal d361 from the image processing device 832 using JPEG, MPEG, or the like.

The memory controller 842 controls the address and access timing of the memory 844 of the CPU, which is constituted by DRAM or the like.

The memory 844 is constituted by DRAM or the like, and is used as storage for work during image processing and the like.

The external I/F 854 is used to output the output image signal d361, or the output image signal d361 compressed by the codec 840, to an external device 856 such as a memory card 859, PC861, etc. The profile information and the like, which are information related to the processed profile data, are output to the image processing device 832 as the input image signal d362. The profile information is the same as described in "Tenth Embodiment", and the external I/F 854 is composed of, for example, a memory card IF/858, PCII/F860, network I/F 862, wireless I/F 864, and the like. In addition, the external I/F 854 does not need to include all the components exemplified here.

The memory card I/F 858 is an interface for connecting a memory card 859 for recording image data, profile information, and the like to the imaging device 820 . PCI/IF 860 is an interface for connecting PC 861 , an external device such as a personal computer, which records image data, profile information, and the like, to imaging device 820 . The network I/F 862 is an interface for connecting the imaging device 820 to a network, and transmitting and receiving image data, profile information, and the like. The wireless I/F 864 is an interface for connecting the imaging device 820 to an external device via a wireless LAN or the like, and for transmitting and receiving image data, profile information, and the like. In addition, the external I/F 854 is not limited to the illustration, and may be, for example, an interface for connecting to the imaging device 820 such as USB or optical fiber.

(image processing device 832)

FIG. 96 shows the configuration of the image processing device 832 . The image processing device 832 has the same configuration as that of the image processing device 723 . In FIG. 96 , the parts having the same functions as those of the image processing device 723 are denoted by the same reference numerals.

The image processing device 832 includes: a color visual processing device 745 for performing color visual processing on the input image data d372; performing color processing; and a profile information output unit 747 that outputs profile information SSI, SCI for specifying profile data used in color visual processing and color processing.

The operation of each unit has been described in the (tenth embodiment), so detailed description is omitted.

In addition, in (tenth embodiment), it is described that the environment information included in the profile information SSI, SCI is "information related to a visual environment for displaying image data after image processing". It may also be information related to the environment in which the shooting was performed.

(Effect of camera 820)

The imaging device 820 includes an image processing device 832 similar to the image processing device 723 described in (the tenth embodiment). Therefore, the same effect as that of the display device 720 (see FIG. 76 ) including the image processing device 723 can be achieved.

(1)

The imaging device 820 includes a profile information output unit 747 (see FIG. 78 ), and can perform image processing using profile data suitable for the acquired environmental information. In particular, since profile data can be selected based on not only automatically detected environment information but also user input environment information, it is possible to perform image processing that is highly visible to the user.

In the case of using a lookup table as profile data, since image processing can be performed by referring to the table, high-speed image processing can be realized.

In the imaging device 820, different image processing is realized by changing the profile data. That is, different image processing can be realized without changing the hardware configuration.

In image processing using profile data, since profile data can be generated in advance, complex image processing can be easily realized.

(2)

The profile information output unit 747 of the image processing device 832 can output different profile information for each of the color visual processing device 745 and the color processing device 746 . Therefore, it is possible to prevent duplication of image processing in the color visual processing device 745 and color processing device 746 , or processing in which effects cancel each other out. That is, processing can be appropriately performed by the image processing device 832 .

(3)

The imaging device 820 includes a display unit 834, and can perform imaging while checking the image processed image. Therefore, the impression of the image at the time of shooting can be brought close to the impression of displaying the image after shooting.

(Modification)

In the imaging device 820, the same modifications as those described in relation to the image processing device 723 or the visual processing device 753 (see FIG. 79 ) are possible in the above-mentioned embodiments. Hereinafter, a modified example of characteristics in the imaging device 820 will be described.

(1)

In the description of (the tenth embodiment), it was explained that the information input unit 748 (see FIG. 78 ) of the profile information output unit 747 is an input device for the user to input environment information.

In the imaging device 820, the information input unit 748 may be a device capable of inputting other information besides the environment information, or after changing it. For example, the information input unit 748 may be a user-input information capable of inputting user-preferred brightness or image quality.

In addition to the information input unit 748, or after modification, the profile information output unit 747 as this modification may include the user input unit 772 ( Refer to Figure 86). A detailed description of the user input unit 772 is omitted since it has already been described in the above-mentioned embodiment.

The output control unit 750 (refer to FIG. 78 ), which is the profile information output unit 747 of this modification, outputs the profile information based on the user input information input from the user input unit 772 and the environment information detected by the environment detection unit 749. SSI, SCI output. More specifically, the output control unit 750 as this modification outputs profile information SSI, SCI by referring to a database of profile data associated with values of user input information and environment information.

In this way, in the imaging device 820, image processing using appropriate profile data according to the user's preference can be realized.

(2)

Among the units of the imaging device 820 described in the above-mentioned embodiments, the parts that realize the same functions may be realized by common software.

For example, the input unit 850 (see FIG. 95 ) of the imaging device 820 may be the same as the information input unit 748 of the profile information output unit 747, the user input unit 772 of the profile information output unit 747 as a modified example, the visual processing The input device 527 of the device 753b (refer to FIG. 83 ), the input device 527 of the visual processing device 753c (refer to FIG. 84 ), and the like are combined devices.

Also, the description of the profile data registration unit 8 of the visual processing device 753 (see FIG. 80 ), the profile data registration unit 521 of the visual processing device 753a (see FIG. 82 ), and the visual processing device 753b (see FIG. 83 ) The document data registration unit 526, the description file data registration unit 531 of the visual processing device 753c (see FIG. 84 ), etc. may be external devices of the image processing device 832, or may be realized by the memory 844 or the external device 856, for example.

Also, the profile data registered in each profile data registration unit or profile data registration device may be registered in each unit in advance, or may be obtained from the external device 856 .

In addition, each profile data registration unit or profile data registration device may also be used as a storage device for storing profile data in the color processing device 746 .

In addition, the profile information output unit 747 may be connected to the outside of the image processing device 832 or the outside of the imaging device 820 by wire or wirelessly.

(3)

The image processing device 832 of the photographing device 820 may also use the profile information used to determine the profile data used in image processing as the input image data d372 or the input image data d372 after image processing together with the output image signal d361 devices that output together.

This will be described using FIGS. 97 to 101 .

(3-1) Configuration of image processing device 886

The configuration of an image processing device 886 as a modified example will be described using FIG. 97 . The image processing device 886 is a device that performs image processing on the input image data d372, displays the processing results on the display unit 834, and simultaneously adds profile information d401 of profile data suitable for image processing to the input image data d372, and outputs the .

The image processing device 886 includes a color visual processing device 888 , a color processing device 889 , a recommended profile information extracting unit 890 , and a profile information adding unit 892 .

The color visual processing device 888 has basically the same function as the color visual processing device 745 described in (the tenth embodiment), and performs visual processing of the input image data d372 similarly to the color visual processing device 745, and converts Color visual processing signal d373 output.

The difference between the color visual processing device 888 and the color visual processing device 745 is that the visual processing device included in the color visual processing device 888 is the same as the visual processing device 1 (refer to FIG. 1 ), the visual processing device 520 (refer to FIG. 6 ), visual processing device 525 (refer to FIG. 7 ), and visual processing device 530 (refer to FIG. 8 ) are basically the same visual processing device, and the visual processing device will recommend description file information SSO outputs this. The details of the recommended profile information SSO will be described later.

The color processing device 889 has basically the same function as the color processing device 746 described in (the tenth embodiment), and performs color processing on the color visually processed signal d373 and the output image data d371 similarly to the color processing device 746. output.

The difference between the color processing device 889 and the color processing device 746 is that the color processing device 889 outputs profile information of profile data used for color processing as recommended profile information SCO. The details of the recommended profile information SCO will be described later.

The recommended profile information extracting unit 890 extracts recommended profile information SSO and SCO, and sets these information as profile information d401.

The profile information adding unit 892 adds profile information d401 to the input image data d372, and outputs it as an output image signal d361.

FIG. 98 shows an example of the format of the output image signal d361 after the profile information adding unit 892 adds the profile information d401.

In FIG. 98(a), profile information d401 is arranged at the head of output image signal d361, and input image data d372 is arranged thereafter. In such a format, all image processing of the input image data d372 is performed using the profile information d401 at the top. Therefore, the profile information d401 only needs to be arranged at one place in the output image signal d361, and the ratio of the profile information d401 in the output image signal d361 can be reduced.

In FIG. 98(b), profile information d401 is arranged for each divided input image data d372. In such a format, different profile data are used in each image processing of the divided input image data d372. Therefore, for example, image processing using profile data can be performed for each scene of the input image data d372, and image processing can be accurately performed.

In addition, in the case of a series of scenes that change continuously, first, scene attribute information is added to the first scene, and then, in a plurality of scenes, change information of brightness and darkness or object change information compared only with the first scene is used as the scene attribute information, it is possible to suppress flickering or a sudden change in image quality when image processing is performed on a moving image.

(3-2) Recommended description file information SSO, SCO

The recommended profile information SSO, SCO is information for specifying each profile data, including label information such as profile data, a number for specifying profile data, and parameter information indicating characteristics of processing profile data at least one of the The profile data, flag information, and parameter information are the same as described in the description of the profile information SSI and SCI.

Also, the profile data specified by the recommended profile information SSO, SCO is profile data for performing any one of the following image processing (a) to image processing (c). The image processing (a) is the visual processing judged by the color visual processing device 888 to be suitable for the input image data d372, or the color processing judged by the color processing device 889 to be suitable for the color visual processing signal d373. Here, in the image processing (a), the image processing "judged to be suitable" is, for example, the image processing used in the color visual processing device 888 and the color processing device 889 , respectively. Image processing (b) is image processing for performing image processing for correcting a difference in characteristics between the display unit 834 of the imaging device 820 and the display device of the standard module, in addition to the image processing (a). The image processing (c) is for correcting the difference in characteristics between the display unit 834 of the imaging device 820 and the display device displaying the image captured by the imaging device 920 in addition to the image processing (a). Image Processing.

When the color visual processing device 888 and the color processing device 889 do not know the display characteristics of the display unit 834 for checking the image at the time of shooting, the profile information of the profile data for image processing (a) is used as Recommended description file information SSO, SCO.

The color visual processing device 888 and the color processing device 889 know the display characteristics of the display unit 834 that displays the image captured by the imaging device 820, but they do not know the display device that displays the image captured by the imaging device 820 (for example, for In the case of the display characteristics of the display device 720, etc.) that display captured and recorded images, the profile information of the profile data subjected to image processing (b) is used as recommended profile information SSO, SCO.

The color visual processing device 888 and the color processing device 889 know the display characteristics of the display unit 834 displaying the image captured by the imaging device 820 and the display device displaying the image captured by the imaging device 820 (for example, the image captured and recorded). In the case of the display characteristics of the display device 720 that performs the display, the profile information of the profile data that performs the image processing (c) is used as the recommended profile information SSO, SCO.

In addition, the above processing is just an example, and the image processing selected in each case is not limited to this.

(3-3) Effects of the image processing device 886

In the image processing device 886, the output image signal d361 including the profile information d401 is output. Therefore, in the apparatus for obtaining the output image signal d361, when image processing is performed on the input image data d372 including the output image signal d361, image processing using appropriate profile data can be performed.

In addition, the profile information d401 includes profile information of profile data for performing any one of image processing (a) to image processing (c). Therefore, for example, the image confirmed on the display unit 834 of the imaging device 820 can be brought close to the image displayed on the display device that obtains the output image signal d361. That is, in the display device that has obtained the profile information of the profile data subjected to the image processing (b), by performing the image processing (b) on the output image signal d361, the difference between the display device and the standard module is performed at the same time Image processing for correction is performed so that the displayed image can be brought close to the image confirmed by the display unit 834 . In addition, in the display device that has obtained the profile information of the profile data subjected to the image processing (c), by performing the image processing (c) on the output image signal d361, the display image can be made to be the same as that confirmed by the display unit 834. The image is close.

(3-4) Variations

The output image signal d361 may further include processing flag information indicating whether or not the input image data d372 included in the output image signal d361 has been image-processed by the image processing device 886 . In this way, the display device that obtains the output image signal d361 can determine whether the input image data d372 included in the output image signal d361 is image-processed data. Therefore, it is possible to prevent excessive image processing in the display device or image processing that cancels out the effect.

(2) Image processing device

In the description of the image processing device 886 described above, "the profile information adding unit 892 adds and outputs the profile information d401 to the input image data d372" has been described.

Here, the profile information adding unit 892 may add the profile information d401 to the output image data d371 that is the result of image processing the input image data d372, and output the result.

FIG. 99 shows an image processing device 894 as a modified example of the image processing device 886 . Parts that realize the same functions as those of the respective parts of the image processing device 886 are assigned the same reference numerals. The image processing device 894 shown in FIG. 99 is characterized in that the profile information d401 is added to the output image data d371 by the profile information adding part 892.

In addition, in the image processing device 894 in FIG. 99 , the profile data specified by the recommended profile information SSO and SCI is used to perform any of the following image processing (a') to image processing (c'). Description file data for image processing. The image processing (a') is the visual processing judged by the color visual processing unit 888 to be suitable for the input image data d372, or the color processing judged by the color processing unit 889 to be suitable for the color visual processing signal d373. Here, in the image processing (a'), the image processing "judged to be suitable" is, for example, the image processing used by the color visual processing device 888 and the color processing device 889 respectively. The image processing (b') is image processing for correcting the difference in characteristics between the display unit 834 of the imaging device 820 and the display device of the standard module. The image processing (c') is image processing for correcting a difference in characteristics between the display unit 834 of the imaging device 820 and the display device that displays the image captured by the imaging device 920 .

The description of the operation of other parts is omitted.

In the image processing device 894, for example, the display device that has obtained the profile information of the profile data subjected to the image processing (a') reproduces the input image data d372 by performing inverse transformation of the image processing (a'). . In addition, in the display device that has obtained the profile information of the profile data for which the image processing (a') is performed, an instruction not to execute the above color visual processing or color processing may be issued thereafter. And, by performing image processing for correcting the difference from the display device of the standard module in the display device that has obtained the profile information of the profile data that performs the above-mentioned image processing (b'), it is possible to make the display image and The image confirmed by the display unit 834 is approached. In addition, the display device that has obtained the profile information of the profile data subjected to the image processing (c') performs the image processing (c') so that the displayed image approximates the image confirmed by the display unit 834.

In addition, the image processing device 894 may output the output image signal d361 including the processing flag information described in (4-1) above. In this way, the display device that obtains the output image signal d361 judges that the output image data d371 included in the output image signal d361 is image-processed data, thereby preventing excessive image processing or image processing with counterproductive effects in the display device.

(3) Image processing device

The image processing device 886 and the image processing device 894 described above may include a user input unit 897 similar to the user input unit 772 (refer to FIG. Enter the device reflected in the selection of the profile data.

100 to 101 show an image processing device 896 and an image processing device 898 including a user input unit 897 . The operation of the user input unit 897 is the same as the operation of the user input unit 772 described in (10th Embodiment) (Modification) (7), and thus detailed description thereof will be omitted.

Among the image processing device 896 and the image processing device 898, the color visual processing device 888 is connected to the visual processing device 753 (refer to FIG. 80 ), the visual processing device 753a (refer to FIG. 82 ), and the visual processing device 753b (refer to FIG. FIG. 83 ) and visual processing device 753c (see FIG. 84 ) are basically the same visual processing device, and include a visual processing device capable of outputting recommended profile information SSO. That is, profile information SSI is obtained from the user input unit 897, and recommended profile information SSO can be output at the same time.

Furthermore, among the image processing device 896 and the image processing device 898 , the color processing device 889 can obtain the profile information SCI from the user input unit 897 and can output the recommended profile information SCO at the same time.

In this way, the image processing device 896 and the image processing device 898 optimize the profile data used for image processing at the time of shooting while viewing the image displayed on the display unit 834 . At this time, since the profile information SSI and SCI can be given to the color visual processing device 888 and the color processing device 889 , it is possible to prevent the effects of processing in each device from being excessive or offset. In addition, through the user input unit 897, finer image processing adjustments can be performed. Furthermore, since only the profile information SSI and SCI necessary for the color visual processing device 888 and the color processing device 889 are provided, the processing information in each device can be reduced and the processing can be performed more easily.

(4)

(4-1)

In the imaging device 820, the image processing device 832 (see FIG. 96) may be a device that obtains security information and switches profile data used for image processing according to the security information. Here, the security information is information indicating whether photography is permitted or the degree of permission in the photography environment of the photography device 820 .

FIG. 102 shows an image processing device 870 as a modified example of the image processing device 832 . The image processing device 870 performs image processing on the input image data d372 and outputs the output image data d371 in the same manner as the image processing device 832 . The difference between the image processing device 870 and the image processing device 832 is that it includes a security information input unit 872 for obtaining security information in an environment captured by the image processing device 870 . In addition, the same code|symbol is attached|subjected to the part common to the image processing apparatus 832, and description is abbreviate|omitted.

The security information input unit 872 is mainly composed of a receiving device that obtains security information through, for example, an input device that allows the user to directly input security information, wireless, infrared, or wired. Furthermore, the security information input unit 872 outputs profile information SSI and SCI based on the obtained security information.

Here, the profile information SSI and SCI are information for specifying the profile data, information such as the profile data and the number specifying the profile data, and parameter information indicating characteristics of processing of the profile data. at least one of the Profile data, tag information, and parameter information are the same as those described in the above-mentioned embodiment.

The output description file information SSI, SCI, when the degree of photographing permission indicated by the security information is higher, the profile data for shooting with higher quality is determined, and when the degree of photographing permission is lower, the profile data for further photographing is determined. Profile data for low-quality captures is determined.

The operation of the image processing device 870 will be described in more detail with reference to FIG. 103 .

FIG. 103 is an explanatory diagram for explaining the operation of the imaging device 820 including the image processing device 870 in the imaging control area 880 for controlling imaging.

In the photographing control area 880 , a photographing prohibited object 883 that prohibits photographing is arranged. The prohibited subject is, for example, a person, a book, or the subject of a portrait right or a copyright. In the shooting control area 880, a security information transmitting device 881 is provided. The security information sending device 881 transmits security information through wireless, infrared rays, and the like.

The imaging device 820 within the imaging control area 880 receives security information through the security information input unit 872 . The security information input unit 872 is a program for judging the photographing permission indicated by the security information. Furthermore, the security information input unit 872 refers to a database or the like storing the association between the value of the degree of photographing permission and the profile data, and obtains the profile information for specifying the profile data corresponding to the value of the degree of photographing permission. SSI, SCI output. For example, in the database, higher imaging permission values are associated with profile data for imaging with higher quality.

In more detail, for example, when the photographing device 820 receives security information with a low degree of photographing permission from the security information transmitting device 881, the security information input unit 872 uses a function to smooth the vicinity of the center of the image or the main area of the image ( The profile information SSI determined by profile data such as gray scale reduction) is output to the color visual processing device 745 . Furthermore, the security information input unit 872 outputs, to the color processing device 746 , profile information SCI for specifying profile data for decolorizing an image. In this way, it is impossible to shoot with an appropriate image quality, and it is possible to protect portrait rights or copyrights.

(4-2) Others

(1)

After receiving the security information, the security information input unit 872 can not only switch the profile data according to the security information, but also stop some functions of the image processing device 870 or the imaging device 820 .

(2)

The security information input unit 872 that has received the security information further obtains the user's authentication information from the input unit 850 of the photographing device 820, etc., and if the user is permitted to photograph, the profile data that can ease the degree of photographing permission can be determined. The description file information SSI, SCI output.

The authentication information of the user is, for example, identification information identified from the user's fingerprint, iridescence, and the like. The security information input unit 872 having obtained the authentication information refers to the database of users permitted to be photographed, and judges whether or not the identified user is a user permitted to be photographed. In addition, at this time, it is also possible to determine the degree of photographing permission based on the user's fee information and the like, and the higher the degree, the higher the quality of photographing.

In addition, the security information may be notified of information for identifying the photographing device 820 permitted to photograph.

(3)

Description file information SSI, SCI, may contain security information. In this case, the color visual processing unit 745 and the color processing unit 746 which have obtained the profile information SSI and SCI select profile data based on the security information.

(4)

The security information input unit 872 may also be used together with the security determination unit 852 .

(Note 1)

The present invention (in particular, the inventions described in the fourth to seventh embodiments) may also have the following expressions. In addition, among the remarks of the attached form described in this column ([1st Remark]), it is subordinate to the remark described in the 1st Remark.

(Contents of Remark No. 1)

(Note 1)

A visual processing device having:

an image area segmentation mechanism, which divides the input image signal into multiple image areas;

The gradation transformation characteristic deriving means is a mechanism for deriving the gradation transformation characteristic for each of the above-mentioned image regions, and uses the distance between the target image region to be derived from the gradation transformation characteristic and the surrounding image regions of the above-mentioned target image region. a grayscale characteristic, deriving the above-mentioned grayscale transformation characteristic of the above-mentioned object image region; and

and a gradation processing unit that performs gradation processing of the image signal based on the derived gradation conversion characteristic.

(Note 2)

According to the visual processing device described in Note 1,

The above-mentioned grayscale transformation characteristic is the grayscale transformation curve,

The gradation transformation characteristic deriving means includes: a histogram creation means for creating a histogram using the gradation characteristics; and a gradation curve creation means for creating the gradation transformation curve based on the created histogram.

(Note 3)

According to the visual processing device described in Note 1,

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from a plurality of gradation conversion tables for performing gradation processing on the image signal,

The gradation processing means includes the plurality of gradation conversion tables as two-dimensional LUTs.

(Note 4)

According to the visual processing device described in Note 3,

The two-dimensional LUT saves the plurality of gradation transformation tables in the order of monotonically increasing or monotonically decreasing the values of the gradation-processed image signal corresponding to the value of the selection signal among all the values of the image signal. .

(Note 5)

According to the visual processing device indicated in Note 3 or 4,

The above-mentioned 2D LUT can be changed by registering profile data.

(Note 6)

According to the visual processing device described in any one of Notes 3 to 5,

The value of the selection signal is derived as a feature amount of an individual selection signal of the selection signal derived for each image area between the target image area and the peripheral image area.

(Note 7)

According to the visual processing device described in any one of Notes 3 to 5,

The selection signal is derived based on a gradation characteristic feature quantity as a derived feature quantity using a gradation characteristic between the target image region and the peripheral image region.

(Remark 8)

According to the visual processing device described in any one of Notes 3 to 7,

The gradation processing unit is a gradation processing execution unit that executes gradation processing of the target image region using the gradation conversion table selected by the selection signal; degree correction mechanism, which has:

a correction means for correcting the gradation of the target pixel based on the gradation processing table selected for an image region including the target pixel to be corrected and an image region adjacent to the image region including the target pixel .

(Remark 9)

According to the visual processing device described in any one of Notes 3 to 7,

The gradation processing means includes: a correction means for correcting the selection signal to derive a correction selection signal for selecting a gradation processing table for each pixel of the image signal; and a gradation processing execution means for using The gradation conversion table selected by the correction selection signal executes gradation processing of the image signal.

(Remark 10)

A visual processing method having:

The image area segmentation step is to divide the input image signal into a plurality of image areas;

The step of deriving the gradation transformation characteristic is a step of deriving the gradation transformation characteristic for each of the above-mentioned image regions, using the distance between the target image region to be derived from the gradation transformation characteristic and surrounding image regions of the above-mentioned target image region. The grayscale characteristics of the above-mentioned grayscale transformation characteristics of the above-mentioned object image area are derived; and

In the gradation processing step, gradation processing of the image signal is performed based on the derived gradation transformation characteristics.

(Note 11)

According to the visual processing method described in Note 10 above,

The above grayscale transformation characteristic is the grayscale transformation curve

The gradation transformation characteristic deriving step includes: a histogram creation step of creating a histogram using the gradation characteristic; and a gradation curve creation step of creating the gradation transformation curve based on the created histogram.

(Note 12)

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from a plurality of gradation conversion tables for performing gradation processing on the image signal,

The gradation processing step is a gradation processing execution step of performing gradation processing on the target image region using the gradation conversion table selected by the selection signal; Steps to correct the degree, with:

In the correcting step, the gradation of the target pixel is corrected based on the gradation processing table selected for an image region including the target pixel to be corrected and an image region adjacent to the image region including the target pixel.

(Note 13)

The visual processing method according to claim 10,

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from a plurality of gradation conversion tables for performing gradation processing on the image signal,

The above-mentioned gradation processing step has: a correction step of correcting the above-mentioned selection signal, deriving a correction selection signal for selecting a gradation processing table for each pixel of the above-mentioned image signal; and a gradation processing execution step of using the above-mentioned correction The gradation conversion table selected by the selection signal is executed to perform gradation processing of the image signal.

(Note 14)

A visual processing program, a visual processing method in which a computer executes the following steps,

The visual processing method has:

The image area segmentation step is to divide the input image signal into a plurality of image areas;

The gradation transformation characteristic deriving step is a step of deriving a gradation transformation region for each of the image regions, using the gradations of the target image region to be derived from the gradation transformation characteristic and the surrounding image regions of the target image region. characteristic, deriving the above-mentioned gray-scale transformation characteristic of the above-mentioned object image region;

In the gradation processing step, gradation processing of the image signal is performed based on the derived gradation transformation characteristics.

(Note 15)

According to the visual processing procedure described in Note 14,

The above-mentioned grayscale transformation characteristic is a grayscale transformation curve,

The gradation transformation characteristic deriving step includes: a histogram creation step of creating a histogram using the gradation characteristic; and a gradation curve creation step of creating the gradation transformation curve based on the created histogram.

(Note 16)

According to the visual processing procedure shown in Remark 14,

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from a plurality of gradation conversion tables for performing gradation processing on the image signal,

The gradation processing step is a gradation processing execution step of performing gradation processing on the target image region using the gradation conversion table selected by the selection signal; Steps to correct the degree, with:

In the correcting step, the gradation of the target pixel is corrected based on the gradation processing table selected for an image region including the target pixel to be corrected and an image region adjacent to the image region including the target pixel.

(Note 17)

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from a plurality of gradation conversion tables for performing gradation processing on the image signal,

The above-mentioned gradation processing step has: a correction step of correcting the above-mentioned selection signal, deriving a correction selection signal for selecting a gradation processing table for each pixel of the above-mentioned image signal; and a gradation processing execution step of using the above-mentioned correction The gradation conversion table selected by the selection signal is executed to perform gradation processing of the image signal.

(Note 18)

According to the visual processing device described in Note 1,

The gradation processing means has a parameter output means for outputting curve parameters of a gradation transformation curve for performing gradation processing on the image signal based on the gradation transformation characteristics,

Perform grayscale processing on the image signal using the grayscale transformation curve determined based on the grayscale transformation determination and the curve parameters.

(Note 19)

A visual processing device as described in Remark 18,

The parameter output unit is a look-up table that stores the relationship between the gray scale transformation characteristics and the curve parameters.

(Remark 20)

The visual processing device described in Remark 18 or 19

The curve parameter includes a value of the grayscale-processed image signal corresponding to a predetermined value of the image signal.

(Remark 21)

A visual processing device according to any one of the above-mentioned curve parameters 18-21,

The curve parameter includes a slope of the gradation transformation curve of the image signal in a predetermined interval.

(Note 22)

A visual processing device according to any one of Notes 18 to 21,

The above-mentioned curve parameters include the coordinates of at least one point that the above-mentioned grayscale transformation curve passes through.

(Note 23)

A visual processing device having:

A spatial processing mechanism is a mechanism that performs spatial processing on an input image signal for each of a plurality of image regions and derives a spatially processed signal. weighting the difference in gradation characteristics between the image region and surrounding image regions of the target image region, performing a weighted average of the gradation characteristics between the target image region and the surrounding image regions; and

A visual processing means that performs visual processing of the target image area based on the gradation characteristics of the target image area and the spatially processed signal.

(Note 24)

According to the visual processing device described in note 23,

The weighting becomes smaller as the absolute value of the difference in the gradation characteristics is larger.

(Note 25)

The weight becomes smaller as the distance between the target image area and the peripheral image area increases.

(Note 26)

A visual processing device according to any one of Notes 23 to 25,

The above-mentioned image area is composed of a plurality of pixels,

The gradation characteristics between the target image area and the peripheral image area are determined as feature quantities of pixel values constituting each image area.

(Description of Remark No. 1)

The visual processing device described in Remark 1 includes image region segmentation means, gradation transformation characteristic derivation means, and gradation processing means. The image area dividing means divides the input image signal into a plurality of image areas. The gradation transformation characteristic deriving mechanism is a mechanism for deriving the gradation transformation characteristic for each image region, using the gradation between the target image region to be derived from the gradation transformation characteristic and the surrounding image regions of the target image region Properties, export the grayscale transformation properties of the object image area. The gradation processing means performs gradation processing of the image signal based on the derived gradation transformation characteristics.

Here, the gradation transformation characteristics are characteristics of gradation processing for each image region. The gradation characteristics refer to, for example, pixel values such as luminance and lightness of each pixel.

In the visual processing device of the present invention, when judging the gradation transformation characteristic of each image region, not only the gradation characteristic of each image region but also the image region of a wide area including surrounding image regions are used. Grayscale characteristics are judged. Therefore, the effect of spatial processing can be added to the gradation processing of each image area, and gradation processing that further improves the visual effect can be realized.

In the visual processing device described in Note 2, according to the visual processing device described in Note 1, the gradation transformation characteristic is a gradation transformation curve. The gradation transformation characteristic deriving means includes: a histogram creation means for creating a histogram using the gradation characteristic; and a gradation curve creation means for creating a gradation transformation curve based on the created histogram.

Here, the so-called histogram is, for example, a distribution corresponding to the gradation characteristics of pixels included in the target image area and the peripheral image area. The gradation curve creating means uses, for example, an integrated curve obtained by integrating histogram values as a gradation transformation curve.

In the visual processing device of the present invention, when creating a histogram, not only the gradation characteristics of each image region but also the gradation characteristics of a wide area including surrounding image regions can be used to create the histogram. Therefore, the number of divisions of the image signal can be increased, the size of the image area can be reduced, and the occurrence of blurred outlines in gradation processing can be suppressed. In addition, it prevents the boundary of the image area from being conspicuous and unnatural.

The visual processing device described in Note 3, according to the visual processing device described in Note 1,

The gradation conversion characteristic is a selection signal for selecting one gradation conversion table from among a plurality of gradation conversion tables for performing gradation processing on an image signal. The grayscale processing mechanism has multiple grayscale transformation tables as 2D LUTs.

Here, the gradation conversion table is, for example, a look-up table (LUT) or the like that stores pixel values of an image signal after gradation processing is performed on pixel values of the image signal.

The selection signal has, for example, a value assigned to one gradation conversion table selected from values divided into a plurality of gradation conversion tables. The gradation processing means refers to the two-dimensional LUT from the value of the selection signal and the pixel value of the image signal, and outputs the pixel value of the image signal subjected to gradation processing.

In the visual processing device of the present invention, gradation processing is performed with reference to a two-dimensional LUT. Therefore, the speed of gradation processing can be increased. Furthermore, since one gradation conversion table is selected from a plurality of gradation conversion tables and gradation processing is performed, appropriate gradation processing can be performed.

The visual processing device shown in Remark 4, according to the visual processing device described in Remark 3, the 2-dimensional LUT, among all the values of the image signal, selects the value of the grayscale processed image signal corresponding to the value of the signal The sequence of monotonically increasing or monotonically decreasing holds multiple grayscale transformation tables.

In the visual processing device of the present invention, for example, the value of the selection signal indicates the degree of gradation conversion.

In the visual processing device described in Remark 5, according to the visual processing device described in Remark 3 or 4, the 2-dimensional LUT is changed by registration of profile data.

Here, the profile data is data stored in a two-dimensional LUT, and has, for example, the pixel value of a gradation-processed image signal as an element.

In the visual processing device of the present invention, by changing the two-dimensional LUT, various changes can be made to the characteristics of gradation processing without changing the hardware configuration.

The visual processing device described in Remark 6 is the visual processing device described in any one of Remarks 3 to 5, wherein the value of the selected signal is derived as a feature quantity of the individually selected signal, and the value of the selected signal is derived as A selection signal derived for each image area between the target image area and the peripheral image area.

Here, the feature amount of an individual selected signal is, for example, an average value (simple average or weighted average), a maximum value, or a minimum value of the selected signals derived for each image region.

In the visual processing device of the present invention, the selection signal corresponding to the target image area is derived as the feature amount of the selection signal corresponding to the image area of the wide area including the peripheral image area. Therefore, the effect of spatial processing on the selection signal can be added, and the boundary of the image region can be prevented from being conspicuous and unnatural.

In the visual processing device described in Note 7, according to the visual processing device described in any one of Notes 3 to 5, the selection signal is derived based on the grayscale characteristic between the image area to be used and the surrounding image area The grayscale characteristic feature quantity of the feature quantity is derived.

Here, the gradation characteristic feature amount is, for example, the average (simple average or weighted average), maximum value, or minimum value of the gradation characteristics in a wide area between the target image region and the peripheral image region.

In the visual processing device of the present invention, the selection signal corresponding to the target image region is derived based on the gradation characteristic feature value corresponding to the wide-area image region including the peripheral image region. Therefore, a spatial processing effect on the selection signal can be added, and the boundary of the image region can be prevented from being conspicuous and unnatural.

The visual processing device described in Note 8 is the visual processing device described in any one of Notes 3 to 7, wherein the grayscale processing mechanism has a grayscale processing executing mechanism and a correction mechanism. The gradation processing actuator executes gradation processing of the target image region using the gradation conversion table selected by the selection signal. The correction mechanism is a mechanism for correcting the gradation of the image signal after gradation processing, based on the difference between the image region including the target pixel to be corrected and the adjacent image region including the target pixel. The selected grayscale processing table corrects the grayscale of the target image.

Here, the adjacent image region may be the same image region as the surrounding image region when deriving the gradation transformation characteristic, or may be a different image region. For example, the adjacent image regions are selected as three image regions that are adjacent to the image region including the target pixel and whose distance from the target pixel is relatively short.

The correction means, for example, corrects the gradation of the image signal that has been gradation-processed using the same gradation conversion table for each target image region. The correction of the target pixel is performed by expressing the influence of each gradation transformation table selected for the adjacent image region, for example, based on the position of the target pixel.

In the visual processing device of the present invention, the gradation of an image signal can be corrected for each pixel. Therefore, it is possible to further improve the visual effect by preventing the boundary of the image region from being conspicuous and unnatural.

The visual processing device described in Note 9 is the visual processing device described in any one of Notes 3 to 7, the grayscale processing mechanism has a correction mechanism and a grayscale processing execution mechanism, and the correction mechanism corrects the selection signal , to derive a correction selection signal for selecting a gradation processing table for each pixel of an image signal. The gradation processing actuator executes gradation processing of the image signal using the gradation conversion table selected by the correction selection signal.

The correction means corrects the selection signal derived for each target image region based on the selection signal derived for the image position and the image region adjacent to the target image region, and derives the selection signal for each pixel.

In the visual processing device of the present invention, the selection signal can be derived for each pixel. Therefore, it is possible to further prevent the boundary of the image region from being conspicuous and unnatural, and to improve the visual effect.

The visual processing method described in Remark 10 includes: an image region segmentation step, a grayscale transformation characteristic derivation step, and a grayscale processing step. In the image area division step, the input image signal is divided into a plurality of image areas. The step of deriving the gradation transformation characteristic is a step of deriving the gradation transformation characteristic for each image region, using the gradation between the target image region to be derived from the gradation transformation characteristic and the surrounding image regions of the target image region Properties, export the grayscale transformation properties of the object image area. In the gradation processing step, gradation processing of the image signal is performed based on the derived gradation transformation characteristics.

Here, the gradation transformation characteristics are characteristics of gradation processing for each image region. The gradation characteristics are, for example, pixel values such as luminance and lightness of each pixel.

In the visual processing method of the present invention, when judging the gradation transformation characteristics of each image region, not only the gradation characteristics of each image region but also the image region of a wide area including surrounding image regions can be used. The grayscale characteristics are judged. Therefore, the grayscale processing of each image area can be added with the effect of space processing, and then the grayscale processing with higher visual effect can be realized.

The visual processing method described in Remark 11 is based on the visual processing method described in Remark 10, and the grayscale transformation characteristic is a grayscale transformation curve. The step of deriving the gray-scale transformation characteristics includes: a histogram making step, using the gray-scale characteristics to make a histogram; and a gray-scale curve making step, making a gray-scale transformation curve based on the made histogram.

Here, the so-called histogram is, for example, a distribution corresponding to the gradation characteristics of pixels included in the target image area and the peripheral image area. The gradation curve creating means uses, for example, an integrated curve obtained by integrating histogram values as a gradation transformation curve.

In the visual processing device of the present invention, when creating a histogram, not only the gradation characteristics of each image region but also the gradation characteristics of a wide area including surrounding image regions can be used to create the histogram. Therefore, the number of divisions of the image signal can be increased, the size of the image area can be reduced, and the occurrence of blurred outlines in gradation processing can be suppressed. In addition, it prevents the boundary of the image area from being conspicuous and unnatural.

The visual processing method described in Remark 12 is based on the visual processing method described in Remark 10, and the grayscale transformation characteristic is used to select one grayscale from a plurality of grayscale transformation tables for grayscale processing of image signals. Select signal for degree transformation table. Furthermore, the gradation processing step includes a gradation processing execution step and a correction step. In the gradation processing execution step, the gradation processing of the target image area is executed using the gradation conversion table selected by the selection signal. The correction step is a step of correcting the gradation of the image signal after the gradation processing, based on the image region of the target pixel to be corrected, the image region adjacent to the image region including the target pixel, and the selected Grayscale processing table, to correct the grayscale of the object pixel.

Here, the gradation conversion table is, for example, a look-up table (LUT) or the like that stores pixel values of an image signal after gradation processing is performed on pixel values of the image signal. The adjacent image region may be the same image region as the surrounding image region when the gradation transformation characteristic is derived, or may be a different image region. For example, the adjacent image regions are selected as three image regions having a shorter distance from the target pixel among image regions adjacent to the image region including the target pixel.

The selection signal has, for example, a value assigned to one gradation conversion table selected from values assigned to each of a plurality of gradation conversion tables. In the gradation processing step, from among the value of the selection signal and the pixel value of the image signal, the pixel value of the image signal subjected to gradation processing is output with reference to the LUT. In the correction step, the gradation of the image signal subjected to gradation processing using the same gradation conversion table is corrected for each target image region. The correction of the target pixel is performed, for example, according to the position of the target pixel so as to express the influence of each gradation transformation table selected for the adjacent image area.

In the visual processing method of the present invention, gradation processing is performed with reference to LUT. Therefore, high-speed gradation processing can be achieved. Furthermore, since one gradation conversion table is selected from a plurality of gradation conversion tables to perform gradation processing, appropriate gradation processing can be performed. Furthermore, the gradation of the image signal can be corrected for each pixel. Therefore, the boundary of the image area is prevented from being conspicuous and unnatural, and the visual effect can be improved.

The visual processing method described in Remark 13 is based on the visual processing method described in Remark 10, and the grayscale transformation characteristic is used to select one of the grayscale transformation tables for performing grayscale processing on the image signal. The selection signal of the grayscale conversion table. Furthermore, the gradation processing step includes a correction step and a gradation processing execution step. In the correcting step, the selection signal is corrected, and the corrected selection signal for selecting the grayscale processing table for each pixel of the image signal is derived. In the gradation processing execution step, the gradation processing of the target image area is executed using the gradation conversion table selected by the correction selection signal.

Here, the gradation conversion table is, for example, a look-up table (LUT) or the like that stores pixel values of an image signal after gradation processing is performed on pixel values of the image signal.

The selection signal has, for example, a value assigned to one gradation conversion table selected from values divided into a plurality of gradation conversion tables. In the gradation processing step, the pixel value of the image signal subjected to gradation processing is output with reference to the two-dimensional LUT from among the value of the selection signal and the pixel value of the image signal. In the correcting step, for example, the selection signal derived for each target image area is corrected based on the selection signal derived for the image position and the image area adjacent to the target image area, and the selection signal for each pixel is derived.

In the visual processing device of the present invention, gradation processing is performed with reference to the LUT. Therefore, the speed of gradation processing can be increased. Furthermore, since one gradation conversion table is selected from a plurality of gradation conversion tables and gradation processing is performed, appropriate gradation processing can be performed. Furthermore, the selection signal can be derived for each pixel. Therefore, it is possible to further prevent the boundary of the image region from being conspicuous and unnatural, and to improve the visual effect.

The visual processing program described in Remark 14 is a computer-executed visual processing method including an image region segmentation step, a gradation transformation characteristic derivation step, and a gradation processing step. In the image area division step, the input image signal is divided into a plurality of image areas. The step of deriving the gradation transformation characteristic is a step of deriving the gradation transformation characteristic for each image region, using the gradation characteristic between the target image region to be derived from the gradation transformation characteristic and the surrounding image regions of the target image region, Exports the grayscale transformation characteristics of the target image area. In the gradation processing step, gradation processing of the image signal is performed based on the derived gradation transformation characteristics.

Here, the gradation transformation characteristics are characteristics of gradation processing for each image region. The gradation characteristic is, for example, the luminance of each pixel and the pixel value of luminance.

In the visual processing program of the present invention, when judging the gradation transformation characteristics of each image region, not only the gradation characteristics of each image region but also the image region of a wide area including surrounding image regions can be used. The grayscale characteristics are judged. Therefore, the effect of spatial processing can be added to the gradation processing of each image area, and further gradation processing with high visual effect can be realized.

The visual processing program described in Remark 15 is based on the visual processing program described in Remark 14, and the grayscale transformation characteristic is a grayscale transformation curve. The step of deriving the gray-scale transformation characteristics includes: a histogram making step, using the gray-scale characteristics to make a histogram; and a gray-scale curve making step, making a gray-scale transformation curve based on the made histogram.

Here, the so-called histogram is a part corresponding to the gradation characteristics of pixels including the target image area and the peripheral image area. The gradation curve creation step is, for example, a gradation transformation curve of an integrated curve obtained by integrating the values of the histogram.

In the visual processing program of the present invention, when creating a histogram, not only the gradation characteristics of each image area but also the gradation characteristics of a wide area including surrounding image areas can be used to create the histogram. . Therefore, the number of divisions of the image signal can be increased, the size of the image area can be reduced, and the generation of blurred outlines due to gradation processing can be suppressed. Also, it is possible to prevent the boundary of the image area from being conspicuous and unnatural.

The visual processing program described in Remark 16 is based on the visual processing program described in Remark 14, and the grayscale transformation characteristic is used to select one grayscale from a plurality of grayscale transformation tables for grayscale processing of image signals. Select signal for degree transformation table. Furthermore, the gradation processing step includes a gradation processing execution step and a correction step. In the gradation processing execution step, the gradation processing of the target image area is executed using the gradation conversion table selected by the selection signal. The correction step is a step of correcting the gradation of the gradation-processed image signal, based on the selected image region including the target pixel to be corrected and the adjacent image region including the target pixel. Grayscale processing table, to correct the grayscale of the object pixel.

Here, the gradation conversion table is, for example, a look-up table (LUT) or the like that stores pixel values of an image signal after gradation processing is performed on pixel values of the image signal. The adjacent image region may be the same image region as the surrounding image region when the gradation transformation characteristic is derived, or may be a different image region. For example, the adjacent image regions are selected as three image regions having a shorter distance from the target pixel among image regions adjacent to the image region including the target pixel.

The selection signal has, for example, a value assigned to one gradation conversion table selected from values assigned to each of a plurality of gradation conversion tables. In the gradation processing step, the LUT is referred to from among the value of the selection signal and the pixel value of the image signal, and the pixel value of the image signal subjected to gradation processing is output. In the correction step, for example, the gradation of the image signal subjected to gradation processing using the same gradation conversion table is corrected for each target image region. The correction of the target pixel is performed, for example, according to the position of the target pixel so as to express the influence of each gradation transformation table selected for the adjacent image area.

In the visual processing method of the present invention, gradation processing is performed with reference to LUT. Therefore, high-speed gradation processing can be achieved. Furthermore, since one gradation conversion table is selected from a plurality of gradation conversion tables to perform gradation processing, appropriate gradation processing can be performed. Furthermore, the gradation of the image signal can be corrected for each pixel. Therefore, the boundary of the image area is prevented from being conspicuous and unnatural, and the visual effect can be improved.

The visual processing program described in Remark 17 is based on the visual processing program described in Remark 14, and the grayscale transformation characteristic is used to select one of a plurality of grayscale transformation tables for grayscale processing of image signals. The selection signal of the grayscale conversion table. Furthermore, the gradation processing step includes a correction step and a gradation processing execution step. In the correcting step, the selection signal is corrected, and the corrected selection signal for selecting the grayscale processing table for each pixel of the image signal is derived. In the gradation processing execution step, the gradation processing of the target image area is executed using the gradation conversion table selected by the correction selection signal.

Here, the gradation conversion table is, for example, a look-up table (LUT) or the like that stores pixel values of an image signal after gradation processing is performed on pixel values of the image signal.

The selection signal has, for example, a value assigned to one gradation conversion table selected from values divided into a plurality of gradation conversion tables. In the gradation processing step, the pixel value of the image signal subjected to gradation processing is output with reference to the two-dimensional LUT from the value of the selection signal and the pixel value of the image signal. In the correcting step, for example, the selection signal derived for each target image area is corrected based on the selection signal derived for the image position and the image area adjacent to the target image area, and the selection signal for each pixel is derived.

In the visual processing device of the present invention, gradation processing is performed with reference to the LUT. Therefore, the speed of gradation processing can be increased. Furthermore, since one gradation conversion table is selected from a plurality of gradation conversion tables and gradation processing is performed, appropriate gradation processing can be performed. Furthermore, the selection signal can be derived for each pixel. Therefore, it is possible to further prevent the boundary of the image region from being conspicuous and unnatural, and to improve the visual effect.

The visual processing device described in Remark 18 is the visual processing device described in Remark 1, the grayscale processing mechanism has a parameter output mechanism, and the curve parameters of the grayscale transformation curve used for grayscale processing of the above-mentioned image signal , output based on the above grayscale transformation characteristics. A grayscale processing mechanism that performs grayscale processing on the image signal using the grayscale transformation curve determined based on the grayscale transformation determination and curve parameters.

Here, the so-called gradation transformation curve also includes at least a portion that is a straight line. The curve parameter is a parameter for distinguishing the gradation transformation curve from other gradation transformation curves. For example, the coordinates on the grayscale transformation curve, the slope and curvature of the grayscale transformation curve, etc. The parameter output mechanism is, for example, a look-up table, which stores the curve parameters corresponding to the gray-scale transformation characteristics; or an operation mechanism, etc., which calculates the curve by using the curve parameters corresponding to the specified gray-scale transformation characteristics to perform calculations such as curve approximation. parameter.

In the visual processing device of the present invention, gradation processing is performed on the image signal according to the gradation transformation characteristic. Therefore, gradation processing can be performed more suitably. Furthermore, there is no need to store the values of all the gradation transformation curves used in the gradation processing in advance, and the gradation transformation curves are determined according to the output curve parameters, and the gradation processing is performed. Therefore, the storage capacity for storing the gradation transformation curve can be reduced.

The visual processing device described in Remark 19 is the visual processing device described in Remark 18, and the parameter output mechanism is a look-up table that stores the relationship between grayscale transformation characteristics and curve parameters.

The lookup table stores the relationship between grayscale transformation characteristics and curve parameters. The grayscale processing mechanism performs grayscale processing on the image signal using the determined grayscale transformation curve.

In the visual processing device of the present invention, gradation processing is performed on the image signal according to the gradation transformation characteristic. Therefore, gradation processing can be performed more suitably. Furthermore, there is no need to store in advance the values of all the gradation transformation curves used, and only the curve parameters are stored. Therefore, the storage capacity for storing the gradation transformation curve can be reduced.

The visual processing device described in Remark 20 is the visual processing device described in Remark 18 or 19, wherein the curve parameters include the value of the grayscale processed image signal corresponding to the specified value of the image signal.

In the gray scale processing mechanism, using the relationship between the specified value of the image signal and the value of the image signal of the object of visual processing, the value of the image signal processed by the gray scale included in the curve parameter is internally classified into nonlinear or Linear, which derives the values of the grayscaled image signal.

In the visual processing device of the present invention, the grayscale transformation curve is determined according to the value of the grayscale processed image signal corresponding to the predetermined value of the image signal, and grayscale processing can be performed.

The visual processing device described in Note 21 is the visual processing device described in any one of Notes 18 to 20, wherein the curve parameters include the slope of the grayscale transformation curve of the image signal in a predetermined interval.

In the gradation processing means, the gradation transformation curve is specified based on the slope of the gradation transformation curve in a predetermined interval of the image signal. Furthermore, using the determined grayscale transformation curve, the value of the grayscale processed image signal corresponding to the value of the image signal is derived.

In the visual processing device of the present invention, the gradation transformation curve can be determined by the slope of the gradation transformation curve in a predetermined interval of the image signal, and gradation processing can be performed.

The visual processing device described in Note 22 is the visual processing device described in any one of Notes 18 to 21, wherein the curve parameters include the coordinates of at least one point that the grayscale transformation curve passes through.

In the curve parameters, the coordinates of at least one point that the grayscale transformation curve passes through are determined. That is, at least one point is determined for the value of the grayscale-processed image signal corresponding to the value of the image signal. In the gradation processing mechanism, using the relationship between the determined value of the image signal and the value of the image signal to be visually processed, the determined value of the gradation-processed image signal is internally classified It is non-linear or linear, so that the value of the grayscale processed image signal is derived.

In the visual processing device of the present invention, the grayscale transformation curve can be determined by at least one point through which the grayscale transformation curve passes, and grayscale processing can be performed.

The visual processing device described in Remark 23 includes: a spatial processing mechanism and a visual processing mechanism. The spatial processing mechanism is a mechanism that performs spatial processing for each of a plurality of image regions in the input image signal, and derives the spatially processed signal. The weighting of the difference in gradation characteristics between the target image region and the surrounding image regions is performed to perform a weighted average of the gradation characteristics between the target image region and the surrounding image regions. A visual processing means for performing visual processing of the target image area based on the gradation characteristics of the target image area and the spatially processed signal.

Here, an image area refers to an area including a plurality of pixels, or pixels, within an image. The gradation characteristics refer to values based on pixels such as luminance, lightness, and the like for each pixel. For example, the gradation characteristic of an image area refers to the average (simple average or weighted average), maximum value, or minimum value of pixel values of pixels included in the image area.

The spatial processing means performs spatial processing of the target image region using the grayscale characteristics of the surrounding image region. In spatial processing, the grayscale characteristics between the object image area and surrounding image areas are weighted and averaged. The weight in the weighted average is set based on the difference in gradation characteristics between the target image area and the peripheral image area.

In the visual processing device of the present invention, it is possible to suppress the influence of image regions having greatly different gradation characteristics in spatially processed signals. For example, the peripheral image area is an image including the boundary of an object, and even when the gradation characteristics of the target image area are greatly different, appropriate spatially processed signals can be derived. As a result, it is possible to especially suppress the occurrence of blurred outlines and the like even in visual processing using spatially processed signals. Therefore, it is possible to implement visual processing that improves the visual effect.

The visual processing device described in Remark 24 is the visual processing device described in Remark 23, and the greater the absolute value of the difference of the gray scale characteristics, the smaller the weight.

Here, the weight may be given as a monotonically decreasing value based on the difference in gradation characteristics, or may be set to a predetermined value by comparing a predetermined threshold value with the difference in gradation characteristics.

In the visual processing device of the present invention, it is possible to suppress the influence of image regions having greatly different gradation characteristics in spatially processed signals. For example, the surrounding image area is an image including the boundary of an object, and even when the gradation characteristics of the target image area are greatly different, an appropriate spatially processed signal can be derived. As a result, especially in the visual processing using the spatially processed signal, the occurrence of blurred outlines and the like can be suppressed. Therefore, it is possible to implement visual processing that improves the visual effect.

The visual processing device described in Remark 25 is the visual processing device described in Remark 23 or 24, the greater the distance between the target image area and the surrounding image area, the smaller the weight.

Here, the weight may be given as a value that decreases monotonically according to the size of the distance between the target image area and the surrounding image area, and the weight may be set to a predetermined value by comparing the distance with a predetermined threshold value. value.

In the visual processing device of the present invention, the spatially processed signal can be suppressed from being affected by peripheral image regions separated from the target image region. Therefore, the surrounding image area is an image including the boundary of an object, and when the grayscale characteristics of the target image area are greatly different, or even if the surrounding image area is far from the target image area, the difference caused by the surrounding image area can be suppressed. Affected, a more suitable spatially processed signal is exported.

The visual processing device described in attachment 26 is the visual processing device described in any one of attachments 23 to 25, wherein the image area is composed of a plurality of pixels. The gradation characteristics between the target image area and surrounding image areas are determined as feature quantities of pixel values constituting each image area.

In the visual processing device of the present invention, when spatial processing is performed for each image area, not only the pixels included in each image area but also pixels included in a wide-area image area including surrounding image areas can be used. The grayscale characteristics of pixels are processed. Therefore, more appropriate spatial processing can be performed. As a result, in the visual processing of the spatially processed signal, in particular, the occurrence of blurred contours and the like can be suppressed. Therefore, it is possible to implement visual processing that improves the visual effect.

(Note 2)

The present invention can also be expressed as follows. In addition, among the remarks of the subordinate form described in this column ("2nd Remark"), it is subordinate to the remark described in the 2nd Remark.

(Contents of Remark No. 2)

(Note 1)

A visual processing device having:

The input signal processing mechanism performs certain processing on the input image signal and outputs the processed signal;

The visual processing means outputs the output signal based on a two-dimensional LUT that provides a relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal.

(Note 2)

According to the visual processing device described in Note 1,

In the above two-dimensional LUT, there is a nonlinear relationship between the image signal and the output signal.

(Note 3)

According to the visual processing device described in Note 2,

In the two-dimensional LUT, there is a nonlinear relationship between both the image signal and the processed signal and the output signal.

(Note 4)

According to the visual processing device described in any one of Notes 1 to 3,

The value of each element of the two-dimensional LUT is determined based on a mathematical formula including an operation for enhancing a value calculated from the image signal and the processed signal.

(Note 5)

According to the visual processing device described in Note 4,

The above-mentioned processed signal is a signal on which the above-mentioned constant processing is performed on an image signal between the pixel of interest and the pixels surrounding the pixel of interest.

(Note 6)

According to the visual processing device described in Note 4 or 5,

The above-mentioned enhanced operation is a nonlinear function.

(Note 7)

According to the visual processing device described in any one of Notes 4 to 6,

The computation of the enhancement is an enhancement function that enhances the difference between the transformation values of the image signal and the processed signal after predetermined transformation.

(Remark 8)

According to the visual processing device described in Note 7,

The value C of each element of the above-mentioned two-dimensional LUT is based on the mathematical formula F2(F1(A ))+F3(F1(A)-F1(B)).

(Remark 9)

According to the visual processing device described in Note 8,

The transformation function F1 described above is a logarithmic function.

(Remark 10)

According to the visual processing device described in Note 8,

The above inverse transform function F2 is a gamma correction function.

(Note 11)

According to the visual processing device described in any one of Notes 4 to 6,

The computation of the enhancement is an enhancement function that enhances the ratio between the image signal and the processed signal.

(Note 12)

According to the visual processing device described in Note 11,

The value C of each element of the two-dimensional LUT is based on the mathematical formula F4(A)×F5(A/ B) and decided.

(Note 13)

According to the visual processing device described in note 12,

The above dynamic range compression function F4 is a monotonically increasing function.

(Note 14)

According to the visual processing device described in note 13,

The above-mentioned dynamic range element function F4 is an upwardly convex function.

(Note 15)

According to the visual processing device described in note 12,

The dynamic range compression function F4 described above is a power function.

(Note 16)

According to the visual processing device described in note 12,

The above-mentioned dynamic range compression function F4 is a proportional function with a proportional coefficient of 1.

(Note 17)

According to the visual processing device described in any one of Notes 12 to 16,

The enhancement function F5 described above is a power function.

(Note 18)

According to the visual processing device described in Note 11,

The above mathematical formula further includes: an operation of performing dynamic range compression on the ratio between the image signal enhanced by the enhancement function and the processed signal.

(Note 19)

According to the visual processing device described in any one of Notes 4 to 6,

The calculation of the enhancement includes a function for enhancing a difference between the image signal and the processed signal based on the value of the image signal.

(Note 20)

According to the visual processing device described in note 19,

The value C of each element of the above-mentioned 2-dimensional LUT is based on the mathematical formula F8(A)+F6(A )×F7(A-B).

(Note 21)

According to the visual processing device described in note 20,

The above dynamic range compression function F8 is a monotonically increasing function.

(Note 22)

According to the visual processing device described in note 21,

The above-mentioned dynamic range compression function F8 is an upward convex function.

(Note 23)

According to the visual processing device described in note 20,

The above-mentioned dynamic range compression function F8 is a power function.

(Note 24)

According to the visual processing device described in note 20,

The above-mentioned dynamic range compression function F8 is a proportional function with a proportional factor of 1.

(Note 25)

According to the visual processing device described in note 19,

The above-mentioned mathematical expression further includes a calculation of adding a value obtained by compressing the dynamic range of the image signal to the value enhanced by the above-mentioned enhancement calculation.

(Note 26)

According to the visual processing device described in any one of Notes 4 to 6,

The operation of the above-mentioned enhancement is an enhancement function for enhancing the difference between the above-mentioned image signal and the above-mentioned processed signal,

The above-mentioned mathematical formula further includes: a calculation of performing grayscale correction on a value obtained by adding the value of the image signal to the value enhanced by the above-mentioned enhancement function.

(Note 27)

According to the visual processing device described in note 26,

The value C of each element of the above-mentioned two-dimensional LUT is based on the mathematical formula F10((A)×F9(A-B )) and decided.

(Note 28)

According to the visual processing device described in any one of Notes 4 to 6,

The operation of the above-mentioned enhancement is an enhancement function for enhancing the difference between the above-mentioned image signal and the above-mentioned processed signal,

The above-mentioned mathematical formula further includes: an operation of adding a value obtained by performing gradation correction on the above-mentioned image signal to the value enhanced by the above-mentioned enhancement function.

(Note 29)

According to the visual processing device described in note 28,

The value C of each element of the two-dimensional LUT is based on the value A of the image signal, the value B of the processed signal, the enhancement function F11, and the gradation correction function F12, based on the mathematical formula F12(A)+F11(A-B) And decided.

(Note 30)

According to the visual processing device described in any one of Notes 1 to 29,

In the two-dimensional LUT, values stored in the image signal and the processed signal having the same value monotonically increase or decrease with respect to the values of the image signal and the processed signal.

(Note 31)

According to the visual processing device described in any one of Notes 1 to 3,

The two-dimensional LUT stores the relationship between the image signal and the output signal as a gradation transformation curve group consisting of a plurality of gradation transformation curves.

(Note 32)

According to the visual processing device described in note 31,

Each of the above gradation transformation curves monotonically increases with respect to the value of the image signal.

(Note 33)

According to the visual processing device described in remark 31 or 32,

The processing signal is a signal for selecting a corresponding grayscale transformation curve from the plurality of grayscale transformation curve groups.

(Note 34)

According to the visual processing device described in note 33,

The value of the processed signal is associated with one grayscale transformation curve included in the plurality of grayscale transformation curve groups.

(Note 35)

According to the visual processing device described in any one of Notes 1 to 34,

In the above-mentioned two-dimensional LUT, profile data created in advance through predetermined calculations is registered.

(Note 36)

A visual processing device as described in Remark 35,

The above-mentioned 2D LUT can be changed by registering the profile data.

(Note 37)

According to the visual processing device described in remark 35 or 36,

It also includes profile data registration means for registering the profile data in the visual processing means.

(Note 38)

A visual processing device as described in Remark 35,

The above-mentioned visual processing mechanism obtains the above-mentioned description file data obtained by an external device.

(Note 39)

According to the visual processing device described in note 38,

The above-mentioned 2-dimensional LUT can be changed by the obtained above-mentioned profile data.

(Note 40)

According to the visual processing device described in remark 38 or 39,

The above-mentioned visual processing mechanism obtains the above-mentioned description file data through a communication network.

(Note 41)

A visual processing device as described in Remark 35,

It also includes profile data creation means that creates the above-mentioned profile data.

(Note 42)

According to the visual processing device shown in Remark 41,

The profile data creating means creates the profile data based on the histogram of the gradation characteristic of the image signal.

(Note 43)

A visual processing device as described in Remark 35,

The profile data registered in the two-dimensional LUT are switched according to predetermined conditions.

(Note 44)

According to the visual processing device described in note 43,

The above-mentioned prescribed conditions are conditions related to brightness.

(Note 45)

According to the visual processing device described in note 44,

The above-mentioned brightness is the brightness of the above-mentioned image signal.

(Note 46)

A visual processing device as described in Remark 45,

A brightness judging means for judging the brightness of the image signal is further provided.

The profile data registered in the two-dimensional LUT is switched according to the judgment result of the brightness judging means.

(Note 47)

According to the visual processing device described in note 44,

There is also a lightness input means that allows the above-mentioned lightness-related conditions to be input,

The profile data registered in the two-dimensional LUT is switched according to the input result of the brightness input means.

(Note 48)

A visual processing device as described in Remark 47,

The brightness input means for inputting the brightness of an output environment of the output signal or the brightness of an input environment of the input signal.

(Note 49)

According to the visual processing device described in note 44,

It also has a lightness detection mechanism that detects at least two types of the above-mentioned brightness,

The profile data registered in the two-dimensional LUT is switched according to the detection result of the lightness detection means.

(Note 50)

Visual processing devices according to Remark 49

The brightness detected by the brightness detection means includes: brightness of the image signal, brightness of an output environment of the output signal, or brightness of an input environment of the input signal.

(Note 51)

According to the visual processing device described in note 43,

It further includes: profile data selection means for selecting the profile data registered in the 2-dimensional LUT,

The profile data registered in the two-dimensional LUT is switched according to the selection result of the profile data selection means.

(Note 52)

According to the visual processing device described in remark 51,

The above-mentioned profile data selection means is an input device for selecting a profile.

(Note 53)

According to the visual processing device described in note 43,

further comprising an image characteristic judging means for judging the image characteristic of the above-mentioned image signal,

The profile data registered in the two-dimensional LUT is switched according to the judgment result of the image characteristic judging means.

(Note 54)

According to the visual processing device described in note 43,

It also has a user identification mechanism that identifies users,

The profile data registered in the above-mentioned 2D LUT is switched according to the identification result of the user identification means.

(Note 55)

According to the visual processing device described in any one of Notes 1 to 54,

The above-mentioned visual processing means performs an interpolation operation on the value stored in the above-mentioned two-dimensional LUT, and outputs the above-mentioned output signal.

(Note 56)

According to the visual processing device described in remark 55,

The interpolation operation is a linear interpolation based on a value of a lower bit of at least one of the image signal or the processed signal represented by a binary number.

(Note 57)

A visual processing device according to any one of Remarks 1 to 56,

The input signal processing unit performs spatial processing on the image signal.

(Note 58)

According to the visual processing device described in remark 57,

The above-mentioned input signal processing means generates an unsharp signal based on the above-mentioned image signal.

(Note 59)

According to the visual processing device described in remark 57 or 58,

In the above-described spatial processing, the average value, maximum value, or minimum value of the image signal is derived.

(Note 60)

According to the visual processing device described in any one of Notes 1 to 59,

The visual processing means performs spatial processing and gradation processing using the inputted image signal and the processed signal.

(Note 61)

A visual processing method having:

The input signal processing step is to perform certain processing on the input image signal, and output the processed signal; and

In the visual processing step, the output signal is output based on a two-dimensional LUT that provides a relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal.

(Note 62)

A visual processing program for performing a visual processing method of the following steps by a computer,

The above visual processing methods include:

The input signal processing step is to perform certain processing on the input image signal, and output the processed signal; and

In the visual processing step, the output signal is output based on a two-dimensional LUT that provides a relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal.

(Note 63)

An integrated circuit, which includes the visual processing device described in any one of Remarks 1-60.

(Note 64)

A display device comprising:

A visual processing device as described in any of Remarks 1-60; and

Display of the display of the output signal output from the visual processing device is performed.

(Note 65)

A photographing device comprising:

the camera agency that took the image; and

The visual processing device described in any one of Remarks 1 to 60 that performs visual processing on an image captured by the imaging mechanism as the image signal.

(Note 66)

A portable information terminal, which has:

A data receiving institution that receives communication or broadcast image data;

The visual processing device described in any one of Remarks 1 to 60 that performs visual processing of the received image data as the image signal; and

A display unit for displaying the image signal after visual processing by the visual processing device.

(Note 67)

A shooting information terminal, which has:

A photography agency, which performs the photography of images;

The visual processing device described in any one of Remarks 1 to 60 that performs visual processing on the image captured by the above-mentioned shooting mechanism as the above-mentioned image signal; and

and a data transmission unit that transmits the above-mentioned visually processed image signal.

(Note 68)

An image processing device that performs image processing of an input image signal that is inputted, comprising:

a profile data creating means for creating profile data used in image processing based on a plurality of profile data for performing different image processing;

and an image processing execution unit that performs the image processing using the profile data created by the profile data creation unit.

(Note 69)

An image processing device that performs image processing of an input image signal that is inputted, comprising:

A description file information output mechanism that outputs description file information for determining the description file data used in the above-mentioned image processing;

An image processing execution unit that performs the image processing using the profile data specified based on the information output from the profile information output unit.

(Note 70)

According to the image processing device described in Remark 69,

The profile information output means outputs the profile information according to a display environment representing the image-processed input image signal.

(Note 71)

According to the image processing device described in Remark 69,

The profile information output means outputs the profile information based on information related to profile data among information contained in the input image signal.

(Note 72)

According to the image processing device described in Remark 69,

The profile information output means outputs the profile information based on the obtained information on the characteristics of the image processing.

(Note 73)

According to the image processing device described in Remark 69,

The profile information output means outputs the profile information based on the information on the environment in which the input image signal is generated.

(Note 74)

According to the image processing device described in Remark 69,

The input image signal includes image data and attribute information of the input image signal,

The above-mentioned description file information output unit outputs the above-mentioned description file information according to the above-mentioned attribute information.

(Note 75)

According to the image processing device described in Remark 74,

The attribute information includes overall attribute information related to the entirety of the image data.

(Note 76)

According to the image processing device described in remark 74 or 75,

The attribute information includes partial attribute information related to a part of the image data.

(Note 77)

According to the image processing device described in Remark 74,

The attribute information includes generation environment attribute information related to an environment in which the input image signal is generated.

(Note 78)

According to the image processing device described in Remark 74,

The attribute information includes medium attribute information related to the medium from which the input image signal is obtained.

(Note 79)

According to the image processing device described in any one of Remarks 68 to 78,

The above description file data is a 2D LUT,

The above-mentioned image processing executing mechanism includes the visual processing device described in any one of Remarks 1-60.

(Note 80)

An image processing device comprising:

an image processing actuator, which performs image processing on the inputted input image signal;

a profile information output means for outputting profile information for determining profile data suitable for image processing of the input image signal being input;

(Note 81)

An integrated circuit comprising the image processing device described in any one of Remarks 60-80.

(Note 82)

A display device comprising:

The image processing device described in any one of Remarks 68 to 80; and

and a display unit for displaying the input image signal subjected to image processing by the image processing device.

(Note 83)

A photographing device comprising:

the filming agency, which carried out the filming of the image; and

The image processing device described in any one of Remarks 68 to 80 of the image processing is performed using the image captured by the imaging means as the input image signal.

(Note 84)

A portable information terminal has:

A data receiving institution that receives communication or broadcast image data;

The image processing device described in any one of Remarks 68 to 80 that performs image processing on the received image data as the input image signal; and

A display unit for displaying the input image signal after image processing by the image processing device.

(Note 85)

A portable information terminal, which has:

A photography agency, which performs the photography of images;

The image processing device described in any one of Remarks 68 to 80 that performs image processing on the image captured by the above-mentioned camera mechanism as the above-mentioned input image signal; and

and a data transmission unit that transmits the input image signal subjected to the image processing.

(Explanation of the second note)

The visual processing device described in Note 1 includes: input signal processing means, and visual processing means. The input signal processing mechanism performs certain processing on the input image signal, and outputs the processed signal. The visual processing means outputs an output signal based on a two-dimensional LUT that provides a relationship between an input image signal and a processed signal, and an output signal that is a visually processed image signal.

Here, the fixed processing refers to, for example, direct or indirect processing of an image signal, including processing of adding transformation to pixel values of an image signal such as spatial processing and gradation processing.

In the visual processing device of the present invention, visual processing is performed using a two-dimensional LUT that describes the relationship between the image signal, the processed signal, and the visually processed output signal. Therefore, a hardware configuration that does not depend on the functions of the two-dimensional LUT can be realized. That is, it is possible to implement a hardware configuration that does not depend on the visual processing realized by the device as a whole.

The visual processing device described in Note 2 is the visual processing device described in Note 1, and in the 2-dimensional LUT, there is a nonlinear relationship between the image signal and the output signal.

Here, the non-linear relationship between the image signal and the output signal is represented by, for example, a nonlinear function of the values of the elements of the two-dimensional LUT with respect to the image signal, or it means that it is difficult to formulate from a function.

In the visual processing device of the present invention, visual processing having visual characteristics of an image signal or visual processing having non-linear characteristics of a device that outputs an output signal can be realized.

The visual processing device described in Note 3 is the visual processing device described in Note 2, and in the two-dimensional LUT, both the image signal and the processed signal have a nonlinear relationship with the output signal.

Here, both the image signal and the processed signal have a nonlinear relationship with the output signal, which means, for example, the value of each element of the 2D LUT is a nonlinear function of two variables corresponding to the image signal and the processed signal. represent, or refer to are difficult to formulate by a function, etc.

In the visual processing device of the present invention, even when the value of the image signal is the same but the value of the processed signal is different, different visual processing can be realized according to the value of the processed signal.

The visual processing device described in Note 4 is the visual processing device described in any one of Notes 1 to 3, wherein the value of each element of the 2D LUT is a value calculated from the image signal and the processed signal based on including enhancement It is determined by the mathematical formula included in the operation.

Here, the value calculated from the image signal and the processed signal is, for example, a value obtained by four arithmetic operations between the image signal and the processed signal, or a value obtained by converting the image signal or the processed signal by a certain function through operation. The resulting value etc. The computation of enhancement is, for example, computation of adjusting gain, computation of suppressing transitional contrast, computation of suppressing small-amplitude noise components, and the like.

In the visual processing device of the present invention, the value calculated from the image signal and the processed signal can be enhanced.

The visual processing device described in Note 5 is the visual processing device described in Note 4, wherein the processed signal is a signal for performing certain processing on an image signal between the pixel of interest and pixels surrounding the pixel of interest.

Here, the fixed processing refers to, for example, spatial processing using neighboring pixels for the pixel of interest, and is processing for deriving an average value, a maximum value, or a minimum value between the pixel of interest and surrounding pixels.

In the visual processing device of the present invention, for example, even when visual processing is performed on a pixel of interest with the same value, different visual processing can be realized due to the influence of surrounding pixels.

The visual processing device described in Note 6 is based on the visual processing device described in Note 4 or 5, and the enhanced operation is a nonlinear function.

In the visual processing device of the present invention, for example, enhancement of visual characteristics of an image signal or enhancement of nonlinear characteristics of a device that outputs an output signal can be realized.

The visual processing device shown in Remark 7 is the visual processing device according to any one of Remarks 4 to 6, and the enhanced calculation is the difference between the transformation values of the image signal and the processed signal in a prescribed transformation Reinforced Reinforcement function.

Here, the enhancement function is, for example, a function for adjusting gain, a function for suppressing transitional contrast, a function for suppressing small-amplitude noise components, and the like.

In the visual processing device of the present invention, after transforming the image signal and the processed signal into different spaces, each difference can be enhanced. In this way, reinforcement with visual characteristics and the like can be realized.

The visual processing device described in Remark 8 is the visual processing device described in Remark 7, and the value C of each element of the 2D LUT is the value A of the image signal, the value B of the processed signal, the transformation function F1, the transformation The inverse transformation function F2 and the strengthening function F3 of the function F1 are determined based on the mathematical formula F2 (F1(A)+F3(F1(A)−F1(B))).

Here, the two-dimensional LUT is a LUT that provides the value C of each element corresponding to two inputs between the value A of the image signal and the value B of the processed signal (hereinafter the same applies to this column). In addition, the value of each signal may be the value of each signal, or may be an approximate value of the value (hereinafter the same applies to this column). The enhancement function F3 is, for example, a function for adjusting gain, a function for suppressing transitional contrast, a function for suppressing small-amplitude noise components, and the like.

Here, the value C of each element is expressed as follows. That is, the value A of the image signal and the value B of the processed signal are transformed into different spatial values by the transformation function F1. The difference between the value of the transformed image signal and the value of the processed signal is expressed, for example, as a sharp signal in a different space or the like. The difference between the transformed image signal and the processed signal enhanced by the enhancement function F3 is added to the transformed image signal. In this way, the value C of each pixel represents a value in which the clear signal component in a different space is enhanced.

In the visual processing device of the present invention, for example, processing such as edge enhancement and contrast enhancement in different spaces can be performed using the value A of the image signal converted into a different space and the value B of the processed signal.

The visual processing device described in Note 9 is the visual processing device described in Note 8, and the transformation function F1 is a logarithmic function.

Here, the visual characteristics of human beings are generally logarithmic. Therefore, by transforming into a logarithmic space and processing the image signal and the processed signal, processing suitable for visual characteristics can be performed.

In the visual processing device of the present invention, it is possible to perform contrast enhancement with high visual effect, or dynamic range compression to maintain local contrast.

The visual processing device described in Note 10 is the visual processing device described in Note 8, and the inverse transformation function F2 is a gamma correction function.

Here, the gamma correction is applied to the image signal by a gamma correction function based on the gamma characteristics of a device that generally inputs and outputs the image signal.

In the visual processing device of the present invention, the gamma correction of the image signal is eliminated by the transformation function F1, and the processing can also be performed according to the linear characteristic. In this way, optical blurring can be corrected.

The visual processing device described in Note 11 is the visual processing device described in any one of Notes 4 to 6, wherein the enhanced operation is an enhanced function that enhances the ratio between the image signal and the processed signal.

In the visual processing device of the present invention, for example, the ratio between the image signal and the processed signal represents the sharp component of the image signal. Thus, for example, visual manipulations that intensify clear components can be performed.

The visual processing device described in Remark 12 is the visual processing device described in Remark 11, wherein the value C of each element of the 2D LUT is the value A of the image signal, the value B of the processed signal, and the dynamic range compression function F4 . The enhancement function F5 is determined based on the mathematical formula F4(A)×F5(A/B).

Here, the value C of each element is expressed as follows. That is, the division amount (A/B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F5(A/B) represents, for example, the enhancement amount of a clear signal. These represent processing equivalent to transforming the value A of the image signal and the value B of the processed signal into a logarithmic space, and performing enhancement processing on each difference, and performing enhancement processing suitable for visual characteristics.

In the visual processing device of the present invention, while compressing the dynamic range as needed, local contrast can be enhanced.

The visual processing device described in Note 13 is the visual processing device described in Note 12, and the dynamic range compression function F4 is a monotonically increasing function.

In the visual processing device of the present invention, local contrast is enhanced while performing a dynamic range compression function using the dynamic range compression function F4 which is an individual increase function.

The visual processing device described in Note 14 is the visual processing device described in Note 13,

The dynamic range compression function F4 is an upward convex function.

In the visual processing device of the present invention, local contrast can be enhanced while performing dynamic range compression using the dynamic range compression function F4 which is an upward convex function.

The visual processing device described in Remark 15 is the visual processing device described in Remark 12, and the dynamic range compression function F4 is a power function.

In the visual processing device of the present invention, it is possible to enhance the local contrast while converting the dynamic range using the dynamic range compression function F4 which is a power function.

The visual processing device described in Note 16 is the visual processing device described in Note 12, and the dynamic range compression function F4 is a proportional function with a proportional coefficient of 1.

In the visual processing device of the present invention, the contrast can be uniformly enhanced from the dark part to the bright part of the image signal. This contrast enhancement is an enhancement process suitable for visual characteristics.

The visual processing device described in Remark 17 is the visual processing device described in any one of Remarks 12 to 17, and the enhancement function F5 is a power function.

In the visual processing device of the present invention, the enhancement function F5 as a power function can be used to transform the dynamic range, and at the same time, the local contrast can be enhanced.

The visual processing device described in Note 18 is the visual processing device described in Note 11, wherein the mathematical formula further includes an operation of performing dynamic range compression on the ratio between the image signal enhanced by the enhancement function and the processed signal.

In the visual processing device of the present invention, for example, it is possible to compress the dynamic range while emphasizing the sharp component of the image signal represented by the ratio between the image signal and the processed signal.

The visual processing device shown in Remark 19 is the visual processing device according to any one of Remarks 4 to 6, the enhanced operation includes emphasizing the difference between the image signal and the processed signal according to the value of the image signal function.

In the visual processing device of the present invention, for example, depending on the value of the image signal, a clear signal of the image signal, which is a difference between the image signal and the processed signal, can be enhanced. Therefore, appropriate enhancement can be performed from the dark part to the bright part of the image signal.

The visual processing device described in Remark 20 is the visual processing device described in Remark 19, wherein the value C of each element of the 2D LUT is the value A of the image signal, the value B of the processed signal, and the enhancement amount adjustment function F6 , the enhancement function F7 and the dynamic range compression function F8 are determined based on the mathematical formula F8(A)+F6(A)×F7(A-B).

Here, the value C of each element is expressed as follows. That is, the difference (A-B) between the value A of the image signal and the value B of the processed signal is expressed as, for example, a clear signal. Also, F7(A-B) represents, for example, the enhancement amount of a clear signal. Furthermore, the enhancement amount is adjusted according to the value A of the image signal by the enhancement amount adjustment function F6, and is added to the value of the image signal whose dynamic range has been compressed as necessary.

In the visual processing device of the present invention, for example, although the value A of the image signal is large, the amount of enhancement can be reduced to maintain the contrast from dark parts to bright parts. Also, even with dynamic range compression, local contrast from dark to bright areas can be maintained.

The visual processing device described in Remark 21 is the visual processing device described in Remark 20, and the dynamic range compression function F8 is a monotonically increasing function.

In the visual processing device of the present invention, it is possible to maintain local contrast while performing dynamic range compression using the dynamic range compression function F8 which is a monotonically increasing function.

The visual processing device described in Remark 22 is the visual processing device described in Remark 21, and the dynamic range compression function F8 is an upward convex function.

In the visual processing device of the present invention, the local contrast can be maintained while dynamic range compression is performed using the dynamic range compression function F8 which is an upward convex function.

The visual processing device described in Remark 23 is the visual processing device described in Remark 20, and the dynamic range compression function F8 is a power function.

In the visual processing device of the present invention, the dynamic range can be transformed using the dynamic range compression function F8 which is a power function while maintaining the local contrast.

The visual processing device described in Remark 24 is the visual processing device described in Remark 20, and the dynamic range compression function F8 is a proportional coefficient of a proportional coefficient of 1.

In the visual processing device of the present invention, the contrast can be uniformly enhanced from the dark part to the bright part of the image signal.

The visual processing device described in Remark 25 is the visual processing device described in Remark 19, and the mathematical formula further includes an operation of adding a value obtained by compressing the dynamic range of the image signal to the value enhanced by the enhanced operation .

In the visual processing device of the present invention, for example, it is possible to compress the dynamic range while enhancing the sharp components of the image signal according to the value of the image signal.

The visual processing device described in Note 26 is the visual processing device described in any one of Notes 4 to 6, wherein the enhanced operation is an enhanced function that enhances the difference between the image signal and the processed signal. The mathematical formula is an operation for performing gradation correction on the value obtained by adding the value of the image signal to the value enhanced by the enhancement function.

In the visual processing device of the present invention, for example, the difference between the image signal and the processed signal represents the sharp component of the image signal. Therefore, it is possible to implement visual processing for performing gradation correction on an image signal that enhances clear components.

The visual processing device described in Remark 27 is the visual processing device described in Remark 26. The value C of each element of the 2D LUT is the value A of the image signal, the value B of the processed signal, the enhancement function F9, the gray The degree correction function F10 is determined based on the mathematical formula F10 (A+F9(A-B)).

Here, the value C of each element is expressed as follows. That is, the difference (A-B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F9(A-B) represents, for example, enhancement processing of a clear signal. Further, it means that the sum of the value A of the image signal and the enhanced clear signal is to be corrected in gray scale.

In the visual processing device of the present invention, the effect of combining contrast enhancement and gradation correction can be obtained.

In the visual processing device described in attachment 28, the calculation for enhancing the visual processing device described in any one of attachments 4 to 6 is an enhancement function that enhances the difference between the image signal and the processed signal. The mathematical formula also includes: an operation of adding a value obtained by performing gradation correction on the image signal to the value enhanced by the enhancement function.

In the visual processing device of the invention, for example, the difference between the image signal and the processed signal represents the sharp component of the image signal. Also, enhancement of clear components and gradation correction of image signals are performed independently. Therefore, regardless of the amount of gradation correction of the image signal, it is possible to enhance the clear components to a certain extent.

The visual processing device described in Remark 29 is based on the visual processing device described in Remark 28, for the value C of each element of the 2D LUT, the value A of the image signal, the value B of the processed signal, the enhancement function F11, the gray The degree correction function F12 is determined based on the mathematical formula F12(A)+F11(A-B).

Here, the value C of each element is expressed as follows. That is, the difference (A-B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F11(A-B) represents, for example, enhancement processing of a clear signal. Furthermore, it indicates that the value of the image signal after the gradation processing is added to the sharp signal after the enhancement processing.

In the visual processing device of the present invention, a certain contrast enhancement can be performed regardless of the gradation correction.

The visual processing device described in Remark 30 is the visual processing device described in any one of Remarks 1 to 29, in which, in a 2-dimensional LUT, the values stored for the image signal and the processed signal of the same value are relatively The values of the image signal and the processed signal have a monotonically increasing or monotonically decreasing relationship.

Here, the values stored in the 2D LUT corresponding to the image signal and the processed signal having the same value represent an outline of the characteristics of the 2D LUT.

In the visual processing device of the present invention, the 2D LUT uses a value that monotonically increases or decreases with respect to the image signal and the processed signal as a value corresponding to the same value of the image signal and the processed signal.

The visual processing device described in Remark 31 is the visual processing device described in any one of Remarks 1 to 3, and the 2-dimensional LUT stores the relationship between the image signal and the output signal as a result of multiple grayscale transformations. A group of grayscale transformation curves composed of curves.

Here, the gradation transformation curve group is a set of gradation transformation curves that perform gradation processing on pixel values such as luminance of an image signal and luminance.

In the visual processing device of the present invention, gradation processing of an image signal can be performed using a gradation transformation curve selected from a plurality of gradation transformation curves. Therefore, more appropriate gradation processing can be performed.

The visual processing device described in Remark 32 is the visual processing device described in Remark 31, wherein each of the gradation transformation curve groups monotonically increases with respect to the value of the image signal.

In the visual processing device of the present invention, gradation processing can be performed using a group of gradation transformation curves that monotonically increase with respect to the value of the image signal.

The visual processing device described in Remark 33 is the visual processing device described in Remark 31 or 32, and the processing signal is a signal for selecting a corresponding grayscale transformation curve from a plurality of grayscale transformation curve groups.

Here, the processed signal is a signal for selecting a grayscale transformation curve, for example, an image signal after spatial processing.

In the visual processing device of the present invention, the gradation processing of the image signal can be performed using the gradation transformation curve selected by processing the signal.

The visual processing device described in Remark 34 is the visual processing device described in Remark 33, wherein the value of the processed signal is associated with at least one grayscale transformation curve included in the plurality of grayscale transformation curve groups.

Here, at least one gradation transformation curve used for gradation processing is selected based on the value of the processed signal.

In the visual processing device of the present invention, at least one gradation transformation curve is selected by processing the value of the signal. Furthermore, the gradation processing of the image signal is performed using the selected gradation transformation curve.

The visual processing device according to attachment 35 is the visual processing device according to any one of attachments 1 to 34, wherein profile data prepared in advance by predetermined calculations are registered in the two-dimensional LUT.

In the visual processing device of the present invention, visual processing is performed using a two-dimensional LUT in which pre-created profile data is registered. In the case of visual processing, processing such as creating profile data is unnecessary, and the execution speed of visual processing can be increased.

The visual processing device described in attachment 36 is based on the visual processing device described in attachment 35, and the 2-dimensional LUT can be changed by registering profile data.

Here, the profile data is data of a two-dimensional LUT that realizes different visual processing.

In the visual processing device of the present invention, various changes can be made to the visual processing realized by registering the profile data. That is, various visual processing can be realized without changing the hardware configuration of the visual processing device.

The visual processing device described in attachment 37 is the visual processing device described in attachment 35 or 36, further comprising a profile data registration means for registering profile data in the visual processing means.

Here, the profile data registration means registers the profile data calculated in advance in the visual processing means according to the visual processing.

In the visual processing device of the present invention, various changes can be made to the format processing realized by registering the profile data. That is, various visual processing can be realized without changing the hardware configuration of the visual processing device.

The visual processing device described in Remark 38 is based on the visual processing device described in Remark 35, and the visual processing mechanism obtains the prepared description file data through an external device.

Profile data is created in advance by an external device. The external device is, for example, a computer having a program capable of creating profile data and a CPU. The visual processing mechanism obtains the description file data. Acquired via, for example, a network or a recording medium. The visual processing means executes visual processing using the acquired profile data.

In the visual processing device of the present invention, visual processing can be executed using profile data created by an external device.

The visual processing device described in Remark 39 is the visual processing device described in Remark 38, and the two-dimensional LUT can be changed through the acquired profile data.

In the visual processing device of the present invention, the obtained profile data is re-registered as a two-dimensional LUT. In this way, the 2D LUT can be changed to realize different visual processing.

The visual processing device described in Remark 40 is the visual processing device described in Remark 38 or 39, and the communication processing mechanism obtains the description file data through a communication network.

Here, the term "communication network" means, for example, a communication connection mechanism such as a dedicated line, a public line, the Internet, or a LAN, and may be wired or wireless.

In the visual processing device of the present invention, visual processing can be realized using profile data obtained through a communication network.

The visual processing device described in Attachment 41 is the visual processing device described in Attachment 35, further comprising: a profile data creating means for creating profile data.

The profile data creating means is a means for creating profile data using, for example, characteristics of image signals and processed signals.

In the visual processing device of the present invention, visual processing can be realized using the profile data created by the profile data creating means.

The visual processing device described in Remark 42 is based on the visual processing device described in Remark 41, wherein the profile data creation mechanism creates profile data based on the histogram of the grayscale characteristic of the image signal.

In the visual processing device of the present invention, visual processing is realized using profile data created based on the histogram of the gradation characteristic of the image signal. Therefore, appropriate visual processing can be realized according to the characteristics of the image signal.

The visual processing device described in attachment 43 is the visual processing device described in attachment 35, wherein the profile data registered in the two-dimensional LUT is switched according to a predetermined condition.

In the visual processing device of the present invention, visual processing is realized using profile data switched according to predetermined conditions. Therefore, more appropriate visual processing can be realized.

The visual processing device described in attachment 44 is the visual processing device described in attachment 43, and the predetermined conditions are conditions related to brightness.

In the visual processing device of the present invention, more appropriate visual processing can be realized based on conditions related to shading.

The visual processing device described in Remark 45 is the visual processing device described in Remark 44, and the brightness is the brightness of the image signal.

In the visual processing device of the present invention, more appropriate visual processing can be realized based on the conditions related to the brightness of the image signal.

The visual processing device described in Attachment 46 is the visual processing device described in Attachment 45, further comprising a lightness determination means for determining lightness and darkness of the image signal. The profile data registered in the 2D LUT is switched according to the judgment result of the lightness judgment means.

Here, the brightness judging means judges the brightness and darkness of the image signal based on, for example, pixel values such as brightness and lightness of the image signal. Furthermore, according to the determination result, the profile data is switched.

In the visual processing device of the present invention, more appropriate visual processing can be realized according to the brightness and darkness of the image signal.

The visual processing device according to attachment 47 is the visual processing device according to attachment 44, further comprising a brightness input means for inputting conditions related to brightness. The profile data registered in the 2D LUT is switched according to the input result of the brightness input mechanism.

Here, the brightness input means is, for example, a wired or wirelessly connected switch or the like for the user to input conditions related to brightness.

In the visual processing device of the present invention, the user judges the conditions related to the brightness, and can switch the profile data through the brightness input means. Therefore, appropriate visual processing can be realized by the user.

The visual processing device described in attachment 48 is the visual processing device described in attachment 47, wherein the brightness input means inputs the brightness of the output environment of the output signal or the brightness of the input environment of the input signal.

Here, the brightness of the output environment means, for example, the brightness of ambient light around a medium from which an output signal is output, such as a computer, a television, a digital camera, a mobile phone, or a PDA, or a medium from which an output signal is output, such as printing paper. The lightness and darkness of the media itself, etc. The brightness of the input environment refers to the brightness of the medium itself into which the input signal is input, such as scanning paper.

In the visual processing device of the present invention, for example, the user judges relevant conditions such as the brightness and darkness of the room, and can switch the profile data through the brightness input mechanism. Therefore, more suitable visual processing for the user can be realized.

The visual processing device described in Attachment 49 is the visual processing device described in Attachment 44, further comprising: a lightness detection mechanism that detects at least two types of lightness and darkness. The profile data registered in the 2D LUT is switched according to the detection result of the lightness detection mechanism.

Here, the so-called lightness detection mechanism is, for example, a mechanism that detects the lightness and darkness of an image signal based on pixel values such as brightness and lightness of the image signal, or a mechanism that detects the lightness and darkness of an output environment or an input environment such as a photosensitive element. , or a mechanism that detects conditions related to brightness and darkness input by the user, or the like. In addition, the brightness of the output environment is, for example, the brightness of ambient light around a medium that outputs an output signal, such as a computer, a television, a digital camera, a mobile phone, or a PDA, or the brightness of the medium itself that outputs an output signal, such as printing paper. lightness etc. The brightness of the input environment refers to the brightness of the medium itself into which the input signal is input, such as scanning paper.

In the visual processing device of the present invention, at least two types of brightness and darkness are detected, and profile data is switched based on these. Therefore, more suitable visual processing can be realized.

The visual processing device described in Remark 50 is based on the visual processing device described in Remark 49, and the shading detected by the shading detection mechanism includes: the shading of the image signal, the shading of the output environment of the output signal, or the shading of the input signal. The lightness and darkness of the input environment of the signal.

In the visual processing device of the present invention, more appropriate visual processing can be realized according to the brightness of the image signal, the brightness of the output environment of the output signal, or the brightness of the input environment of the input signal.

The visual processing device described in attachment 51 is the visual processing device described in attachment 43, further comprising profile data selection means for allowing a person to select profile data registered in the two-dimensional LUT. The profile data registered in the 2D LUT is switched according to the selection result of the profile data selection mechanism.

The profile data selection mechanism enables the user to select profile data. Furthermore, in the visual processing device, visual processing is realized using the selected profile data.

In the visual processing device of the present invention, the user selects a description file according to preference, and visual processing can be realized.

The visual processing device described in attachment 52 is the visual processing device described in attachment 51, wherein the profile data selection means is an input device for selecting a profile.

Here, the input device is, for example, a switch built in or connected to the visual processing device by wire or wirelessly.

In the visual processing device of the present invention, the user can select a favorite description file by using the input device.

The visual processing device described in Attachment 53 is the visual processing device described in Attachment 43, further comprising an image characteristic judgment means for judging the image characteristic of the image signal. The profile data registered in the 2D LUT is switched according to the judgment result of the image characteristic judging means.

The image characteristic judging means judges image characteristics such as brightness, lightness, or spatial frequency of the image signal. The visual processing device implements visual processing using the profile data switched according to the judgment result of the image characteristic judging means.

In the visual processing device of the present invention, the image characteristic judging mechanism automatically selects the description file data corresponding to the image characteristic. Therefore, visual processing can be realized using profile data more suitable for image signals.

The visual processing device according to attachment 54 is the visual processing device according to attachment 43, further comprising a user identification means for identifying a user. The profile data registered in the 2D LUT is switched according to the identification result of the user identification mechanism.

The user identification means is, for example, an input device for identifying a user, a camera, or the like.

In the visual processing device of the present invention, visual processing suitable for a user identified by the user identification means can be realized.

The visual processing device described in Remark 55 is the visual processing device described in any one of Remarks 1 to 54, wherein the visual processing mechanism performs an interpolation operation on the value stored in the 2-dimensional LUT, and outputs an output signal.

The 2-dimensional LUT stores the values of image signals or processed signals at predetermined intervals. By interpolating the value of the 2D LUT corresponding to the interval including the value of the input image signal or the value of the processed signal, the value of the output signal corresponding to the value of the input image signal or the value of the processed signal value is output.

In the visual processing device of the present invention, it is not necessary to store the values of the 2D LUT for all possible values of the image signal or the processed signal, so that the storage capacity for the 2D LUT can be reduced.

The visual processing device described in Remark 56 is the visual processing device described in Remark 55, and the interpolation operation is linear interpolation based on the value of at least one of the low-order bits of the image signal represented by a binary number or the processed signal. repair.

The 2-dimensional LUT stores a value corresponding to the high-order bit value of an image signal or a processed signal. The visual processing mechanism performs linear interpolation on the value of the 2-dimensional LUT corresponding to the interval including the value of the input image signal or the value of the processed signal by using the value of the lower bit of the image signal or the processed signal, thereby outputting signal output.

In the visual processing device of the present invention, more accurate visual processing can be realized while storing a two-dimensional LUT with a smaller storage capacity.

The visual processing device described in Remark 57 is the visual processing device described in any one of Remarks 1 to 56, which receives a signal processing mechanism and performs spatial processing on the image signal.

In the visual processing device of the present invention, visual processing can be realized by a 2-dimensional LUT using an image signal and a spatially processed image signal.

The visual processing device described in Remark 58 is the visual processing device described in Remark 57, wherein the input signal processing mechanism generates the unsharp signal from the image signal.

Here, the so-called unsharp signal refers to a signal directly or indirectly subjected to spatial processing on an image signal.

In the visual processing device of the present invention, visual processing can be realized by a 2-dimensional LUT using the image signal and the unsharp signal.

The visual processing device described in attachment 59 is the visual processing device described in attachment 57 or 58, which derives the average value, maximum value, or minimum value of the image signal during spatial processing.

Here, the average value may be, for example, a simple average of image signals or a weighted average.

In the visual processing device of the present invention, the image signal and the average value, maximum value, or minimum value of the average signal can be used, and visual processing can be realized by a two-dimensional LUT.

The visual processing device described in Remark 60 is the visual processing device described in any one of Remarks 1 to 59, and the visual processing mechanism uses the input image signal and processing signal to perform spatial processing and grayscale processing .

In the visual processing device of the present invention, spatial processing and gradation processing can be realized simultaneously by using a two-dimensional LUT.

The visual processing method described in Note 61 includes: an input signal processing step; and a visual processing step. In the input signal processing step, a certain processing is performed on the input image signal, and the processed signal is output. In the visual processing step, an output signal is output based on a two-dimensional LUT that provides a relationship between the input image signal and the processed signal, and an output signal that is the visually processed image signal.

Here, the term "certain processing" means, for example, direct or indirect processing of an image signal, including processing of converting pixel values of an image signal such as spatial processing or gradation processing.

In the visual processing method of the present invention, visual processing is performed using a two-dimensional LUT describing the relationship between the image signal, the processed signal, and the visually processed output signal. Therefore, visual processing can be accelerated.

The visual processing program described in Remark 62 is a visual processing program for executing the visual processing method by a computer, and causes the computer to execute the visual processing method including an input signal processing step and a visual processing step. In the input signal processing step, a certain processing is performed on the input image signal, and the processed signal is output. In the visual processing step, an output signal is output based on a two-dimensional LUT based on a relationship between the input image signal and the processed signal, and an output signal that is a visually processed image signal.

Here, the fixed processing refers to, for example, direct or indirect processing of an image signal, including processing of converting pixel values of an image signal such as spatial processing or gradation processing.

In the visual processing program of the present invention, visual processing is performed using a two-dimensional LUT describing the relationship between the image signal, the processed signal, and the visually processed output signal. Therefore, visual processing can be accelerated.

The integrated circuit described in Remark 63 includes the visual processing device described in any one of Remarks 1-60.

In the integrated circuit of the present invention, the visual processing device described in any one of Remarks 1 to 60 can be obtained.

In the integrated circuit of the present invention, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 can be obtained.

The display device described in Attachment 64 includes: the visual processing device described in any one of Attachments 1 to 60, and a display mechanism that displays an output signal output from the visual processing device.

In the display device of the present invention, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 can be obtained.

The photographing device described in Remark 65, comprising: a photographing mechanism that captures an image, and the visual processing described in any one of Remarks 1 to 60 that uses the image captured by the photographing mechanism as an image signal to perform visual processing device.

In the imaging device of the present invention, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 can be obtained.

The portable information terminal described in Remark 66 is equipped with: a data receiving mechanism that receives image data for communication or broadcast; and performs visual processing on the received image data as an image signal according to any one of Remarks 1 to 60. The above-mentioned visual processing device; and a display mechanism for displaying the image signal after visual processing by the above-mentioned visual processing device.

In the portable information terminal of the present invention, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 can be obtained.

The portable information terminal described in Remark 67 includes: a photographing mechanism for photographing images;

The visual processing device described in any one of Remarks 1 to 60 that uses the image captured by the camera as an image signal for visual processing; and

A data transmission mechanism that transmits an image signal that is visually processed.

In the portable information terminal of the present invention, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 can be obtained.

The image processing device described in Note 68 is an image processing device that performs image processing on an input image signal that is input, and includes profile data creating means and image processing executing means. The profile data creating means creates profile data used for image processing based on a plurality of profile data for performing different image processing. The image processing executing means performs image processing using the profile data created by the profile data creating means.

Here, image processing refers to, for example, visual processing such as spatial processing and gradation processing, or color processing such as color conversion (hereinafter the same applies to this column).

Also, the profile data refers to, for example, coefficient matrix data for performing calculations on an input image signal; or table data for storing values of an input image signal after image processing has been performed on the values of the input image signal (hereinafter referred to in this same column).

The image processing device of the present invention creates new profile data based on a plurality of profile data. Therefore, even if the profile data prepared in advance is small, many different image processes can be performed. That is, the storage capacity for storing profile data can be reduced.

The image processing device described in Note 69 is an image processing device that performs image processing on an input image signal that is input, and includes profile information output means and image processing execution means. The profile information output means outputs profile information for specifying profile data used in image processing. The image processing execution unit performs image processing using the specified profile data based on the information output from the profile information output unit.

Here, the profile information refers to, for example, profile data, information such as a number specifying the profile data, parameter information indicating characteristics of processing of the profile data, and information for specifying other profile data. wait.

In the image processing device of the present invention, profile data is controlled based on profile information to perform image processing.

In the image processing device described in Remark 70, in the image processing device described in Remark 69, the profile information output unit outputs the profile information according to a display environment for displaying the image-processed input image signal.

Here, the display environment refers to, for example, the brightness or color temperature of ambient light, the display device, the size of the displayed image, the positional relationship between the user who can see the displayed image and the displayed image, User-related information, etc.

In the image processing device of the present invention, image processing according to the display environment can be performed.

The image processing device shown in Remark 71 is based on the image processing device described in Remark 69, and the profile information output mechanism outputs the profile information according to the information related to the profile data among the information included in the input image signal.

The information related to the profile data means, for example, profile data, information such as a number specifying the profile data, parameter information indicating the characteristics of processing of the profile data, and information for specifying other profile data. information etc.

In the image processing apparatus of the present invention, image processing can be performed by obtaining information related to profile data from an input image signal.

The image processing device described in Remark 72 is the image processing device described in Remark 69, wherein the profile information output unit outputs the profile information based on the obtained information related to the characteristics of the image processing.

The so-called information related to the characteristics of image processing refers to information about the characteristics of parameters of image processing, such as the values of parameters in the adjustment of brightness, image quality, and color.

In the image processing apparatus of the present invention, for example, image processing can be performed by inputting information related to characteristics of image processing according to the user's preference.

The image processing device described in Remark 73 is the image processing device described in Remark 69, wherein the profile information output unit outputs the profile information based on the information related to the environment in which the input image signal is generated.

The information on the environment in which the input image signal is generated includes, for example, information on the shooting environment when the input image signal is recorded by shooting, shooting permission information in the shooting environment, and the like.

In the image processing device of the present invention, image processing can be performed based on information on the environment in which the input image signal is generated.

The image processing device described in attachment 74 is the image processing device described in attachment 69, wherein the input image signal includes image data and attribute information of the input image signal. Describes the file information output mechanism. According to the attribute information, the description file information is output.

In the image processing device of the present invention, image processing can be performed based on attribute information of an input image signal. Therefore, image processing suitable for an input image signal can be performed.

The image processing device described in Attachment 75 is the image processing device described in Attachment 74, and the attribute information includes overall attribute information related to the entire image data.

The overall attribute information is, for example, information related to creation of the entire image data, information related to the content of the entire image data, and the like.

In the image processing device of the present invention, image processing can be performed based on the overall attribute information. Therefore, image processing suitable for image data can be performed.

The image processing device described in attachment 76 is the image processing device described in attachment 74 or 75, and the attribute information includes attribute information related to a part of the image data.

The partial attribute information includes, for example, information related to a part of the scene content of the image data.

In the image processing device of the present invention, image processing can be performed based on partial attribute information. Therefore, image processing suitable for image data can be performed.

The image processing device described in attachment 77 is the image processing device described in attachment 74, and the attribute information includes generation environment attribute information related to an environment in which the input image signal is generated.

The generation environment attribute information refers to information related to the environment in which the input image signal is captured, recorded, and created, for example, information about the environment when the input image signal is generated, or information about the operation of equipment used to generate the input image signal.

In the image processing device of the present invention, image processing can be performed based on the generated environment attribute information. Therefore, image processing suitable for an input image signal can be performed.

The image processing device described in Remark 78 is the image processing device described in Remark 74, and the attribute information refers to media attribute information related to the medium from which the input image signal is obtained.

The medium attribute information refers to information related to a medium from which an input image signal is obtained, such as a broadcast medium, a communication medium, and a recording medium.

In the image processing device of the present invention, image processing can be performed based on the medium attribute information. Therefore, image processing suitable for the properties of the medium can be performed.

The image processing device described in Remark 79 is the image processing device according to any one of Remarks 68 to 78, wherein the profile data is a 2-dimensional LUT. The image processing execution mechanism includes the visual processing device described in any one of Remarks 1-60.

In the image processing device of the present invention, the same effect as that of the image processing device described in any one of Remarks 68 to 78 can be obtained. Furthermore, the same effect as that of the visual processing device described in any one of Remarks 1 to 60 is obtained.

The image processing device described in Remark 80 includes: an image processing execution unit, a profile information output unit, and a profile information output unit. The image processing actuator performs image processing on the input image signal. The profile information output means outputs profile information for specifying profile data for performing image processing suitable for the input image signal. The profile information adding means adds profile information to the input image signal or the input image signal after image processing by the image processing executing means, and outputs the profile information.

With the image processing device of the present invention, it is possible to perform a process of associating an input image signal or an input image signal subjected to image processing by an image processing actuator with profile information. Therefore, an apparatus that obtains a signal to which profile information is added can easily perform appropriate image processing on the signal.

The integrated circuit described in Remark 81, including the image processing device described in any one of Remarks 68 to 80.

In the integrated circuit of the present invention, the same effect as that of the image processing device described in any one of Remarks 68 to 80 can be obtained.

The display device according to attachment 82, comprising: the image processing device according to any one of attachments 68 to 80; and a display unit for displaying an input image signal subjected to image processing by the image processing device.

In the display device of the present invention, the same effect as that of the image processing device described in any one of Remarks 68 to 80 can be obtained.

The photographing device described in Remark 83, comprising: a photographing mechanism that captures an image; and the image described in any one of Remarks 68 to 80 that uses the image captured by the photographing mechanism as an input image signal and performs image processing Processing device.

In the imaging device of the present invention, the same effect as that of the image processing device described in any one of Remarks 68 to 80 can be obtained.

The portable information terminal shown in Remark 84 is equipped with: a data receiving mechanism that receives image data transmitted by communication or broadcast; and performs image processing on the received image data as the above-mentioned input image signal in Remarks 68 to 80. The image processing device according to any one of the above; and a display unit for displaying the input image signal after image processing by the image processing device.

In the portable information terminal of the present invention, the same effect as that of the image processing device described in any one of Remarks 68 to 80 can be obtained.

The portable information terminal described in Remark 85, comprising: a photographing mechanism that captures images; and any one of Remarks 68 to 80 that performs image processing on the image captured by the photographing mechanism as an input image signal. an image processing device; and a data transmission mechanism that transmits the image-processed input image signal.

In the portable information terminal of the present invention, the same effect as that of the image processing device described in any one of Notes 60 to 80 can be obtained.

(Note 3)

The present invention (especially the inventions described in the first to third embodiments) can be expressed as follows. In addition, among the remarks of the subordinate form described in this column ("3rd Remark"), the remark described in the 3rd Remark is subordinate.

(Contents of Remark 3)

(Note 1)

A visual processing device, which has:

an input signal processing mechanism, which performs spatial processing on the input image signal, and outputs the processed signal; and

The signal calculation means outputs an output signal based on a calculation that emphasizes the difference between the respective values of the image signal and the processed signal converted by a predetermined conversion.

(Note 2)

According to the visual processing device described in Note 1,

The above-mentioned signal operation means is based on the mathematical formula F2(F1(A)+F3( F1(A)-F1(B))), calculate the value C of the output signal.

(Note 3)

According to the visual processing device described in Note 2,

The transformation function F1 described above is a logarithmic function.

(Note 4)

According to the visual processing device described in Note 2,

The above inverse transform function F2 is a gamma correction function.

(Note 5)

According to the visual processing device described in any one of Notes 2 to 4,

The above-mentioned signal calculation unit includes: a signal space conversion unit that converts the signal space of the image signal and the processed signal; The difference signal is subjected to enhancement processing; and the inverse transformation means performs signal space inverse transformation on the added signal between the transformed image signal and the enhanced difference signal to output the output signal.

(Note 6)

A visual processing device, which has:

An input signal processing mechanism, which performs spatial processing on the input image signal, and outputs the processed signal;

The signal calculation means outputs an output signal based on a calculation that enhances a ratio between the image signal and the processed signal.

(Remark 7)

According to the visual processing device described in Note 6,

The signal calculation unit outputs the output signal based on the calculation that further compresses the dynamic range of the image signal.

(Remark 8)

According to the visual processing device described in remark 6 or 7,

The above-mentioned signal operation means calculates the value of the output signal based on the mathematical formula F4(A)×F5(A/B) with respect to the value A of the image signal, the value B of the processed signal, the dynamic range compression function F4, and the enhancement function F5 c.

(Remark 9)

According to the visual processing device described in Note 8,

The above-mentioned dynamic range compression function F4 is a proportional function with a proportional coefficient of 1.

(Remark 10)

According to the visual processing device described in Note 8,

The above dynamic range compression function F4 is a monotonically increasing function.

(Note 11)

According to the visual processing device described in Note 10,

The above-mentioned dynamic range compression function F4 is an upward convex function.

(Note 12)

According to the visual processing device described in Note 8,

The dynamic range compression function F4 described above is a power function.

(Note 13)

According to the visual processing device described in note 12,

The exponent of the power function in the above-mentioned dynamic range compression function F4 is determined based on a target contrast value which is a target contrast value when displaying an image, and an actual contrast value which is a contrast value in a display environment when displaying an image. .

(Note 14)

According to the visual processing device described in any one of Notes 8 to 13,

The enhancement function F5 described above is a power function.

(Note 15)

According to the visual processing device described in Note 14,

The exponent of the power function in the enhancement function F5 is determined based on a target contrast value which is a target value of contrast when displaying an image, and an actual contrast value which is a contrast value in a display environment when displaying an image.

(Note 16)

According to the visual processing device described in remark 14 or 15,

When the value A of the image signal is larger than the value B of the processed signal, the exponent of the power function in the enhancement function F5 is a value that monotonically decreases with respect to the value A of the image signal.

(Note 17)

According to the visual processing device described in remark 14 or 15,

When the value A of the image signal is smaller than the value B of the processed signal, the exponent of the power function in the enhancement function F5 is a value that monotonically increases with respect to the value A of the image signal.

(Note 18)

According to the visual processing device described in remark 14 or 15,

When the value A of the image signal is larger than the value B of the processed signal, the exponent of the power function in the enhancement function F5 is a value that monotonically increases with respect to the value A of the image signal.

(Note 19)

According to the visual processing device described in remark 14 or 15,

The exponent of the power function in the enhancement function F5 is a value that increases monotonically with respect to the absolute value of the difference between the value A of the image signal and the value B of the processed signal.

(Note 20)

According to the visual processing device described in any one of Notes 14 to 19,

At least one of the maximum value or the minimum value of the enhancement function F5 is limited within a predetermined range.

(Note 21)

According to the visual processing device described in Note 8,

The above-mentioned signal calculation unit includes: an enhancement processing unit that performs enhancement processing on a division signal obtained by dividing the image signal by the processing signal; and an output processing unit based on the image signal and the division signal obtained by the enhancement process. Process the signal and output the above-mentioned output signal.

(Note 22)

According to the visual processing device described in note 21,

The output processing means performs multiplication processing on the image signal and the division-processed signal subjected to the gradation processing.

(Note 23)

According to the visual processing device described in note 21,

The output processing means includes a DR compression means for performing dynamic range (DR) compression on the image signal, and performs multiplication processing on the image signal compressed by the DR and the division signal after the enhancement processing.

(Note 24)

The visual processing device according to any one of Remarks 8-23, further comprising:

a first conversion mechanism that converts the input image data in the first specified range into the second specified range, and uses it as the above-mentioned image signal;

The second conversion mechanism converts the above-mentioned output signal in the third predetermined range into the fourth predetermined range as an output image signal,

The above-mentioned second predetermined range is determined based on a target contrast value which is a target value of contrast when displaying an image,

The above-mentioned third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

(Note 25)

According to the visual processing device described in Note 24,

The dynamic range compression function F4 is a function for converting the image signal in the second predetermined range into the output signal in the third predetermined range.

(Note 26)

According to the visual processing device described in remark 24 or 25,

The first conversion means converts the minimum value and maximum value of the first predetermined range into respective minimum values and maximum values of the second predetermined range.

The second conversion means converts the minimum value and maximum value of the third predetermined range into the minimum value and maximum value of the fourth range, respectively.

(Note 27)

According to the visual processing device described in note 26,

The transformations in the first transformation mechanism and the second transformation mechanism are each linear transformations.

(Note 28)

According to the visual processing device described in any one of Notes 24 to 27,

It also includes setting means for setting the above-mentioned third predetermined range.

(Note 29)

According to the visual processing device described in note 28,

The above-mentioned setting means includes: storage means for storing a dynamic range of a display device that displays an image; and measurement means for measuring brightness of ambient light in a display environment when an image is displayed.

(Note 30)

According to the visual processing device described in note 28,

The setting means includes a measuring means for measuring the luminance of the display device performing image display at the time of black-level display and white-level display in a display environment.

(Note 31)

A visual processing device, which has:

an input signal processing mechanism that performs image processing on the input image signal and outputs the processed signal; and

A signal calculation means that outputs an output signal based on a calculation that enhances a difference between the image signal and the processed signal based on the value of the image signal.

(Note 32)

According to the visual processing device described in note 31,

The signal calculation means outputs the output signal based on a calculation of adding a value obtained by compressing the dynamic range of the image signal to the value enhanced by the enhancement calculation.

(Note 33)

According to the visual processing device described in remark 31 or 32,

The above-mentioned signal operation means is based on the mathematical formula F8(A)+F6(A)× F7(A-B), calculate the value C of the output signal.

(Note 34)

According to the visual processing device described in note 33,

The above-mentioned dynamic range compression function F8 is a proportional function with a proportional factor of 1.

(Note 35)

According to the visual processing device described in note 33,

The above dynamic range compression function F8 is a monotonically increasing function.

(Note 36)

A visual processing device as described in Remark 35,

The above-mentioned dynamic range compression function F8 is an upward convex function.

(Note 37)

According to the visual processing device described in note 33,

The above-mentioned dynamic range compression function F8 is a power function.

(Note 38)

According to the visual processing device described in note 33,

The above-mentioned signal arithmetic unit includes: an enhancement processing unit that performs enhancement processing corresponding to pixel values of the above-mentioned image signal on a difference signal between the above-mentioned image signal and the above-mentioned processed signal; and an output processing unit based on the above-mentioned image signal and the difference signal after the above-mentioned enhancement processing, and output the above-mentioned output signal.

(Note 39)

According to the visual processing device described in note 33,

The output processing means performs addition processing between the image signal and the enhanced difference signal.

(Note 40)

According to the visual processing device described in note 38,

The output processing means includes a DR compression means for compressing a dynamic range (DR) of the image signal, and performs an addition process on the image signal compressed by the DR and the difference signal after the enhancement processing.

(Note 41)

A visual processing device, which has:

an input signal processing mechanism that performs image processing on the input image signal and outputs the processed signal; and

A signal calculation means that outputs an output signal based on a calculation that adds a gradation-corrected value of the image signal to a value emphasizing a difference between the image signal and the processed signal.

(Note 42)

According to the visual processing device described in note 41,

The above-mentioned signal operation means calculates the value of the output signal based on the mathematical formula F12(A)+F11(A-B) with respect to the value A of the image signal, the value B of the processed signal, the transformation function F11, and the gradation correction function F12 c.

(Note 43)

According to the visual processing device described in remark 42,

The above-mentioned signal operation unit has: an enhancement processing unit that performs enhancement processing on a difference signal between the above-mentioned image signal and the above-mentioned processed signal; The processed difference signal is added and output as an output signal.

(Note 44)

A visual processing method having:

The first conversion step is to convert the input image data in the first specified range into the second specified range, and use it as an image signal;

The signal calculation step is based on at least one of calculations that include dynamic range compression of the image signal, or calculations that enhance the ratio between the image signal and a processed signal that is spatially processed on the image signal, and the third The output signal output within the specified range;

In the second conversion step, the above-mentioned output signal in the above-mentioned third predetermined range is converted into a fourth predetermined range, and as an output image signal,

The above-mentioned second predetermined range is determined based on a target contrast value which is a target value of contrast when displaying an image,

The above-mentioned third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

(Note 45)

A visual processing device having:

The first conversion mechanism converts the input image data in the first specified range into the second specified range, and uses it as an image signal;

A signal calculation unit that performs a calculation based on at least one of a calculation including dynamic range compression of the image signal or a calculation that enhances a ratio between the image signal and a processed signal obtained by spatially processing the image signal. 3 The output signal output within the specified range;

The second conversion mechanism converts the above-mentioned output signal in the above-mentioned third predetermined range into a fourth predetermined range, and uses it as an output image signal,

The above-mentioned second predetermined range is determined based on a target contrast value which is a target value of contrast when displaying an image,

The above-mentioned third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

(Note 46)

A visual processing program for causing a computer to perform visual processing, having:

The first conversion step is to convert the input image data in the first specified range into the second specified range, and use it as an image signal;

The signal calculation step is based on at least one of calculations including dynamic range compression of the image signal, or enhancement of the ratio of the image signal to a processed signal obtained by spatially processing the image signal, and the third predetermined range The output signal output;

In the second conversion step, the above-mentioned output signal in the above-mentioned third predetermined range is converted into a fourth predetermined range, and as an output image signal,

The above-mentioned second predetermined range is determined based on a target contrast value which is a target value of contrast when displaying an image,

The above-mentioned third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

(Description of Remark 3)

The visual processing device described in Note 1 includes: input signal processing means and signal calculation means. The input signal processing unit performs spatial processing on the input image signal and outputs the processed signal. The signal calculation means outputs an output signal based on a calculation that intensifies the difference between the values of the image signal and the processed signal converted by a predetermined conversion.

Here, the so-called spatial processing refers to the processing of applying a low-frequency spatial filter to the input image signal, or deriving the average value, maximum value, or minimum value between the pixel of interest and surrounding pixels of the input image signal, etc. Processing, etc. (the same applies to this column below). Also, the enhancement calculation refers to, for example, a calculation for adjusting gain, a calculation for suppressing excessive contrast, a calculation for suppressing small-amplitude noise components, etc. (hereinafter the same applies to this column).

In the visual processing device of the present invention, after transforming the image signal and the processed signal into different spaces, each difference can be enhanced. In this way, reinforcement with visual characteristics and the like can be realized.

The visual processing device described in Note 2 is the visual processing device described in Note 1, the signal operation mechanism, and its value A of the image signal, the value B of the processed signal, the transformation function F1, and the inverse transformation function of the transformation function F1 F2, the strengthening function F3, calculates the value C of the output signal based on the mathematical formula F2 (F1(A)+F3(F1(A)-F1(B))).

The enhancement function F3 is, for example, a function for adjusting gain, a function for suppressing excessive contrast, a function for suppressing small-amplitude noise components, and the like.

The value C of the output signal is expressed as follows. That is, the value A of the image signal and the value B of the processed signal are transformed into values in another space by the transformation function F1. The difference between the value of the transformed image signal and the value of the processed signal represents, for example, a sharp signal in a different space or the like. The difference between the transformed image signal and the processed signal enhanced by the enhancement function F3 is added to the transformed image signal. In this way, the value C of the output signal represents a value obtained by enhancing clear components in different spaces.

In the visual processing device of the present invention, for example, by using the value A of the image signal converted into different spaces and the value B of the processed signal, processing such as edge enhancement and contrast enhancement in different spaces can be realized.

The visual processing device described in Note 3 is the visual processing device described in Note 2, and the transformation function F1 is a logarithmic function.

Here, the visual characteristics of human beings are generally logarithmic. Therefore, by transforming into a logarithmic space and processing the image signal and the processed signal, processing suitable for visual characteristics can be performed.

In the visual processing device of the present invention, it is possible to perform contrast enhancement with high visual effect, or dynamic range compression to maintain local contrast.

The visual processing device described in Note 4 is the visual processing device described in Note 2, and the inverse transform function F2 is a gamma correction function.

Generally, an image signal is subjected to gamma correction by a gamma correction function according to the gamma characteristics of a device that inputs and outputs the image signal.

In the visual processing device of the present invention, the gamma correction of the image signal is eliminated by the transformation function F1, and the processing can also be performed according to the linear characteristic. In this way, optical blurring can be corrected.

The visual processing device described in Note 5 is the visual processing device described in any one of Notes 2 to 6, and the signal calculation mechanism includes: a signal space transformation mechanism, an enhancement processing mechanism, and an inverse transformation mechanism. The signal space changing mechanism transforms the image signal and the signal space of the processed signal. The enhanced processing mechanism performs enhanced processing on the difference signal between the transformed image signal and the transformed processed signal. The inverse transformation means performs inverse transformation of the signal space on the added signal between the transformed image signal and the enhanced difference signal, and outputs an output signal.

In the visual processing device of the present invention, the signal space conversion means uses the conversion function F1 to convert the signal space between the image signal and the processed signal. The enhancement processing mechanism uses the enhancement function F3 to perform enhancement processing on the difference signal between the transformed image signal and the transformed processed signal. The inverse transform means uses the inverse transform function F2 to inversely transform the signal space of the added signal between the transformed image signal and the enhanced difference signal.

The visual processing device described in Note 6 includes: input signal processing means and signal calculation means. The input signal processing unit performs spatial processing on the input image signal and outputs the processed signal. The signal calculation means outputs an output signal based on a calculation that strengthens the ratio between the image signal and the processed signal.

In the visual processing device of the present invention, for example, the ratio between the image signal and the processed signal indicates the sharp component of the image signal. Thus, for example, visual manipulations that intensify clear components can be performed.

The visual processing device described in Note 7 is the visual processing device described in Note 6, and the signal calculation mechanism, which outputs an output signal based on further calculation of dynamic range compression of the image signal.

In the visual processing device of the present invention, for example, it is possible to compress the dynamic range while enhancing the clear component of the image signal represented by the ratio between the image signal and the processed signal.

The visual processing device described in Remark 8 is the visual processing device described in Remark 6 or 7, and the signal operation mechanism is for the value A of the image signal, the value B of the processed signal, the dynamic range compression function F4, and the enhancement function F5 , based on the mathematical formula F4(A)×F5(A/B), the value C of the output signal is calculated.

Here, the value C of the output signal is expressed as follows. That is, the division amount (A/B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F5(A/B) represents, for example, the enhancement amount of a clear signal. This means that the value A of the image signal and the value B of the processed signal are transformed into a logarithmic space, and the enhancement processing is performed on each difference, and the enhancement processing suitable for the visual characteristics is performed.

In the visual processing device of the present invention, the dynamic range can be compressed and the local contrast can be enhanced at the same time as required.

The visual processing device described in Remark 9 is based on the compression of the dynamic range described in Remark 8, and the dynamic range compression function F4 is a proportional function with a proportional coefficient of 1.

In the visual processing device of the present invention, the contrast can be enhanced from the dark part to the bright part of the image signal. This contrast enhancement is an enhancement process suitable for visual characteristics.

The visual processing device described in Note 10 is the visual processing device described in Note 8, and the dynamic range compression function F4 is a monotonically increasing function.

In the visual processing device of the present invention, while performing dynamic range compression using the dynamic range compression function F4 which is a monotonically increasing function, local contrast enhancement can be realized.

The visual processing device described in Remark 11 is the visual processing device described in Remark 10, and the dynamic range compression function F4 is an upward convex function.

In the visual processing device of the present invention, dynamic range compression is performed using the dynamic range compression function F4 which is an upward convex function, and local contrast can be enhanced at the same time.

The visual processing device described in Note 12 is the visual processing device described in Note 8, and the dynamic range compression function F4 is a power function.

In the visual processing device of the present invention, the dynamic range compression function F4 as a power function can be used to transform the dynamic range while enhancing the local contrast.

The visual processing device described in Remark 13 is the visual processing device described in Remark 12, wherein the exponent of the power function in the dynamic range compression function F4 is based on a target contrast value that is a target value of contrast when displaying an image, and an actual contrast value which is a contrast value in a display environment when an image is displayed is determined.

Here, the target contrast value is a target value of contrast when displaying an image, and is, for example, a value determined by the dynamic range of a display device that displays an image. The actual contrast value is a contrast value in a display environment when an image is displayed, for example, a value determined by the contrast of an image displayed on a display device in the presence of ambient light.

In the visual processing device of the present invention, the image signal having a dynamic range equal to the target contrast value can be compressed into a dynamic range corresponding to the actual contrast value through the dynamic range compression function F4.

The visual processing device described in Remark 14 is the visual processing device according to any one of Remarks 8 to 13, and the enhancement function F5 is a power function.

In the visual processing device of the present invention, by using the enhancement function F5 which is a power function, the local contrast can be enhanced, and the dynamic range can be transformed visually.

The visual processing device described in Remark 15 is the visual processing device described in Remark 14, wherein the exponent of the power function in the enhancement function F5 is based on a target contrast value that is a target value of contrast when displaying an image, and When displaying an image, an actual contrast value is determined as a contrast value in a display environment.

In the visual processing device of the present invention, the enhancement function F5, which is a power function, can be used to enhance the local contrast and visually change the dynamic range.

The visual processing device described in Remark 16 is based on the visual processing device described in Remark 14 or 15. When the value A of the image signal is greater than the value B of the processed signal, the exponent of the power function in the enhancement function F5 is relatively The value A of the image signal decreases monotonically.

In the visual processing device of the present invention, it is possible to weaken the enhancement of the local contrast in the high-brightness portion of the pixel of interest that is brighter than surrounding pixels in the image signal. Therefore, so-called highlighting in the visually processed image can be suppressed.

In the visual processing device described in Remark 17, according to the visual processing device described in Remark 14 or 15, when the value A of the image signal is smaller than the value B of the processed signal, the exponent of the power function in the strengthening function F5 is A value that increases monotonically with respect to the value A of the image signal.

In the visual processing device of the present invention, it is possible to weaken the enhancement of the local contrast in the low-luminance portion of the pixel of interest which has a lower luminance than surrounding pixels in the image signal. Therefore, it is possible to suppress so-called reverse blacking in the visually processed image.

The visual processing device described in Remark 18 is based on the visual processing device described in Remark 14 or 15. When the value A of the image signal is greater than the value B of the processed signal, the exponent of the power function in the enhancement function F5 is relatively The value A of the image signal is a value that increases monotonically.

In the visual processing device of the present invention, it is possible to weaken the enhancement of local contrast in a low-luminance portion of a pixel of interest that is brighter than surrounding pixels in an image signal. Therefore, deterioration of the so-called S/N ratio in the visually processed image can be suppressed.

The visual processing device described in Remark 19 is the visual processing device described in Remark 14 or 15, wherein the exponent of the power function in the enhancement function F5 is the difference between the value A of the relative image signal and the value B of the processed signal The absolute value of the value, a monotonically increasing value.

Here, as a value that increases monotonously with respect to the absolute value of the difference between the value A of the image signal and the value B of the processed signal, it can also be defined as the ratio between the value A of the image signal and the value B of the processed signal. The closer it is to 1, the more it increases.

In the visual processing device of the present invention, in the image signal, the local contrast in the pixel of interest that has a smaller brightness difference with the surrounding pixels can be particularly enhanced, and it is not necessary to adjust the local contrast in the image signal that has a smaller brightness difference with the surrounding pixels. Local contrast in large pixels of interest is over-emphasized.

The visual processing device in Remark 20 is the visual processing device according to any one of Remarks 14 to 19, wherein at least one of the maximum value or the minimum value of the enhancement function F5 is limited within a predetermined range.

In the visual processing device of the present invention, the enhancement amount of the local contrast is limited within an appropriate range.

The visual processing device according to Remark 21 is the visual processing device according to Remark 8, and the signal calculation mechanism includes: an enhancement processing mechanism and an output processing mechanism. The enhancement processing means performs enhancement processing on the divided signal obtained by dividing the image signal by the processed signal. The output processing means outputs an output signal based on the image signal and the enhanced division signal.

In the visual processing device of the present invention, the enhancement processing means performs enhancement processing using the enhancement function F5 on the divided signal obtained by dividing the image signal by the processed signal. The output processing means outputs the output signal based on the image signal and the division processing signal.

The visual processing device described in Remark 22 is the visual processing device described in Remark 21, wherein the output processing mechanism performs multiplication processing on the image signal and the enhanced division signal.

In the visual processing device of the present invention, the dynamic range compression function F4 is, for example, a proportional function with a proportional coefficient of 1.

The visual processing device described in Remark 23 is the visual processing device described in Remark 21, the output processing mechanism includes a DR compression mechanism for performing dynamic range (DR) compression on the image signal, and compresses the DR compressed image signal and The multiplication processing is carried out on the division processing signal after the enhancement processing.

In the visual processing device of the present invention, the DR compression means performs dynamic range compression of the image signal using the dynamic range compression function F4.

The visual processing device according to attachment 24 is the visual processing device according to any one of attachments 8 to 23, further comprising: a first conversion mechanism and a second conversion mechanism. The first conversion means converts the input image data in the first predetermined range into a second predetermined range as image data. The second conversion means converts the output signal in the third predetermined range into a fourth predetermined range, and uses it as output image data. The second predetermined range is determined based on a target contrast value that is a target value of contrast when displaying an image. The third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

In the visual processing device of the present invention, the overall dynamic range of the image can be compressed to the actual contrast lowered by the ambient light, while the target contrast value can be locally maintained. Therefore, the visual effect of the visually processed image is improved.

The visual processing device described in attachment 25 is the visual processing device described in attachment 24, wherein the dynamic range compression function F4 is a function for converting an image signal in the second predetermined range into an output signal in the third predetermined range.

In the visual processing device of the present invention, the dynamic range of the entire image is compressed up to the third predetermined range by the dynamic range compression function F4.

The visual processing device shown in Remark 26 is based on the visual processing device shown in Remark 24 or 25, wherein the first conversion mechanism converts the minimum value and maximum value in the first specified range into the minimum value and maximum value in the second specified range, respectively. Each of the maximum values. The second conversion means converts the minimum value and maximum value of the third predetermined range into the minimum value and maximum value of the fourth predetermined range, respectively.

The visual processing device described in Attachment 27 is the visual processing device described in Attachment 26, wherein the transformations in the first transformation means and the second transformation means are linear transformations, respectively.

The visual processing device according to attachment 28 is the visual processing device according to any one of attachments 24 to 27, further comprising setting means for setting the third predetermined range.

In the visual processing device of the present invention, the third predetermined range can be set according to the display environment of the display device that displays the image. Therefore, correction of ambient light can be performed more appropriately.

The visual processing device shown in Remark 29 is based on the visual processing device shown in Remark 28, the setting mechanism includes: a storage mechanism that stores the dynamic range of the display environment for image display; and a measurement mechanism that The brightness of ambient light in the display environment when displaying an image was measured.

In the visual processing device of the present invention, the brightness of ambient light is measured, and the actual contrast value can be determined according to the measured brightness and the dynamic range of the display device.

The visual processing device described in Remark 30 is based on the visual processing device described in Remark 29, the setting mechanism includes the measuring mechanism, which controls the black level display and white power of the display device for image display in the display environment. The brightness of the flat display is measured.

In the visual processing device of the present invention, it is possible to measure the luminance in the display environment when the black level is displayed and when the white level is displayed, so as to determine the actual contrast value.

The visual processing device described in Note 31 includes input signal processing means and signal calculation means. The input signal processing mechanism performs spatial processing on the input image signal and outputs the processed signal. The signal operation means outputs an output signal based on an operation that enhances a difference between the image signal and the processed signal based on the value of the image signal.

In the visual processing device of the present invention, for example, the sharp component of the image signal which is the difference between the image signal and the processed signal can be enhanced according to the value of the image signal. Therefore, appropriate enhancement can be performed from the dark part to the bright part of the image signal.

The visual processing device described in Remark 32 is based on the visual processing device described in Remark 31, and the signal operation mechanism is based on adding the value obtained by compressing the dynamic range of the image signal to the value enhanced by the enhancement operation. operation, output the output signal.

In the visual processing device of the present invention, for example, it is possible to compress the dynamic range while enhancing the sharp components of the image signal and the like according to the value of the image signal.

The visual processing device described in Remark 33 is the visual processing device described in Remark 31 or 32, and the signal operation mechanism is for the value A of the image signal, the value B of the processed signal, the enhancement amount adjustment function F6, and the enhancement function F7 . The dynamic range compression function F8 calculates the value C of the output signal based on the mathematical formula F8(A)+F6(A)×F7(A-B).

Here, the value C of the output signal is expressed as follows. That is, the difference (A-B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F7(A-B) represents, for example, the amount of enhancement of a clear signal. Furthermore, the enhancement amount is adjusted according to the value A of the image signal by the enhancement amount adjustment function F6, and is added to the image signal subjected to dynamic range compression as necessary.

In the visual processing device of the present invention, for example, although the value of the image signal A is large, the contrast from dark parts to bright parts can be maintained by reducing the amount of enhancement or the like. In addition, even in the case of performing dynamic range compression, local contrast from dark parts to bright parts can be maintained.

The visual processing device shown in Remark 34 is the visual processing device described in Remark 33, and the dynamic range compression function F8 is a proportional function with a proportional coefficient of 1.

In the visual processing device of the present invention, the contrast can be uniformly enhanced from the dark part to the bright part of the image signal.

The visual processing device described in Remark 35 is the visual processing device described in Remark 33, and the dynamic range compression function F8 is a monotonically increasing function.

In the visual processing device of the present invention, the dynamic range compression function F8, which is a monotonically increasing function, can be used to perform dynamic range compression while maintaining local contrast.

The visual processing device described in Remark 36 is the visual processing device described in Remark 35, and the dynamic range compression function F8 is an upward convex function.

In the visual processing device of the present invention, it is possible to perform dynamic range compression while maintaining contrast by using the dynamic range compression function F8 which is an upward convex function.

The visual processing device described in Remark 37 is the visual processing device described in Remark 33, and the dynamic range compression function F8 is a power function.

In the visual processing device of the present invention, the dynamic range compression function F8, which is a power function, can be used to perform dynamic range conversion while maintaining local contrast.

The visual processing device described in Remark 38 is the visual processing device described in Remark 33, wherein the signal calculation means includes an enhancement processing means and an output processing means. The enhancement processing mechanism performs enhancement processing corresponding to the pixel value of the image signal for the difference signal between the image signal and the processed signal. The output processing mechanism outputs the output signal based on the image signal and the enhanced difference signal.

In the visual processing device of the present invention, the enhancement processing means performs enhancement processing using the enhancement function F7 whose enhancement amount is adjusted by the enhancement amount adjustment function F6. The output processing mechanism outputs the output signal based on the image signal and the difference signal.

The visual processing device described in Remark 39 is based on the visual processing device described in Remark 38, wherein the output processing mechanism performs addition processing on the image signal and the enhanced difference signal.

In the visual processing device of the present invention, the dynamic range compression function F8 is, for example, a proportional function with a proportional factor of 1.

The visual processing device described in Remark 40 is the visual processing device described in Remark 38, the output processing mechanism includes a DR compression mechanism for performing dynamic range (DR) compression on the image signal, and the image signal compressed by DR and Addition processing is performed on the enhanced difference signal.

In the visual processing device of the present invention, the DR compression means compresses the dynamic range of the image signal using the dynamic range compression function F8.

The visual processing device shown in Note 41 includes: input signal processing means and signal calculation means. The input signal processing unit performs spatial processing on the input image signal, and outputs the processed signal. The signal calculation means outputs an output signal based on calculation of adding a value obtained by performing gradation correction on the image signal to a value emphasizing the difference between the image signal and the processed signal.

In the visual processing device of the present invention, for example, the difference between the image signal and the processed signal represents the sharp component of the image signal. In addition, the enhancement of clear components and the gradation correction of image signals are performed independently. Therefore, regardless of the amount of gradation correction of the image signal, it is possible to enhance the clear components to a certain extent.

The visual processing device described in Remark 42 is based on the visual processing device described in Remark 41, and the signal operation mechanism is based on the value A of the image signal, the value B of the processed signal, the enhancement function F11, and the grayscale correction function F12, Based on the mathematical formula F12(A)+F11(A-B), the value C of the output signal is calculated.

Here, the value C of the output signal is expressed as follows. That is, the difference (A-B) between the value A of the image signal and the value B of the processed signal represents, for example, a sharp signal. Also, F11(A-B) represents, for example, enhancement processing of a clear signal. Furthermore, it means adding the gradation-corrected image signal and the enhanced clear signal.

In the visual processing device of the present invention, a certain contrast enhancement can be performed regardless of the gradation correction.

The visual processing device described in Remark 43 is the visual processing device described in Remark 42, wherein the signal calculation unit includes an enhancement processing unit and an addition processing unit. The enhanced processing mechanism performs enhanced processing on the difference signal between the image signal and the processed signal. The addition processing mechanism performs addition processing on the grayscale-corrected image signal and the enhanced difference signal, and outputs it as an output signal.

In the visual processing device of the present invention, the enhancement processing mechanism performs enhancement processing on the difference signal using the enhancement function F11. The addition processing means performs addition processing on the image signal after the grayscale correction processing using the grayscale correction function F12 and the difference signal after the grayscale processing.

The visual processing method described in Note 44 includes a first conversion step, a signal calculation step, and a second conversion step. The first conversion step is to convert the input image data in the first specified range into the second specified range, and use it as an image signal; the signal operation step is based on the operation including dynamic range compression of the image signal, or the enhancement of the image signal and the corresponding At least one of the calculations of the ratio between the processed signals after the image signal is subjected to spatial processing is to output the output signal in the third specified range; the second conversion step is to convert the output signal in the third specified range into a fourth specified range As the output image signal, the second specified range is determined based on the target contrast value as the target value of the contrast when performing image display, and the third specified range is determined based on the target value of contrast when performing image display. Determined as the actual contrast value of the contrast value in the display environment.

In the visual processing method of the present invention, for example, the overall dynamic range of the image can be compressed until the actual contrast value is reduced due to the presence of ambient light, while the target contrast value can be locally maintained. Therefore, the visual effect of the visually processed image is improved.

The visual processing device described in Note 45 includes a first conversion means, a signal calculation means, and a second conversion means. The first conversion mechanism converts the input image data in the first specified range into the second specified range as an image signal; the signal operation step is based on the operation including dynamic range compression of the image signal, or the enhancement of the image signal and the At least one of the calculations of the ratio between the processed signals after the image signal is subjected to spatial processing is to output the output signal in the third specified range; the second conversion step is to convert the output signal in the third specified range into a fourth specified range and as an output image signal, the second predetermined range is determined based on the target contrast value which is the target value of the contrast when performing image display, and the third predetermined range is determined based on the The actual contrast value is determined as the actual contrast value in the display environment.

In the visual processing device of the present invention, for example, the overall dynamic range of the image can be compressed until the actual contrast value is reduced due to the presence of ambient light, while the target contrast value can be locally maintained. Therefore, the visual effect of the visually processed image is improved.

The visual processing program shown in Remark 46 is a visual processing program for causing a computer to execute visual processing, and causes the computer to execute a visual processing method including a first conversion step, a signal calculation step, and a second conversion step.

In the first conversion step, the input image data in the first predetermined range is converted into a second predetermined range, and the image signal is obtained. The signal calculation step is based on at least one of the calculations including the calculation of compressing the dynamic range of the image signal or the calculation of enhancing the ratio between the image signal and the processed signal after the spatial processing of the image signal, and the output of the third predetermined range signal output. In the second conversion step, the output signal in the third predetermined range is converted into a fourth predetermined range, and is output as image data. The second predetermined range is determined based on a target contrast value that is a target value of contrast when displaying an image. The third predetermined range is determined based on an actual contrast value that is a contrast value in a display environment when an image is displayed.

In the visual processing program of the present invention, for example, the overall dynamic range of the image can be compressed until the actual contrast value is reduced due to the presence of ambient light, while the target contrast value can be locally maintained. Therefore, the visual effect of the visually processed image is improved.

(industrial availability)

With the visual processing device of the present invention, a hardware configuration independent of the implemented visual processing can be provided, as a visual processing device, especially a visual processing device that performs visual processing such as spatial processing or grayscale processing of image signals. The processing unit is useful.

Claims (46)

1. A visual processing device, comprising: an input signal processing unit that performs pixel value conversion processing for adding conversion to pixel values of the input image signal, and outputs the processed signal; and A visual processing means for converting all the inputted image signals and the processed signal to an output signal that is the visually processed image signal based on a conversion means transforming the image signal, outputting the output signal, The conversion relationship given by the conversion mechanism is determined based on the conversion of changing brightness, The transformation that changes the shading is a transformation that outputs the output signal that is monotonically decreased with respect to the processed signal. 2. The visual processing device according to claim 1, wherein: The processed signal is a signal for performing the pixel value conversion process on the pixel of interest included in the image signal and pixels surrounding the pixel of interest. 3. The visual processing device according to claim 1, wherein: The conversion relationship given by the conversion mechanism is a non-linear relationship between at least a part of the image signal or at least a part of the processed signal and at least a part of the output signal. 4. The visual processing device according to claim 3, characterized in that: The conversion relationship given by the conversion mechanism is a nonlinear relationship between both the image signal and the processed signal and the output signal. 5. The visual processing device according to any one of claims 1-4, characterized in that: The conversion relationship given by the conversion means is determined based on an operation that enhances a value calculated from the image signal and the processed signal. 6. The visual processing device according to claim 5, characterized in that: The enhancement operation is a non-linear function. 7. The visual processing device according to claim 5, characterized in that: The enhancement operation is a transformation using a transformed value of the image signal or the processed signal. 8. The visual processing device according to claim 5, wherein: The enhancement operation is an enhancement function that enhances the difference between each transformed value after transforming the image signal and the processed signal. 9. The visual processing device according to claim 5, wherein: The enhancement operation is an enhancement function that enhances the ratio of the image signal to the processed signal. 10. The visual processing device according to claim 1, wherein: The transformation that changes the shading is a transformation that changes the level or gain of the image signal. 11. The visual processing device according to claim 1, wherein: The transformation for changing the brightness is a transformation determined based on the processed signal. 12. The visual processing device according to claim 1, wherein: The transformation mechanism stores the relationship between the image signal and the output signal as a group of grayscale transformation curves composed of a plurality of grayscale transformation curves. 13. The visual processing device according to claim 12, characterized in that, The processing signal is a signal for selecting a corresponding grayscale transformation curve from the plurality of grayscale transformation curve groups. 14. The visual processing device according to claim 13, characterized in that, The value of the processed signal is associated with at least one grayscale transformation curve contained in the plurality of grayscale transformation curve groups. 15. The visual processing device according to claim 1, wherein: The conversion means is constituted by a look-up table in which profile data created in advance by a predetermined calculation is registered. 16. The visual processing device according to claim 15, characterized in that, The lookup table can be changed by registration of profile data. 17. The visual processing device of claim 15, wherein: It further includes profile data registration means for registering the profile data in the visual processing means. 18. The visual processing device of claim 17, wherein: The visual processing mechanism obtains the description file data created by an external device. 19. The visual processing device of claim 18, wherein: The look-up table can be altered by the acquired profile data. 20. A visual processing device according to claim 18 or 19, characterized in that The visual processing mechanism obtains the description file data via a communication network. 21. The visual processing device of claim 15, wherein: It further includes profile data creating means for creating the profile data. 22. The visual processing device of claim 21, wherein: The profile data creating means creates the profile data based on a histogram of the grayscale characteristic of the image signal. 23. The visual processing device of claim 15, wherein: The profile data registered in the lookup table is switched according to predetermined conditions. 24. The visual processing device of claim 23, wherein: The prescribed conditions refer to conditions related to lightness and darkness. 25. The visual processing device of claim 24, wherein: The brightness is the brightness of the image signal. 26. The visual processing device of claim 25, wherein: further comprising a brightness judging means for judging the brightness of the image signal, The profile data registered in the lookup table is switched according to the judgment result of the brightness judging means. 27. The visual processing device of claim 24, wherein: further comprising lightness input means for inputting conditions related to the lightness, The description file data registered in the lookup table is switched according to the input result of the brightness input means. 28. The visual processing device of claim 27, wherein: The brightness input means inputs the brightness of the output environment of the output signal or the brightness of the input environment of the input signal. 29. The visual processing device of claim 24, wherein: further comprising a lightness detection mechanism, the lightness detection mechanism detects at least two kinds of said lightness and darkness, The description file data registered in the lookup table is switched according to the detection result of the lightness detection mechanism. 30. The visual processing device of claim 29, wherein: The brightness detected by the brightness detection mechanism includes: the brightness of the image signal; the brightness of the output environment of the output signal, or the brightness of the input environment of the input signal. 31. The visual processing device of claim 23, wherein: further comprising profile data selection means for selecting the profile data registered in the lookup table, The profile data registered in the lookup table is switched according to the selection result of the profile data selection means. 32. The visual processing device of claim 31, wherein: The profile data selection means is an input device for selecting a profile. 33. The visual processing device of claim 23, wherein: further comprising image characteristic judging means for judging the image characteristics of the image signal, The profile data registered in the lookup table is switched according to the judgment result of the image characteristic judging means. 34. The visual processing device of claim 23, wherein: further having a user identification mechanism that identifies the user, The description file data registered in the lookup table is switched according to the identification result of the user identification means. 35. The visual processing device of claim 15, wherein: The visual processing mechanism performs an interpolation operation on the values stored in the look-up table, and outputs the output signal. 36. The visual processing device of claim 35, wherein: The interpolation operation is a linear interpolation performed based on a value of at least one lower bit of the image signal or the processed signal represented by a binary number. 37. The visual processing device according to any one of claims 1-4, characterized in that: The input signal processing mechanism performs spatial processing on the image signal. 38. The visual processing device of claim 37, wherein: The input signal processing unit generates an unsharp signal based on the image signal. 39. The visual processing device of claim 37, wherein: In the spatial processing, an average value, a maximum value or a minimum value of the image signal is derived. 40. The visual processing device according to any one of claims 1-4, characterized in that: The visual processing unit performs spatial processing and gradation processing using the input image signal and the processing signal. 41. A visual processing method, comprising: an input signal processing step of performing pixel value conversion processing of adding conversion to pixel values of the input image signal to the input image signal, and outputting the processed signal; and a visual processing step of converting all the inputted image signals and the processed signal to an output signal which is the visually processed image signal based on a transformation mechanism transforming the image signal, outputting the output signal, The conversion relationship given by the conversion mechanism is determined based on the conversion of changing brightness, The transformation that changes the shading is a transformation that outputs the output signal that is monotonically decreased with respect to the processed signal. 42. An integrated circuit comprising the visual processing device according to any one of claims 1-4. 43. A display device, comprising: The visual processing device according to any one of claims 1-4; and and a display unit for displaying the output signal output from the visual processing device. 44. A photographing device, comprising: a photography agency, which takes the image; and The visual processing device according to any one of claims 1 to 4, which performs visual processing using the image captured by the imaging means as the image signal. 45. A portable information terminal, having: A data receiving mechanism that receives image data transmitted through communication or broadcasting; The visual processing device according to any one of claims 1 to 4, which uses the received image data as the image signal to perform visual processing; and A display unit for displaying the image signal visually processed by the visual processing device. 46. A portable information terminal, having: a photography agency, which performs the image capture; The visual processing device according to any one of claims 1 to 4, which uses the image captured by the camera as the image signal to perform visual processing; and and a data sending unit that sends the image signal after the visual processing.

Images