JP5183440B2 - Gradation correction apparatus and imaging apparatus - Google Patents

Description

  The present invention relates to a gradation correction apparatus that performs gradation correction of an image, and particularly relates to an apparatus used in an imaging apparatus represented by a digital still camera, a digital video camera, and the like.

  In general, in an imaging apparatus such as a digital camera, when a bright area and a dark area exist simultaneously on the screen, a contrast between the two areas can be obtained, but the contrast in each area tends to be small. Therefore, a gradation correction method for enhancing the contrast in the region has been proposed.

  As an effective gradation correction method that is widely known, there is gradation correction by histogram equalization. This gradation correction is performed by the following procedure. First, a histogram of an image to be corrected is created, a gradation conversion curve that emphasizes the contrast of gradation with a large number of elements in the histogram is obtained, and gradation correction is performed using the gradation conversion curve.

  A method of creating a histogram for each of a plurality of areas instead of one histogram for an image and performing gradation conversion based on the histogram corresponding to each area is referred to as gradation correction by local histogram equalization. In general, it is known that tone correction by local histogram equalization is more effective.

  However, tone correction by histogram equalization has a drawback due to tone conversion based on the histogram distribution. Specifically, when a subject that occupies a wide area with the same brightness exists on the screen, the histogram distribution has a very large number of gradation elements corresponding to the brightness of the subject, and as a result, There is a property that a gradation conversion curve for correcting gradation can be obtained. For this reason, there is a problem that the effect of gradation conversion is hardly effective for a subject having a small area.

  In addition, even if a large area of the subject is originally close to uniform with no shading, the contrast is enhanced by the tone conversion, so that the subject with the originally uniform shading is unnaturally shaded or noisy. There is also a problem that emphasizes only the ingredients. This is because tone correction by histogram equalization strongly depends on the area of the subject.

To deal with this problem, in the gradation correction method described in Patent Document 1, the texture is analyzed in advance, the image is divided into areas with the same texture as one area, and a histogram is obtained for each area. Correction is being performed. In addition, density variation in each area is determined before gradation correction, and the effect of gradation conversion is weakened by adjusting the element clip value of the histogram for areas determined to have sufficient density variation. Processing is in progress.
Japanese Patent No. 3465226

  However, the above gradation correction method solves a subject with a small area, but does not solve the problem with a uniform subject with a large area. Also, dividing the region by analyzing the texture in advance results in different region division results every time depending on the scene of the image, and the process becomes complicated.

  In view of the above background, an object of the present invention is to provide a gradation correction apparatus that obtains a gradation conversion curve that does not depend on the area of a subject and performs optimal gradation conversion.

  The gradation correction apparatus according to the present invention is a gradation correction apparatus that performs gradation correction on an input image, divides the image into a plurality of areas, and sets values representing pixel values in each area as area values. Based on the area value calculation unit, an edge amount calculation unit that calculates an edge amount indicating the amount of the edge component in each area, and the area value and the edge amount. A histogram generation unit that generates a histogram of area values in a first predetermined range, and a gradation conversion curve generation that generates a gradation conversion curve corresponding to the area based on the histogram corresponding to the area to be calculated And a gradation conversion unit that converts pixel values in the area to be calculated using the gradation conversion curve generated by the gradation conversion curve generation unit.

  With this configuration, the tone conversion curve is obtained based on the area value histogram obtained using the edge amount as well, so that an area where there is a change in tone (edge) in the image before tone correction. Area) with a large amount of tone). In addition, since calculation of a histogram with a large processing load is performed using area values that represent the pixel values of the area, the storage capacity and processing load required for the gradation correction processing can be greatly reduced.

  In the gradation correction apparatus of the present invention, the histogram generation unit calculates a value unique to each area within the first predetermined range based on an edge amount of the area, and the area value belongs to the same gradation. It has a configuration for generating a histogram by adding unique values.

  With this configuration, the size of the edge amount in each area can be appropriately reflected in the histogram of area values.

  In the gradation correction apparatus of the present invention, the histogram generation unit has a configuration for calculating a value specific to each area within the first predetermined range based on a distance from the area to be calculated.

  With this configuration, the distance from the area to be calculated to each area within the first predetermined range can be reflected in the histogram.

  The gradation correction apparatus according to the present invention includes an edge summation unit that obtains the sum of the edge amounts of areas within a second predetermined range centered on the area to be calculated, and the gradation conversion curve generation unit is configured to calculate A gradation conversion curve corresponding to the area is generated based on the histogram corresponding to the target area and the total edge.

  With this configuration, the gradation conversion curve is obtained based not only on the area value but also on the total amount of edges, so that the area where gradation changes exist in the image before gradation correction is focused on. Tone correction can be performed. Note that the second predetermined range may be the same as or different from the first predetermined range.

  A gradation correction apparatus according to another aspect of the present invention is a gradation correction apparatus that performs gradation correction on an input image, divides the image into a plurality of areas, and represents pixel values in each area. An area value calculation unit that calculates a value as an area value, an edge amount calculation unit that calculates an edge amount indicating the amount of an edge component in each of the areas, and a calculation target area based on the area value. A histogram generation unit that generates a histogram of area values in a first predetermined range, an edge total unit that calculates a sum of edge amounts of areas within a second predetermined range centered on the area to be calculated, A gradation conversion curve generation unit that generates a gradation conversion curve corresponding to the area based on the histogram and the edge total corresponding to the area to be calculated, and the gradation conversion curve generation unit The using gradation conversion curve has a configuration in which a gradation conversion unit for converting the pixel value in the calculation target area.

  With this configuration, the gradation conversion curve is obtained based not only on the area value but also on the total amount of edges, so that the area where gradation changes exist in the image before gradation correction is focused on. Tone correction can be performed.

  In the gradation correction apparatus of the present invention, the gradation conversion curve generation unit sets an upper limit value and a lower limit value of the number of elements of each gradation according to the size of the edge sum, and the number of elements exceeds the upper limit value. For the key, the number of elements is set as the upper limit, and for gradations where the number of elements is lower than the lower limit, the histogram is shaped with the number of elements as the lower limit. A cumulative histogram in which numbers are accumulated is generated, and a gradation conversion curve is obtained based on the generated cumulative histogram.

  With this configuration, when the gradation conversion curve is generated from the histogram, the maximum and minimum inclinations of the gradation conversion curve are adjusted for each gradation according to the size of the total edge, and the degree of gradation correction according to the scene Can be adjusted.

  In the gradation correction apparatus of the present invention, the gradation conversion curve generation unit has a configuration in which the upper limit value of the number of elements of each gradation is set larger and the lower limit value is set smaller as the edge sum is larger.

  With this configuration, the larger the sum of the edges, the more relaxed gradation correction can be performed by relaxing restrictions on the maximum inclination and minimum inclination of the gradation conversion curve.

  In the gradation correction apparatus according to the present invention, the edge summation unit obtains the sum of the edge amounts of the areas within the second predetermined range from the calculation target area to the area to which the edge amount is to be added. It has the structure which weights according to distance.

  With this configuration, when calculating the total edge in the second predetermined range, the influence of the edge amount on the total edge can be adjusted according to the distance from the area to be calculated to each area.

  In the gradation correction apparatus of the present invention, the edge amount calculation unit has a configuration for adjusting the edge amount calculated according to the brightness of the area.

  With this configuration, it is possible to control the gradation of a brightness target that should be subjected to gradation correction.

  The image pickup apparatus of the present invention has a configuration including an image pickup element and the above-described gradation correction apparatus that performs gradation correction on an image picked up by the image pickup element.

  With this configuration, similar to the above-described gradation correction apparatus, it is possible to perform gradation correction mainly on an area where a change in gradation exists in an image before gradation correction. Various configurations of the gradation correction apparatus of the present invention can also be applied to the imaging apparatus of the present invention.

  The gradation correction program of the present invention is a gradation correction program that performs gradation correction on an input image, and divides the image into a plurality of areas, and is a value that represents the pixel value in each area. Based on the area value, the edge amount indicating the amount of the edge component in each area, and the area value and the edge amount. Generating a histogram of area values in a range, generating a gradation conversion curve corresponding to the area based on the histogram corresponding to the area to be calculated, and using the generated gradation conversion curve And a step of converting pixel values in the area to be calculated.

  The computer-readable recording medium of the present invention has a configuration in which the above-described gradation correction program is recorded.

  The gradation correction method of the present invention is a gradation correction method for performing gradation correction on an input image, and divides the image into a plurality of areas, and values representing pixel values in each area are represented by area values. In a predetermined range centered on the area to be calculated based on the area value and the edge amount based on the step of calculating the edge amount indicating the amount of the edge component in each area A step of generating an area value histogram, a step of generating a gradation conversion curve corresponding to the area based on the histogram corresponding to the area to be calculated, and a calculation target using the generated gradation conversion curve And a step of converting pixel values in the area.

  With these configurations, similar to the above-described gradation correction apparatus, it is possible to perform gradation correction focusing on an area where a change in gradation exists in an image before gradation correction. Various configurations of the gradation correction apparatus of the present invention can be applied to the gradation correction program, recording medium, and gradation correction method of the present invention.

  According to the present invention, since the gradation conversion curve is obtained based on the histogram of the area value obtained also using the edge amount, the area where the gradation change exists in the image before the gradation correction. It has an excellent effect that gradation correction can be performed with priority.

  Hereinafter, an imaging apparatus including a gradation correction apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment. The imaging apparatus includes a lens 1 for imaging light from a subject on the imaging device 3, a diaphragm unit 2 that controls the amount of light incident on the imaging device 3, an imaging device 3 that photoelectrically converts the formed image, An A / D converter 4 that converts an analog signal obtained from the image sensor 3 into a digital signal, a preprocessing circuit 5 that performs OB subtraction, white balance adjustment, and the like on the A / D converted video signal, and post-processing YC separation circuit 6 that separates the video signal into a luminance signal and a color difference signal, a gradation correction unit 100 that performs gradation correction on the luminance signal and the color difference signal output from YC separation circuit 6, and Y that processes the luminance signal A signal processing circuit 7 and a C signal processing circuit 8 for processing color difference signals are provided. The gradation correction unit 100 corresponds to the gradation correction apparatus of the present invention.

  The gradation correction unit 100 extracts an edge signal from the pre-processed video signal, and an area for dividing the edge signal extracted by the edge extraction circuit 10 and the pre-processed video signal into a plurality of areas, respectively. For each of the areas divided by the dividing circuit 11, the area dividing circuit 11, an area value calculating circuit 12 for calculating an area value from the video signal, and an edge amount calculating circuit 13 for calculating an edge amount from the edge signal are provided. The area value is a value of a video signal representing the area, and for example, an average value or a median value of the pixel values of all the pixels included in the area can be used. The edge amount is a value of an edge signal representing an area, and for example, an average value or a median value of the edge signals of all pixels included in the area can be used.

  In addition, the gradation correction unit 100 includes a storage unit 14 that stores the area value calculated by the area value calculation circuit 12 and the edge amount calculated by the edge amount calculation circuit 13 for one frame period.

  Further, the gradation correction unit 100 is based on the edge sum calculation circuit 15 that calculates the sum of the edge amounts in the local area centered on each area based on the edge amount stored in the storage unit 14, and based on the area value and the edge amount. A local histogram calculation circuit 16 for calculating a histogram in a local region centered on each area, and an edge sum calculated by the edge sum calculation circuit 15 for each histogram calculated by the local histogram calculation circuit 16. A histogram shaping circuit 17 that performs shaping processing such as clip processing and smoothing, and a gradation conversion curve generation circuit 18 that generates a gradation conversion curve for each area from the histogram after shaping. Here, the histogram shaping circuit 17 and the gradation conversion curve generation circuit 18 correspond to the “gradation conversion curve generation unit” recited in the claims. The gradation conversion curve obtained by the gradation conversion curve generation circuit 18 is a gradation conversion curve for each area. That is, the gradation conversion curve calculated using the histogram in the local area centered on the predetermined area is a gradation conversion curve corresponding to the predetermined area.

  The tone correction unit 100 uses a tone conversion curve to convert the pre-processed video signal value L, and a plurality of tone-converted video signals obtained by the tone conversion circuit 19. A linear interpolation circuit 20 that performs linear interpolation calculation of values, a ratio calculation circuit 21 that calculates a ratio between the pre-processed video signal value L and the gradation-converted video signal value L ′, and the calculated ratio as a YC separation circuit. And a circuit for multiplying the luminance signal and the color signal generated in step.

  An operation of gradation correction in the imaging apparatus configured as described above, that is, an operation until the luminance signal and the color signal are multiplied by the conversion ratio obtained from the edge extraction circuit 10 via the ratio calculation circuit 21 will be described.

  A video signal obtained by imaging by the imaging device 3 is input to the YC separation circuit 6 and the gradation correction unit 100 via the A / D conversion unit 4 and the preprocessing circuit 5.

  The edge extraction circuit 10 of the gradation correction unit 100 extracts an edge signal from the input image. In the present embodiment, the edge signal extraction greatly affects the gradation correction result. Here, two methods are taken up as effective edge signal extraction methods.

  FIG. 2 is a diagram for explaining a first edge signal extraction method. First, an edge extraction filter is applied to the input image. It is also effective to remove a noise component in the screen in advance by applying a low-pass filter or the like before applying the edge extraction filter. Here, a known filter such as a Sobel filter or a Canny edge filter can be used as the edge extraction filter. As a result of these filters, an extraction result by the filter is obtained. Next, a final edge signal extraction result is obtained by multiplying this result by an edge adjustment coefficient. The edge adjustment coefficient is a gain by which the edge extraction result by the filter is multiplied according to the pixel value of each pixel of the input image. In the example illustrated in FIG. 2, the gain increases as the pixel value of the input image decreases, and as a result, the edge signal having a low pixel value increases. In this way, the edge adjustment coefficient plays a role of adjusting which gradation edge signal is extracted. Thereby, it becomes possible to extract an edge according to the brightness of the area, and for example, only an edge signal existing in a dark area on the screen can be extracted with priority. Edge signal extraction along the purpose of gradation correction can be performed.

  FIG. 3 is a diagram for explaining a second edge signal extraction method. First, a low pass filter is applied to the input image. This processing by the low-pass filter is processing for preventing a noise component from being erroneously determined as an edge signal. Next, using each pixel in the screen as a target pixel, a difference value between the target pixel and eight neighboring pixels is calculated. The difference value is an absolute value of the difference between the two pixels. As a noise countermeasure, when the obtained difference value is smaller than the threshold value diffTH, there is no difference.

  Next, the eight difference values obtained in the above processing are divided by the average value of the two pixels calculated for each difference value. For example, considering the target pixel and the upper left pixel A1, first, the difference 1 ′ is obtained from the absolute value of the difference between the two pixels, and the difference 1 ′ is further divided by the average value of the target pixel and the upper left pixel A1 as the difference 1. Become. Similarly, the difference 1 to the difference 8 are obtained. The sum of the obtained differences 1 to 8 is used as a difference meter, and when the difference meter is equal to or higher than THmin and lower than THmax, the target pixel is determined to be an edge signal.

  The above is the second edge signal extraction method. This extraction method has characteristics that it is easy to extract edges of dark areas on the screen and it is difficult to extract edges of bright areas. This characteristic is suitable for gradation correction used mainly for the purpose of improving the visibility of dark areas.

  Next, the area division circuit 11 divides the input image and the edge extraction result extracted by the edge extraction circuit 10 into a plurality of areas. The area division of the input image and the edge extraction result is the same, and at least one pixel belongs to one area.

  FIG. 4 is a diagram illustrating an example of area division. FIG. 4 shows an example in which the image is divided into a total of 4800 areas of 80 horizontal and 60 vertical when the image size is 640 pixels horizontal and 480 pixels vertical. At this time, one area includes a total of 64 pixels of 8 horizontal pixels and 8 vertical pixels.

  Subsequently, for each area divided by the area dividing circuit 11, the area value calculating circuit 12 calculates an average value of the pixel values belonging to the area, and sets the calculated average value as the area value. This area value is used in the following processing.

  Similarly, in the edge amount calculation circuit 13, for each area divided by the area division circuit 11, an average value of the magnitudes of the edge signals belonging to the area is calculated, and the calculated average value is used as the edge amount. . In the following processing, this edge amount is used.

  By making these areas, it is not necessary to perform calculation using pixel values or edge signals in units of pixels, so the load of calculation processing can be reduced. Further, the area value and the edge amount are subjected to gradation reduction as necessary. In this embodiment, the area value is reduced to 5 bits and the edge amount is reduced to 2 bits. Thereby, the memory capacity can be reduced when the area value and the edge amount are stored in the storage unit 14.

  Through the processing so far, the area value and the edge amount of each area whose gradation has been reduced are stored in the storage unit 14 of the gradation correction unit 100. The gradation correction unit 100 obtains a gradation conversion curve corresponding to each area using the area value and edge amount of each area. The processing for obtaining the gradation conversion curve is performed by the edge total calculation circuit 15, the local histogram calculation circuit 16, the histogram shaping circuit 17, and the gradation conversion curve generation circuit 18.

  The edge sum total calculation circuit 15 calculates the sum of the edge amounts in the local region centered on each area, using the edge amounts stored in the storage unit 14.

  FIG. 5 is a diagram for explaining the calculation of the edge sum. The edge sum total calculation circuit 15 calculates the sum of the edge amounts of the areas in the predetermined range after weighting according to the distance from the target area for the area in the predetermined range centered on the target area. .

  In the example shown in FIG. 5, the edge amount is multiplied by K1 for each area within the range E1 of the horizontal distance D1 and the vertical distance D1 centering on the attention area EC. Similarly, the edge amount is multiplied by K2 for each area in the range E2 that is within the horizontal distance D2 and the vertical distance D2 around the attention area EC and is not included in the range E1. The edge amount is multiplied by K3 for each area in the range E3 that is within the horizontal distance D3 and the vertical distance D3 around the attention area EC and is not included in the ranges E1 and E2. The edge amount is multiplied by K4 for each area in the range E4 that is in the range of the horizontal distance D4 and the vertical distance D4 around the attention area EC and is not included in the ranges E1 to E3. Then, the total value of the edge amounts multiplied by these weighting factors is defined as the total edge.

  Note that an area on the screen that is not included within the range E4 with the attention area EC as the center is not involved in the edge sum calculated in this attention area. Here, the weighting factors K1 to K4 are the largest in the weighting factor K1 and are sequentially decreased toward the weighting factor K4, and the distances D1 to D4 are the distances in which the distance D1 is the smallest and traveled toward the distance D4. Enlarge. As a result, as the distance from the attention area EC increases, the influence on the total edge in the attention area EC can be reduced. The weighting factors K1 to K4 and the horizontal and vertical distances D1 to D4 are based on experience, K1 = 4, K2 = 3, K3 = 2, K4 = 1, D1 = 12, D2 = 25, D3 = 40, and D4 = 60. It is desirable to set the value of.

  Here, in the present embodiment, when gradation conversion is performed, a gradation conversion curve of a total of four areas including an area including a pixel to be subjected to gradation conversion and three areas adjacent to the area is used. Perform gradation correction. The tone correction unit 100 independently calculates the edge sum and local histogram for each of the four areas of interest from the edge sum calculation circuit 15 to the tone conversion circuit 19, and generates a tone conversion curve based on the sum. To obtain the gradation conversion result.

  FIG. 6 is a diagram illustrating four attention areas. As shown in FIG. 6, the four areas of interest are an area (i, j), an area (i + 1, j), an area (i, j + 1), an area (i, j), a distance from the pixel of interest K currently being processed to the center of gravity position. Area (i + 1, j + 1) is selected. The edge summation calculation circuit 15 to the gradation conversion circuit 19 are processed in parallel, and the linear interpolation circuit 20 performs linear interpolation calculation using four gradation-converted video signal values. One gradation-converted video signal The value L ′ is obtained.

  The processing focused on the four areas is executed in parallel, but the processing contents are the same. Therefore, in the description of the edge total calculation circuit 15 to the gradation conversion circuit 19, the processing of one area will be focused on and described.

  The local histogram calculation circuit 16 calculates a histogram in a local region centered on each area using the area value and edge amount stored in the storage unit 14.

  FIG. 7 is a diagram for explaining calculation of a local histogram. The local histogram calculation circuit 16 obtains a unique value of each area according to the distance from the area of interest to each area and the edge amount of each area for an area within a predetermined range centered on the area of interest. Then, a histogram is calculated by adding area-specific values for each gradation. It should be noted that the area-specific value is not counted as one when counting as the number of gradation elements to which the area value of the area belongs, but is multiplied by the weight W and counted as “W”. Is a value corresponding to.

In the example shown in FIG. 7, for each area within the range E1 of the horizontal distance D1 and the vertical distance D1 with the attention area EC as the center, an area-specific value W1 is obtained by the following equation.

  Based on the distance weight coefficient origW1 applied in the range E1, the weight is obtained by adjusting the coefficient origW1 by the edge amount according to the edge amount of each area. The coefficient G1 is a coefficient for adjusting the edge amount. The same value is applied to the coefficient origW1 and the coefficient G1 within the range E1, but since the edge amount varies depending on each area, the area-specific value W1 is different even within the range E1.

Similarly, for each area in the range E2 that is in the range of the horizontal distance D2 and the vertical distance D2 around the attention area EC and is not included in the range E1, an area-specific value W2 is obtained by the following equation.

For each area in the range E3 that is in the range of the horizontal distance D3 and the vertical distance D3 around the area of interest EC and not included in the ranges E1 and E2, an area-specific value W3 is obtained by the following equation.

For each area within the range E4 that is in the range of the horizontal distance D4 and the vertical distance D4 around the attention area EC and is not included in the ranges E1 to E3, an area-specific value W4 is obtained by the following equation.

The area on the screen that is not included within the range E4 with the attention area EC as the center is not involved in the local histogram calculated in this attention area. Here, the distance weight coefficients origW1 to origW4 have the largest distance weight coefficient origW1 and the smaller the coefficients sequentially toward the distance weight coefficient origW4, and the distances D1 to D4 have the smallest distance D1 and go to the distance D4. It is appropriate to increase the distance sequentially. Thereby, the influence on the local histogram in the attention area EC can be reduced as the distance from the attention area EC increases. Further, the edge amount adjustment coefficients G1 to G4 are multiplied by the distance weighting coefficient, and thus are appropriately set in consideration of the distance weighting coefficient. Each coefficient has a value of about orig W1 = 14, orig W2 = 6, orig W3 = 2, orig W4 = 1, G1 = 0.29, G2 = 0.5, G3 = 1, G4 = 1. Is desirable. The distances D1 to D4 are the same as those in the edge total calculation circuit 15. Since the edge amount is reduced to 2 bits, it takes a value of 0 to 3.

  FIG. 8 shows a case where the attention area is near the edge of the image. As shown in FIG. 8, when the area of interest is near the edge of the image, a predetermined range (area reference range) centered on the area may protrude from the image. In this case, the range outside the screen may be treated as having no value, or another process may be performed as the edge process. For example, a method of caring for the area of the local histogram considering that the area of the outermost frame of the screen extends infinitely outside the screen, or the outer side of the screen is folded inside with the outermost frame area of the screen symmetrical A method of caring for the range of the local histogram can be considered.

  Next, the histogram shaping circuit 17 performs processing on the local histogram calculated by the local histogram calculation circuit 16. The histogram shaping circuit 17 performs three processes: (1) a process of distributing the number of elements to upper and lower gradations, (2) a MAX clip of the number of elements, and (3) a MIN clip of the number of elements.

  FIG. 9 is a diagram for explaining a process of distributing the number of elements of the histogram to the upper and lower gradations. In the distribution process, the histogram is smoothed by distributing the number of elements of each gradation to the upper and lower gradations in all 32 gradations of the histogram. In the present embodiment, since the pixel values of 8 × 8 pixels included in the area are processed by one area value, the pixel values of various values included in the actual image are rounded. Yes. The distribution process is a process based on the idea that the rounded portion is restored by using the area value.

  In the distribution process of the number of elements, a value obtained by multiplying the number of elements of the target gradation by a weighting factor V1 is added to the gradation that is one gradation above and below the target gradation, and the gradation that is two gradations above and below is added. A value multiplied by the weighting factor V2 is added to the key, and a value multiplied by the weighting factor V3 is added to the tone separated by three tones above and below, so that the weight is determined according to the tone difference from the target tone. A coefficient is set, and the number of elements of the target gradation is distributed to the upper and lower gradations.

  FIG. 9A shows an example of a histogram before distribution processing. In FIG. 9B, a weighting factor of V1 = 0.75, V2 = 0.50, V3 = 0.25 is set for the histogram shown in FIG. It is a figure which shows the result of having performed the process which discards the distribution part of. The weight coefficients V1, V2, and V3 are preferably V1 ≧ V2 ≧ V3 from experience.

  FIG. 9C to FIG. 9E are diagrams showing the progress in the middle of obtaining the histogram in FIG. 9B by the distribution process from the histogram shown in FIG. 9A. FIG. 9C is a diagram showing an example in which the value 100 at the gradation 0 of the histogram shown in FIG. 9A is distributed to the upper and lower gradations. As shown in FIG. 9C, a value of 100 × V1 (0.75) = 75 is added to the gradation 1, and 100 × V2 (0.5) = 75 for the gradation 2. A value of 50 is added, and for gradation 3, a value of 100 × V3 (0.25) = 25 is added. Note that since gradation 0 is the lowest gradation, there is no gradation lower than gradation 0. In this case, the distribution outside the gradation is lost.

FIG. 9 (d) shows an example in which the value 200 at gradation 1 of the histogram shown in FIG. 9 (a) is distributed to the upper and lower layers, as in FIG. 9 (c). The example which distributes the value 150 in the key 2 to an upper and lower hierarchy is shown. The sum shown in FIG. 9C, FIG. 9D, and FIG. 9E is taken to obtain the result shown in FIG. 9B. As seen in FIG. 9B, the histogram is smoothed by the distribution process.
Next, the clip of the number of elements performed by the histogram shaping circuit 17 will be described.

  FIG. 10A and FIG. 10B are diagrams for explaining the clip of the number of elements. FIG. 10A shows a histogram before clip processing, and FIG. 10B shows a histogram after clip processing. The histogram shaping circuit 17 sets the MAX clip value and the MIN clip value for the number of elements independently for each gradation, and performs the clip process for the number of elements. That is, the histogram shaping circuit 17 subtracts the number of elements exceeding the MAX clip value for each gradation to the MAX clip value, and adds the number of elements less than the MIN clip value to the MIN clip value.

  Next, the MAX clip value and the MIN clip value will be described. The setting of the MAX clip value and the MIN clip value is largely related to the gradation correction result. Basically, the larger the MAX clip value or the smaller the MIN value clip, the greater the degree of gradation correction and the greater the contrast improvement. Clip values are set independently for each gradation.

  The MAX clip value and the MIN clip value are adjusted by the edge sum of the area of interest obtained by the edge sum calculation circuit 15 with reference to a preset value.

  FIG. 11A and FIG. 11B are diagrams for explaining the adjustment of the size of the edge sum, the MAX clip value, and the MIN clip value. FIG. 11A is a diagram for explaining the adjustment of the MAX clip value. When the edge sum is large, the MAX clip value is made larger than the reference value, and when the edge sum is small, the MAX clip value is made smaller than the reference value.

  FIG. 11B is a diagram for explaining adjustment of the MIN clip value. When the edge sum is large, the MIN clip value is made smaller than the reference value, and when the edge sum is large, it is made larger than the reference value. This adjustment method is based on the following idea. In areas where the sum of edges is small, it is less necessary for the subject to be nearly uniform and to expand the gradation. Accordingly, the MAX clip value is decreased and the MIN clip value is increased so that the degree of gradation correction is reduced. On the contrary, since the area where the sum of edges is large is an area where it is highly necessary to expand the gradation, the MAX clip value is larger than the reference and the MIN clip value is larger than the reference so that the degree of gradation correction becomes larger. Make it smaller.

  Next, the gradation conversion curve generation circuit 18 generates a gradation conversion curve from the histogram shaped by the histogram shaping circuit 17.

  FIG. 12A to FIG. 12C are diagrams for explaining the generation of the gradation conversion curve. FIG. 12A shows a histogram shaped by the histogram shaping circuit 17. The gradation conversion curve generation circuit 18 accumulates the histogram shown in FIG. 12A from the lower gradations, and creates the accumulated histogram shown in FIG. Next, the gradation conversion curve generation circuit 18 normalizes the number of elements of the highest gradation of the cumulative histogram so that it becomes the number of gradations of the video signal value, and converts the gradation conversion curve shown in FIG. Generate.

  In the example shown in FIGS. 12A to 12C, normalization is performed by setting the number of gradations of the video signal value to 256 gradations, and a gradation conversion curve is generated from the normalized cumulative histogram. The gradation conversion curve is a broken line curve having 0 as the origin and connecting other than the origin by 32 equally spaced control points.

  Through the processing so far, a gradation conversion curve corresponding to each area is generated. Using this gradation conversion curve, gradation correction can be performed by gradation conversion of pixels in the area. In the present embodiment, processing as described below is performed in order to solve the problem that an area boundary line is generated by gradation correction using a gradation conversion curve for each area.

The gradation conversion circuit 19 performs gradation conversion of the preprocessed video signal value L using the generated gradation conversion curve.
FIG. 13 is a diagram illustrating an example of gradation conversion. In conversion based on the gradation conversion curve, a value corresponding to the pre-processed video signal value L is read from the gradation conversion curve and used as an output value after gradation conversion. Since the gradation conversion curve is a broken line curve composed of 32 control points, an interpolation calculation is performed between the control points to calculate an output value.

  The gradation conversion circuit 19 performs gradation conversion of the target pixel using the gradation conversion curves of four areas (see FIG. 6) including the target pixel. As a result, the gradation conversion circuit 19 obtains four gradation conversion results for one pixel.

  The linear interpolation circuit 20 performs linear interpolation processing on the four gradation conversion results obtained by the gradation conversion circuit 19 to obtain gradation correction results for the pixels.

FIG. 14 is a diagram for explaining processing by the linear interpolation circuit 20. As shown in FIG. 14, the linear interpolation circuit includes each of the pixel of interest K and the areas of interest (i, j), area (i + 1, j), area (i, j + 1), and area (i + 1, j + 1). Interpolation is performed using the distance from the center of gravity. Here, the horizontal and vertical sizes of the area are R, the horizontal distance between the area of interest (i, j) and the pixel of interest K is p, the vertical distance is q, and the area (i, j) and area (i + 1, j) When the gradation conversion results of area (i, j + 1) and area (i + 1, j + 1) are sequentially LA, LB, LC, and LD, the signal value L ′ after linear interpolation is obtained by the following equation.

Next, based on the video signal value L output from the preprocessing circuit 5 and the video signal value L ′ after gradation conversion obtained by the linear interpolation circuit 20, the ratio calculation circuit 21 (after gradation conversion). Video signal value L ′) / (preprocessed video signal value L) to calculate the conversion ratio. The gradation correction unit 100 multiplies the luminance signal (Y signal) and color difference signal (C signal) generated by the YC separation circuit 7 by the conversion ratio obtained by the ratio calculation circuit 21.
The operation of the gradation correction unit 100 has been described above.

  Since the imaging apparatus according to the present embodiment obtains the gradation conversion curve using not only the area value but also the edge amount, an area in which a change in gradation exists in an image before gradation correction. Gradation correction can be performed with emphasis on gradation.

  Since the imaging apparatus of this embodiment calculates a histogram with a large processing load for each area, it realizes gradation correction by local histogram equalization that greatly reduces the storage capacity and processing load required for processing. it can.

  The imaging apparatus according to the present embodiment calculates a histogram from an area value and an edge amount within a predetermined range including the area when obtaining a gradation conversion curve corresponding to the area. Compared to the case where the gradation conversion curve is obtained using the histogram calculated from the edge signal, the gradation conversion curve can be obtained using a wider range of data.

  When obtaining the gradation conversion curve corresponding to the area, the imaging apparatus according to the present embodiment is based on the distance from the area with respect to the area value and edge amount within a predetermined range including the area when calculating the local histogram. Since the weights are adjusted step by step, there is no sudden histogram fluctuation depending on the area. Therefore, the gradation conversion curve also changes smoothly depending on the area, and a result with no sense of incongruity is obtained after gradation correction.

  In the imaging apparatus according to the present embodiment, the histogram shaping circuit 17 performs the histogram distribution process, so that it is possible to reduce gradation collapse caused by processing a plurality of pixels in the area with the area value.

  In the imaging device of the present embodiment, the linear correction result of the pixel is obtained by linearly interpolating the gradation conversion result obtained using the gradation conversion curve corresponding to each area. A gradation correction value in consideration of the gradation conversion curve is obtained. Thereby, it is possible to make it difficult for an area boundary line to occur in the image after gradation correction.

  As described above, the image correction apparatus including the gradation correction apparatus has been described in detail with respect to the gradation correction apparatus of the present invention, but the present invention is not limited to the above-described embodiment.

  In the above-described embodiment, an example in which the gradation correction apparatus is configured by a circuit has been described. However, a program that realizes the function of each circuit of the gradation correction unit 100 described above is also included in the scope of the present invention.

  In the above-described embodiment, the OB subtraction and the white balance adjustment are described as the processing performed by the preprocessing circuit 5. However, these two processings may not be performed, and processing other than these two processing is performed by the preprocessing circuit. May be. For example, shading correction or gamma correction may be included in the preprocessing.

  In the embodiment described above, two methods have been described as the edge extraction circuit 10, but the edge extraction method is not limited to this. Any filter can be used for the edge extraction as long as the differential component of the pixel value can be extracted, and it is also effective to use a combination of the two means described above.

  In the above-described embodiment, an example in which the screen size is 640 pixels horizontal and 480 pixels vertical and the screen is divided into a total of 4800 areas of 80 horizontal and 60 vertical is described. The size and the number of area divisions are not limited to the above example. Further, although the average value of the pixel values in the area is used as the area value, the method for calculating the average value is not particularly limited. For example, an average value of all the pixels in the area may be obtained, or some pixels in the area may be sampled to obtain an average value of the pixels. From experience, the result tends to be more stable when the average value of as many pixels as possible is obtained. Further, the value used as the pixel value is not particularly limited. For example, the luminance value in each pixel may be calculated and used.

  In the above-described embodiment, the area value is 5 bits, the edge amount is 2 bits, the histogram is 32 gradations, and the gradation conversion curve is a 32-point broken line curve. However, the present invention is not limited to this. For example, it is effective to reduce the storage capacity and the circuit scale to set the gradation of the histogram to 16 gradations and obtain the gradation conversion curve with a 16-point broken line curve. On the contrary, if the histogram is 64 gradations and the gradation conversion curve is a 64-point line curve, more accurate gradation conversion is possible.

  As for the edge amount, if the number of bits is increased, the histogram can be weighted with higher accuracy. Further, although the gradation conversion curve is a polygonal line curve with equal intervals, it may not be equal intervals. For example, by increasing the number of gradations on the lower side and taking the number of gradations on the upper side sparsely, it is possible to improve the accuracy of the broken line curve only on the lower side. The same applies to the gradation of the histogram.

  In the above-described embodiment, the range referred to when calculating the local histogram is a square, but the shape of the reference range is not limited to a square. For example, a rhombus or a circle can be used.

  In the above-described embodiment, weights are switched stepwise in the four reference ranges when calculating the local histogram, but the number of steps is not necessarily four.

  Further, in the above-described embodiment, the reference ranges at the time of calculating the local histogram and the edge total are the same, but they are not necessarily the same.

  In the above-described embodiment, the distribution element to the lowest gradation or the nonexistent gradation exceeding the highest gradation is discarded in the histogram distribution process, but it is not always necessary to discard it. For example, it is also effective to add a distribution element to a nonexistent gradation exceeding the lowest gradation to the lowest gradation and add a distribution element to a nonexistence gradation exceeding the highest gradation to the highest gradation. It is a simple method. In addition, although the example in which the number of elements is distributed for each upper and lower three gradations has been described, the number of gradations is not limited to this.

  As described above, the present invention has an excellent effect that gradation correction can be focused on an area where a change in gradation exists in an image before gradation correction. This is useful for a gradation correction device used in an imaging device represented by a camera, a digital video camera, or the like.

Schematic configuration diagram of a gradation correction apparatus in an embodiment The figure explaining the 1st edge extraction method in embodiment The figure explaining the 2nd edge extraction method in embodiment The figure explaining the area division | segmentation in embodiment The figure explaining edge sum total calculation in an embodiment The figure explaining four area selection in an embodiment The figure explaining local histogram calculation in an embodiment The figure explaining the state where the reference range in the local histogram calculation in the embodiment extends outside the screen (A) A diagram showing an example of a histogram before distribution processing, (b) a diagram showing an example of a histogram after distribution processing, (c) a diagram showing an example of distribution processing of gradation 0, and (d) a gradation 1 The figure which shows the example of a distribution process, The figure which shows the example of the distribution process of (e) gradation 2 (A) The figure which shows the example of the histogram before a clip process, (b) The figure which shows the example of the histogram after a clip process (A) A figure explaining adjustment of a MAX clip value, (b) A figure explaining adjustment of a MIN clip value (A) A diagram illustrating an example of a histogram, (b) a diagram illustrating an example of a cumulative ratio histogram, and (c) a diagram illustrating an example of a gradation conversion curve. FIG. 6 is a diagram illustrating an example of gradation conversion in an embodiment The figure explaining the linear interpolation process of the four gradation conversion results in embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens 2 Aperture part 3 Image pick-up element 4 A / D converter 5 Preprocessing circuit 6 YC separation circuit 7 Y signal processing circuit 8 C signal processing circuit 10 Edge extraction circuit 11 Area division circuit 12 Area value calculation circuit 13 Edge amount calculation circuit DESCRIPTION OF SYMBOLS 14 Memory | storage part 15 Edge sum total calculation circuit 16 Local histogram calculation circuit 17 Histogram shaping circuit 18 Tone conversion curve generation circuit 19 Tone conversion circuit 20 Linear interpolation circuit 21 Ratio calculation circuit 100 Tone correction part of embodiment

Claims (10)

A gradation correction apparatus that performs gradation correction on an input image,
An area value calculation unit that divides an image into a plurality of areas and calculates a value that represents a pixel value in each area as an area value;
An edge amount calculation unit for calculating an edge amount indicating an amount of an edge component in each of the areas;
Based on the area value, the edge amount, and the distance from the area to be calculated, a value specific to each area in the first predetermined range centered on the area to be calculated is calculated, and the area value has the same gradation. A histogram generation unit that generates a histogram of area values by adding the area-specific values to which they belong ;
A gradation conversion curve generation unit that generates a gradation conversion curve corresponding to the area based on a histogram corresponding to the area to be calculated;
A gradation conversion unit that converts pixel values in an area to be calculated using the gradation conversion curve generated by the gradation conversion curve generation unit;
A gradation correction apparatus comprising: An edge summation unit for obtaining the sum of the edge amounts of areas within a second predetermined range centered on the area to be calculated;
The gradation correction apparatus according to claim 1 , wherein the gradation conversion curve generation unit generates a gradation conversion curve corresponding to an area based on a histogram and an edge total corresponding to the area to be calculated. The gradation conversion curve generation unit sets an upper limit value and a lower limit value for the number of elements of each gradation in accordance with the size of the total edge, and for a gradation whose number of elements exceeds the upper limit value, the number of elements is set as the upper limit value. For gradations where the number of elements falls below the lower limit, the histogram is shaped with the number of elements as the lower limit, and a cumulative histogram is generated by accumulating the number of elements for each gradation and the number of elements for the gradations lower than that gradation. The gradation correction apparatus according to claim 2 , wherein a gradation conversion curve is obtained based on the generated cumulative histogram. The gradation correction apparatus according to claim 3 , wherein the gradation conversion curve generation unit sets the upper limit value of the number of elements of each gradation to be larger and the lower limit value to be smaller as the edge sum is larger. The edge summation unit performs weighting according to a distance from an area to be calculated to an area to which an edge amount is to be added when obtaining a sum of edge amounts of areas within the second predetermined range. The gradation correction apparatus according to any one of 1 to 4 . The gradation correction apparatus according to claim 1 , wherein the edge amount calculation unit adjusts an edge amount calculated according to brightness of an area. An image sensor;
A gradation correction device according to any one of claims 1 to 6, the image captured to gradation correction by the image pickup device,
An imaging apparatus comprising: A gradation correction program that performs gradation correction on an input image,
Dividing an image into a plurality of areas and calculating a value representative of a pixel value in each area as an area value;
Calculating an edge amount indicating an amount of an edge component in each of the areas;
Based on the area value, the edge amount, and the distance from the calculation target area, a value specific to each area in a predetermined range centered on the calculation target area is calculated, and the area value belongs to the same gradation Generating a histogram of area values , adding unique values ;
Generating a gradation conversion curve corresponding to the area based on a histogram corresponding to the area to be calculated;
Converting the pixel value in the area to be calculated using the generated gradation conversion curve;
Gradation correction program that executes   A computer-readable recording medium on which the gradation correction program according to claim 8 is recorded. A gradation correction method for performing gradation correction on an input image,
Dividing an image into a plurality of areas and calculating a value representative of a pixel value in each area as an area value;
Calculating an edge amount indicating an amount of an edge component in each of the areas;
Based on the area value, the edge amount, and the distance from the calculation target area, a value specific to each area in a predetermined range centered on the calculation target area is calculated, and the area value belongs to the same gradation Generating a histogram of area values , adding unique values ;
Generating a gradation conversion curve corresponding to the area based on a histogram corresponding to the area to be calculated;
Converting the pixel value in the area to be calculated using the generated gradation conversion curve;
A gradation correction method comprising:

Images