JP2009004887A - Image correction method and image correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide image correction technique for actualizing suitable image correction by generating for an input image to be corrected, an image correction curve with both image features of an area of high spatial frequency and image features of an area of low spatial frequency taken into consideration. <P>SOLUTION: Sub-matrixes of first size and second size smaller than it are used for an image for correction arithmetic corresponding to the input image to generate first/second mosaic processing images, from which a first image feature quantity group and a second image feature quantity group are found. Then a merged correction curve is generated by merging a rough image correction curve generated based upon the first image feature quantity group and a fine image correction curve generated based upon the second image feature quantity group, and the input image is corrected with the merged correction curve. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image correction technique for correcting an input image into a proper image using a correction curve.

  In the field of photographic print output, which is a typical field for correcting input images to appropriate images, photographic images (data) obtained by digitizing photographic images formed on photographic film using a film scanner, and digital cameras After image correction such as density correction and color correction is performed on the shot image (data) obtained by digitizing the shot image directly with a digital shooting device such as this, it is converted to print data, and a photo is taken based on this print data The mainstream is a digital photo processing technique in which a print unit is driven and a photographed image is printed on a photosensitive material (printing paper) with a light beam or formed on a printing paper with color ink. In order to obtain an appropriate shot image under various photography conditions, it is necessary to appropriately set shooting conditions such as shutter speed, aperture setting, weather, and light source type. The actual situation is that the photographer is not consciously aware of this, and an inappropriate photographed image in which the color tone or contrast of the subject is not properly obtained is input to the photographic printing apparatus. There are many things. The main image corrections used to reproduce the acquired input captured image as an appropriate output captured image, such as a photographic print, are color correction, density correction, and contrast correction. These corrections are adjusted and combined correctly. Image correction is required.

  Image correction that can select the part of the image that requires color tone and contrast correction from the photographed image acquired by photography, and improve the accuracy of color tone and contrast reproducibility for the selected image part as much as possible As one of the above, the image data whose frequency is less than a predetermined ratio with respect to the maximum frequency of the gradation histogram for each color is excluded for each color, and the gradation width of the remaining image data is extended or compressed to the reference gradation width, There is a technique in which either or both of color balance and contrast are corrected to some extent by correcting and converting pixel values of image data in accordance with the extension or compression of the gradation width (see, for example, Patent Document 1).

  Also, as image correction that efficiently performs color correction, density correction, and contrast correction, a density / contrast conversion curve (conversion table) is first created based on the low-resolution data of the captured image and a predetermined density / contrast correction algorithm. Then, a color correction conversion curve (conversion table) is created based on the low-resolution data of the captured image corrected using the density / contrast conversion curve and a predetermined color correction algorithm. There is also known one that corrects high-resolution image data of a captured image for the purpose of photographic print output using an integrated correction conversion curve created by fusing correction conversion curves (see, for example, Patent Document 2).

  Further, the selected representative color RGB signal is used as input data, and the RGB signal obtained by reading the representative color duplicate image output by the output device (printer) again by the input device (scanner) is used as a teacher. A neural network is constructed by using it as data, and each pixel value of the input image is input to the input layer, so that the ink amount of the printer is output from the output layer through the intermediate layer. There is also known an image processing apparatus that attempts to obtain (see, for example, Patent Document 3).

  As a technique using a neural network for image correction, an image correction obtained by inputting an image signal into this neural network is further provided with a neural network that has previously learned an image signal as input data and an image correction amount as a teacher value. An image forming apparatus that obtains a stable output image by correcting an image signal using the amount is also known (see, for example, Patent Document 4).

JP 2004-145399 A (paragraph numbers 0023-0029) JP 2006-157562 A (paragraph number 0026-0029, FIG. 6) Japanese Patent Laid-Open No. 09-168095 (paragraph numbers 0050-0057, 0066, FIG. 4) Japanese Unexamined Patent Publication No. 09-018716 (paragraph numbers 0039-0040, FIG. 1)

  When constructing an algorithm for generating an appropriate image correction curve according to an input image, first, a large number of sample images are prepared. Then, an image correction in which each sample image is manually set as a sample appropriate image is manually performed by an expert, the image feature amount group calculated from each sample image, the pixel value of the original image generated through the manual image correction, and the appropriate image An algorithm for generating an image correction curve is constructed by statistically or deductively evaluating the correspondence (image correction curve) with the pixel value of. Such an algorithm construction method is known not only from the above-mentioned patent document but also from many documents. Also, when applying neural network technology to image correction, the image feature quantity group calculated from each sample image is used as input data, and the pixel value of the original image and the pixel value of the appropriate image created through this manual image correction The neural network is learned using the correspondence relationship with the teacher data. By giving the image feature amount calculated from the input image to be corrected to the input layer of the neural network constructed through such learning, an image correction curve suitable for this input image is output, more precisely this image. An output value that defines the correction curve is output. In any case, by using the generated image correction curve (which can also be called an image correction amount matrix, a conversion table, or a conversion formula) to correct the input image as the original image, an appropriate photographic print can be obtained. A corrected image suitable for output can be obtained. Sample images used during neural network learning are selected to have various image characteristics so that input images having various image characteristics can be appropriately corrected. Many of them have both spatial frequency components in the high frequency range, and a neural network that uses the image feature group used to generate a conventional image correction curve as an input value is sufficiently satisfactory for such an input image. The problem that an image correction curve that can be generated cannot be generated has been revealed.

  In view of the above situation, an object of the present invention is to generate an image correction curve that takes into consideration both an image feature of a region having a high spatial frequency and an image feature of a region having a low spatial frequency for an input image to be corrected. Thus, an image correction technique for realizing proper image correction is provided.

  In an image correction method for correcting an input image to a proper image using a correction curve, in order to solve the above-described problem, the method according to the present invention includes a first-size submatrix for a correction calculation image corresponding to the input image. Generating a first mosaic processed image using the steps, obtaining a first image feature quantity group from the first mosaic processed image, and generating a coarse image correction curve based on the first image feature quantity group Generating a second mosaic-processed image using a sub-matrix of a second size smaller than the first size for the correction calculation image corresponding to the input image; and a second from the second mosaic-processed image Obtaining an image feature amount group; generating a dense image correction curve based on the second image feature amount group; and the coarse image correction curve and the dense image. A step of fusing a correction curve to generate a fusion correction curve, and a step of image correction the input image in said fusion correction curve.

  In the above method, first, a mosaic processed image having two different spatial frequency components is generated from two input sub-matrices having different sizes from an input image to be corrected. That is, the first mosaic processed image using the larger first size sub-matrix has a lower spatial frequency component than the second mosaic processed image using the smaller second size sub-matrix. Therefore, the first mosaic processed image is a coarse image, and the second mosaic processed image is a dense image. After that, when an image feature amount group is obtained from the first mosaic processed image using a pre-defined calculation and a rough image correction curve is generated based on the image feature amount group, the coarse image correction curve becomes a global image of the input image. It becomes a correction curve based on the trend. Furthermore, when an image feature amount group is similarly obtained from the second mosaic processed image and a dense image correction curve is generated based on the image feature amount group, the dense image correction curve is a correction curve based on the detailed tendency of the input image. It becomes. Therefore, the fusion correction curve generated by fusing the coarse image correction curve and the dense image correction curve becomes a correction curve that fits both the global trend and the detailed trend of the input image, and this fusion curve. By performing image correction using the correction curve, a corrected image that satisfies both the correction evaluation from the overall viewpoint and the correction evaluation from the detailed viewpoint can be obtained.

  Here, the correction curve is generated based on two mosaic processed images generated using two different size sub-matrices, but the mosaic processed image generated using different sub-matrices is also used. May be added. In other words, in the present invention, two or more correction curves generated based on two or more mosaic processing images generated using two or more different sub-matrices are fused to generate a correction curve to be finally used. The theme is to do.

  Furthermore, the term “correction curve” used in this specification is a general term for a function for converting pixel values of an original image into pixel values of an appropriate image, such as a correction formula and a correction matrix often used in the field of image processing. Furthermore, a data group for creating such a conversion function also belongs to the term. In other words, the correction curve here defines the correspondence between the pixel value of the original image and the pixel value of the appropriate image, and a specific embodiment of the correction curve is a hardware called LUT (Look Up Table). Modules built with hardware and / or software are common. Further, in the specification, image data as digital data, a print image based on this image data, and a monitor display image are also collectively referred to simply as “image” unless otherwise specifically required.

  As a suitable method for generating an image correction curve from a group of image features, a neural network is recommended from the viewpoint of high speed processing and high accuracy. For this reason, in one preferred embodiment according to the present invention, in the step of generating the coarse image correction curve, a plurality of sample images prepared are generated using the first-size sub-matrix. A coarse image neural network is used in which a first image feature group obtained from one mosaic processed image is used as an input value and an image correction curve for creating an appropriate image from the sample image is learned as a teacher value, and the dense image correction is performed. In the step of generating a curve, the second image feature quantity group obtained from the second mosaic processing image generated using the second-size sub-matrix with respect to the sample image is used as an input value and from the sample image Using a dense image neural network that has learned an image correction curve to create an appropriate image as a teacher value It is.

  Regarding the fusion of the rough image correction curve and the dense image correction curve, depending on the quality specifications required for the final image print output, there are cases where a rough image correction curve is preferred or a dense image correction curve is preferred. . Accordingly, in one preferred embodiment of the present invention, the fusion rate of the coarse image correction curve and the dense image correction curve for creating the fusion correction curve is configured to be adjustable. At that time, the spatial frequency component of the input image is calculated through spatial frequency analysis, and the spatial frequency component is compared with the spatial frequency component of the average image. It is convenient to adjust the fusion rate so that the image is fused, and the image close to the low frequency is fused near the coarse image correction curve.

  When the input image is a photographed image for photographic print output, the overall balance and the balance between the subject located at the center and the surrounding background are important. In consideration of this, it is advantageous that the image feature quantity group includes both the area-dependent image feature quantity that depends on the divided area of the input image and the area-independent image feature quantity that does not depend on the divided area.

  In an image correction apparatus that corrects an input image to a proper image by using a correction curve, in order to solve the above-described problem, the apparatus according to the present invention includes a first-size sub-matrix for a correction calculation image corresponding to the input image. And a mosaic processing unit for generating a second mosaic processed image using a sub-matrix of a second size smaller than the first size, and a first mosaic processed image from the first mosaic processed image. An image feature amount calculation unit that obtains an image feature amount group and obtains a second image feature amount group from the second mosaic processed image, and a coarse image correction that generates a coarse image correction curve based on the first image feature amount group A curve generation unit; a dense image correction curve generation unit that generates a dense image correction curve based on the second image feature amount group; the coarse image correction curve and the dense image correction curve; A fusion unit configured to generate a fusion correction curve by fusing consists image correction unit and an image correcting said input image in said fusion correction curve. Naturally, this image correction apparatus can also obtain all the effects described in the above image correction method, and can further incorporate the above-described additional techniques. Also here, a final correction curve is generated by fusing two or more correction curves generated based on two or more mosaic images generated using two or more different sub-matrices. Of course, doing is included.

  In particular, when a neural network technique is employed to generate an image correction curve from a group of image feature quantities, the coarse image correction curve generation unit performs the first size of the prepared sample images. A coarse image neural network in which a first image feature group obtained from a first mosaic processed image generated using a sub-matrix is used as an input value and an image correction curve for creating an appropriate image from the sample image is learned as a teacher value. And the dense image correction curve generation unit uses, as an input value, a second image feature amount group obtained from a second mosaic processed image generated using the second size sub-matrix for the sample image. And a dense image neural network in which an image correction curve for creating an appropriate image from the sample image is learned as a teacher value The over wards.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, an image correction curve generation algorithm for generating an image correction curve based on an image feature amount group which is an important component of the correction technique according to the present invention will be described. State the principle. In the present invention, a neural network is proposed as a particularly preferable image correction curve generation algorithm, but the present invention is not limited to the neural network. Other algorithms based on statistical processing can also be applied.

  In any case, first, as schematically shown in FIG. 1, a large number of images acquired under various shooting conditions are prepared as sample images, and these sample shot images are used as original images. On the other hand, a plurality of mosaic processed images are generated using sub-matrices having different sizes (# 01). In FIG. 1, the first mosaic processed image is generated using the first size sub-matrix, and the second mosaic processed image is generated using the second size sub-matrix smaller than the first size. Further mosaic processing images may be generated using the sub-matrix of the third and fourth sizes. The following processing is similarly performed on the two generated mosaic processed images.

Using the first mosaic processed image as an original image, image correction is manually performed so that an appropriate image is obtained through display on a monitor or a test print (# 02). Next, by comparing and evaluating the pixel values of the original image and the appropriate image obtained by manual image correction, an image correction amount that defines the correspondence between the pixel value of the original image and the pixel value of the appropriate image, and consequently Generates a rough image correction curve defined by the image correction amount (# 03). At that time, in the case of an 8-bit color image, the image correction amount for deriving the pixel value of the appropriate image from the pixel value of each pixel of the original image is 256 pixel value conversion coefficients for each color: β (0). 255), but instead of obtaining the image correction amount (pixel value conversion coefficient) for all 256 pixel values, the image correction amount may be obtained and the rest may be calculated by interpolation. . Since the correspondence between the pixel value of the original image and the pixel value of the appropriate image is most easily understood when considered as an image correction curve: f (x), the term image correction curve will be used hereinafter. In general, such a correction curve can be satisfactorily approximated by a quartic equation, for example.
f (x) = ax ^ 4 + bx ^ 3 + cx ^ 2 + dx + e (x represents the input gradation (0 to 255), ^ represents power)
Thus, the correspondence (correction curve) between the pixel value of the original image and the pixel value of the appropriate image can be represented by at most five coefficients: a, b, c, d, and e.

  Next, a first image feature quantity group: α0... Αn is calculated from the first mosaic processed image that is the original image (# 04). The image feature quantity group includes feature quantities that depend on the image area such as the average density value, maximum density value, and minimum density value in the entire or partial area of the image, and the integrated frequency value of the gradation value partial area obtained from the histogram of each color. A feature quantity that does not depend on the image area is used. The coefficient defining the coarse image correction curve obtained in step # 03 and the first image feature quantity group obtained in step # 04 output the coarse image correction curve while using the first image feature quantity group as input data. Since it is used as learning data for constructing a rough image correction curve generation algorithm (preferably a neural network) as data, the operations from Step # 02 to Step # 04 are performed from all prepared sample images. This is performed for one mosaic processed image. Then, the rough image correction is performed by performing an iterative simulation that adjusts the internal variable so that the coefficient defining the image correction curve becomes the output data while the image feature amount group of the first mosaic processed image is the output data, and thousands of times. A curve generation algorithm is constructed (# 05). When the image correction curve generation algorithm is a neural network, in the learning process, a coefficient that defines the image correction curve is called a teacher value, and the image feature amount group of the first mosaic processed image is used as input data and a coarse image is used. Using the coefficient that defines the correction curve as a teacher value, iterative learning is performed several thousand times or more, and the input / output relationship of the coarse image neural network is constructed.

  Further, the second mosaic processing image generated in step # 01a is used as the original image, and the same processing as the processing from step # 02 to step # 04 described above is performed (# 02a to # 04a). Then, the second image feature group (γ0... Γn) obtained in step # 04a is used as input data, and the dense image correction curve (δ0... Δn) obtained in step # 03a is used as output data. Using the learning data, a dense image correction curve generation algorithm, preferably a dense image neural network is constructed (# 05a). Of course, a further image correction curve generation algorithm, preferably an image neural network, may be constructed using a further mosaic-processed image generated using a sub-matrix of a different size as the original image.

  The rough image correction curve generation algorithm and the dense image correction curve generation algorithm, in which the input / output relationship is determined in this way, in this example, the rough image neural network and the detailed area in which the global balance is important If a dense image neural network in which balance is regarded as important is constructed, the input image to be corrected can be corrected to an appropriate image using these two neural networks. The principle of this image correction is schematically shown in FIG. First, a correction calculation image is generated by performing a reduction process to a predetermined size as necessary from an image input for correction purposes (# 10). Then, a mosaic process is performed from the input image or the correction calculation image using the first size sub-matrix to generate a first mosaic processed image (# 11). A first image feature amount group is calculated from the first mosaic processed image (# 12). By inputting the first image feature quantity group to the coarse image neural network (# 13), a coefficient that defines the coarse image correction curve, and as a result, a coarse image correction curve is obtained (# 14).

  Similarly, a mosaic process is performed from the input image or the correction calculation image using a second-size sub-matrix to generate a second mosaic process image (# 11a), and the second image feature is generated from the second mosaic process image. A quantity group is calculated (# 12a). By inputting the second image feature quantity group to the dense image neural network (# 13a), a coefficient defining the dense image correction curve, and consequently, a dense image correction curve is obtained (# 14a).

  Next, the generated coarse image correction curve (coefficient defining the coarse image correction curve) and the dense image correction curve (coefficient defining the dense image correction curve) are fused at a predetermined fusion rate to obtain a fusion correction curve. (# 15). This fusion rate may be fixed or may be arbitrarily adjusted according to preference, but may be automatically adjusted based on the spatial frequency analysis value of the input image. Next, the created fusion correction curve is set as a use correction curve (# 16), and the input image is corrected using the fusion correction curve (# 17). Further, print data is generated from the corrected image, and the print data is transferred to the print unit, whereby an image print is output (# 18).

  Next, an example of a specific procedure for constructing the above-described neural network will be described with reference to FIGS. The rough image neural network and the dense image neural network have the same construction procedure. FIG. 3 shows a schematic structure of the neural network. This neural network consists of an input layer having 89 input elements, an intermediate layer having 45 intermediate elements, and an output layer having 60 output elements. Image feature groups of input images: α0 to α88 (first image feature groups) and γ0 to γ88 (second image feature groups) are input to the input elements, and the output elements are as shown in FIG. The output value as a coefficient for defining the image correction curve indicated by 20 representative points each defining the correction curve for each color, that is, R component output values β (1) to β (20) (coarse image correction curve), δ (1) to δ (20) (dense image correction curve); G component output values β (21) to β (40) (rough image correction curve), δ (21) to δ (40) (dense image correction) Curve); B component output values β (41) to β (60), δ (41) to δ (60) (dense image correction curve) are output. A known approximate curve creation algorithm can be used as an algorithm for generating a smooth correction curve for each color using the output value.

The definition contents of the first image feature quantity group: α0 to α88 and the second image feature quantity group: γ0 to γ88 adopted at the time of learning of the neural network are the same, and the target images are different (first mosaic processing) Only the first image feature value group will be described below.
α0: Average density value of the entire image α1: Average density value of the central portion of the image (see FIG. 5A) α2: Average density value of the peripheral portion of the image (see FIG. 5A) α3: Upper portion of the image ( Average density value α4 in FIG. 5 (b)): Average density value α5 in the lower side of the image (see FIG. 5 (b)): Average density value α6 in the left side of the image (see FIG. 5 (c)) : Average density value α7 at the right side of the image (see FIG. 5C): Maximum density value α8 of the entire image α8: Maximum density value at the center of the image (see FIG. 5A) α9: Image peripheral part (see FIG. 5) 5) (see (a) of FIG. 5): maximum density value α11 of the upper side of the image (see (b) of FIG. 5): maximum density value α12 of the lower side of the image (see (b) of FIG. 5): Maximum density value α13 at the left side of the image (see FIG. 5C): Maximum density value α14 at the right side of the image (see FIG. 5C): Minimum density value α15 of the entire image: Central portion of the image (see FIG. 5) (See (a)) Density value α16: Minimum density value α17 at the peripheral portion of the image (see FIG. 5A): Minimum density value α18 at the upper side portion of the image (see FIG. 5B): Lower side portion of the image (see FIG. 5B) ))) Minimum density value α19: minimum density value α20 on the left side of the image (see FIG. 5C): minimum density value α21 on the right side of the image (see FIG. 5C): pixels adjacent vertically Mean value α22: density standard deviation α23 of the entire image α23: maximum value of the entire image regarding the R color component α24: maximum value of the entire image regarding the G color component α25: B color component The maximum value of the entire image relating to the R color component α27: The minimum value of the entire image relating to the G color component α28: The minimum value of the entire image relating to the B color component α29 to α48: The histogram level relating to the R color component Obtained by integrating the frequency of gradation values included in each area (see FIG. 6) obtained by dividing the gradation value axis into 20 parts. Integrated frequency values α49 to α68: G integrated frequency values α69 to α88 obtained by integrating the frequencies of the gradation values included in each region (see FIG. 6) obtained by dividing the gradation value axis of the histogram relating to the G color component into 20: B integrated frequency value obtained by integrating the frequency of the gradation value included in each region (see FIG. 6) obtained by dividing the gradation value axis of the histogram related to the B color component into 20 image features from α0 to α28 are images. The feature amount depends on the region, and the image feature amount from α29 to α88 is a feature amount independent of the image region. Further, in the calculation of the first image feature value group and the second image feature value group from the mosaic processed image, the mosaic block (sub-matrix size) is calculated as one pixel. As the density value, a value obtained by adding the values of each pixel of the R, G, and B color components and dividing by 3 is adopted.

FIG. 7 shows a construction routine for a neural network having the above input / output configuration.
First, the prepared sample image is read (# 20), each image feature amount described above is calculated using the mosaic processing image created from the sample image (# 21), and these are used as an image feature amount group. Store in the memory (# 22). Manual image correction by an expert is performed while displaying the read sample image on the monitor (# 23). A printout of the corrected image is performed as necessary (# 24), and it is checked whether the corrected image is appropriate (# 25). Manual correction is repeated until the corrected image is determined to be appropriate, and when an appropriate corrected image is obtained, an image correction curve that converts the pixel value of the original image into the pixel value of the appropriate corrected image based on the image correction (Coefficient defining image correction curve) is created and stored in the memory (# 26). The processing from step # 20 to step # 26 is executed for all sample images for which preparation is made (# 27 Yes branch). When the processing for all the sample images is completed (# 27 No branch), the image feature amount group and the image correction curve of the mosaic processed image obtained so far are applied to the neural network (# 28). The neural network learns the image feature group as input data and the image correction curve as teacher data, and determines the input / output relationship (coupling coefficient and threshold) (# 29). By using such a neural network construction routine, a coarse image neural network and a dense image neural network, and a further super coarse image or ultra dense image neural network are constructed as necessary.

  Next, a photographic printing apparatus equipped with an image correction apparatus using the image correction technique described with reference to FIGS. 1 and 2 and handling captured images as images will be described. FIG. 8 is an external view showing the photographic printing apparatus. This photographic printing apparatus includes a printing station 1B as a photographic printing section that performs exposure processing and development processing on the photographic paper P, a developed photographic film 2a, and the like. The operation station 1A is configured to process a captured image taken from an image input medium such as a digital camera memory card 2b and generate / transfer print data used in the print station 1B.

  This photo printing apparatus is also called a digital minilab. As can be understood from FIG. 9, the printing station 1B pulls out the rolled photographic paper P stored in the two photographic paper magazines 11 and prints it with the sheet cutter 12. The back print unit 13 prints print processing information such as color correction information and frame number on the back side of the photographic paper P, and the print exposure unit 14 cuts the print paper P into the size. A photographed image is exposed on the surface of the photographic paper P, and the exposed photographic paper P is sent to a processing tank unit 15 having a plurality of development processing tanks for development processing. After drying, the photographic printing paper P, that is, the photographic prints P, sent to the sorter 17 from the transverse feed conveyor 16 at the upper part of the apparatus is collected in a plurality of trays of the sorter 17 in a state of being sorted in order units (see FIG. 8). ).

  A photographic paper transport mechanism 18 is laid to transport the photographic paper P at a transport speed in accordance with various processes for the photographic paper P described above. The photographic paper transport mechanism 18 is composed of a plurality of nipping and transporting roller pairs including a chucker type photographic paper transport unit 18a disposed before and after the print exposure unit 14 in the photographic paper transport direction.

  The print exposure unit 14 applies R (red), G (green), and B (blue) to the printing paper P conveyed in the sub-scanning direction based on print data from the operation station 1A along the main scanning direction. A line exposure head for irradiating laser beams of the three primary colors (1) is provided. The processing tank unit 15 includes a color developing tank 15a for storing a color developing processing liquid, a bleach-fixing tank 15b for storing a bleach-fixing processing liquid, and a stabilizing tank 15c for storing a stable processing liquid.

  At the upper position of the desk-like console of the operation station 1A, there is a film scanner 20 capable of acquiring photographed image data (hereinafter simply referred to as a photographed image) from a photographed image frame of the photographic film 2a with a resolution exceeding 2000 dpi. A media reader 21 that is arranged and acquires captured images from various semiconductor memories or CD-Rs used as an image recording medium 2b mounted on a digital camera or the like functions as the controller 3 of this photographic printing apparatus. Built into a general-purpose computer. The general-purpose personal computer is also connected with a monitor 23 for displaying various information, and a keyboard 24 and a mouse 25 as operation input devices used as an operation input unit used for various settings and adjustments.

  The controller 3 of this photographic printing apparatus uses a CPU as a core member and constructs a functional unit for performing various operations of the photographic printing apparatus by hardware and / or software, as shown in FIG. As described above, the functional unit particularly related to the present invention includes an image input unit 31 that takes a photographed image read by the film scanner 20 or the media reader 21 and performs preprocessing necessary for the next processing, Graphic user interface (hereinafter abbreviated as GUI) that generates a control command from creation of a graphic operation screen including a window, various operation buttons, and the like, and user operation input through such a graphic operation screen (using the keyboard 24, mouse 25, etc.) Sent from the GUI unit 33 and the GUI unit 33 A print management unit 32 that performs image processing on a captured image transferred from the image input unit 31 to the memory 30 in order to generate desired print data based on a control command or an operation command directly input from the keyboard 24 or the like. Video control for generating a video signal for causing the monitor 23 to display graphic data sent from the GUI source 33, a simulated image as a print source image, an expected finished print image, and a pre-judge print operation such as color correction A print data generation unit 36 for generating print data suitable for the print exposure unit 14 installed in the print station 1B based on the processed photographed image having undergone the image processing, C shot images or processed shot images that have been processed Etc. formatter 37 for formatting into a form for writing to -R and the like.

  When the photographic image recording medium is the film 2a, the image input unit 31 separately sends the scan data for the pre-scan mode and the main scan mode to the memory 30, and performs preprocessing according to each purpose. When the captured image recording medium is the memory card 2b, when the captured image includes a thumbnail image (low resolution data), the captured image main data (for use in displaying a list on the monitor 23) ( The thumbnail image is sent to the memory 30 separately from the high-resolution data. If a thumbnail image is not included, a reduced image is created from this data and sent to the memory 30 as a thumbnail image. Further, the input captured image used for calculating the above-described feature amount set at high speed from the input captured image is converted into a correction processing captured image (for example, 256 × 384 pixels) having a smaller image size. And a mosaic processing function.

  The print management unit 32 includes a print order processing unit 40 that manages the print size, the number of prints, and the like, and an image processing unit 50 that performs various types of image processing on the captured image developed in the memory 30.

  The image processing unit 50 includes a first image correction unit 60 in which a photo retouching program for manually performing various image processing on the captured image developed in the memory 30 and a neural network as described above. A second image correction unit 70 that performs automatic image correction on an input photographed image developed in the memory 30 using an image correction technique is basically incorporated in the form of a program.

  As shown in FIG. 11, the second image correction unit 70 having a function as an image correction apparatus according to the present invention has a correction calculation photographed image (256) created from the input photographed image developed in the memory 30. The first mosaic processed image is generated using the 24 × 16 sub-matrix as the first size and the second mosaic processed image using the 48 × 32 sub-matrix as the second size. , A mosaic processing unit 71 that generates a first image feature amount group from the first mosaic processed image, and an image feature amount calculation unit 72 that obtains a second image feature amount group from the second mosaic processed image, and a first image feature By inputting a coarse image neural network unit 73a that generates a coarse image correction curve by inputting a quantity group, and by inputting a second image feature quantity group The dense image neural network unit 73b that generates the image correction curve, and the generated coarse image correction curve and the dense image correction curve are fused with the fusion rate set by the fusion rate adjustment unit 76 to generate a fusion correction curve. A fusion unit 75 and an image correction unit 77 for correcting an input photographed image using the fusion correction curve are provided. The image correction unit 77 includes an image correction execution unit 77a that corrects an input captured image based on the correction curve set by the correction curve setting unit 77b. The fusion rate adjustment by the fusion rate adjustment unit 76 can be manually adjusted through the GUI unit 33, but can also be automatically adjusted based on the spatial frequency characteristics obtained through the spatial frequency analysis of the input photographed image. It is.

  The size of the sub-matrix for the mosaic processing used in the mosaic processing unit 71 is set by the sub-matrix size setting unit 71a. As the size used here, 12 × 8, 36 × 24, and the like are suitable in addition to the above in consideration of the size of the correction calculation photographed image (256 × 384 image).

  A photo print process in the photo printing apparatus configured as described above will be described with reference to the flowchart of FIG. First, a captured image input to perform a photographic print output is read (# 50), and the mosaic processing unit 71 uses the correction processing captured image created from the captured image, and the mosaic processing unit 71 performs the first mosaic processing image and the second mosaic processing. A processed image is generated (# 51). The image feature amount computing unit 71 calculates the first image feature amount group: α0 to α88 from the first mosaic processed image, and calculates the second image feature amount group: γ0 to γ88 from the second mosaic processed image (# 52). ). By giving the first image feature amount group to the input element of the coarse image neural network unit 73a (# 53), an image correction amount suitable for the coarse image: β (n) is outputted as an output value, and the coarse image correction curve is obtained. It is created (# 54). Similarly, by giving the second image feature quantity group to the input element of the dense image neural network unit 73b (# 55), an image correction amount: δ (n) suitable for the dense image is output as an output value, and the dense image is output. A correction curve is created (# 56).

  A fusion rate for fusing the coarse image correction curve and the dense image correction curve is determined (# 57). The fusion rate may be a constant set in advance, or may be a variable that changes according to image characteristics such as a spatial frequency characteristic of the input photographed image. For example, it can be obtained by an arithmetic expression as shown by fusion rate = g (image characteristics), but the function at that time may be a discontinuous function or a continuous function. Further, the fusion rate may be different for each region such as a shadow region, an intermediate region, and a highlight region.

  When the fusion rate is determined, a fusion correction curve of the coarse image correction curve and the dense image correction curve is created with the fusion rate (# 58). The created fusion correction curve is given to the correction curve setting unit 77b of the image correction unit 77 (# 59). Subsequently, the input correction image is corrected using the correction curve (fusion correction curve or individual correction curve) set by the image correction execution unit 77a (# 60), and the print data generation unit 36 is corrected from the corrected shooting image. Generates print data (# 61). Print processing is performed by transferring the print data to the print station 1B (# 62), and a photo print is output (# 63).

  In the description of the above-described embodiment, the example in which the image correction technology according to the present invention is incorporated in a photo printing apparatus called a minilab installed in a DP shop has been taken up. However, it is installed in a convenience store or a DP shop storefront. The image processing software may be incorporated into various photographic printing apparatuses such as a self-service photographic printing apparatus, or may be incorporated into a program of the image processing software as one of the photographed image correction functions of the image processing software.

  In the above-described embodiment, the print station 1B serving as the image print output unit exposes the photographic image to the photographic paper P by the print exposure unit 14 including the laser type exposure engine, and prints after the exposure. Although the so-called silver salt photographic printing method for developing the paper P was adopted, of course, the printing method of the photographed image corrected by the present invention is not limited to such a method. And various photographic printing methods such as an inkjet printing method in which an image is formed by ejecting ink onto paper and a thermal transfer method using a thermal transfer sheet.

Explanatory drawing explaining the basic principle used with the image correction technique by this invention Explanatory drawing explaining the basic principle used with the image correction technique by this invention Schematic diagram showing the configuration of a neural network Explanatory drawing explaining the image correction curve as the output of the neural network Explanatory drawing explaining the image feature amount depending on the image area Explanatory drawing explaining the image feature amount depending on the histogram Flow chart showing the construction routine of the neural network External view of photographic printing apparatus adopting image correction technology according to the present invention Schematic diagram schematically showing the configuration of the print station of the photo printing device Functional block diagram explaining the functional elements built in the controller of the photo printing device 1 is a functional block diagram showing the functional configuration of an image correction unit that employs the image correction technology of the present invention. Flow chart showing the procedure of photo print processing

Explanation of symbols

1B: Print station (print section)
31: Image input unit 36: Print data generation unit 70: Second image correction unit (image correction device)
71: Mosaic processing unit 71a: Submatrix size setting unit 72: Image feature amount calculation unit 73a: Coarse image neural network unit 73b: Dense image neural network unit 74a: Coarse image correction curve creation unit 74b: Dense image correction curve creation unit 75 : Fusion unit 76: fusion rate adjustment unit 77: image correction unit 77 a: image correction execution unit 77 b: correction curve setting unit

Claims (7)

In an image correction method for correcting an input image to a proper image using a correction curve,
Generating a first mosaiced image using a first-size sub-matrix for a correction calculation image corresponding to the input image;
Obtaining a first image feature quantity group from the first mosaic processed image;
Generating a coarse image correction curve based on the first image feature amount group;
Generating a second mosaic processed image using a sub-matrix of a second size smaller than the first size for the correction calculation image corresponding to the input image;
Obtaining a second image feature quantity group from the second mosaic processed image;
Generating a dense image correction curve based on the second image feature quantity group;
Fusing the coarse image correction curve and the dense image correction curve to generate a fusion correction curve;
Image correcting the input image with the fusion correction curve;
An image correction method comprising: In the step of generating the rough image correction curve, a first image feature amount group obtained from a first mosaic processed image generated using the first-size sub-matrix is input to a number of prepared sample images. A coarse image neural network unit that learns, as a teacher value, an image correction curve that creates an appropriate image from the sample image and generates a dense image correction curve in the step of generating the dense image correction curve. The second image feature amount group obtained from the second mosaic processed image generated using the second size sub-matrix is used as an input value, and an image correction curve for creating an appropriate image from the sample image is learned as a teacher value. 2. The image compensation according to claim 1, wherein a dense image neural network unit is used. Right way.   The image correction method according to claim 1, wherein a fusion rate between the coarse image correction curve and the dense image correction curve for creating the fusion correction curve is adjustable.   The image correction method according to claim 3, wherein the fusion rate is adjusted based on a spatial frequency analysis value of the input image.   The first image feature quantity group and the second image feature quantity group are an area-independent image feature quantity that depends on the divided area of the correction calculation image and an area-independent area that does not depend on the divided area of the correction calculation image, respectively. The image correction method according to any one of claims 1 to 4, wherein both image feature amounts are included. In an image correction apparatus that corrects an input image to a proper image using a correction curve,
A first mosaic processed image is generated using a first size sub-matrix for the correction calculation image corresponding to the input image, and a second mosaic is generated using a second size sub-matrix smaller than the first size. A mosaic processing unit for generating a processed image;
An image feature quantity computing unit for obtaining a first image feature quantity group from the first mosaic processed image and obtaining a second image feature quantity group from the second mosaic processed image;
A coarse image correction curve generation unit for generating a coarse image correction curve based on the first image feature quantity group;
A dense image correction curve generating unit for generating a dense image correction curve based on the second image feature amount group;
A fusion unit for fusing the coarse image correction curve and the dense image correction curve to generate a fusion correction curve;
An image correction unit that corrects the input image with the fusion correction curve;
An image correction apparatus comprising: The coarse image correction curve generation unit uses, as an input value, a first image feature amount group obtained from a first mosaic processed image generated using the first size sub-matrix for a large number of prepared sample images. And a coarse image neural network unit that learns, as a teacher value, an image correction curve that creates an appropriate image from the sample image, and the dense image correction curve generation unit has a sub-size of the second size with respect to the sample image. A dense image neural network unit having a second image feature amount group obtained from a second mosaic processed image generated using a matrix as an input value and learning an image correction curve for creating an appropriate image from the sample image as a teacher value The image correction apparatus according to claim 6, wherein the image correction apparatus is an image correction apparatus.

Images