JP4605468B2 - Image processing apparatus, image processing method, learning apparatus, learning method, and program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, a learning device, a learning method, and a program, and more particularly to an image processing device, an image processing method, and a learning device that can accurately improve the quality of an artificial image. And a learning method and program.

  The present applicant has previously proposed a classification adaptation process (see, for example, Patent Document 1). The class classification adaptive processing classifies the target pixel, which is the pixel of the second image obtained from the first image, into a class according to the pixel values of the plurality of pixels in the predetermined area of the input first image. The input is performed by calculating a linear linear expression of a prediction coefficient corresponding to the class, which is obtained in advance for each class by learning processing, and pixel values of a plurality of pixels in a predetermined area of the first image. This is a process for obtaining the second image from the first image.

  For example, if the first image is an image including noise and the second image is an image in which noise is reduced, the classification adaptation process functions as a noise removal process, and the first image If it is an SD (Standard Definition) image and the second image is an HD (High Definition) image with a higher resolution than the SD image, this classification adaptation processing is a resolution that converts a low-resolution image into a high-resolution image. Functions as a conversion process.

  Further, in the conventional learning process, in order to enable prediction of general moving images mainly based on broadcasting, images that exist in the natural world, which are not artificial images described later, are captured as they are as teacher images and student images. A natural image that is an image obtained by this is used. Therefore, when the first image is a natural image, a high-definition and beautiful second image is predicted from the first image by performing the class classification adaptive processing using the prediction coefficient obtained by the learning processing. can do.

JP 9-167240 A

  However, when the first image is an artificial image such as a character or a simple figure, when the class classification adaptive processing is performed using the prediction coefficient obtained by the learning processing using the natural image, the edge originally stands sufficiently. The sense of fineness of the current location is further emphasized, or flat noise is recognized as a correct waveform, and the fineness of the noise is emphasized. As a result, ringing occurs or noise is emphasized.

  Here, the artificial image is an artificial image such as a character or a simple figure with few gradations and clear phase information indicating the position of the edge, that is, many flat portions. In other words, the artificial image is an image in which there is not much gradation such as characters and simple figures, and information indicating the position of the contour or the like is dominant.

  The present invention has been made in view of such a situation, and makes it possible to accurately improve the quality of an artificial image.

The image processing apparatus according to the first aspect of the present invention is an image for obtaining an output image having a high quality from an input image, which is an artificial image having a small gradation and a clear edge, out of the input images. In the processing device, a first class tap extraction that extracts, from the input image, a first class tap composed of a plurality of pixels that are used to classify a target pixel that is a pixel of the output image into a first class. Means for calculating a difference between pixel values of adjacent pixels of the first class tap, and a first classifying a target pixel corresponding to the difference into the first class based on the difference. And a second class tap extraction unit that extracts, from the input image, a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class. , Based on the pixel value of each pixel of the second class taps, the pixel of interest corresponding to the second class taps, and a second classifying means for classifying the second class, the first class Based on the selection means for selecting one of the first class of the pixel of interest and the second class of the pixel of interest as the class of the pixel of interest, and using a plurality of artificial images Storage means for storing the prediction coefficient for each of the first classes and the prediction coefficient for each of the second classes obtained by learning, the input image, and the first or selected as the class of the target pixel Computation means for obtaining the output image from the input image by performing computation using the second class of prediction coefficients.

Said first classifying means, based on the absolute value of the difference, it is possible to classify the target pixel corresponding to the difference in the first class.

Said first classifying means, based on the ratio of the absolute value of the difference, it is possible to classify the target pixel corresponding to the difference in the first class.

The image processing method according to the first aspect of the present invention is an image for obtaining an output image with high quality from an input image, which is an artificial image having a small gradation and a clear edge, out of the input images. In the processing method, a first class tap composed of a plurality of pixels used for classifying a pixel of interest as a pixel of the output image into a first class is extracted from the input image, and the first class is extracted. In order to obtain a pixel value difference between adjacent pixels of the tap, classify the target pixel corresponding to the difference into the first class, and classify the target pixel into the second class based on the difference A second class tap composed of a plurality of pixels to be used is extracted from the input image, and based on the pixel value of each pixel of the second class tap, the target pixel corresponding to the second class tap The classified into the second class, based on the first class, said first class of the pixel of interest, either one of the second class of the pixel of interest, the pixel of interest Class of the pixel of interest among the prediction coefficient for each of the first class and the prediction coefficient for each of the second class acquired by learning using the input image and a plurality of artificial images And calculating the output image from the input image by calculating using the first or second class prediction coefficient selected as.

The program according to the first aspect of the present invention is a process for obtaining an output image having a high quality artificial image, which is an artificial image with small gradation and clear edges, from the input image. A program to be executed by a computer, wherein a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the output image into a first class is extracted from the input image, A difference between pixel values of adjacent pixels of the first class tap is obtained, and based on the difference, a target pixel corresponding to the difference is classified into the first class, and the target pixel is classified into a second class. A second class tap composed of a plurality of pixels used to classify the second class tap is extracted from the input image, and based on the pixel value of each pixel of the second class tap, the second class tap is extracted. The target pixel corresponding to Stapp, classified into the second class, based on the first class, said first class of the pixel of interest, any of the second class of the pixel of interest One of them as the class of the pixel of interest, and the prediction coefficient for each of the first class and the prediction coefficient for each of the second class obtained by learning using the input image and a plurality of artificial images. Among them, the method includes a step of calculating the output image from the input image by calculating using the prediction coefficient of the first or second class selected as the class of the target pixel.

In the first aspect of the present invention, a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of an output image into a first class is extracted from the input image, A difference between pixel values of adjacent pixels of one class tap is obtained, and based on the difference, a target pixel corresponding to the difference is classified into the first class, and the target pixel is classified into the second class. A second class tap composed of a plurality of pixels used in the above is extracted from the input image, and based on the pixel value of each pixel of the second class tap, the target pixel corresponding to the second class tap is is classified into the second class, based on the first class, the first class of the subject pixel, either one of the second class of the pixel of interest, is selected as the class of the subject pixel, the input image And multiple The prediction coefficient of the first or second class selected as the class of the pixel of interest among the prediction coefficient for each first class and the prediction coefficient for each second class acquired by learning using the processed image. The output image is obtained from the input image by using the calculation.

The learning device according to the second aspect of the present invention learns a prediction coefficient used when performing a conversion process for converting an artificial image, which is an artificial image with few gradations and sharp edges, into a high-quality artificial image. A learning device, comprising: a teacher image that is a target artificial image after the conversion process; an acquisition unit that acquires a student image corresponding to the artificial image before the conversion process; and a pixel of the teacher image A first class tap extracting means for extracting a first class tap composed of a plurality of pixels used for classifying a certain target pixel into a first class from the student image; and the extracted first class tap Difference calculating means for obtaining a difference between pixel values of adjacent pixels of the class tap, first classification means for classifying a target pixel corresponding to the difference into the first class based on the difference, and the attention Second class tap extraction means for extracting from the student image a second class tap composed of a plurality of pixels used to classify a prime into a second class; and each pixel of the second class tap Second classifying means for classifying the target pixel corresponding to the second class tap into the second class based on the pixel value of the second class tap; and the second class means of the target pixel based on the first class . A selection unit that selects one of the first class and the second class of the target pixel as the class of the target pixel, and a prediction that includes a plurality of pixels that are used to obtain the target pixel Predictive tap extracting means for extracting a tap from the student image, and the first or second selected as a class of pixel of interest corresponding to the predictive tap based on the extracted predictive tap For each class, and a calculating means for calculating the prediction coefficients for statistically minimizing the prediction error of the pixel of interest is determined using the prediction coefficients.

In the second aspect of the present invention, a teacher image that is a target artificial image after the conversion process and a student image corresponding to the artificial image before the conversion process are acquired, and the attention image is a pixel of the teacher image A first class tap composed of a plurality of pixels used for classifying a pixel into the first class is extracted from the student image, and a difference between pixel values of adjacent pixels of the extracted first class tap is extracted. A second class tap composed of a plurality of pixels used to classify the pixel of interest into the second class based on the difference, the pixel of interest corresponding to the difference is classified into the first class Is extracted from the student image, and based on the pixel value of each pixel of the second class tap, the pixel of interest corresponding to the second class tap is classified into the second class, and based on the first class Featured image One of the first class and the second class of the target pixel is selected as the class of the target pixel, and a prediction tap including a plurality of pixels used for obtaining the target pixel is a student image. For each first or second class selected as the class of the target pixel corresponding to the prediction tap based on the extracted prediction tap, the prediction error of the target pixel obtained using the prediction coefficient is statistically calculated. The prediction coefficient that minimizes the result is calculated.

  According to the present invention, the quality of an artificial image can be improved accurately.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

The image processing apparatus according to the first aspect of the present invention has a small gradation and a clear edge in the input image from an input image (for example, an HD image input from the output phase converter 112 in FIG. 1). In an image processing apparatus (for example, the artificial image prediction unit 132 in FIG. 10) that obtains an output image (for example, an artificial high quality image) obtained by improving the quality of an artificial image that is an artificial image, attention is given to pixels of the output image. First class tap extraction means for extracting, from the input image, a first class tap composed of a plurality of pixels used for classifying pixels into the first class, and adjacent to the first class tap. Difference calculating means for obtaining a pixel value difference between pixels, and first classification means for classifying the target pixel corresponding to the difference into the first class based on the difference (for example, the class classification unit in FIG. 10) 51), second class tap extraction means for extracting, from the input image, a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class, and Based on the pixel value of each pixel of the class tap, second classifying means for classifying the target pixel corresponding to the second class tap into the second class (for example, class classifier 861 in FIG. 23) If, on the basis of the first class, said first class of the pixel of interest, either one of the second class of the pixel of interest, selecting means for selecting as a class of the pixel of interest ( For example, the prediction coefficient for each of the first classes and the prediction coefficient for each of the second classes (for example, the prediction coefficient W _n ) acquired by learning using a class selection unit 863) and a plurality of artificial images in FIG. ) Storage means (for example, the prediction coefficient memory 654 in FIG. 10), the input image, and the prediction coefficient of the first or second class selected as the class of the target pixel, Computation means for obtaining the output image from the input image (for example, the prediction unit 655 in FIG. 10).

Said first classifying means, the absolute value of the difference (e.g., the adjacent pixel difference absolute value) on the basis of, classifying a pixel of interest that corresponds to the difference in the first class (e.g., the classification of FIG. 16 processing).

The first classifying unit classifies the target pixel corresponding to the difference into the first class based on a ratio of absolute values of the differences (for example, horizontal difference ratio) (for example, class classification in FIG. 22). processing).

In the image processing method according to the first aspect of the present invention, an input image (for example, an HD image input from the output phase converter 112 in FIG. 1) has a small gradation and a clear edge from the input image. In an image processing method for obtaining an output image (for example, an artificial high quality image) in which an artificial image, which is an artificial image, has a high quality, it is used to classify a pixel of interest that is a pixel of the output image into a first class. A first class tap composed of a plurality of pixels is extracted from the input image, a difference between pixel values of adjacent pixels of the first class tap is obtained, and the difference is handled based on the difference. Classifying the target pixel to be classified into the first class (for example, step S701 in FIG. 15), and selecting a second class tap composed of a plurality of pixels used to classify the target pixel into the second class. , Extracted from the input image, based on the pixel value of each pixel of the second class taps, the pixel of interest corresponding to the second class taps are classified into the second class, the first Based on the class , one of the first class of the pixel of interest and the second class of the pixel of interest is selected as the class of the pixel of interest, and the input image and a plurality of artificial Of the prediction coefficient (for example, prediction coefficient W _n ) for each first class and the prediction coefficient for each second class acquired by learning using an image, the first pixel selected as the class of the pixel of interest It includes the step of obtaining the output image from the input image (for example, step S706 in FIG. 15) by calculating using the prediction coefficient of the first or second class.

The learning device according to the second aspect of the present invention provides an artificial image (for example, an HD image input from the output phase conversion unit 112 in FIG. 1), which is an artificial image with few gradations and clear edges, with high quality. A learning device (for example, the learning device 811 in FIG. 17) that learns a prediction coefficient (for example, the prediction coefficient W _n ) that is used when performing a conversion process for converting into an artificial image (for example, an artificial high quality image) An acquisition means (for example, a student image generation unit 821 in FIG. 17) for acquiring a teacher image to be a target artificial image after the conversion process and a student image corresponding to the artificial image before the conversion process; First class tap extraction means for extracting a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the teacher image into a first class from the student image (for example, In FIG. A class tap extraction unit of the class classification unit 822 in FIG. 17 configured in the same manner as the last tap extraction unit 671, and a difference calculation unit that calculates a difference in pixel values between adjacent pixels of the extracted first class tap. First classifying means for classifying the target pixel corresponding to the difference into the first class based on the difference (for example, the class classification unit in FIG. 17 configured similarly to the ADRC processing unit 673 in FIG. 11) 822 ADRC processing unit) and second class tap extraction means for extracting from the student image a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class. , Based on the pixel value of each pixel of the second class tap, second classifying means for classifying the target pixel corresponding to the second class tap into the second class, and the first class Based on And selecting means for selecting one of the first class of the pixel of interest and the second class of the pixel of interest as the class of the pixel of interest, and used for obtaining the pixel of interest. Corresponding to the prediction tap based on the prediction tap extraction means (for example, the prediction tap extraction unit 831 in FIG. 18) that extracts prediction taps composed of a plurality of pixels and the extracted prediction tap. Computing means for computing the prediction coefficient that statistically minimizes the prediction error of the pixel of interest obtained using the prediction coefficient for each of the first or second class selected as the class of the pixel of interest For example, a normal equation generation unit 832) of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of an image conversion apparatus 101 to which the present invention is applied. The image conversion apparatus 101 includes a cyclic IP conversion unit 111, an output phase conversion unit 112, a natural image prediction unit 131, an artificial image prediction unit 132, a natural image / artificial image determination unit 114, and a synthesis unit 133. The cyclic IP conversion unit 111 includes an IP conversion unit 121 and a cyclic conversion unit 122.

  An interlaced SD image to be processed is input to the IP conversion unit 121 and the cyclic conversion unit 122 of the cyclic IP conversion unit 111.

  Based on a predetermined method, the IP conversion unit 121 converts an input interlaced SD image (hereinafter also referred to as an input image) into a progressive SD image (hereinafter also referred to as an intermediate image). The progressive SD image is supplied to the cyclic conversion unit 122.

  The cyclic conversion unit 122 obtains a motion vector between an input image and a progressive SD image (hereinafter also referred to as an output image) output from the cyclic conversion unit 122 one frame before. The cyclic conversion unit 122 weights and adds the pixel value of the input image and the pixel value of the image that has been subjected to motion compensation to the output image based on the obtained motion vector, using the cyclic coefficient. Improve image quality. The recursive conversion unit 122 converts the intermediate image into an output image that is a higher-quality progressive SD image, and supplies the output image to the output phase conversion unit 112. Note that the cyclic coefficient is a reliability indicating whether each pixel of the intermediate image is in a position where the pixel exists in the input image before conversion, the magnitude of the motion vector in the vertical direction, and the probability of the motion vector. It is set based on the probability.

  The output phase conversion unit 112 performs interpolation in the horizontal direction and the vertical direction on the SD image having the first number of pixels supplied from the cyclic conversion unit 122, so that the second number larger than the first number of pixels. An HD image having the number of pixels is generated. The output phase conversion unit 112 supplies the HD image to the natural image prediction unit 131, the artificial image prediction unit 132, and the natural image / artificial image determination unit 114.

  The image processing unit 113 processes the HD image into a high quality image based on the artificial image supplied from the natural image / artificial image determination unit 114, and outputs a high quality HD image as a processing result.

  The natural image / artificial image determination unit 114 determines whether each pixel of the HD image supplied from the output phase conversion unit 112 belongs to an area classified as an artificial image or an area classified as a natural image. The determination result is output to the image processing unit 113 as an artificial image degree. That is, the artificial degree of art is a value of 0 to 1 indicating the ratio of the artificial image in the natural image in the region classified between the artificial image and the natural image.

  The natural image prediction unit 131 predicts, from the HD image supplied from the output phase conversion unit 112, an HD image (hereinafter, referred to as a natural high quality image) in which the natural image of the HD image is of high quality. Specifically, the natural image prediction unit 131 classifies the target pixel, which is a pixel of a natural high quality image obtained from the HD image, into a class suitable for the feature of the natural image, according to the feature of the HD image. Then, the natural image prediction unit 131 performs an operation using a prediction coefficient for predicting a natural high-quality image corresponding to the class and the HD image, thereby providing an HD image supplied from the output phase conversion unit 112. Predict natural high-quality images. The natural image prediction unit 131 supplies the natural high quality image to the synthesis unit 133.

  Similar to the natural image prediction unit 131, the artificial image prediction unit 132 uses an HD image (hereinafter referred to as artificial high quality) obtained from the HD image supplied from the output phase conversion unit 112 to a high quality image of the HD image. (Referred to as image). Specifically, the artificial image prediction unit 132 classifies the target pixel, which is a pixel of an artificial high quality image obtained from the HD image, into a class suitable for the feature of the artificial image according to the feature of the HD image. Then, the artificial image predicting unit 132 performs an operation using a prediction coefficient for predicting the artificial high quality image corresponding to the class and the HD image, and thereby the HD image supplied from the output phase converting unit 112. To predict artificial high quality images. The artificial image prediction unit 132 outputs the artificial high quality image to the synthesis unit 133.

  Based on the determination result supplied from the natural-image / artificial-image determination unit 114, the synthesis unit 133 supplies the pixel value of each pixel of the natural high-quality image supplied from the natural-image prediction unit 131 and the artificial-image prediction unit 132. A pixel value of each pixel of the artificial high-quality image to be synthesized is synthesized at a ratio corresponding to the artificial picture degree, and an HD image obtained as a result of the synthesis is output.

  Next, image conversion processing executed by the image conversion apparatus 101 will be described with reference to the flowchart of FIG. This process is started, for example, when input of an interlaced SD image is started from the outside.

  In step S1, the IP conversion unit 121 performs IP conversion. Specifically, the IP conversion unit 121 IP-converts an interlaced input image into a progressive intermediate image based on a predetermined method, and supplies the IP-converted intermediate image to the cyclic conversion unit 122.

  In step S2, the cyclic conversion unit 122 performs a cyclic conversion process. Specifically, a motion vector between the input image and the output image output from the cyclic conversion unit 122 one frame before is obtained. The cyclic conversion unit 122 weights and adds the pixel value of the input image and the pixel value of the image that has been subjected to motion compensation to the output image based on the obtained motion vector, using the cyclic coefficient. Improve image quality. The recursive conversion unit 122 converts the intermediate image into an output image that is a higher-quality progressive SD image, and supplies the output image to the output phase conversion unit 112.

  In step S3, the output phase converter 112 performs an output phase conversion process. Specifically, the output phase conversion unit 112 generates an HD image by performing interpolation in the horizontal direction and the vertical direction on the SD image supplied from the cyclic conversion unit 122. The output phase conversion unit 112 supplies the HD image to the natural image prediction unit 131, the artificial image prediction unit 132, and the natural image / artificial image determination unit 114.

  In step S4, the natural image prediction unit 131 performs natural image prediction processing. Details of the natural image prediction process will be described later with reference to FIG. 6, and a natural high quality image is predicted from the HD image by this process, and the natural high quality image is supplied to the synthesis unit 133.

  In step S5, the artificial image prediction unit 132 performs an artificial image prediction process. Details of the artificial image prediction process will be described later with reference to FIG. 15, and by this process, an artificial high quality image is predicted from the HD image, and the artificial high quality image is output to the synthesis unit 133.

  In step S6, the natural-image / artificial-image determination unit 114 performs a natural-image / artificial-image determination process. The details of the natural image prediction processing will be described later with reference to FIG. 35, but the natural image artificial image determination unit 114 is a region classified as an artificial image for each pixel of the HD image supplied from the output phase conversion unit 112. Alternatively, it is determined which of the areas classified as natural images belongs to, and the determination result is output to the synthesizing unit 133 as an artificial image degree.

  In step S7, the synthesizer 133 synthesizes the images. Specifically, the synthesis unit 133 determines the pixel value of each pixel of the natural high-quality image supplied from the natural image prediction unit 131 based on the determination result supplied from the natural image / artificial image determination unit 114, and the artificial image. The pixel values of the respective pixels of the artificial high quality image supplied from the prediction unit 132 are combined at a ratio corresponding to the artificial image quality. The combining unit 133 outputs the combined image to a subsequent apparatus.

  In addition, when performing image conversion of a plurality of images continuously, the processes in steps S1 to S7 described above are repeatedly executed.

  FIG. 3 is a block diagram illustrating a configuration example of the natural image prediction unit 131 in FIG.

  3 includes a class tap extraction unit 551, an ADRC (Adaptive Dynamic Range Coding) processing unit 552, a coefficient seed memory 553, a prediction coefficient generation unit 554, a prediction coefficient memory 555, a prediction tap extraction unit 556, and A high-quality natural high-quality image that is composed of a prediction calculation unit 557 and removes noise from the natural image of the HD image from the progressive HD image supplied from the output phase conversion unit 112 in FIG. Predict images.

  The progressive HD image supplied from the output phase conversion unit 112 in FIG. 1 is input to the natural image prediction unit 131, and the HD image is supplied to the class tap extraction unit 551 and the prediction tap extraction unit 556.

  The class tap extraction unit 551 sequentially sets pixels constituting the natural high-quality image obtained from the input HD image as the target pixel, and uses the input HD image used to classify the target pixel into a class. Some of the constituent pixels are extracted as class taps. The class tap extraction unit 551 supplies the extracted class tap to the ADRC processing unit 552.

  The ADRC processing unit 552 performs the ADRC processing on the pixel value of the pixel constituting the class tap supplied from the class tap extraction unit 551 as a characteristic of the class tap waveform, and the result is obtained. Detects ADRC code as a class tap feature.

In the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the pixels constituting the class tap are detected, and DR = MAX-MIN is set as the local dynamic range of the set, and this dynamic range Based on DR, the pixel values constituting the class tap are requantized to K bits. That is, the pixel value of each pixel forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided by DR / 2 ^K.

  A bit string obtained by arranging the pixel values of the K-bit pixels constituting the class tap in a predetermined order is output as an ADRC code. Therefore, for example, when a class tap is subjected to 1-bit ADRC processing, the pixel value of each pixel constituting the class tap is divided by the average value of the maximum value MAX and the minimum value MIN, and the result of the division is The fractional part is rounded down, so that the pixel value of each pixel is 1 bit. That is, the pixel value of each pixel is binarized. Then, a bit string in which the 1-bit pixel values are arranged in a predetermined order is output as an ADRC code.

  The ADRC processing unit 552 determines a class based on the detected ADRC code, classifies the target pixel into a class, and supplies the class to the prediction coefficient memory 555. For example, the ADRC processing unit 552 supplies the ADRC code as it is to the prediction coefficient memory 555 as a class.

  The coefficient seed memory 553 stores coefficient seeds for each class acquired by learning described later with reference to FIGS. 7 to 9. The prediction coefficient generation unit 554 reads the coefficient seed from the coefficient seed memory 553. The prediction coefficient generation unit 554, based on the parameter h for determining the horizontal resolution input by the user and the parameter v for determining the vertical resolution, calculates a polynomial including the parameters h and v. The prediction coefficient is generated from the read coefficient type and supplied to the prediction coefficient memory 555 for storage.

  The prediction coefficient memory 555 reads out the prediction coefficient of the class according to the class supplied from the ADRC processing unit 552 and supplies it to the prediction calculation unit 557.

  Note that the prediction tap and the class tap may have the same tap structure or different tap structures.

  The prediction tap extraction unit 556 extracts pixels constituting the HD image used for predicting the pixel value of the target pixel from the input HD image as a prediction tap. The prediction tap extraction unit 556 supplies the prediction tap to the prediction calculation unit 557.

  The prediction calculation unit 557 uses a prediction tap supplied from the prediction tap extraction unit 556 and a prediction coefficient supplied from the prediction coefficient memory 555 to calculate a linear linear expression that obtains a predicted value of the true value of the target pixel. The prediction calculation is performed. Thereby, the prediction calculation unit 557 obtains the predicted value of the pixel value of the target pixel, that is, the pixel value of the pixels constituting the natural high-quality image, and outputs the pixel value to the synthesis unit 133.

  FIG. 4 shows an example of a tap structure of class taps extracted by the class tap extraction unit 551 of FIG.

  In the figure, white circles represent pixels constituting a class tap among HD image pixels supplied from the output phase conversion unit 112, dotted circles represent pixels not constituting a class tap, and black circles represent a target pixel. Represents. The same applies to FIG. 5 described later.

  In FIG. 4, a class tap is composed of nine pixels. That is, five pixels p60, p61, p64, p67, p68 arranged every other pixel in the vertical direction and pixels p64 arranged every other pixel in the horizontal direction around the pixel p64 constituting the HD image corresponding to the target pixel q6. The excluded four pixels p62, p63, p65, and p66 constitute a so-called cross-shaped class tap.

  FIG. 5 shows an example of the tap structure of the prediction tap extracted by the prediction tap extraction unit 556 of FIG.

  In FIG. 5, the prediction tap is comprised by 13 pixels. That is, in FIG. 5, among the HD images supplied from the output phase conversion unit 112, five pixels p80, p82, p86, p90, which are arranged every other pixel in the vertical direction around the pixel p86 corresponding to the target pixel q8. p92, 4 pixels p84, p85, p87, p88 excluding the pixels p86 arranged every other pixel in the horizontal direction, and 2 pixels p81, p89 excluding the pixels p85 arranged every other pixel in the vertical direction around the pixel p85, A substantially rhombic prediction tap is constituted by two pixels p83 and p91 excluding the pixels p87 arranged every other pixel in the vertical direction around the pixel p87.

  4 and 5, the nine pixels p60 to p68 constituting the class tap and the 13 pixels p80 to p92 constituting the prediction tap are arranged every other pixel in the vertical direction (vertical direction) or the horizontal direction (horizontal direction). However, the interval is not limited to this, and the interval depends on the ratio of the number of pixels of the SD image before conversion and the HD image after conversion in the output phase converter 112, that is, the interpolation magnification. To change.

  For example, when the output phase converter 112 converts the image so that the number of pixels in the horizontal direction and the vertical direction is doubled, as shown in FIGS. When a class tap and a prediction tap are configured from pixels arranged at intervals of two pixels, a class tap and a prediction tap are configured by only one of the interpolated pixel and the pixel existing before the interpolation. For example, the accuracy of the result of the prediction process by the natural image prediction unit 131 is improved as compared with a case where the class tap and the prediction tap are composed of pixels arranged at intervals of one pixel.

  Next, the details of the natural image prediction process in step S4 of FIG. 2 performed by the natural image prediction unit 131 of FIG. 3 will be described with reference to FIG.

  First, in step S551, the class tap extraction unit 551 selects one of the pixels constituting the natural high quality image obtained from the HD image input from the output phase conversion unit 112 of FIG. 1 as the target pixel. To do.

  In step S552, the class tap extraction unit 551 selects some of the pixels constituting the input HD image used for classifying the pixel of interest selected in step S551 as shown in FIG. A class tap is extracted from the HD image and supplied to the ADRC processing unit 552.

  In step S553, the ADRC processing unit 552 performs ADRC processing on the pixel values of the pixels constituting the class tap supplied from the class tap extraction unit 551, and uses the resulting ADRC code as the class tap feature. To detect.

  In step S554, the ADRC processing unit 552 determines a class based on the ADRC code, thereby classifying the pixel of interest into a class, and supplies the class to the prediction coefficient memory 555.

  In step S555, the predicted coefficient generation unit 554 reads the coefficient seed from the coefficient seed memory 553.

  In step S556, the prediction coefficient generation unit 554 calculates a prediction coefficient from the coefficient seed read from the coefficient seed memory 553 using a polynomial including the parameters h and v based on the parameters h and v input by the user. Generated and supplied to the prediction coefficient memory 555 for storage. Details of processing for generating a prediction coefficient from this coefficient type will be described later.

  In step S557, the prediction coefficient memory 555 reads the prediction coefficient of the class based on the class supplied from the ADRC processing unit 552, and supplies the prediction coefficient to the prediction calculation unit 557.

  In step S558, the prediction tap extraction unit 556 extracts, from the input HD image, the pixels constituting the HD image used to predict the pixel value of the target pixel as shown in FIG. 5 as the prediction tap. . The prediction tap extraction unit 556 supplies the prediction tap to the prediction calculation unit 557.

  In step S559, the prediction calculation unit 557 uses the prediction tap supplied from the prediction tap extraction unit 556 and the prediction coefficient supplied from the prediction coefficient memory 555 to obtain a linear primary value for obtaining the true value prediction value of the target pixel. Perform prediction calculations such as formulas.

  In step S560, the prediction calculation unit 557 outputs the predicted value of the pixel value of the target pixel obtained as a result of the prediction calculation, that is, the pixel value of the pixels constituting the natural high-quality image to the synthesis unit 133.

  In step S561, the class tap extraction unit 551 determines whether or not all the pixels constituting the natural high quality image obtained from the input HD image are the target pixels. If it is determined in step S561 that all the pixels have not yet been set as the target pixel, in step S562, the class tap extraction unit 551 newly selects a pixel constituting the natural high-quality image that has not been set as the target pixel. Is determined as the pixel of interest, the process returns to step S552, and the same process is repeated thereafter. On the other hand, if the class tap extraction unit 551 determines in step S561 that all the pixels are the target pixel, the natural image prediction process ends.

  As described above, the natural image prediction unit 131 predicts and outputs a natural high-quality image from the HD image supplied from the output phase conversion unit 112. That is, the natural image prediction unit 131 converts the HD image into a natural high quality image and outputs it.

  As described above, in the image conversion apparatus 101 in FIG. 1, the output phase conversion unit 112 converts the SD image supplied from the cyclic conversion unit 122 into an HD image output from the natural image prediction unit 131, Since the image is supplied to the image prediction unit 131, the number of pixels of the image before and after the prediction is the same, and the position of the pixel of the image before and after the prediction is not shifted.

  Therefore, the natural image predicting unit 131 can predict the pixel value of the target pixel using the prediction tap including the HD image pixel having the same phase as the phase of the target pixel which is a pixel of the natural high quality image. As a result, the natural image prediction unit 131 can accurately predict a natural high quality image and perform high-accuracy image conversion. That is, the output phase conversion unit 112 and the natural image prediction unit 131 accurately convert an image that is an SD image input from the cyclic conversion unit 122 into a natural high quality image that is a high quality HD image having a different number of pixels. can do.

  Further, since the natural image prediction unit 131 classifies the pixel value of the pixel constituting the class tap as a class tap waveform characteristic, the target pixel is classified into the class, and thus classifies the feature of the natural image with relatively few flat portions. Classification suitable for As a result, the natural image prediction unit 131 can accurately improve the quality of the natural image of the HD images.

  Next, prediction calculation in the prediction calculation unit 557 in FIG. 3 and learning of a prediction coefficient used for the prediction calculation will be described.

  Now, a prediction tap is extracted from the input HD image, and a pixel value of a pixel (hereinafter, appropriately referred to as a natural high quality image pixel) constituting a natural high quality image is predicted using the prediction tap and the prediction coefficient. For example, if a linear primary prediction calculation is employed as the predetermined prediction calculation, the pixel value y of the natural high-quality image pixel is obtained by the following linear primary expression.

However, in Expression (1), x _n is a pixel value of an nth HD image pixel (hereinafter, referred to as an HD image pixel as appropriate) that constitutes a prediction tap for a natural high quality image pixel having a pixel value y. Wn represents the _nth prediction coefficient multiplied by the pixel value of the nth HD image pixel. In equation (1), the prediction tap is assumed to be composed of N HD image pixels x ₁ , x ₂ ,..., X _N.

  The pixel value y of the natural high-quality image pixel can be obtained not by the linear primary expression shown in Expression (1) but by a higher-order expression of the second or higher order.

Now, when the true value of the pixel value of the natural high quality image pixel of the k-th sample is expressed as y _k and the predicted value of the true value y _k obtained by the equation (1) is expressed as y _k ′, the prediction error e _k is expressed by the following equation.

Since the predicted value y _k ′ of the formula (2) is obtained according to the formula (1), the following formula is obtained by replacing the predicted value y _k ′ of the formula (2) according to the formula (1).

In Equation (3), x _{n, k} represents the nth HD image pixel constituting the prediction tap for the natural high quality image pixel of the kth sample.

0 prediction error e _k of the formula (3) or formula (2), i.e. the prediction coefficient W _n statistically minimizes found is the optimal to predict the high-quality natural image pixel, all natural It is generally difficult to obtain such a prediction coefficient W _n for high quality image pixels.

Therefore, if, for example, the least square method is adopted as a standard indicating that the prediction coefficient W _n is optimum, the optimum prediction coefficient W _n is obtained by calculating the sum E of square errors expressed by the following equation. It can be obtained by minimizing.

However, in the formula (4), K is, HD image pixels x ₁ constituting the true value y _k of the pixel values of the high-quality image, the prediction taps for the true value y _{k, k,} x _{2, k,} · .., X _{N, k} represents the number of samples, that is, the number of learning samples.

The minimum value of the sum E of square errors in equation (4) is given by W _n , which is 0 as a result of partial differentiation of the sum E by the prediction coefficient W _n as shown in equation (5).

Therefore, when the above equation (3) is partially differentiated by the prediction coefficient W _n , the following equation is obtained.

  From the equations (5) and (6), the following equation is obtained.

By substituting equation (3) into e _k in equation (7), equation (7) can be expressed by the normal equation shown in equation (8).

The normal equation of Expression (8) can be solved for the prediction coefficient W _n by using, for example, a sweeping method (Gauss-Jordan elimination method) or the like.

By solving the normal equation of equation (8) for each class, a prediction coefficient W _n that minimizes the sum E of square errors can be obtained for each class as an optimal prediction coefficient.

  Also, a polynomial for generating a prediction coefficient in the prediction coefficient generation unit 554 in FIG. 3 and learning of a coefficient type used for the polynomial will be described.

As an expression for generating a prediction coefficient using the input parameters h and v and the coefficient type, for example, if a polynomial is adopted, the prediction coefficient W _n for each class and each combination of the parameters h and v Is obtained by the following polynomial.

However, in Equation (9), w _{n, k} (k = 1, 2,..., 9) constitutes a prediction tap for the natural high quality image pixel having the pixel value y represented by Equation (14). , The coefficient of the k-th term among the coefficient seeds for generating the n-th prediction coefficient W _n multiplied by the pixel value x _n of the pixel of the n-th HD image.

Here, when the true value of the nth prediction coefficient corresponding to the parameters h and v is expressed as W _vhn and the estimated value of the true value W _vhn obtained by the equation (9) is expressed as W _vhn ′, The estimation error e _vhn is expressed by the following equation.

Since the estimated value W _vhn ′ of the equation (10) is obtained according to the equation (9), the following equation is obtained by replacing the estimated value W _vhn ′ of the equation (10) according to the equation (9).

In Equation (11), w _{vhn, k} represents the coefficient of the k-th term among the coefficient types for generating the prediction coefficient W _vhn . In the equation (11), t _k is defined by the following equation.

The prediction error e _vhn in equation (10) or equation (11) is 0, that is, the coefficient type w _{vhn, k} that statistically minimizes is optimal for estimating the prediction coefficient. In general, it is difficult to obtain such a coefficient seed w _{vhn, k} .

Therefore, if the least square method is adopted as a standard indicating that the coefficient type w _{vhn, k} is optimal, for example, the optimal coefficient type w _{vhn, k} is the square error represented by the following equation. Can be obtained by minimizing the sum E of

  However, in Formula (13), V represents the number of types of parameter v, and H represents the number of types of parameter h.

The minimum value of the sum E of square errors in equation (13) is given by w _{vhn, k, where} 0 is a partial differentiation of the sum E with a coefficient seed w _{vhn, k} as shown in equation (14).

Here, if X _kl and Y _k are defined by the following equations (15) and (16), respectively, equation (14) can be rewritten as a normal equation of the following equation (17).

The normal equation of the equation (17) can be solved for the coefficient seed wn _{, k} by using, for example, a sweeping method (Gauss-Jordan elimination method) or the like.

By solving the normal equation of Equation (17) for each class, the coefficient type wn _{, k} that minimizes the sum E of square errors can be obtained for each class as the optimum coefficient type.

FIG. 7 is a block diagram illustrating a configuration example of a learning device 601 that performs learning to obtain the coefficient seed w _{n, k} for each class by building and solving the normal equation of Expression (17).

  7 includes a band limiting filter 611, a class tap extraction unit 612, an ADRC processing unit 613, a prediction tap extraction unit 614, a normal equation generation unit 615, a prediction coefficient generation unit 616, a normal equation generation unit 617, and coefficient types. The decision unit 618 and the coefficient seed memory 619 are configured.

  The learning device 601 receives a plurality of teacher images that are read from a database (not shown) and become target natural images after prediction processing, and the teacher images are supplied to the band limiting filter 611 and the normal equation generation unit 615. Is done. In addition, parameters h and v are input to the learning device 601 from the outside based on a user instruction, and are supplied to the band limiting filter 611 and the normal equation generation unit 615. Note that every time a single teacher image is input to the learning device 601, all combinations of parameters h and v are input to the learning device 601.

  The band limiting filter 611 performs filter processing for limiting the vertical and horizontal bands of the teacher image acquired from a database (not shown) according to parameters h and v input from the outside. Thereby, a student image corresponding to the natural image before the prediction process is performed for each combination of the parameters h and v is generated. For example, when there are nine types of parameters h and nine types of parameters v, the band limiting filter 611 uses 81 types from one teacher image according to parameters h and v input from the outside. Generate student images.

  The band limiting filter 611 supplies the student image to the class tap extraction unit 612 and the prediction tap extraction unit 614.

  The class tap extraction unit 612 is configured in the same manner as the class tap extraction unit 551 in FIG. The class tap extraction unit 612 sequentially selects the pixels constituting the teacher image as the target teacher pixel, and the class tap extraction unit 551 of FIG. 3 extracts the target teacher pixel from the student image for the target teacher pixel as shown in FIG. The class tap having the same tap structure as the class tap to be extracted is extracted and supplied to the ADRC processing unit 613.

  The ADRC processing unit 613 performs ADRC processing or the like on the pixel values of the pixels constituting the class tap supplied from the class tap extraction unit 551, and detects the resulting ADRC code as a class tap feature. The ADRC processing unit 613 determines a class based on the ADRC code, and supplies the class to the normal equation generation unit 615.

  The prediction tap extraction unit 614 is configured similarly to the prediction tap extraction unit 556 of FIG. The prediction tap extraction unit 614 uses, as prediction taps, the pixels constituting the student image used to predict the pixel value of the teacher pixel of interest as shown in FIG. 5 from the student image supplied from the band limiting filter 611. Extract. The prediction tap extraction unit 556 supplies the prediction tap to the normal equation generation unit 615.

The normal equation generation unit 615 is supplied from the ADRC processing unit 613 using the input teacher image and the prediction tap pair supplied from the prediction tap extraction unit 614 as a learning pair used for learning the prediction coefficient W _n. When the normal equation of the above equation (8) is established for each class and for each combination of parameters h and v input from the outside, the normal equation is supplied to the prediction coefficient generation unit 616.

The prediction coefficient generation unit 616 solves the normal equation for each class supplied from the normal equation generation unit 615, thereby calculating the prediction coefficient W _n that statistically minimizes the prediction error for each combination of the parameters h and v, and It calculates | requires for every class and supplies it to the normal equation production | generation part 617. FIG.

Based on the prediction coefficient W _vhn from the prediction coefficient generation unit 616, the normal equation generation unit 617 generates the above-described normal equation of Expression (17) for each class and outputs it to the coefficient type determination unit 618. The coefficient seed generation unit 618 solves the normal equation of the equation (17) for each class, obtains the coefficient seed w _{n, k} for each class, and stores it in the coefficient seed memory 619. The coefficient seed stored in the coefficient seed memory 619 is stored in the coefficient seed memory 553 of FIG.

  Next, with reference to FIG. 8, the positional relationship between the pixels of the teacher image and the student image will be described.

  In FIG. 8, rhombuses represent pixels of the teacher image, and white circles represent pixels of the student image. In the figure, the horizontal axis represents the horizontal position, and the vertical axis represents the vertical position.

  As shown in FIG. 8, the horizontal and vertical positions of the pixels of the teacher image and the student image are the same. That is, the teacher image and the student image are in phase.

  Next, the learning process of the learning device 601 in FIG. 7 will be described with reference to the flowchart in FIG.

  First, in step S601, the band limiting filter 611 is input by performing filter processing for limiting the vertical and horizontal bands of the input teacher image according to the input parameters h and v, respectively. A student image is generated from the teacher image and supplied to the class tap extraction unit 612 and the prediction tap extraction unit 614.

  In step S602, the class tap extraction unit 612 selects one of the pixels constituting the teacher image as a notable teacher pixel, similarly to the class tap extraction unit 551 in FIG.

  In step S <b> 603, the class tap extraction unit 612 extracts a class tap as illustrated in FIG. 4 from the student image, and supplies it to the ADRC processing unit 613 in the same manner as the class tap extraction unit 551.

  In step S604, the ADRC processing unit 613 performs ADRC processing on the pixel values of the pixels constituting the class tap. In step S605, the ADRC processing unit 613 determines a class based on the ADRC code obtained as a result of the ADRC process, and supplies the class to the normal equation generation unit 615.

  In step S606, the prediction tap extraction unit 614 applies a prediction tap as shown in FIG. 5 from the student image supplied from the band limiting filter 611 for the teacher pixel of interest, as in the prediction tap extraction unit 556 of FIG. Extracted and supplied to the normal equation generation unit 615.

  In step S <b> 607, the normal equation generation unit 615 extracts a target teacher pixel from the input teacher image, and configures the prediction tap configured for the target teacher pixel and the target teacher pixel supplied from the prediction tap extraction unit 614. The addition of Expression (8) for the student image to be performed is performed for each combination of the parameters h and v and for each class supplied from the ADRC processing unit 613.

  In step S608, the class tap extraction unit 612 determines whether the pixels of all input teacher images are the target teacher pixels. If it is determined in step S608 that all the teacher image pixels are not yet the target teacher pixels, in step S609, the class tap extraction unit 612 determines that among the teacher image pixels that are not yet the target teacher pixels, It is newly determined as an attention teacher pixel. Then, the process returns to step S603, and the same process is repeated thereafter.

  On the other hand, when it is determined in step S608 that all the teacher image pixels are the target teacher pixels, in step S610, the normal equation generation unit 615 determines whether each of the combinations of the parameters h and v obtained by the processing so far is performed. In addition, the matrix on the left side and the vector on the right side in Equation (8) for each class are supplied to the prediction coefficient generation unit 616 as normal equations.

In step S <b> 611, the prediction coefficient generation unit 616 supplies a class composed of a matrix on the left side and a vector on the right side in Expression (8) for each combination of parameters h and v and supplied from the normal equation generation unit 615. Each normal equation is solved, and a prediction coefficient W _vhn for each combination of parameters h and v and for each class is obtained and output to the normal equation generation unit 617.

In step S612, the normal equation generation unit 617 generates the normal equation of Expression (17) for each class based on the prediction coefficient W _vhn and _outputs the normal equation to the coefficient type determination unit 618.

In step S613, the coefficient seed determination unit 618 solves the normal equation of Expression (17) to obtain the coefficient seed wn _{, k} for each class. In step S614, the coefficient seed memory 619 stores the coefficient seed wn _{, k} . This coefficient seed is stored in the coefficient seed memory 553 of FIG.

As described above, the natural image prediction unit 131 predicts a natural high-quality image using the prediction coefficient W _n generated according to the coefficient type obtained by learning using the natural image. The quality of the natural image of the HD images supplied from the unit 112 can be improved accurately.

  In addition, the natural image prediction unit 131 classifies the pixel value of the pixels constituting the class tap as a class tap waveform feature according to the feature, and classifies the pixel of interest into a class. Can be accurately classified. As a result, the natural image prediction unit 131 predicts a natural high-quality image from the HD image using the prediction coefficient generated by the coefficient type acquired by learning for each class, and thereby includes noise included in the natural image. Etc. can be removed more accurately, and a higher quality natural high quality image can be output.

  FIG. 10 shows a detailed configuration example of the first embodiment of the artificial image prediction unit 132 of FIG.

  10 includes a class classification unit 651, a coefficient seed memory 652, a prediction coefficient generation unit 653, a prediction coefficient memory 654, and a prediction unit 655, and a progressive method supplied from the output phase conversion unit 112. From the HD image, noise and the like are removed from the artificial image of the HD image to predict a high-quality artificial high-quality image.

  The HD image supplied from the output phase conversion unit 112 is input to the class classification unit 651 and the prediction unit 655. The class classification unit 651 sequentially sets a pixel constituting the artificial high quality image obtained from the HD image as a target pixel, and sets the target pixel as one of several classes according to the phase characteristics of the HD image. Classify into one of the classes. The class classification unit 651 supplies the classified class to the prediction coefficient memory 654.

  The coefficient seed memory 652 is composed of, for example, a ROM (Read Only Memory) or the like, and stores a coefficient seed for each class acquired by learning described later with reference to FIGS.

The prediction coefficient generation unit 653 generates a prediction coefficient W _n from the coefficient seed w _{n, k} read from the coefficient seed memory 652 by using the polynomial of Expression (9) including parameters h and v input by the user. And stored in the prediction coefficient memory 654.

The prediction coefficient memory 654, based on the class supplied from the classification unit 651 reads from the prediction coefficient W _n of the class, and supplies the prediction unit 655.

The prediction unit 655 uses the HD image and the prediction coefficient W _n supplied from the prediction coefficient memory 654 to perform a predetermined prediction calculation for obtaining a true predicted value of the target pixel. Thereby, the prediction unit 655 obtains the predicted value of the pixel value of the target pixel, that is, the pixel value of the pixels constituting the artificial high quality image, and outputs the pixel value to the synthesis unit 133 in FIG.

  FIG. 11 is a block diagram illustrating a detailed configuration example of the class classification unit 651 in FIG.

  The class classification unit 651 in FIG. 11 includes a class tap extraction unit 671, a difference calculation unit 672, and an ADRC processing unit 673.

  The class tap extraction unit 671 extracts some of the pixels constituting the HD image used for classifying the pixel of interest into a class as class taps, and supplies them to the difference calculation unit 672.

  The difference calculation unit 672 includes an absolute value (hereinafter referred to as a difference between pixel values) of two adjacent pixels (hereinafter referred to as adjacent pixels) among the pixels constituting the class tap supplied from the class tap extraction unit 671. Is calculated for each adjacent pixel as a feature of the phase of the class tap. The difference calculation unit 672 supplies the adjacent difference absolute value of each adjacent pixel to the ADRC processing unit 673.

  The ADRC processing unit 673 performs 1-bit ADRC processing on the adjacent difference absolute value supplied from the difference calculation unit 672. Specifically, the ADRC processing unit 673 divides the adjacent difference absolute value of the class tap by the average value of the maximum value MAX and the minimum value MIN, and rounds off the decimal part of the division result to thereby calculate the adjacent difference absolute value. Set the value to 1 bit. That is, the ADRC processing unit 673 binarizes the adjacent difference absolute value.

  Then, the ADRC processing unit 673 determines a bit string in which the 1-bit pixel values are arranged in a predetermined order as the class of the target pixel. Therefore, the class is phase information representing the position of the edge in the class tap. That is, the class has a value obtained by degenerating the phase of the class tap. The ADRC processing unit 673 supplies the determined class to the prediction coefficient memory 654 in FIG.

  As described above, the class classification unit 651 classifies the pixel of interest into a class according to the feature of the phase, using the adjacent difference absolute value of each neighboring pixel as the feature of the phase of the class tap.

  FIG. 12 shows an example of a tap structure of class taps extracted by the class tap extraction unit 671 of FIG. Note that the tap structure of the class tap may be a structure other than that shown in FIG.

  In FIG. 12, the pixel p124 corresponding to the target pixel in the HD image supplied from the output phase conversion unit 112 in FIG. 1 is adjacent to the pixel 2 in the upward, leftward, rightward, and downward directions, respectively. The so-called cross-shaped class taps are configured from the pixels p120, p121, p122, p123, p125, p126, p127, and p128.

  The difference calculation unit 672 in FIG. 11 includes pixels p120 and p121, p121 and p124, p122 and p123, p123 and p124, p124 and p125, which are adjacent pixels among the nine pixels p120 to p128 constituting the class tap. Eight adjacent difference absolute values d0 to d7 of p125 and p126, p124 and p127, and p127 and p128 are calculated and supplied to the ADRC processing unit 673. As a result, a class represented by 8 bits is output from the ADRC processing unit 673.

  FIG. 13 is a block diagram illustrating a detailed configuration example of the prediction unit 655 of FIG.

  The prediction unit 655 of FIG. 13 includes a prediction tap extraction unit 691 and a prediction calculation unit 692.

  The prediction tap extraction unit 691 extracts pixels constituting an HD image used for predicting the pixel value of the target pixel as a prediction tap.

  Specifically, the prediction tap extraction unit 691 is a spatially close position to a pixel of the HD image corresponding to the target pixel, for example, a pixel of the HD image that is spatially closest to the target pixel. Are extracted as prediction taps from the HD image. The prediction tap extraction unit 691 supplies the prediction tap to the prediction calculation unit 692.

  Note that the prediction tap and the class tap may have the same tap structure or different tap structures.

  In addition to the prediction tap supplied from the prediction tap extraction unit 691, the prediction calculation unit 692 is supplied with a prediction coefficient from the prediction coefficient memory 654 of FIG. The prediction calculation unit 692 uses the prediction tap and the prediction coefficient to perform the prediction calculation shown in Expression (1) to obtain the true predicted value of the target pixel. Thereby, the prediction calculation unit 692 obtains the predicted value of the pixel value of the target pixel, that is, the pixel value of the pixels constituting the artificial high quality image, and outputs the pixel value to the synthesis unit 133 in FIG.

  FIG. 14 shows an example of the tap structure of the prediction tap extracted by the prediction tap extraction unit 691 of FIG. Note that the tap structure of the prediction tap may be a structure other than that shown in FIG.

  In FIG. 14, the prediction tap is comprised by 13 pixels. That is, in FIG. 14, among the HD images supplied from the output phase conversion unit 112, five pixels p140, p142, p146, p150, and p152 arranged in the vertical direction centering on the pixel p146 corresponding to the pixel of interest, Three pixels p141, p145, p149, p143, p147, and p151 that are arranged in the vertical direction around the pixels p145 and p147 adjacent to the left and right of the corresponding pixel p146, and the pixel p146 that corresponds to the target pixel are on the left and right. A substantially rhombic prediction tap is formed from the pixels p144 and p148 separated by two pixels.

  Next, with reference to FIG. 15, the details of the artificial image prediction process in step S5 of FIG. 2 performed by the artificial image prediction unit 132 of FIG.

  First, in step S701, the class classification unit 651 classifies a target pixel, which is a predetermined pixel among pixels constituting the obtained artificial high quality image, according to the phase characteristics of the HD image corresponding to the target pixel. Class classification processing is performed to classify into Details of this class classification processing will be described later with reference to FIG.

In step S702, the coefficient seed memory 652 reads the coefficient seed wn _{, k} and outputs it to the prediction coefficient generation unit 653. In step S703, the prediction coefficient generation unit 653 uses the polynomial of the equation (9) including the parameters h and v based on the parameters h and v input by the user to calculate the prediction coefficient from the coefficient type wn _{, k.} W _n is generated and supplied to the prediction coefficient memory 555 for storage.

In step S _< b _> 704, the prediction coefficient memory 654 reads the prediction coefficient W _{n of the} class based on the class classified by the class classification unit 651, and supplies it to the prediction calculation unit 692 of the prediction unit 655.

  In step S705, the prediction tap extraction unit 691 selects pixels constituting the HD image used to predict the pixel value of the target pixel as shown in FIG. 14 from the HD image supplied from the output phase conversion unit 112. , Extracted as a prediction tap, and supplied to the prediction calculation unit 692.

In step S706, the prediction computation unit 692, the prediction taps supplied from the prediction tap extracting unit 691, by using the prediction coefficients W _n supplied from the prediction coefficient memory 654, a prediction computation shown in equation (1) By doing so, the pixel values of the pixels constituting the artificial high quality image are obtained. In step S707, the prediction calculation unit 692 outputs the pixel values of the pixels constituting the artificial high quality image obtained in step S706 to the synthesis unit 133 in FIG.

  In step S708, the class classification unit 651 determines whether or not all the pixels constituting the artificial high quality image are the target pixels, and it is determined that all the pixels constituting the artificial high quality image are not yet the target pixels. In such a case, in step S709, the class classification unit 651 newly determines a pixel of interest that is not yet a pixel of interest from among the pixels constituting the artificial high quality image, returns the processing to step S701, and so on. Repeat the process.

  On the other hand, in step S708, when the class classification unit 651 determines that all the pixels constituting the artificial high quality image are the target pixels, the artificial image prediction process ends.

  As described above, the artificial image prediction unit 132 predicts and outputs an artificial high quality image from the HD image supplied from the output phase conversion unit 112. That is, the artificial image prediction unit 132 converts the HD image into an artificial high quality image and outputs it.

  Next, the class classification processing in step S701 in FIG. 15 will be described with reference to the flowchart in FIG.

  In step S721, the class tap extraction unit 671 (FIG. 11) of the class classification unit 651 classifies some of the pixels constituting the HD image used for classifying the target pixel as shown in FIG. This is extracted as a tap and supplied to the difference calculation unit 672.

  In step S722, the difference calculation unit 672 calculates the adjacent difference absolute value of each adjacent pixel among the pixels constituting the class tap supplied from the class tap extraction unit 671, and calculates the adjacent difference absolute value of each adjacent pixel. This is supplied to the ADRC processing unit 673.

  In step S723, the ADRC processing unit 673 performs 1-bit ADRC processing on the adjacent difference absolute value supplied from the difference calculation unit 672. The ADRC processing unit 673 classifies the target pixel into a class by determining the bit string obtained as a result as a class. Then, the ADRC processing unit 673 supplies the class to the prediction coefficient memory 654 in FIG. Thereafter, the processing returns to step S701 in FIG.

  FIG. 17 is a block diagram illustrating a configuration example of the learning device 811 that performs learning for obtaining the coefficient seed stored in the coefficient seed memory 652 of FIG.

  17 includes a student image generation unit 821, a class classification unit 822, a generation unit 823, a coefficient generation unit 824, a normal equation generation unit 825, a coefficient type determination unit 826, and a coefficient type memory 827.

This learning device 811 learns the coefficient seed in the same manner as the learning of the coefficient seed wn _{, k} stored in the coefficient seed memory 553 (FIG. 3) of the natural picture prediction unit 131 described above. That is, the learning device 811 employs an HD image corresponding to the target artificial image after the prediction process as the teacher image, and an HD image corresponding to the artificial image before the prediction process is performed as the student image. For each combination of parameters h and v, for each combination of parameters h and v, and for each class, by solving the normal equation of equation (8) described above for each class and for each class based on a user instruction A prediction coefficient W _vhn that is a prediction coefficient W _n for each is _obtained .

Then, by solving the normal equation of the above-described equation (17) generated for each class by the prediction coefficient W _{vhn, the} coefficient type _{wn , k} for each class is generated and stored.

  A plurality of teacher images read from a database (not shown) are input to the learning device 811, and the teacher images are supplied to the student image generation unit 821 and the generation unit 823. In addition, the parameters h and v are input to the learning device 811 and supplied to the student image generation unit 821 and the generation unit 823.

  The student image generation unit 821 is configured by, for example, a low-pass filter or the like, and degrades the quality of the teacher image, which is an artificial image acquired from a database (not shown), according to the parameters h and v. A student image for each combination is generated. The student image generation unit 821 supplies the student image to the class classification unit 822 and the generation unit 823.

  The class classification unit 822 is configured in the same manner as the class classification unit 651 in FIG. The class classification unit 822 sequentially uses the pixels constituting the teacher image as the target teacher pixel, and the same teacher tap pixel as the class tap (FIG. 12) extracted by the class tap extraction unit 671 in FIG. Class taps of the tap structure are extracted.

  Then, the class classification unit 822 calculates the adjacent difference absolute value of each adjacent pixel among the pixels constituting the class tap, and performs 1-bit ADRC processing on the adjacent difference absolute value. The class classification unit 822 determines the bit string obtained as a result as the class of the teacher pixel of interest, and supplies it to the generation unit 823.

  The generation unit 823 uses a pair of the teacher image and the student image supplied from the student image generation unit 821 as a learning pair used for learning of the prediction coefficient, for each combination of parameters h and v input from the outside, and a class. For each class supplied from the classification unit 822, when the normal equation shown in Expression (8) is established, the normal equation is supplied to the coefficient generation unit 824.

The coefficient generation unit 824 calculates the prediction coefficient W _vhn for each combination of parameters h and v and for each class by solving a normal equation for each combination of parameters h and v and for each class supplied from the generation unit 823. And output to the normal equation generation unit 825.

The normal equation generation unit 825 generates a normal equation of Expression (17) for each class based on the prediction coefficient W _vhn and outputs it to the coefficient type determination unit 826. The coefficient seed determination unit 826 calculates the coefficient seed wn _{, k} by solving the normal equation and stores it in the coefficient seed memory 827. The coefficient seed w _{n, k} stored in the coefficient seed memory 827 is stored in the coefficient seed memory 652 of FIG.

  FIG. 18 is a block diagram illustrating a detailed configuration example of the generation unit 823 in FIG.

  The generation unit 823 in FIG. 18 includes a prediction tap extraction unit 831 and a normal equation generation unit 832.

  The learning pair student image supplied to the generation unit 823 is supplied to the prediction tap extraction unit 831, and the teacher image is supplied to the normal equation generation unit 832.

  The prediction tap extraction unit 831 sequentially sets the pixels constituting the learning pair teacher image as the attention teacher pixel, and the prediction tap extraction unit 691 in FIG. 13 extracts the attention teacher pixel from the learning pair student image. A prediction tap having the same tap structure as that of the tap (FIG. 14) is extracted and supplied to the normal equation generation unit 832.

  The normal equation generation unit 832 extracts the target teacher pixel from the teacher image, and targets the target teacher pixel and the student image constituting the prediction tap configured for the target teacher pixel supplied from the prediction tap extraction unit 831. Is added for each combination of parameters h and v input from the outside and for each class supplied from the class classification unit 822.

  Then, the normal equation generation unit 832 performs the above-mentioned addition using the pixels of all the teacher images input to the learning device 811 as the target teacher pixels, so that the normal equations shown in Equation (8) are obtained for each class. , The normal equation is supplied to the coefficient generation unit 824 in FIG.

  Next, the learning process of the learning device 811 in FIG. 17 will be described with reference to the flowchart in FIG.

  First, in step S741, the student image generation unit 821 generates a student image from the input teacher image according to the parameters h and v input from the outside, and supplies the student image to the class classification unit 822 and the generation unit 823. .

  In step S742, the class classification unit 822 selects a target teacher pixel, which is a predetermined pixel in the teacher image, in accordance with the phase characteristics of the student image corresponding to the target teacher pixel, as in the class classification process of FIG. Class classification processing for classifying into classes. The class classification unit 822 supplies the determined class to the normal equation generation unit 832 (FIG. 18) of the generation unit 823.

  In step S743, the prediction tap extraction unit 831 in FIG. 18 extracts prediction taps from the student image supplied from the student image generation unit 821 for the teacher pixel of interest, and supplies the prediction tap to the normal equation generation unit 832.

  In step S744, the normal equation generation unit 832 extracts the target teacher pixel from the input teacher image, and configures the target teacher pixel and the prediction tap configured for the target teacher pixel supplied from the prediction tap extraction unit 831. The addition of Expression (8) for the student image to be performed is performed for each combination of parameters h and v and for each class supplied from the class classification unit 822.

  In step S745, the class classification unit 822 determines whether or not the pixels of all input teacher images are the target teacher pixels. If it is determined in step S745 that all the teacher image pixels have not yet been set as the target teacher pixel, in step S746, the class classification unit 822 newly sets the teacher image pixel not yet set as the target teacher pixel as the target teacher pixel. Determine as. Then, the process returns to step S742, and the same process is repeated thereafter.

  On the other hand, if it is determined in step S745 that all the teacher image pixels are the target teacher pixels, in step S747, the normal equation generation unit 832 determines each combination of the parameters h and v obtained by the processing so far. In addition, the matrix on the left side and the vector on the right side in Equation (8) for each class are supplied to the coefficient generation unit 824 as normal equations.

In step S748, the coefficient generation unit 824 solves the normal equation that is supplied from the generation unit 823 and is configured by the matrix on the left side and the vector on the right side in Expression (8) for each combination of parameters h and v and for each class. The prediction coefficient W _vhn for each combination of parameters h and v and for each class is obtained and output to the normal equation generation unit 825.

In step S749, the normal equation generation unit 825 generates a normal equation of Expression (17) for each class based on the prediction coefficient W _vhn and _outputs the normal equation to the coefficient type determination unit 826. In step S750, the coefficient type determination unit 826 solves the normal equation of the equation (17) for each class and obtains the coefficient type wn _{, k} for each class. In step S751, the coefficient seed memory 827 stores the coefficient seed wn _{, k} . This coefficient seed is stored in the coefficient seed memory 652 in FIG.

  As described above, the artificial image prediction unit 132 predicts the artificial high quality image using the prediction coefficient generated by the coefficient type obtained by learning using the artificial image, and is supplied from the output phase conversion unit 112. It is possible to accurately improve the quality of the artificial image among the HD images to be displayed.

  Further, since the artificial image prediction unit 132 uses the position of the edge of the class tap as a phase feature and classifies the pixel of interest into a class according to the feature, attention is paid to an artificial image with few gradations and clear phase information. Pixels can be classified accurately. As a result, the artificial image prediction unit 132 predicts an artificial high-quality image from the HD image using a prediction coefficient generated by the coefficient type obtained by learning for each class, and thereby includes noise included in the artificial image. And the like can be more accurately removed, and a higher quality artificial high quality image can be output.

  Next, a second embodiment of the artificial image prediction unit 132 will be described.

The second embodiment of the artificial image prediction unit 132 is configured by providing a class classification unit 840 of FIG. 20 described later, instead of the class classification unit 651 of the artificial image prediction unit 132 of FIG. In this case, the learning device 811 for obtaining the coefficient seed w _{n, k} stored in the coefficient seed memory 652 is provided with a class classification unit configured similarly to the class classification unit 840 instead of the class classification unit 822. Consists of.

  FIG. 20 is a block diagram illustrating a detailed configuration example of the class classification unit according to the second embodiment of the artificial image prediction unit 132.

  The class classification unit 840 of FIG. 20 includes a class tap extraction unit 671, a difference calculation unit 672, and a class determination unit 841. The class classification unit 840 includes, among the pixels constituting the class tap, a ratio of adjacent difference absolute values of adjacent pixels adjacent in the horizontal direction (hereinafter referred to as a horizontal difference ratio) and an adjacent difference of adjacent pixels adjacent in the vertical direction. The absolute value ratio (hereinafter referred to as the vertical difference ratio) is used as a class tap phase feature, and class classification is performed according to the feature. Note that the same components as those in FIG. 11 are denoted by the same reference numerals, and the description thereof will be omitted to avoid repetition.

  The class determination unit 841 is supplied with the adjacent difference absolute value calculated by the difference calculation unit 672. The class determining unit 841 calculates a horizontal difference ratio that represents the horizontal phase of the class tap and a vertical difference ratio that represents the vertical phase of the class tap. The class determining unit 841 uses the horizontal difference ratio and the vertical difference ratio as the characteristics of the phase of the class tap, and determines the class of the target pixel according to the characteristics.

  FIG. 21 shows an example of the tap structure of class taps extracted by the class tap extraction unit 671 of FIG. The tap structure of the class tap can be a structure other than that shown in FIG.

  In FIG. 21, among the HD images supplied from the output phase converter 112, the pixel p162 corresponding to the pixel of interest, and one pixel p160 adjacent to the pixel in the upward, leftward, rightward, and downward directions, respectively. From p161, p163, and p164, a so-called cross-shaped class tap is formed.

  The difference calculation unit 672 includes four adjacent difference absolute values of adjacent pixels p161 and p162, p162 and p163, p160 and p162, and p162 and p164 among the five pixels p160 to p164 constituting the class tap. d40 to d43 are calculated and supplied to the class determining unit 841.

  The class determination unit 841 calculates the ratio (d40 / d41) of the adjacent difference absolute values d40 and d41 of the pixels p161 and p162, p162 and p163, which are adjacent pixels adjacent in the horizontal direction, as the horizontal difference ratio. Further, the class determining unit 841 calculates the ratio (d42 / d43) of the adjacent difference absolute values d42 and d43 of the pixels p160 and p162, p162 and p164, which are adjacent pixels adjacent in the vertical direction, as the vertical difference ratio. Then, the class determining unit 841 determines a class according to the horizontal difference ratio and the vertical difference ratio.

  Specifically, the class determination unit 841 determines, for example, eight types of classifications (subclasses) in which the horizontal difference ratio and the vertical difference ratio of the target pixel are assigned with classification numbers 1 to 8 according to the values, for example. Divide either. Then, the class determining unit 841 determines a combination of the classification number divided according to the horizontal difference ratio and the classification number divided according to the vertical difference ratio as a class. Accordingly, at this time, the target pixel is classified into 64 (= 8 × 8) classes. Of course, the number of classifications is not limited to eight.

  Next, a class classification process in which the class classification unit 840 in FIG. 20 classifies the target pixel into a class will be described with reference to the flowchart in FIG. This class classification process corresponds to the class classification process in step S701 in FIG.

  In step S761, the class tap extraction unit 671 of the class classification unit 840 extracts the class tap (FIG. 21) from the input HD image and supplies the class tap to the difference calculation unit 672.

  In step S <b> 762, the difference calculation unit 672 calculates the adjacent difference absolute value of each adjacent pixel among the pixels constituting the class tap supplied from the class tap extraction unit 671, and supplies the calculated value to the class determination unit 841.

  In step S763, the class determination unit 841 calculates the horizontal difference ratio and the vertical difference ratio from the adjacent difference absolute value supplied from the difference calculation unit 672.

  In step S764, the class determination unit 841 determines a class based on the horizontal difference ratio and the vertical difference ratio. The class determination unit 841 supplies the class to the prediction coefficient memory 654 in FIG. Thereafter, the processing returns to step S701 in FIG.

  Next, a third embodiment of the artificial image prediction unit 132 will be described.

The third embodiment of the artificial image prediction unit 132 is configured by providing a class classification unit 851 of FIG. 23 described later instead of the class classification unit 651 of the artificial image prediction unit 132 of FIG. In this case, the learning device 811 for obtaining the coefficient seed w _{n, k} stored in the coefficient seed memory 652 is provided with a class classification unit configured similarly to the class classification unit 851 instead of the class classification unit 822. Consists of.

  FIG. 23 is a block diagram illustrating a detailed configuration example of the class classification unit according to the third embodiment of the artificial image prediction unit 132.

  The class classification unit 851 in FIG. 23 includes a class classifier 861, a class classifier 862, and a class selection unit 863. The class classification unit 851 not only performs class classification according to the phase characteristics of the HD image, but also performs class classification according to the waveform characteristics of the HD image, and selects one of the resulting classes. Output.

  The HD image supplied from the output phase converter 112 is input to the class classifier 861 and the class classifier 862.

  The class classifier 861 sequentially sets a pixel constituting the artificial high quality image as a target pixel, and the target pixel is selected from any of several classes according to the waveform characteristics of the HD image corresponding to the target pixel. Class (hereinafter referred to as waveform classification class). The class classifier 861 performs classification suitable for classifying features of an artificial image having relatively few flat portions, that is, having relatively many edges, that is, an artificial image close to a natural image, among the artificial images. The class classifier 861 supplies the classified waveform classification class to the class selection unit 863.

  The class classifier 862 is configured in the same manner as the class classification unit 651 in FIG. The class classifier 862 classifies into a class (hereinafter referred to as a phase classification class) according to the phase characteristics of the HD image corresponding to the target pixel. The class classifier 862 performs classification suitable for classifying the features of an artificial image having a relatively large number of flat portions, that is, having relatively few edges, that is, an artificial image like an artificial image. The class classifier 862 supplies the classified phase classification class to the class selection unit 863.

  Based on the phase classification class supplied from the class classifier 862, the class selection unit 863 selects one of the waveform classification class supplied from the class classifier 861 and the phase classification class supplied from the class classifier 862. This is selected and output to the prediction coefficient memory 654 of FIG.

  FIG. 24 is a block diagram illustrating a detailed configuration example of the class classifier 861 in FIG.

  The class classifier 861 in FIG. 23 includes a class tap extraction unit 871 and an ADRC processing unit 872.

  The HD image is supplied to the class tap extraction unit 871, and the class tap extraction unit 871 extracts the pixels constituting the HD image used for classifying the target pixel into the waveform classification class as a class tap, and the ADRC processing unit 872.

The ADRC processing unit 872 performs 1-bit ADRC processing on the pixel value of the pixel constituting the class tap supplied from the class tap extraction unit 871 as the characteristic of the class tap waveform. Then, the ADRC processing unit 872 determines a bit string in which 1-bit pixel values obtained as a result of the ADRC processing are arranged in a predetermined order as the waveform classification class of the target pixel. Therefore, the waveform classification class is a value obtained by degenerating the waveform of the class tap. The ADRC processing unit 872 supplies the determined waveform classification class to the prediction coefficient memory 654 in FIG.

  As described above, the class classifier 861 classifies the pixel of interest into the waveform classification class according to the waveform characteristics of the class tap.

  FIG. 25 shows an example of the structure of class taps extracted by the class tap extraction unit 871. The tap structure of the class tap can be a structure other than that shown in FIG.

  In FIG. 25, the class tap extracted by the class tap extraction unit 871 is the pixel p184 corresponding to the target pixel in the HD image supplied from the output phase conversion unit 112, and the pixel tap of the pixel, as in FIG. It consists of two pixels p180, p181, p182, p183, p185, p186, p187, and p188 that are adjacent in the upward, leftward, rightward, and downward directions, respectively.

  The ADRC processing unit 872 performs 1-bit ADRC processing on the pixel values of the nine pixels p180 to p188 supplied from the class tap extraction unit 871, and uses the resulting 9-bit bit string as a waveform classification class. decide.

  FIG. 26 is a block diagram illustrating a detailed configuration example of the class selection unit 863 in FIG.

  The class selection unit 863 in FIG. 26 includes a table conversion unit 891 and a switch 892.

  The 9-bit waveform classification class output from the class classifier 861 is supplied to one input A of the switch 892, and the 8-bit phase classification class output from the class classifier 862 includes the table conversion unit 891 and the switch 892. To the other input B.

  The table conversion unit 891 associates an 8-bit phase classification class supplied from the class classifier 862 with a 1-bit select signal in a built-in memory (not shown), thereby converting the 8-bit phase classification class into one bit. A degeneration table for degeneration is stored in the select signal. Based on the degeneration table, the table conversion unit 891 converts the 8-bit phase classification class supplied from the class classifier 862 into a 1-bit select signal, and switches the select signal to the switch 892. Supply as a signal.

  For example, in the degeneration table, for a phase classification class in which 5 or more bits out of 8 bits are 1, that is, a class in which the number of edges is larger than the reference value, 1 of the select signal is more than 5 out of 8 bits. The phase classification class having a small number of bits of 1, that is, the class having a smaller number of edges than the reference value is configured by associating 0 of the select signal with each other. Thereby, for example, when the value of the phase classification class is (10110010) (the number of 1s is four), the table conversion unit 891 outputs a select signal indicating 0, and the value of the phase classification class is (00110111). If (the number of 1s is 5), the table conversion unit 891 outputs a select signal representing 1.

  The switch 892 responds to a select signal supplied from the table conversion unit 891, and when the select signal is 1, that is, an HD image configured by class taps is an artificial image having more edges than the reference value, that is, a natural image. Is selected, the waveform classification class supplied to the input A is selected, and when the select signal is 0, that is, the HD image constituted by class taps is an artificial image having a number of edges smaller than the reference value, that is, In the case of an artificial image that seems to be an artificial image, the phase classification class supplied to the input B is selected and output to the prediction coefficient memory 654 of FIG.

  Next, a class classification process in which the class classification unit 851 in FIG. 23 classifies the target pixel will be described with reference to the flowchart in FIG. This class classification process corresponds to the class classification process in step S701 in FIG.

  In step S781, the class classifier 861 of the class classification unit 851 sequentially sets the pixels constituting the artificial high quality image as the target pixel, and sets the target pixel according to the characteristics of the waveform of the HD image corresponding to the target pixel. Then, class classification processing for classifying into waveform classification classes is performed. Details of this class classification processing will be described later with reference to FIG.

  In step S782, the class classifier 862 performs the same class classification process as in FIG. Since the description is repeated, it is omitted. In step S783, the class selection unit 863 selects the waveform classification class supplied from the class classifier 861 and the phase classification class supplied from the class classifier 862 based on the phase classification class supplied from the class classifier 862. A class selection process for selecting either one is performed. Thereafter, the processing returns to step S701 in FIG. Details of this class selection processing will be described later with reference to FIG.

  Next, the class classification process in step S781 in FIG. 27 will be described with reference to the flowchart in FIG.

  In step S801, the class tap extraction unit 871 of the class classifier 861 extracts a class tap (FIG. 25) from the input HD image and supplies the class tap to the ADRC processing unit 872.

  In step S802, the ADRC processing unit 872 performs 1-bit ADRC processing on the pixel value of the class tap supplied from the class tap extraction unit 871. The ADRC processing unit 872 determines the 8-bit bit string obtained as a result as the waveform classification class. The ADRC processing unit 872 supplies the waveform classification class to the input A of the class selection unit 863, and returns the processing to step S781 in FIG.

  As a result, among the artificial images, classification of a class suitable for the feature of the artificial image having many edges, that is, close to the natural image is performed.

  Next, the class selection processing in step S783 in FIG. 27 will be described with reference to the flowchart in FIG.

  In step S821, the table conversion unit 891 (FIG. 26) of the class selection unit 863 degenerates the 8-bit phase classification class supplied from the class classifier 862 into a 1-bit select signal.

  In step S822, the switch 892 determines whether the select signal supplied from the table conversion unit 891 is “1”. If it is determined in step S822 that the select signal is 1, that is, if the HD image composed of class taps is an artificial image close to a natural image with many edges, in step S823, the switch 892 A waveform classification class is selected from the waveform classification class supplied from 861 to the input A and the phase classification class supplied to the input B, and is output to the prediction coefficient memory 654.

  On the other hand, if it is determined in step S822 that the select signal is 0, that is, if the HD image formed by class taps is a representative artificial image with few edges, in step S824, the switch 892 performs waveform classification. The phase classification class is selected from the class and the phase classification class, and is output to the prediction coefficient memory 654.

  As described above, the class selection unit 863 selects the waveform classification class when the HD image configured by clustering is an artificial image having many edges, that is, a flat portion, and the HD image configured by class taps is selected. In the case of an artificial image with few edges, that is, many flat parts, the phase classification class is selected.

  That is, the class classifier 861 outputs the waveform classification class classified according to the waveform characteristics of the class tap when the HD image formed by the class tap is an artificial image with a small flat portion. Among the artificial images with small gradation and clear phase information, the target pixel corresponding to the class tap can be accurately classified into a class even in a complex image portion with a small flat portion.

  As a result, the artificial image prediction unit 132 obtains an artificial high quality image from the HD image using the prediction coefficient obtained by learning for each class, thereby further reducing noise included in the artificial image of the HD image. It is possible to accurately remove and output a higher quality artificial high quality image.

  In the above description, the class classifier 862 in FIG. 23 is configured in the same manner as the class classification unit 651 in FIG. 11, but is configured in the same manner as the class classification unit 840 in FIG. Also good.

  In the class classification unit 851 of FIG. 23, the class classifier 861 is disposed before the class selection unit 863, but may be disposed after the class selection unit 863. In this case, when the select signal is 1, the class selection unit 863 instructs the class classifier 861 to perform class classification, and when the select signal is 0, the phase classification class supplied from the class classifier 862. Is output. Thereby, the class classifier 861 only needs to perform class classification when the select signal is 1, and the processing load on the class classifier 861 can be reduced.

  In the above description, the natural image prediction unit 131 and the artificial image prediction unit 132 predict a high-quality natural high-quality image or artificial high-quality image from which noise is removed from an HD image including noise. A high-quality high-resolution image may be predicted from the resolution HD image. For example, the natural image prediction unit 131 and the artificial image prediction unit 132 predict an image in which the number of pixels is quadrupled from the input image, or predict an image in which the number of pixels is reduced, that is, the pixels are thinned out. You can also That is, the present invention can be used when various image processing is performed.

  Furthermore, the natural image prediction unit 131 and the artificial image prediction unit 132 can also predict an original image from the input compressed image. In this case, the learning device 601 and the learning device 811 use, for example, a compressed image that is broadcast, a compressed natural image or an artificial image that is recorded on a DVD (Digital Versatile Disk) as a student image, and the student image. Learning is performed using the original image as a teacher image.

  In the above description, the coefficient type is learned, but the prediction coefficient itself may be learned. In this case, the natural image prediction unit 131 and the artificial image prediction unit 132 perform prediction using the prediction coefficient itself acquired by learning.

  Next, a detailed configuration example of the natural-image / artificial-image determination unit 114 will be described with reference to FIG. The natural image / artificial image determination unit 114 includes a wide area feature amount extraction unit 911, a wide area artificial image degree calculation unit 912, an artificial image degree generation unit 913, a narrow area feature amount extraction unit 914, and a narrow area artificial image degree calculation unit 915. Is done.

  The wide area feature amount extraction unit 911 includes a BEP (Broad Edge Parameter) extraction unit 931, a BFP (Broad Flat Parameter) extraction unit 932, and a feature amount synthesis unit 933, and the HD image supplied from the output phase conversion unit 112. A wide-area feature amount is extracted from each pixel and supplied to the wide-area artificial image degree calculation unit 912.

  The BEP extraction unit 931 sets, for each pixel of the HD image supplied from the output phase conversion unit 112, a value obtained based on the dynamic range of the pixel value of the pixel in the reference region set corresponding to the target pixel. The feature amount is extracted as a feature amount BEP and supplied to the BFP extraction unit 932 and the feature amount synthesis unit 933. A detailed configuration example of the BEP extraction unit 931 will be described later with reference to FIG. Hereinafter, the pixels in the reference area and the feature amount BEP are also simply referred to as reference pixels and BEP. Further, in the following description, the target pixel is a pixel of an HD image supplied from the output phase conversion unit 112 and is a pixel to be processed in the natural image / artificial image determination unit 114.

  The BFP extraction unit 932, for each pixel of the HD image supplied from the output phase conversion unit 112, based on the absolute value of the pixel value difference between adjacent pixels of the reference pixel set corresponding to the target pixel and the BEP The obtained value is extracted as a feature quantity BFP constituting the wide area feature quantity and supplied to the feature quantity synthesis unit 933. A detailed configuration example of the BFP extraction unit 932 will be described later with reference to FIG. Hereinafter, the feature amount BFP is also simply referred to as BFP.

  The feature amount synthesis unit 933 synthesizes the BEP supplied from the BEP extraction unit 931 and the BFP supplied from the BFP extraction unit 932, generates a wide area feature amount, and supplies it to the wide area artificial image degree calculation unit 912. That is, the wide-area feature amount is a feature amount that includes the above-described two types of feature amounts, BEP and BFP, and indicates the features of a wide range of images to which the target pixel belongs.

The wide-area artificial image degree calculation unit 912 includes a feature amount separation unit 951, a wide-area boundary line comparison unit 952, a wide-area boundary line memory 953, and an internal division calculation unit 954, and is supplied from the wide-area feature amount extraction unit 911. Based on the feature amount, the wide-area artificial image degree Art _b is calculated and supplied to the artificial image degree generation unit 913.

  The feature amount separation unit 951 separates feature amounts constituting the wide area feature amount supplied from the wide area feature amount extraction unit 911 and supplies the separated feature amounts to the wide area boundary comparison unit 952. That is, in this example, since the wide area feature amounts are BEP and BFP, the feature amount separation unit 951 separates the BEP and BFP and supplies them to the wide area boundary comparison unit 952.

  The wide area boundary line comparison unit 952 is statistically obtained in advance from a plurality of artificial images and natural images based on the two types of feature amount values supplied from the feature amount separation unit 951, and has a wide area boundary line memory. A comparison is made with a wide boundary line indicating whether the image belongs to an artificial image or a natural image stored in 953, and the comparison result is supplied to the internal division calculation unit 954.

Based on the comparison result supplied from the wide area boundary line comparison unit 952, the internal division calculation unit 954 calculates a wide area artificial degree Art _b corresponding to the wide area feature amount, and supplies it to the artificial image degree generation unit 913.

  The narrow area feature amount extraction unit 914 includes a PNDP (Primary Narrow Discrimination Parameter) extraction unit 971, an SNDP (Secondary Narrow Discrimination Parameter) extraction unit 972, and a feature amount synthesis unit 973, and is supplied from the output phase conversion unit 112. A narrow area feature amount is extracted from each pixel of the HD image and supplied to the narrow area artificial image degree calculation unit 915.

  For each pixel of the HD image supplied from the output phase conversion unit 112, the PNDP extraction unit 971 is based on the dynamic range of the pixel value of the pixel in the long tap region included in the reference region set corresponding to the target pixel. The obtained value is extracted as a feature quantity PNDP constituting the narrow area feature quantity, and supplied to the SNDP extraction unit 972 and the feature quantity synthesis unit 973. A detailed configuration example of the PNDP extraction unit 971 will be described later with reference to FIG. Hereinafter, the feature amount PNDP is also simply referred to as PNDP.

  The SNDP extraction unit 972, for each pixel of the HD image supplied from the output phase conversion unit 112, is a pixel of a pixel in a short tap region included in a long tap region included in a reference region set corresponding to the target pixel A feature quantity SNDP obtained based on the dynamic range of values and PNDP is extracted as a feature quantity constituting the narrow area feature quantity, and is supplied to the feature quantity synthesis unit 973. A detailed configuration example of the SNDP extraction unit 972 will be described later with reference to FIG. Hereinafter, the feature amount SNDP is also simply referred to as SNDP.

  The feature amount combining unit 973 combines the PNDP supplied from the PNDP extraction unit 971 and the SNDP supplied from the SNDP extraction unit 972, generates a narrow region feature amount, and supplies the narrow region artificial image degree calculation unit 915 To do. That is, the narrow area feature amount is a feature amount composed of the above-described two types of feature amounts, PNDP and SNDP, and is a feature amount indicating a feature of a narrow range image to which the target pixel belongs.

The narrow area artificial degree-of-image calculation unit 915 includes a feature amount separation unit 991, a narrow region boundary line comparison unit 992, a narrow region boundary line memory 993, and an internal division calculation unit 994, and is supplied from the narrow region feature amount extraction unit 914. On the basis of the narrow area feature value, the narrow area artificial image degree Art _n is calculated and supplied to the artificial image degree generating unit 913.

  The feature amount separation unit 991 separates the feature amounts constituting the narrow region feature amount supplied from the narrow region feature amount extraction unit 914, and supplies the separated feature amounts to the narrow region boundary line comparison unit 992. That is, in this example, since the narrow area feature quantities are PNDP and SNDP, the feature quantity separating unit 991 separates PNDP and SNDP, respectively, and supplies them to the narrow area boundary comparison unit 992.

  The narrow area boundary line comparison unit 992 is statistically obtained in advance from a plurality of artificial images and natural images based on the values of the two types of feature values supplied from the feature amount separation unit 991, and the narrow area boundary line 992 The result is compared with a narrow boundary line indicating whether the image belongs to an artificial image or a natural image stored in the line memory 993, and the comparison result is supplied to the internal calculation unit 994.

Based on the comparison result supplied from the narrow boundary line comparison unit 992, the internal division calculation unit 994 calculates the narrow region artificial image degree Art _n corresponding to the narrow region feature amount, and sends it to the artificial image degree generation unit 913. Supply.

The articulation degree generation unit 913 obtains the wide-area articulation degree Art _b supplied from the wide-area articulation degree calculation unit 912 and the narrow-area articulation degree Art _n supplied from the narrow-area articulation degree calculation unit 915. The resultant image is combined to generate an artificial art degree Art and output to the image processing unit 113.

  Next, a detailed configuration example of the BEP extraction unit 931 will be described with reference to FIG.

  31 includes a buffer 1011, a reference pixel extraction unit 1012, a weight calculation unit 1013, an inter-pixel difference calculation unit 1014, a multiplication unit 1015, a storage unit 1016, a maximum / minimum value extraction unit 1017, and a BEP calculation unit. 1018.

  The buffer 1011 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1012 as necessary. The reference pixel extraction unit 1012 sequentially reads out the reference pixels for each target pixel and supplies them to the weight calculation unit 1013 and the inter-pixel difference calculation unit 1014. Here, the reference pixel is a pixel in a predetermined area set in association with the target pixel, and is, for example, all pixels in a range of n pixels × n pixels centering on the target pixel. Note that the reference pixels may be a plurality of pixels set in association with the target pixel, and needless to say, other numbers or arrangement of pixels may be used.

The weight calculation unit 1013 calculates a weight corresponding to the distance from the target pixel for each reference pixel supplied from the reference pixel extraction unit 1012 and supplies the weight to the multiplication unit 1015. Pixel difference calculating unit 101 4 supplies to the multiplication unit 1015 obtains a difference value between the target pixel for each reference pixel supplied from the reference-pixel extracting portion 1012. The multiplication unit 1015 multiplies the weight supplied from the weight calculation unit 1013 by the difference value between the target pixel and the reference pixel supplied from the inter-pixel difference calculation unit 1014 and causes the accumulation unit 1016 to accumulate the result.

The maximum value / minimum value extraction unit 1017 obtains a value obtained by multiplying the difference value between all the reference pixels accumulated in the accumulation unit 1016 and the target pixel by a weight according to the distance between the corresponding reference pixel and the target pixel. When all are accumulated, the maximum value and the minimum value are extracted from them and supplied to the BEP calculation unit 1018. The BEP calculation unit 1018 supplies a reference value corresponding to the difference value between the reference pixel and the target pixel, that is, the difference dynamic range obtained from the difference between the reference pixel and the target pixel, supplied from the maximum value / minimum value extraction unit 1017. The difference value of the value multiplied by the weight according to the distance between the pixel and the target pixel is output as the BEP of the target pixel. A value obtained by multiplying the differential dynamic range by the weight is also referred to as a dynamic range corresponding to the distance from the pixel of interest or simply as a differential dynamic range.

  Next, a detailed configuration example of the BFP extraction unit 932 will be described with reference to FIG.

  32 includes a buffer 1031, a reference pixel extraction unit 1032, a difference calculation unit 1033 between adjacent pixels, a function conversion unit 1034, a weight calculation unit 1035, a multiplication unit 1036, a storage unit 1037, and a BFP calculation unit 1038. It is configured.

  The buffer 1031 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1032 as necessary. The reference pixel extraction unit 1032 sequentially reads out the reference pixel for each target pixel and supplies the reference pixel difference calculation unit 1033 and the weight calculation unit 1035 to each other.

  The inter-adjacent pixel difference calculation unit 1033 supplies an absolute difference value between adjacent pixels to the function conversion unit 1034 for all the reference pixels supplied from the reference pixel extraction unit 1032. Based on the BEP supplied from the BEP extraction unit 931, the function conversion unit 1034 sets the conversion function, and sets the adjacent pixel difference absolute value supplied from the adjacent pixel difference calculation unit 1033. The data is converted by the conversion function and supplied to the multiplication unit 1036.

  The weight calculation unit 1035 calculates a weight corresponding to the distance from the target pixel for each position of the reference pixels supplied from the reference pixel extraction unit 1032 and supplies the weight to the multiplication unit 1036. The multiplication unit 1036 multiplies the weight supplied from the weight calculation unit 1035 and the function-converted reference inter-pixel difference absolute value supplied from the function conversion unit 1034, and causes the accumulation unit 1037 to accumulate the result.

  The BFP calculation unit 1038 cumulatively adds the multiplication results stored in the storage unit 1037 and outputs the result as the BFP of the target pixel.

  Next, a detailed configuration example of the PNDP extraction unit 971 will be described with reference to FIG.

33 includes a buffer 1051, a reference pixel extraction unit 1052, a long tap extraction unit 1053, a pixel value storage unit 1054, a maximum / minimum value extraction unit 1055, and a PNDP calculation unit 1056.

  The buffer 1051 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1052 as necessary. The reference pixel extraction unit 1052 sequentially reads out the reference pixels for each target pixel and supplies the reference pixels to the long tap extraction unit 1053.

  The long tap extraction unit 1053 extracts a pixel from a region that is included in the reference pixel region of the reference pixels, that is, is the same as or smaller than the reference pixel region, and the pixel value storage unit 1054 To supply. The long tap extraction unit 1053 also supplies the long tap information to the short tap extraction unit 1073 of the SNDP extraction unit 972. Further, the long tap extraction unit 1053 supplies the fact to the maximum value / minimum value extraction unit 1055 at the stage where all the extraction has been completed for the long taps.

  The maximum value / minimum value extraction unit 1055 extracts the maximum value and the minimum value from all the pixel values of the long tap accumulated in the pixel value accumulation unit 1054 and supplies the extracted value to the PNDP calculation unit 1056.

  The PNDP calculation unit 1056 obtains the pixel value dynamic range of the reference pixel from the difference value between the maximum value and the minimum value of the pixel value supplied from the maximum value / minimum value extraction unit 1055, and outputs it as the PNDP of the target pixel. Here, the pixel value dynamic range is a value obtained by subtracting the minimum value from the maximum value of all the pixel values included in the predetermined area.

  Next, a detailed configuration example of the SNDP extraction unit 972 will be described with reference to FIG.

  34 includes a buffer 1071, a reference pixel extraction unit 1072, a short tap extraction unit 1073, a pixel value storage unit 1074, a maximum / minimum value extraction unit 1075, a DR calculation unit 1076, a DR storage unit 1077, and an SNDP. The selection unit 1078 is configured.

  The buffer 1071 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1072 as necessary. The reference pixel extraction unit 1072 sequentially reads out the reference pixels for each target pixel and supplies them to the short tap extraction unit 1073.

  The short tap extraction unit 1073 is an area including the target pixel based on the long tap information supplied from the long tap extraction unit 1053, and is the same as or more than the area included in the long tap. Also, pixels are extracted from a plurality of small areas and supplied to the pixel value storage unit 1074. That is, for one long tap, a plurality of patterns may be set for the short tap. In addition, the short tap extraction unit 1073 supplies the fact to the SNDP selection unit 1078 when extraction of all the patterns for a plurality of short taps is completed.

  The maximum value / minimum value extraction unit 1075 extracts the maximum value and the minimum value from all the pixel values included in the short tap accumulated in the pixel value accumulation unit 1074 and supplies the extracted value to the DR calculation unit 1076.

  The DR calculation unit 1076 obtains a pixel value dynamic range from the difference between the maximum value and the minimum value among all the pixel values included in the short tap supplied from the maximum value / minimum value selection unit 1075, and the DR accumulation unit 1077. To accumulate.

  When the SNDP 1079 is notified by the short tap extraction unit 1073 that extraction has been completed for all the short taps of the pattern, the pixels calculated for each short tap of all the patterns stored in the DR storage unit 1077 Select the minimum value from the value dynamic range and output as SNDP.

  Next, the natural image / artificial image determination process will be described with reference to the flowchart of FIG.

  In step S831, the BEP extraction unit 931 of the wide area feature amount extraction unit 911 performs BEP extraction processing, extracts BEP for each pixel from the HD image supplied from the output phase conversion unit 112, and outputs it to the feature amount synthesis unit 933. Supply.

  Here, the BEP extraction processing by the BEP extraction unit 931 in FIG. 31 will be described with reference to the flowchart in FIG.

  In step S851, the buffer 1011 temporarily stores the HD image supplied from the output phase converter 112.

  In step S852, the reference pixel extraction unit 1012 sets an unprocessed pixel as a target pixel from the buffer 1011 and reads out a reference pixel set corresponding to the target pixel in step S853, thereby calculating a weight calculation unit. 1013 and the inter-pixel difference calculation unit 1014. Here, the reference pixel is, for example, a pixel in a predetermined region set in association with the target pixel. For example, as shown in FIG. 37, n pixels × n pixels centered on the target pixel. All the pixels in the range.

  In FIG. 37, the white circles on the coordinates in the two-dimensional space in the x and y directions centered on the noticed pixel Q (0, 0) of the black circle in the reference region R in the image are the reference pixels Q ( i, j), and a pixel within a range surrounded by n × n dotted lines centered on the pixel of interest Q (0,0) is a reference pixel Q (i, j) (− (n− 1) / 2 ≦ i ≦ (n−1) / 2, − (n−1) / 2 ≦ j ≦ (n−1) / 2: i and j are integers).

  In step S854, the inter-pixel difference calculation unit 1014 calculates a pixel value difference value between the pixel of interest and the unprocessed reference pixel, and supplies the pixel value difference value to the multiplication unit 1015. That is, in FIG. 37, the inter-pixel difference calculation unit 1014 calculates (Q (0,0) −Q (i, j)) as the inter-pixel difference value and supplies the calculated value to the multiplication unit 1015. Note that Q (0,0) and Q (i, j) in the inter-pixel difference values are the pixel values of the target pixel Q (0,0) and the reference pixel Q (i, j), respectively.

In step S855, the weight calculation unit 1013 calculates a weight according to the distance between the target pixel and an unprocessed reference pixel, and supplies the weight as a calculation result to the multiplication unit 1015. More specifically, for example, when the reference pixel for which the inter-pixel difference value has been obtained is the reference pixel Q (i, j), the weight calculation unit 1013 uses, for example, the coordinate parameter as the weight w _d = a / (i ² + j ² ) (a: constant) is calculated. Note that the weight w _d may be calculated by a method other than the above-described method because the value may be determined so as to increase as the distance increases and decreases as the distance increases. For example, the weight w _d = a / √ (i ² + j ² ) (a: constant) may be used.

In step S856, the multiplication unit 1015 multiplies the inter-pixel difference value supplied from the inter-pixel difference calculation unit 1014 by the weight supplied from the weight calculation unit 1013, and causes the accumulation unit 1016 to accumulate the result. That is, in the case of FIG. 37, the multiplication unit 1015 supplies the inter-pixel difference value (Q (0,0) −Q (i, j)) supplied from the inter-pixel difference calculation unit 1014 and the weight calculation unit 1013. Multiplying by the weight w _d that has been performed, the storage unit 1016 stores w _d × (Q (0,0) −Q (i, j)) as a multiplication result.

  In step S857, the inter-pixel difference calculation unit 1014 determines whether there is an unprocessed reference pixel. If there is an unprocessed reference pixel, the process returns to step S854. That is, the processes in steps S854 to S857 are repeated for all reference pixels.

In step S857, there is no unprocessed reference pixel, that is, the difference value from the target pixel is obtained for all the reference pixels, and a weight w _d corresponding to the distance is set and multiplied and accumulated. If it is determined that all are stored in the unit 1016, in step S858, the inter-pixel difference calculation unit 1014 notifies the maximum value / minimum value extraction unit 1017 that the processing has been completed for all reference pixels. The maximum value / minimum value extraction unit 1017 extracts the maximum value and the minimum value from the multiplication results stored in the storage unit 1016 and supplies the extracted value to the BEP calculation unit 1018.

  In step S859, the BFP calculation unit 1018 calculates the difference dynamic range of the reference pixel, which is a value obtained by subtracting the minimum value from the maximum value of the multiplication result supplied from the maximum value / minimum value extraction unit 1017, as a BEP for the target pixel. And supplied to the BFP extraction unit 932 and the feature amount synthesis unit 933.

  In step S860, the reference pixel extraction unit 1012 determines whether BEPs have been obtained for all the pixels in the image stored in the buffer 1011. In step S860, when BEP is not calculated | required about all the pixels of an image, a process returns to step S852. That is, the processes in steps S852 to S860 are repeated until BEPs are obtained for all the pixels of the image stored in the buffer 1011. If it is determined in step S860 that BEPs have been obtained for all the pixels in the image, the BEP extraction process ends.

  Through the above processing, a BEP that is a feature amount representing a wide edge in an image is obtained as a BEP. In other words, the BEP expresses the differential dynamic range in the reference pixel with respect to the target pixel, and becomes a large value when an edge exists in the wide area to which the target pixel belongs, and conversely, a small value when there is no edge. Will take.

  Now, the description returns to the flowchart of FIG.

  When the BEP is extracted from the image by the BEP extraction process in step S831, the BFP extraction unit 932 of the wide area feature quantity extraction unit 911 executes the BFP extraction process in step S832, and is supplied from the output phase conversion unit 112. A BFP is extracted for each pixel from the image and supplied to the feature amount synthesis unit 933.

  Here, the BFP extraction processing by the BFP extraction unit 932 in FIG. 32 will be described with reference to the flowchart in FIG. 38.

  In step S871, the buffer 1031 temporarily stores the HD image supplied from the output phase converter 112.

  In step S872, the reference pixel extraction unit 1032 sets an unprocessed pixel as a target pixel from the buffer 1011 and reads out a reference pixel set corresponding to the target pixel in step S873, so that it is not between adjacent pixels. The difference calculation unit 1033 and the weight calculation unit 1035 are supplied.

In step S874, the difference calculation unit 1033 between adjacent pixels calculates a difference absolute value of pixel values between unprocessed reference pixels (including the target pixel), and supplies the calculated difference to the function conversion unit 1034. That is, in FIG. 39, the inter- adjacent pixel difference calculation unit 1033 calculates e = | Q (i + 1, j) −Q (i, j) | Part 1034. Note that Q (i + 1, j) and Q (i, j) in the inter-pixel difference absolute values are the pixel values of the reference pixels Q (i + 1, j) and Q (i, j), respectively. Also, the black circles in FIG. 39 are the target pixel Q (0, 0).

  Furthermore, the difference absolute value e between adjacent pixels of the reference pixel is a difference absolute value of a pixel value of each pixel indicated by a circle in FIG. 39 with a pixel adjacent in the x direction and the y direction. Accordingly, the difference value e between adjacent pixels of the reference pixel Q (i, j) is | Q (i + 1, j) −Q (i, j) |, | Q (i, j + 1) −Q (i, j). |, | Q (i-1, j) -Q (i, j) |, | Q (i, j-1) -Q (i, j) | The However, since the pixels at the edge of the region where the reference pixel is set are not adjacent to the reference pixel in either the positive or negative direction of the xy direction, the difference absolute value between the three types of adjacent pixels Is set. Further, among the reference pixels, the reference image at the four corners of the region where the reference pixel is set is not adjacent to the reference pixel in either the positive or negative direction of the xy direction. Two types of absolute difference values between adjacent pixels are set. As a result, when n pixels × n reference pixels are set around the pixel of interest, 2n (n−1) inter-pixel difference absolute values are set.

  In step S875, the function conversion unit 1034 uses the adjacent pixel difference absolute value e supplied from the adjacent pixel difference calculation unit 1033 based on the function f set by the BEP supplied from the BEP extraction unit 931. The data is converted and supplied to the multiplication unit 1036. More specifically, the function conversion unit 1034 converts a value based on the following equation (18) set by the BEP supplied from the BEP extraction unit 931, and outputs the converted value to the multiplication unit 1036. .

In Expression (18), when the difference absolute value e between adjacent pixels is larger than the threshold th = BEP / b, 0 is output to the multiplication unit 1036, and the difference absolute value e between adjacent pixels is equal to or less than the threshold th = BEP / b. In some cases, 1 is output to the multiplier 1036. Here, b is a constant. The BEP used for calculating the threshold th is the BEP of the target pixel. Further, the threshold th may be a threshold using a BEP other than the threshold th = BEP / b, as long as the BEP tendency can be used. For example, the threshold th = (√BEP) / b or the threshold th = (BEP ² / b etc. may be used.

In step S876, the weight calculator 1035, and the pixel of interest, a weight corresponding to the distance between the center position between the pixels to calculate the adjacent-pixel difference absolute value is calculated, and the calculation result of the weight w _e multiplication unit 1036 is To supply. More specifically, for example, when the reference pixels for which the adjacent pixel difference absolute value is obtained are the reference pixels Q (i, j) and Q (i + 1, j), the weight calculation unit 1013 sets the coordinate parameters. used, for example, w _e = a / as weights ((i + 1/2) 2 + j 2): to calculate the (a constant). The weight w _e is the distance-dependent value is only needs to be determined, may be calculated in addition to the above-described manner, for example, the weight _{w e = a / √ ((} i + 1/2) 2 + j 2 ) (A: constant) or the like.

In step S877, the multiplication unit 1036 multiplies the adjacent-pixel difference absolute value e which is supplied from the function transformer 1034, a weight w _e which is supplied from the weight calculator 1035, at step S878, the storage unit 1037. That is, in the case of FIG. 39, the multiplication unit 1036, 1 or 0 is a conversion result supplied from the function transformer 1034 multiplies the weight w _e which is supplied from the weight calculator 1035, a multiplication result to accumulate w _e or 0 in the storage unit 103 7.

  In step S879, the inter-adjacent pixel difference calculation unit 1033 determines whether there is an unprocessed reference pixel, that is, there is a pixel that is not obtained even though the adjacent pixel difference absolute value is set. If it exists, the process returns to step S874. At this time, the processing of steps S874 to S879 is repeated for all the absolute values of differences between adjacent pixels. That is, as described above, as shown in FIG. 39, when the reference pixel is set to n pixels × n pixels, 2n (n−1) absolute difference values between adjacent pixels are obtained. Therefore, the processes in steps S874 to S879 are repeated 2n (n-1) times.

  In step S879, there is no unprocessed reference pixel, that is, all adjacent pixel difference absolute values are obtained and function-converted, weights corresponding to the distances are set, they are multiplied, and the storage unit If it is determined that all are stored in 1037, in step S880, the adjacent pixel difference calculation unit 1033 notifies the BFP calculation unit 1038 that the processing has been completed for all reference pixels. The BFP calculation unit 1038 obtains the sum of the multiplication results accumulated in the accumulation unit 1037 and outputs it as BFP.

  In step S881, the reference pixel extraction unit 1032 determines whether BFP has been obtained for all the pixels in the image stored in the buffer 1011. In step S881, when BFP is not calculated | required about all the pixels of an image, a process returns to step S872. That is, the processes in steps S872 to S881 are repeated until BFP is obtained for all the pixels of the image stored in the buffer 1031. If it is determined in step S881 that the BFP has been obtained for all the pixels in the image, the BFP extraction process ends.

Through the above processing, BFP is obtained as a feature value representing a wide flat portion in the image. That, BFP is, if not the weight w _e is corresponding to the distance, if uniformly was 1, among the reference pixels, and values which the number counted in the small adjacent-pixel difference absolute value than the threshold value th Become. As a result, the BFP value increases when the wide range to which the pixel of interest belongs is flat, and conversely, the value decreases when there are many edges or the like and the value is not flat.

  In the above description, the example of obtaining the absolute difference value between all adjacent pixels of the reference pixel has been described in calculating the absolute difference value between adjacent pixels. However, the relationship between the pixel values between adjacent pixels is clear. The absolute value of the difference between adjacent pixels may be obtained by other methods, for example, between pixels adjacent in either the x direction or the y direction in FIG. You may make it obtain | require the absolute difference value of a pixel value, and about each reference pixel, the sum of the absolute difference value between adjacent pixels with each pixel adjoining up and down, the adjacent pixel with each pixel adjacent to right and left The sum of absolute differences between pixels or the sum of absolute differences between pixels with pixels adjacent in the vertical and horizontal directions may be used as the absolute difference between adjacent pixels. Further, in the following description, there is a process for obtaining the absolute value of the difference between adjacent pixels, but it is the same that various formats may be used as described above.

  In addition, the function conversion unit 1034 described above converts the difference absolute value e between adjacent pixels by a function as shown in Expression (18) and outputs the converted value, but the conversion function is expressed by Expression (18). For example, as shown in FIG. 40, the difference in the absolute value of the difference between adjacent pixels is converted into a value that can be clarified as shown in FIG. You may make it acquire the effect similar to the case where such a function is used.

Furthermore, the conversion function f may be, for example, as shown in FIG. 40 as described above. However, the conversion function f may be associated with the BEP. For example, the absolute difference e between adjacent pixels is a threshold value. When th1 = BEP / b1 or more, f (e) = A (A is a constant). When the absolute difference e between adjacent pixels is equal to or less than the threshold th2 = BEP / b2, f (e) = B (B is And the difference absolute value e between adjacent pixels is larger than the threshold th2 = BEP / b2 and smaller than the threshold th1 = BEP / b1, f (e) = (B−A) · (e−th1) / You may make it be (th2-th1) + A. The thresholds th1 and th2 only need to use the BEP tendency. For example, the threshold th1 = (BEP) ² / b1, th2 = (BEP) ² / b2, or the threshold th1 = (√BEP) / b1, You may make it utilize th2 = (√BEP) / b2.

  Now, the description returns to the flowchart of FIG.

  When the BFP is obtained by the process of step S832, in step S833, the feature amount combining unit 933 combines the BEP supplied from the BEP extracting unit 931 and the BFP supplied from the BFP extracting unit 932, This is supplied to the wide-area artificial image degree calculation unit 912 as a wide-area feature amount.

In step S834, the wide-area artificial degree-of-gravity calculation unit 912 calculates a wide-area artificial degree of art Art _b by executing a wide-area artificial degree-of-gravity calculation process based on the wide-area feature amount supplied from the wide-area feature amount extraction unit 911. Then, the image is supplied to the artificial image degree generation unit 913.

  Here, with reference to the flowchart of FIG. 41, the wide-area artificial image degree calculation processing by the wide-area artificial image degree calculation unit 912 of FIG.

  In step S891, the feature amount separation unit 951 obtains the wide area feature amount supplied from the wide area feature amount extraction unit 911, separates the BEP and the BFP, and supplies them to the wide area boundary line comparison unit 952. .

  In step S <b> 892, the wide area boundary line comparison unit 952 reads information on the wide area line from the wide area line memory 953.

  In step S893, the wide area boundary line comparison unit 952 uses the unprocessed pixel as the target pixel, extracts the BEP and BFP of the target pixel, and plots them on the two-dimensional plane of the BEP-BFP. Compare the positional relationship with the boundary line.

  Here, the wide boundary line is statistically obtained by plotting BEP and BFP extracted from multiple artificial images and natural images as (BEP, BFP) in a two-dimensional space consisting of BEP axes and BFP axes. Is a boundary generated from the distribution of. There are two types of wide boundary lines, wide area artificial image boundary lines and wide area natural image boundary lines. Among these, the wide-area artificial image boundary line is a boundary line between a region where only the wide-area feature amount of the artificial image is plotted and a region where the wide-area feature amount of the artificial image and the natural image is mixed. The wide area natural image boundary line is a boundary line between an area where only the wide area feature amount of the natural image is plotted and an area where the wide area feature amount of the artificial image and the natural image is mixed. Therefore, as shown in FIG. 42, the area composed of the BEP axis and the BFP axis is based on the wide area artificial image boundary line and the wide area natural image boundary line, and the wide area artificial image area, the wide area natural image area, and the artificial image and natural image area. The image is divided into three types of areas, that is, a wide mixed area of images. Here, in FIG. 42, the wide-area artificial image boundary line is the curve L1 in the figure, and the wide-area natural image boundary line is the curve L2 in the figure. Hereinafter, the curves L1 and L2 are also referred to as a wide-area artificial image boundary line L1 and a wide-area natural image boundary line L2 as appropriate. In FIG. 42, the area above the wide area artificial image boundary line L1 is a wide area artificial image area, and the area between the wide area artificial image boundary line L1 and the wide area natural image boundary line L2 is a wide area mixed area. Yes, the area below the wide area natural image boundary L2 is the wide area natural image area.

  In step S894, the wide area boundary line comparison unit 952 determines whether or not the position where the BEP and BFP of the target pixel are plotted is the wide area artificial image area. For example, when the position where the BEP and BFP of the target pixel are plotted is the position B1 in FIG. 42, the wide area boundary comparison unit 952 determines that the pixel belongs to the wide area artificial image area in step S895. The calculation unit 954 is notified.

In step S896, the internal division calculation unit 954 obtains a notification from the wide area boundary line comparison unit 952 that the wide area feature amount of the pixel of interest belongs to the wide area artificial image area, and sets the wide area art degree Art _b to 1. The image is supplied to the artificial image degree generation unit 913, and the process proceeds to step S902.

  In step S894, for example, when the position where the BEP and BFP of the pixel of interest are plotted is not the position B1 in FIG. 42 but the position B2 or B3, the wide area boundary comparison unit 952 places the pixel in the wide area artificial image area. The process proceeds to step S897 because it does not belong.

  In step S897, the wide area boundary comparison unit 952 determines whether or not the position where the BEP and BFP of the pixel of interest are plotted is the wide natural image area. For example, when the position where the BEP and BFP of the pixel of interest are plotted is the position B3 in FIG. 42, in step S898, the wide area boundary line comparison unit 952 determines that the pixel belongs to the wide natural image area. The calculation unit 954 is notified.

In step S899, the internal division calculation unit 954 obtains a notification from the wide area boundary line comparison unit 952 that the wide area feature amount of the pixel of interest belongs to the wide area natural image area, and sets the wide area artificial art degree Art _b to 0. The image is supplied to the artificial image degree generation unit 913, and the process proceeds to step S902.

  In step S897, for example, when the position where the BEP and BFP of the pixel of interest are plotted is not the position B3 in FIG. 42 but the position B2, the wide area boundary comparison unit 952 determines that the pixel is also in the wide area artificial image area. Assuming that the image does not belong to the natural image area, the process proceeds to step S900.

  In step S900, the wide area boundary comparison unit 952 calculates the internal division ratio between the wide area artificial image boundary line L1 and the wide area natural image boundary line L2 while the pixel belongs to the wide area mixed area. 954 is notified. That is, in the case of FIG. 42, since the ratio of the position B2 to the distance to the wide-area artificial image boundary line L1: distance to the wide-area natural image boundary line L2 is 1-q: q, this information is calculated internally. Section 954 is notified.

In step S <b> 901, the internal division calculation unit 954 obtains the internal division ratio notification together with the fact that the wide area feature amount of the pixel of interest from the wide area boundary line comparison unit 952 belongs to the wide area mixed area. q is supplied to the artificial-image-ratio generating unit 913 as the wide-area artificial image degree Art _b , and the process proceeds to step S902.

  In step S902, the wide area boundary comparison unit 952 determines whether or not an unprocessed pixel exists, and the unprocessed pixel, that is, the wide area feature amount is a wide area artificial image area, a wide area natural image area, or Compared to which of the wide-area mixed area, it is determined whether or not a wide-area artificial image degree is required, and if it is determined that the wide-area artificial image degree is not required for all pixels, the process is Return to step S893.

On the other hand, when the wide-area artificial image degree Art _b is obtained for all the pixels, the process ends.

Through the above process, wide area feature values consisting of BEP and BFP are read for all pixels, plotted in the BEP-BFP space, and statistically obtained from the distribution of multiple artificial images and natural images. Based on the positional relationship between the plot position (BEP, BFP) in the BEP-BFP space and the wide-area artificial image boundary line and wide-area natural image boundary line by comparing with the boundary line and wide-area natural image boundary line The image degree Art _b is calculated. In the above description, an example in which the internal ratio q is set as the wide-area artificial image degree Art _b when it exists in the wide-area mixed area has been described. However, the internal ratio should be a value that reflects the tendency as a trend. Therefore, as the wide-area artificial image, for example, q ² or √q may be used.

  Now, the description returns to the flowchart of FIG.

In step S835, PNDP extracting portion 971 of the narrow-range feature extraction unit 91 4 performs the PNDP extraction processing, feature synthesizer extracts the PNDP for each pixel from the HD image supplied from the output phase converter 112 933.

  Here, the PNDP extraction processing by the PNDP extraction unit 971 in FIG. 31 will be described with reference to the flowchart in FIG.

  In step S911, the buffer 1051 temporarily stores the HD image supplied from the output phase converter 112.

  In step S912, the reference pixel extraction unit 1052 sets an unprocessed pixel as a target pixel from the buffer 1051, and reads a reference pixel set corresponding to the target pixel in step S913 to extract a long tap. Supplied to the unit 1053.

  In step S914, the long tap extraction unit 1053 extracts the long tap from the reference pixel, and in step S915, the long tap pixel value is accumulated in the pixel value accumulation unit 1054, and further supplied to the SNDP extraction unit 972. Here, the long tap is, for example, a pixel in a predetermined area set in association with the target pixel included in the reference pixel. For example, as shown in FIG. M pixels × m pixels. In FIG. 44, a range of n pixels × n pixels (n> m) indicates a range of reference pixels. In FIG. 44, the white circle on the coordinates in the two-dimensional space in the x and y directions centered on the target pixel P (0, 0) of the black circle in the image represents the long tap P (i, j). A pixel within a range surrounded by m × m dotted lines with the pixel of interest P (0, 0) as the center is represented by a long tap P (i, j) (− (m−1) / 2 ≦ It is shown that i ≦ (m−1) / 2, − (m−1) / 2 ≦ j ≦ (m−1) / 2: i and j are integers.

In step S916, the pixel value storage unit 1054 determines whether there is an unprocessed long tap. If there is an unprocessed long tap, the process returns to step S915. That is, the processing in steps S915 and S916 is repeated for all long taps.

If it is determined in step S916 that there are no unprocessed long taps, that is, pixel values are extracted for all long taps and all are stored in the pixel value storage unit 1054, the pixel values are determined in step S917. The accumulation unit 1054 notifies the maximum value / minimum value extraction unit 1055 that the processing has been completed for all the long taps. In response to this, the maximum value / minimum value extraction unit 1055 extracts the maximum value and the minimum value from the pixel values stored in the pixel value storage unit 1054, and outputs them to the PNDP calculation unit 1056. Supply.

  In step S918, the PNDP calculation unit 1056 calculates a long tap pixel value dynamic range, which is a value obtained by subtracting the minimum value from the maximum value supplied from the maximum value / minimum value extraction unit 1055, as the PNDP for the pixel of interest, This is supplied to the feature value composition unit 933.

  In step S919, the reference pixel extraction unit 1052 determines whether PNDP has been obtained for all the pixels in the image stored in the buffer 1051. In step S919, when the PNDP has not been obtained for all the pixels of the image, the process returns to step S912. That is, the processes of steps S912 to S919 are repeated until PNDP is obtained for all the pixels of the image stored in the buffer 1051. If it is determined in step S919 that PNDP has been obtained for all pixels in the image, the PNDP extraction process ends.

  Through the above processing, PNDP, which is a feature amount representing a narrow edge in an image, is obtained. That is, PNDP expresses the pixel value dynamic range within the long tap for the pixel of interest, and if the edge exists in the narrow area to which the pixel of interest belongs, it becomes a large value, and conversely, if there is no edge, It will take a small value.

  Now, the description returns to the flowchart of FIG.

  When the PNDP is extracted from the image by the PNDP extraction process in step S835, the SNDP extraction unit 972 of the narrow area feature quantity extraction unit 914 executes the SNDP extraction process and is supplied from the output phase conversion unit 112 in step S836. SNDP is extracted for each pixel from the HD image and supplied to the feature amount synthesis unit 933.

  Here, the SNDP extraction processing by the SNDP extraction unit 972 in FIG. 34 will be described with reference to the flowchart in FIG.

  In step S941, the buffer 1071 temporarily stores the HD image supplied from the output phase converter 112.

  In step S942, the reference pixel extraction unit 1072 sets an unprocessed pixel as a target pixel from the buffer 1051, and reads out a reference pixel set corresponding to the target pixel in step S943 to extract a short tap. To the unit 1073.

  In step S944, the short tap extraction unit 1073 extracts a short tap group from the reference pixel based on the reference pixel and the long tap information supplied from the PNDP extraction unit 971. Here, the short tap is a pixel included in the long tap and is a plurality of pixels including the pixel of interest. For example, among the reference pixels, the pixel included in the long tap is the pixel of interest. This is a pixel in a predetermined area set in association with each other. For example, as shown by a pattern A in FIG. 46, 5 pixels including four pixels adjacent to each other in the horizontal direction and the vertical direction centering on the target pixel. A short tap ST1 composed of pixels, or a short tap ST2 composed of nine pixels of 3 pixels × 3 pixels in the horizontal and vertical directions with the target pixel as the lower right corner as shown in the pattern B, or a pattern C As shown in the figure, the pixel of interest is a short tap ST3 consisting of 9 pixels of 3 pixels × 3 pixels in the horizontal direction and the vertical direction with the pixel of interest at the upper left corner. Or a short tap ST4 consisting of 9 pixels of 3 pixels × 3 pixels in the horizontal and vertical directions, or 3 pixels in the horizontal and vertical directions with the target pixel as the upper right corner as shown by the pattern E A short tap ST5 composed of 9 pixels of 3 pixels may be used. A plurality of patterns of short taps is referred to as a short tap group. Therefore, when the short taps ST1 to ST5 in FIG. 46 are used as short taps, it can be said that a short tap group composed of five patterns of short taps is configured. In FIG. 46, a circle indicates a pixel, a black circle indicates a target pixel, and pixels within a range surrounded by a solid line are pixels constituting a short tap. Is shown.

  In step S945, the short tap extraction unit 1073 extracts the pixel value of one unprocessed short tap and stores the pixel value in the pixel value storage unit 1074.

  In step S946, the short tap extraction unit 1073 determines whether there is an unprocessed short tap. If there is an unprocessed short tap, the process returns to step S945. That is, the processes in steps S945 and S946 are repeated for all the pixels included in one short tap.

  If it is determined in step S946 that there are no unprocessed short taps, that is, all pixel values of one short tap are all stored in the pixel value storage unit 1074, the pixel value is determined in step S947. The extraction unit 1074 notifies the maximum value / minimum value extraction unit 1075 that the process of accumulating all the pixels of one short tap is completed. The maximum value / minimum value extraction unit 1075 extracts the value that becomes the maximum value and the value that becomes the minimum value from the pixel values of one short tap stored in the pixel value storage unit 1074, and sends them to the DR calculation unit 1076. Supply.

  In step S948, the DR calculation unit 1076 calculates the pixel value dynamic range of one short tap, which is a value obtained by subtracting the minimum value from the maximum value of the pixel value supplied from the maximum value / minimum value extraction unit 1075, for the target pixel. The pixel value DR (pixel value dynamic range) is calculated and stored in the DR storage unit 1077 in step S949.

  In step S950, the short tap extraction unit 1073 determines whether or not an unprocessed short tap group remains. If it is determined that the unprocessed short tap group remains, the process returns to step S944. That is, the processes in steps S944 to S950 are repeated until it is determined that DR has been obtained for all the short tap groups. Therefore, for example, when the short tap group has the five patterns shown in FIG. 46, the processes in steps S944 to S950 are repeated five times.

  In step S950, when all the short tap groups have been processed, that is, when DR is obtained for all the short taps, in step S951, the short tap extraction unit 1073 informs the SNDP selection unit 1078 to that effect. Notice. Further, the SNDP selection unit 1078 compares the DRs of the respective short tap groups stored in the DR storage unit 1077, selects the minimum DR as the SNDP, and supplies the SNDP to the feature amount combining unit 973.

  In step S952, the reference pixel extraction unit 1072 determines whether SNDP has been obtained for all the pixels in the image stored in the buffer 1071. In step S952, when SNDP is not calculated | required about all the pixels of an image, a process returns to step S942. That is, the processes in steps S942 to S952 are repeated until SNDPs are obtained for all the pixels of the image stored in the buffer 1071. If it is determined in step S952 that the SNDP has been obtained for all the pixels in the image, the SNDP extraction process ends.

  Through the above processing, the SNDP that is a feature amount representing the degree of the narrow flat portion to which each pixel belongs in the image is obtained. That is, SNDP is the flatness of the flattest region among the regions represented by short taps in the long taps by the minimum value of the pixel value dynamic range of a plurality of short taps in the long taps for the target pixel. Since the degree is expressed, the flat portion included in the narrow area to which the pixel of interest belongs is flatter, and the smaller the value, the smaller the flat portion, the larger the flat portion.

  Now, the description returns to the flowchart of FIG.

  When the SNDP is obtained by the processing in step S836, in step S837, the feature amount combining unit 973 combines the PNDP supplied from the PNDP extracting unit 971 and the SNDP supplied from the SNDP extracting unit 972, The narrow area feature quantity is supplied to the narrow area artificial image degree calculation unit 915.

In step S838, the narrow area artificial degree-of-gravity calculation unit 915 executes the narrow area artificial degree-of-gravity calculation process based on the narrow area feature amount supplied from the narrow area feature amount extraction unit 914, and the narrow area artificial degree of view. Art _n is calculated and supplied to the artificial image degree generation unit 913.

  Here, with reference to the flowchart of FIG. 47, the narrow area artificial image degree calculation processing by the narrow area artificial image degree calculation unit 915 of FIG. 30 will be described.

  In step S971, the feature amount separation unit 991 acquires the narrow region feature amount supplied from the narrow region feature amount extraction unit 914, separates the PNDP and the SNDP, and separates them from the narrow region boundary line comparison unit 992. To supply.

  In step S <b> 972, the narrow area boundary comparison unit 992 reads the narrow area boundary line information from the narrow area boundary line memory 993.

  In step S973, the narrow boundary line comparison unit 992 uses an unprocessed pixel as a pixel of interest, extracts the PNDP and SNDP of the pixel of interest, and plots them on a two-dimensional plane of PNDP-SNDP. Compare the positional relationship with the narrow boundary line.

  Here, the narrow boundary line is a statistic obtained by plotting PNDP and SNDP extracted from a plurality of artificial images and natural images as (PNDP, SNDP) in a two-dimensional space consisting of PNDP axes and SNDP axes. A boundary line generated from the above distribution. The narrow boundary line includes two types of boundary lines: a narrow artificial image boundary line and a narrow natural image boundary line. Among these, the narrow-area artificial image boundary line is the boundary line between the area where only the narrow-area feature amount of the artificial image is plotted and the area where the narrow-area feature amount of the artificial image and the natural image is mixed. It is. The narrow natural image boundary line is the boundary line between the area where only the narrow area feature amount of the natural image is plotted and the area where the narrow area feature amount of the artificial image and the natural image is mixed. is there. Therefore, the region composed of the PNDP axis and the SNDP axis is, as shown in FIG. 48, based on the narrow artificial image boundary and the narrow natural image boundary, and the narrow artificial image region, the narrow natural image region, and It is divided into three types of areas, which are a narrow-area mixed area of an artificial image and a natural image. Here, in FIG. 48, the narrow natural image boundary line is the curve L11 in the figure, and the narrow artificial image boundary line is the curve L12 in the figure. Hereinafter, the curves L11 and L12 are also referred to as a narrow natural image boundary line L11 and a narrow artificial image boundary line L12 as appropriate. In FIG. 48, the region above the narrow natural image boundary line L11 is the narrow natural image region, and the region between the narrow natural image boundary line L11 and the narrow artificial image boundary line L12 is The narrow area mixed area, and the area below the narrow area artificial image boundary L12 is the narrow area artificial image area.

  In step S974, the narrow area boundary comparison unit 992 determines whether or not the position where the PNDP and SNDP of the target pixel are plotted is the narrow area artificial image area. For example, when the position where the PNDP and SNDP of the target pixel are plotted is the position N3 in FIG. 48, in step S975, the narrow boundary line comparison unit 992 indicates that the pixel belongs to the narrow artificial image area. Notify the internal calculation unit 994.

In step S976, when the internal division calculation unit 994 obtains a notification from the narrow area boundary line comparison unit 992 that the narrow area feature amount of the target pixel belongs to the narrow area artificial image area, the narrow area artificial degree of art Art _{n is set} to 1 and is supplied to the artificial degree-of-image generation unit 913, and the process proceeds to step S982.

  In step S974, for example, when the position where the PNDP and SNDP of the pixel of interest are plotted is not the position N3 in FIG. 48 but the position N2 or N1, the narrow boundary line comparison unit 992 has the pixel narrow-area artificial image. The processing proceeds to step S977, assuming that it does not belong to the area.

  In step S977, the narrow area boundary comparison unit 992 determines whether or not the position where the PNDP and SNDP of the target pixel are plotted is the narrow natural image area. For example, if the position where the PNDP and SNDP of the pixel of interest are plotted is the position N1 in FIG. 48, the narrow boundary line comparison unit 992 confirms that the pixel belongs to the wide natural image area in step S978. The minute calculation unit 994 is notified.

In step S979, when the internal division calculation unit 994 obtains notification from the narrow area boundary line comparison unit 992 that the narrow area feature amount of the target pixel belongs to the narrow area natural image area, the narrow area artificial degree of art Art _{n is set} to 0 and supplied to the artificial-image-degree generation unit 913, and the process proceeds to step S982.

  In step S977, for example, when the position where the PNDP and SNDP of the pixel of interest are plotted is not the position N1 in FIG. 48 but the position N2, the narrow boundary line comparison unit 992 places the pixel in the narrow artificial image area. Are not belonging to the narrow natural image area, and the process proceeds to step S980.

  In step S980, the narrow area boundary comparison unit 992 calculates the internal ratio between the narrow area natural image boundary line L11 and the narrow area artificial image boundary line L12 while the pixel belongs to the narrow area mixed region. Notify the internal calculation unit 994. That is, in the case of FIG. 48, since the ratio of the position N2 to the distance to the narrow natural image boundary line L11: the distance to the narrow artificial image boundary line L12 is p: 1−p. The minute calculation unit 994 is notified.

In step S981, the internal division calculation unit 994 obtains the notification of the internal division ratio together with the fact that the narrow region feature amount of the target pixel from the narrow region boundary line comparison unit 992 belongs to the narrow region mixed region. The internal ratio p is supplied to the artificial-image-ratio generating unit 913 as the narrow-area artificial image degree Art _n , and the process proceeds to step S982.

  In step S982, the narrow area boundary comparison unit 992 determines whether or not an unprocessed pixel exists, and the unprocessed pixel, that is, the narrow area feature amount is a narrow area artificial image area or a narrow area natural image. It is determined whether it belongs to a region or a narrow-area mixed region, and whether or not a narrow-area artificial image degree is required. If determined, the process returns to step S973.

On the other hand, when the narrow-area artificial image degree Art _n is obtained for all the pixels, the process ends.

Through the above processing, the narrow area feature quantity consisting of PNDP and SNDP is read for all pixels, plotted in the PNDP-SNDP space, and statistically obtained from the distribution of multiple artificial images and natural images Compared with the artificial image boundary line and the narrow natural image boundary line, the positional relationship between the plot position (PNDP, SNDP) in the PNDP-SNDP space and the narrow artificial image boundary line and the narrow natural image boundary line Based on this, the narrow-area artificial image degree Art _n is calculated. In the above description, the example in which the internal ratio p is set as the narrow-area artificial image degree Art _n when it exists in the narrow mixed area has been described. However, the internal ratio should be a value that reflects the tendency. Therefore, the wide-area artificial image may be, for example, p ² or √p.

  Now, the description returns to the flowchart of FIG.

In step S <b> 839, the artificial image creation unit 913 performs the wide-area artificial image Art _b supplied from the wide-area artificial image degree calculation unit 912 and the narrow-area artificial image degree supplied from the narrow-area artificial image degree calculation unit 915. Based on Art _n , the following equation (19) is calculated to generate an artificial art degree Art and supply it to the image processing unit 113.

  Here, α, β, and γ are constants. Further, since the artificial art degree Art is a value set on the assumption that 0 ≦ Art ≦ 1, when the calculation result of Expression (19) indicates that the artificial art degree Art is 1 or more, the artificial art degree generator In step 913, the artificial art degree Art is clipped to 1 and supplied to the image processing unit 113.

  With the above processing, in the processing of step S7 in FIG. 2 described above, the synthesis unit 133 calculates each of the natural high quality images supplied from the natural image prediction unit 131 by, for example, calculating the following equation (20). The pixel value of the pixel and the pixel value of each pixel of the artificial high quality image supplied from the artificial image predicting unit 132 are combined at a ratio according to the artificial image degree.

Here, Pix is a pixel value of each pixel of the high-quality image that is finally synthesized and generated, Pix _nat is a natural image supplied from the natural image prediction unit 131, and Pix _art is It is an artificial image supplied from the artificial image prediction unit 132, and Art is a coefficient of the artificial image degree.

  According to the above, as a natural image in an image, a pixel that has been subjected to image processing as an artificial image and a pixel that has been subjected to image processing as a natural image are synthesized according to the degree of artificial image. It is possible to distinguish an area including the element and an area including an element as an artificial image, and to perform optimum processing on each area.

  In the above, as in the BEP extraction process described with reference to the flowchart of FIG. 38, the difference value between the maximum value and the minimum value of the difference value between the reference pixel and the target pixel is obtained, thereby obtaining the difference dynamic of the reference pixel. The example of extracting the BEP by calculating the range has been described. However, since it is only necessary to obtain a part where the change in the pixel value of the reference pixel is large, other methods may be used, and the pixel value of the pixel included in the reference region may be determined. The pixel value dynamic range may be used. If the pixel value dynamic range is used as the BEP, the BEP value may change greatly depending on whether or not the reference pixel near the boundary of the reference region includes the pixel belonging to the edge region. . Therefore, as described above, for BEP, a more preferable result can be expected when the differential dynamic range is weighted according to the distance.

  In addition, as the BEP, for example, the absolute difference values of the pixel values between adjacent pixels of each reference pixel may be rearranged in ascending order and may be obtained using the upper absolute difference value.

  FIG. 49 shows the BEP extraction in which the absolute value of the pixel value between adjacent pixels of each reference pixel is rearranged in ascending order and is obtained as a BEP that represents the presence of the edge of the reference pixel using the upper absolute difference value. A configuration example of the unit 931 is shown.

  The BEP extraction unit 931 in FIG. 49 includes a buffer 1101, a reference pixel extraction unit 1102, an adjacent pixel difference absolute value calculation unit 1103, a rearrangement unit 1104, an upper extraction unit 1105, and a BEP calculation unit 1106.

The buffer 1101 is basically the same as the buffer 1011 in FIG. 31, temporarily stores the HD image supplied from the output phase converter 112, and sequentially supplies it to the reference pixel extractor 1102 as necessary. . Reference-pixel extracting portion 1102 is the same as the basic reference-pixel extracting portion 1012 in FIG. 31, are sequentially supplied to the adjacent-pixel difference absolute value calculator 1103 reads out reference pixels for each subject pixel.

  The inter-adjacent pixel difference absolute value calculation unit 1103 calculates the absolute difference value of the pixel value between pixels adjacent in the vertical and horizontal directions among the reference pixels supplied from the reference pixel extraction unit 1102 and supplies the difference absolute value to the rearrangement unit 1104. To do. The rearrangement unit 1104 rearranges the absolute difference values between adjacent pixels of the reference pixel supplied from the adjacent pixel difference absolute value calculation unit 1103 in ascending order and supplies the rearrangement absolute values to the upper extraction unit 1105.

  The upper extraction unit 1105 extracts values of the upper N1 to N2 ranks based on the information rearranged in ascending order of the absolute value of the difference between adjacent pixels supplied from the rearrangement unit 1104, and performs BEP calculation. To the unit 1106. The BEP calculation unit 1106 uses, as the BEP, the average value of the highest N1 to N2 rank values based on the information supplied from the higher extraction unit 1105 and rearranged in the ascending order of the difference value between adjacent pixels. It is obtained and supplied to the feature value composition unit 933.

  Here, the BEP extraction processing by the BEP extraction unit 931 in FIG. 49 will be described with reference to the flowchart in FIG.

  In step S1001, the buffer 1101 temporarily stores the HD image supplied from the output phase converter 112.

  In step S1002, the reference pixel extraction unit 1102 sets an unprocessed pixel as a target pixel from the buffer 1101, and reads out a reference pixel set corresponding to the target pixel in step S1003, so that it is not between adjacent pixels. This is supplied to the difference absolute value calculation unit 1103.

  In step S <b> 1004, the inter-adjacent-pixel difference absolute value calculation unit 1103 calculates a difference absolute value of pixel values between adjacent pixels (including the target pixel) of the unprocessed reference pixel, and supplies the calculated difference to the rearrangement unit 1104.

  In step S <b> 1005, the rearrangement unit 1104 accumulates the adjacent pixel difference absolute value supplied from the adjacent pixel difference absolute value calculation unit 1103.

  In step S1006, the adjacent pixel difference absolute value calculation unit 1103 determines whether or not the difference absolute value between all adjacent pixels in the reference pixel has been obtained. For example, the difference absolute value between all adjacent pixels is obtained. If it is determined that it is not, the process returns to step S1004.

  That is, the processes in steps S1004 to S1006 are repeated until the absolute difference value between all adjacent pixels is obtained. In step S1004, when the absolute difference values between all adjacent pixels are obtained and it is determined that all the difference pixels are stored in the rearrangement unit 1104, in step S1007, the rearrangement unit 1104 determines the difference between adjacent pixels. The absolute value calculation results are rearranged in ascending order and supplied to the upper extraction unit 1105. That is, the rearrangement unit 1104 rearranges the adjacent pixel difference absolute values in ascending order as shown in FIG. In FIG. 51, the horizontal axis indicates the ranking, and the vertical axis indicates the adjacent pixel difference absolute value.

  In step S <b> 1008, the upper extraction unit 1105 extracts the adjacent pixel difference absolute values in the higher N1 to N2 ranks from the adjacent pixel difference absolute values rearranged in the ascending order supplied from the rearrangement unit 1104. Then, the data is supplied to the BEP calculation unit 1106. That is, for example, in the case of FIG. 51, it is extracted from the absolute value of the difference between adjacent pixels of the highest N1 to N2 ranks in the figure.

  In step S <b> 1009, the BEP calculation unit 1106 calculates the average value of the difference values between adjacent pixels in the highest N1 to N2 ranks as BEP, and outputs the BEP to the feature amount synthesis unit 933.

  In step S1010, the reference pixel extraction unit 1102 determines whether or not a BEP has been obtained for all the pixels in the image stored in the buffer 1101. In step S1010, when BEP is not calculated | required about all the pixels of an image, a process returns to step S1002. That is, the processes in steps S1002 to S1010 are repeated until BEP is obtained for all the pixels of the image stored in the buffer 1101. If it is determined in step S1010 that the BEP has been obtained for all the pixels in the image, the BEP extraction process ends.

  Through the above processing, a BEP that is a feature amount that represents a wide edge in an image is obtained. That is, BEP is the average value of the highest N1 to N2 ranks of the absolute difference value between adjacent pixels, and a relatively large value among the absolute difference values between adjacent pixels of the reference pixel represents a wide-area edge. Extracted as a quantity.

  In the above description, the example in which the average value of the top N1 to N2 ranks is set as BEP has been described. However, it is only necessary to obtain a wide-area feature amount using the absolute value of the difference between adjacent pixels in the reference pixel. The calculation method may be other than the above method. For example, the product sum may be used by adding a weight to each absolute difference value between adjacent pixels instead of the average value, and further, as shown in FIG. As described above, the Nth order values from the top N1 th to the N2 th order may be used as BEP as they are.

  In the above, as in the BFP extraction process described with reference to the flowchart of FIG. 38, the difference absolute value between adjacent pixels of the reference pixel is compared with a predetermined threshold th. Although an example in which a predetermined value is set by a function and the sum of all reference pixels is set to BFP has been described, it is only necessary to obtain a small change in the pixel value of the reference pixel. The difference absolute values of the pixel values between adjacent pixels of each reference pixel may be rearranged in ascending order, and may be obtained using the lower difference absolute value.

  FIG. 52 shows a BFP in which absolute values of pixel values between adjacent pixels of each reference pixel are rearranged in ascending order, and a BFP expressing the presence of a flat portion of the reference pixel is obtained using the lower absolute difference value. A configuration example of the extraction unit 932 is shown.

BFP extracting portion 932 of FIG. 52, the buffer 1121, the reference-pixel extracting portion 1122, an adjacent-pixel difference absolute value calculating unit 1123, rearrangement unit 1124, and a lower position extracting unit 1125, and the BFP calculator 1126.

The buffer 1121 is basically the same as the buffer 1031 in FIG. 32, temporarily stores the HD image supplied from the output phase converter 112, and sequentially supplies it to the reference pixel extractor 1122 as necessary. . Reference-pixel extracting portion 1122 is of basically the same as the reference-pixel extracting unit 1032 in FIG. 32, are sequentially supplied to the weighting adjacent-pixel difference absolute value calculator 1123 reads out reference pixels for each subject pixel .

  The inter-adjacent pixel difference absolute value calculation unit 1123 obtains the absolute difference value of the pixel value between pixels adjacent in the vertical and horizontal directions among the reference pixels supplied from the reference pixel extraction unit 1122 and supplies the difference absolute value to the rearrangement unit 1124. To do. The rearrangement unit 1124 rearranges the absolute difference values between adjacent pixels of the reference pixel supplied from the adjacent pixel difference absolute value calculation unit 1123 in ascending order and supplies the rearrangement unit 1124 to the lower extraction unit 1125.

  The lower order extraction unit 1125 extracts the values of the lower order n1 to n2 based on the information rearranged in ascending order of the difference value between adjacent pixels supplied from the rearrangement unit 1124, and performs BFP calculation. Supplied to the unit 1126. Based on the information supplied from the lower extraction unit 1125 and rearranged in the ascending order of the difference value between adjacent pixels, the BFP calculation unit 1126 uses the average value of the values of the lower n1 to n2 ranks as the BFP. It is obtained and supplied to the feature value composition unit 933.

  Here, the BFP extraction processing by the BFP extraction unit 932 in FIG. 52 will be described with reference to the flowchart in FIG.

  In step S1031, the buffer 1121 temporarily stores the HD image supplied from the output phase converter 112.

  In step S1032, the reference pixel extraction unit 1122 sets an unprocessed pixel as a target pixel from the buffer 1121, and reads out a reference pixel set corresponding to the target pixel in step S1033 so that the pixel between adjacent pixels is read. This is supplied to the difference absolute value calculation unit 1123.

  In step S <b> 1034, the inter-adjacent-pixel difference absolute value calculation unit 1123 calculates an absolute difference value of pixel values between adjacent pixels of the unprocessed reference pixel (including the target pixel), and supplies the calculated difference to the rearrangement unit 1124.

  In step S <b> 1035, the rearrangement unit 1124 accumulates the adjacent pixel difference absolute value supplied from the adjacent pixel difference absolute value calculation unit 1123.

  In step S1036, the adjacent pixel difference absolute value calculation unit 1123 determines whether or not the difference absolute value between all adjacent pixels in the reference pixel has been obtained. For example, the difference absolute value between all adjacent pixels is obtained. If it is determined that it is not, the process returns to step S1034.

  That is, the processes in steps S1034 to S1036 are repeated until the absolute difference value between all adjacent pixels is obtained. In step S1034, if the absolute difference values between all adjacent pixels are obtained and it is determined that all are stored in the rearrangement unit 1124, in step S1037, the rearrangement unit 1124 determines the difference between adjacent pixels. The absolute value calculation results are rearranged in ascending order and supplied to the lower extraction unit 1125. That is, the rearrangement unit 1124 rearranges the adjacent pixel difference absolute values in ascending order as shown in FIG. In FIG. 54, the horizontal axis indicates the ranking, and the vertical axis indicates the adjacent pixel difference absolute value.

  In step S1038, the lower-order extraction unit 1125 extracts the difference absolute values between adjacent pixels ranked n1 to n2 from the lower-order adjacent pixel difference absolute values rearranged in ascending order supplied from the rearrangement unit 1124. Then, the data is supplied to the BFP calculation unit 1126. That is, for example, in the case of FIG. 54, it is extracted from the absolute value of the difference between adjacent pixels in the lower order n1 to n2 in the figure.

  In step S <b> 1039, the BFP calculation unit 1126 calculates the average value of the difference values between adjacent pixels of the lowest n1 to n2 ranks as BFP, and outputs it to the feature amount synthesis unit 933.

  In step S1040, the reference pixel extraction unit 1122 determines whether or not BFP has been obtained for all the pixels in the image stored in the buffer 1121. In step S1040, when BFP is not calculated | required about all the pixels of an image, a process returns to step S1032. That is, the processes in steps S1032 to S1040 are repeated until BFP is obtained for all the pixels of the image stored in the buffer 1121. If it is determined in step S1040 that the BFP has been obtained for all the pixels in the image, the BFP extraction process ends.

  Through the above processing, a BFP that is a feature amount representing a wide flat portion in an image is obtained. That is, BFP is an average value in the order of the lowest n1 to n2 of the absolute difference value between adjacent pixels, and a relatively small value among the absolute difference values between adjacent pixels of the reference pixel represents a wide flat portion. It is extracted as a feature value.

  In the above description, an example in which the average value of the lowest n1 to n2 ranks is set to BFP has been described. However, since a wide-area feature amount using the absolute value of the difference between adjacent pixels in the reference pixel may be obtained. The method of obtaining may be other than the above method. For example, a product sum may be used by adding a weight to each absolute difference value between adjacent pixels instead of the average value, and further, as shown in FIG. As described above, the values of the nth ranks before and after the lowest n1th to n2nd ranks may be used as BFP as they are.

  In the above description, the PNDP has been described with respect to the example obtained from the long tap pixel value dynamic range included in the reference pixel as in the PNDP extraction process described with reference to the flowchart of FIG. The pixel value dynamic range may be used.

  FIG. 55 shows a configuration example of the PNDP extraction unit 971 in which the PNDP is obtained by using the pixel value dynamic range of the reference pixel.

  55 includes a buffer 1141, a reference pixel extraction unit 1142, a pixel value extraction unit 1143, a pixel value storage unit 1143, a maximum / minimum value extraction unit 1144, and a PNDP calculation unit 1145.

  The buffer 1141 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1142 as necessary. The reference pixel extraction unit 1142 sequentially reads out the reference pixels for each pixel of interest and causes the pixel value storage unit 1143 to store the pixel values.

  The maximum value / minimum value extraction unit 1144 extracts the maximum value and the minimum value from the pixel values of all the reference pixels accumulated in the pixel value accumulation unit 1143 and supplies the extracted value to the PNDP calculation unit 1145.

  The PNDP calculation unit 1145 obtains the pixel value dynamic range of the reference pixel from the value obtained by subtracting the minimum value from the maximum value of the pixel values of all reference pixels supplied from the maximum value / minimum value extraction unit 1144, and calculates the PNDP of the target pixel. Output as.

  Next, the PNDP extraction process by the PNDP extraction unit 971 in FIG. 55 will be described with reference to the flowchart in FIG.

  In step S1051, the buffer 1141 temporarily stores the HD image supplied from the output phase converter 112.

  In step S1052, the reference pixel extraction unit 1142 sets an unprocessed pixel as a target pixel from the buffer 1141, and reads out a reference pixel set corresponding to the target pixel in step S1053 and outputs the pixel value. Is stored in the pixel value storage unit 1143 in step S1054.

  In step S1055, the reference pixel extraction unit 1142 determines whether or not there is an unprocessed reference pixel, that is, whether or not the pixel values of all reference pixels are accumulated in the pixel value accumulation unit 1143, and the unprocessed reference pixel is determined. If there is a reference pixel, the process returns to step S1054. That is, the processing in steps S1054 and S1055 is repeated for all reference pixels.

  If it is determined in step S1055 that there is no unprocessed reference pixel, that is, pixel values are extracted for all reference pixels and all are stored in the pixel value storage unit 1143, in step S1056, the reference pixels The extraction unit 1142 notifies the maximum value / minimum value extraction unit 1144 that the processing has been completed for all the reference pixels. In response to this, the maximum value / minimum value extraction unit 1144 extracts the maximum value and the minimum value from the pixel values stored in the pixel value storage unit 1054, and outputs the extracted value to the PNDP calculation unit 1145. Supply.

  In step S1057, the PNDP calculation unit 1145 sets the pixel value dynamic range of the reference pixel, which is a value obtained by subtracting the minimum value from the maximum value of the pixel value supplied from the maximum value / minimum value extraction unit 1144, as the PNDP for the target pixel. Calculate and supply to the feature value composition unit 933.

  In step S <b> 1058, the reference pixel extraction unit 1142 determines whether PNDP has been obtained for all pixels in the image stored in the buffer 1141. In step S1058, when the PNDP has not been obtained for all the pixels of the image, the process returns to step S1052. That is, the processes in steps S1052 to S1058 are repeated until the PNDP is obtained for all the pixels of the image stored in the buffer 1141. If it is determined in step S1058 that PNDP has been obtained for all pixels in the image, the PNDP extraction process ends.

  Through the above processing, PNDP, which is a feature amount representing a narrow edge in an image, is obtained. In other words, the PNDP expresses the pixel value dynamic range in the reference pixel with respect to the target pixel, and when the edge exists in the narrow area to which the target pixel belongs, it becomes a large value, and conversely, when the edge does not exist, It will take a small value.

  Furthermore, in the above description, an example in which the minimum value of the short-tap pixel value dynamic range is extracted as the SNDP as in the SNDP extraction process described with reference to the flowchart of FIG. 45 has been described. As long as the tendency of the flat part can be used as the feature amount, the difference absolute value between the reference pixel and the target pixel is converted by a function in accordance with the path from the reference pixel to the target pixel, as in the BFP extraction unit 932 in FIG. It is also possible to add a weight, compare with a threshold value, add the comparison results, and extract as SNDP.

  In FIG. 57, the absolute value of the difference between the reference pixel and the target pixel is converted by a function, a weight corresponding to the path from the reference pixel to the target pixel is added, compared with a threshold value, and the comparison result is added as SNDP. The example of a structure of the SNDP extraction part 972 made to extract is shown.

57 includes a buffer 1161, a reference pixel extraction unit 1162 , an inter-pixel difference calculation unit 1163, a function conversion unit 1164, a weight calculation unit 1165, a multiplication unit 1166, an accumulation unit 1167, and an SNDP calculation unit 1168. Has been.

  The buffer 1161 temporarily stores the HD image supplied from the output phase conversion unit 112 and sequentially supplies the HD image to the reference pixel extraction unit 1162 as necessary. The reference pixel extraction unit 1162 sequentially reads out the reference pixels for each target pixel and supplies them to the inter-pixel difference calculation unit 1163 and the weight calculation unit 1165.

  The inter-pixel difference calculation unit 1163 supplies, to the function conversion unit 1164, the absolute difference values of all the reference pixels supplied from the reference pixel extraction unit 1162 and the target pixel. The function conversion unit 1164 converts the inter-pixel difference absolute value supplied from the inter-pixel difference calculation unit 1163 with the set conversion function and supplies the converted difference value to the multiplication unit 1166.

The weight calculation unit 1165 includes an interpolation pixel generation unit 1165a and a difference calculation unit 1165b, and calculates a weight corresponding to the distance from the target pixel for each reference pixel position supplied from the reference pixel extraction unit 1162. This is supplied to the multiplier 1166. More specifically, the weight calculation unit 1165 controls the difference calculation unit 1165b to integrate the absolute difference value between pixels existing on the path between the target pixel and the reference pixel, and based on the integration result, the weight w _{Find s} . At this time, the weight calculation unit 1165 controls the interpolation pixel generation unit 1165a to generate a pixel by interpolation on the path between the target pixel and the reference pixel as necessary.

The multiplying unit 1166 multiplies the weight w _s supplied from the weight calculating unit 1165 and the function-converted inter-pixel difference absolute value supplied from the function converting unit 1164, and causes the accumulating unit 1167 to accumulate the result.

  The SNDP calculation unit 1168 cumulatively adds the multiplication results stored in the storage unit 1167 and outputs the result as the SNDP of the target pixel.

  Here, the SNDP extraction processing by the SNDP extraction unit 972 in FIG. 57 will be described with reference to the flowchart in FIG.

  In step S1081, the buffer 1161 temporarily stores the HD image supplied from the output phase converter 112.

  In step S1082, the reference pixel extraction unit 1162 sets an unprocessed pixel as a target pixel from the buffer 1161, and reads out a reference pixel set corresponding to the target pixel in step S1083 to obtain an inter-pixel difference. The data is supplied to the calculation unit 1163 and the weight calculation unit 1165.

  In step S <b> 1084, the inter-pixel difference calculation unit 1163 calculates the absolute difference between the pixel values of the pixel of interest with respect to the pixel of interest for an unprocessed reference pixel, i.e., a reference pixel for which the absolute value of the difference of the pixel of interest is not calculated. The value g is calculated and supplied to the function converter 1164.

  In step S <b> 1085, the function conversion unit 1164 converts the inter-pixel difference absolute value g supplied from the inter-pixel difference calculation unit 1163 based on a preset function F, and supplies the converted inter-pixel difference absolute value g to the multiplication unit 1166. More specifically, for example, the function conversion unit 1164 converts the value based on the function F (g) indicated by the solid line in FIG. 59 and outputs the converted value to the multiplication unit 1166. In the function F (g) indicated by the solid line in FIG. 59, the inter-pixel difference absolute value g is converted by the function F to be larger than when it is smaller than a predetermined value and smaller than when it is larger than a predetermined value. That is, the value changes with a predetermined value as a boundary.

  In step S1086, the weight calculation unit 1165 determines whether there is a pixel to be interpolated between the target pixel and the reference pixel. More specifically, for example, as shown in FIG. 60, when the target pixel P (0, 0) and the reference pixel P (2, 4) indicated by a double circle in the figure, the target pixel Interpolation pixels P (0.5,1) and P (1.5,3) are required on the path between P (0,0) and reference pixel P (2,4), so interpolation is necessary. It is determined that a correct pixel exists. In this case, the process proceeds to step S1087.

In step S1087, the weight calculation unit 1165 controls the interpolation pixel generation unit 1165a to generate an interpolation pixel. For example, in the case of the target pixel P (0, 0) and the reference pixel P (2, 4) in FIG. 60, the interpolation pixel generation unit 1165a performs interpolation pixel P (0.5, 1), P (1.5, For example, the pixel values of 3) are average values such as (P (0,1) + P (1,1)) / 2, (P (1,3) + P (1,1)) / 2, respectively. Ask. The interpolation method is not limited to this, and other methods may be used. For example, a sum may be obtained by adding weights.

  In step S1086, for example, when the pixel of interest P (0, 0) in FIG. 60 and the reference pixel P (3, 3) indicated by the cross in the drawing are referred to, the pixel of interest P (0, 0) is referred to. Since there are interpolated pixels P (1,1) and P (2,2) on the path to the pixel P (3,3), there is no pixel that needs to be interpolated, so there is no pixel that needs to be interpolated. It will be determined. In this case, the process of step S1087 is skipped.

In step S1088, the weight calculation unit 1165 controls the difference calculation unit 1165b to calculate the sum D of absolute differences between pixels on the path between the target pixel and the reference pixel, and the weight w _g = 1 from the sum D. -D / H (H is a constant) is obtained and supplied to the multiplier 1166. That is, in the case of the target pixel P (0,0) and the reference pixel P (2,4) in FIG. 60, the difference calculation unit 1165b calculates | P (0,0) −P (0.5,1) | | P (0.5,1) -P (1,2) |, | P (1,2) -P (1.5,3) |, | P (1.5,3) -P (2, 4) The sum D of | is calculated. Further, the weight calculator 1165 calculates the weight w _g = 1−D / H and supplies it to the multiplier 1166. The weight w _g may be a value associated with a parameter such as PNDP. For example, the weight w _g = 1−D / PNDP or the weight w _g = 1−D / (√PNDP). Also good.

In step S1089, the multiplication unit 1166 multiplies the inter-pixel difference absolute value g supplied from the function conversion unit 1164 and the weight w _g supplied from the weight calculation unit 1165, and in step S1090, the accumulation unit 1167. To accumulate.

  In step S1091, the inter-pixel difference calculation unit 1163 determines whether or not an unprocessed reference pixel exists. If there is, the process returns to step S1084. That is, the absolute value of the inter-pixel difference between all the reference pixels and the target pixel is obtained by repeating the processes in steps S1084 to S1091.

  If it is determined in step S1091 that there are no unprocessed reference pixels, in step S1092, the inter-pixel difference calculation unit 1163 notifies the SNDP calculation unit 1168 that the processing has been completed for all reference pixels. The SNDP calculation unit 1168 calculates the sum of the multiplication results stored in the storage unit 1167 and outputs it as SNDP.

  In step S1093, the reference pixel extraction unit 1162 determines whether SNDP has been obtained for all the pixels in the image stored in the buffer 1161. In step S1093, when the SNDP has not been obtained for all the pixels of the image, the process returns to step S1082. That is, the processes in steps S1082 to S1093 are repeated until SNDPs are obtained for all the pixels of the image stored in the buffer 1161. If it is determined in step S1093 that the SNDP has been obtained for all the pixels in the image, the SNDP extraction process ends.

  Through the above processing, SNDP, which is a feature amount representing a narrow flat portion in an image, is obtained. That is, the SNDP has a large value when the narrow area to which the pixel of interest belongs is flat, and conversely has a small value when there are many edges and the like and is not flat.

In the above, the conversion function F may be, for example, as shown in FIG. 59 as described above. However, the conversion function F may be associated with the PNDP. When th11 = PNDP / b11 or more, F (g) = AA (AA is a constant), and when the inter-pixel difference absolute value g is equal to or less than the threshold th2 = PNDP / b12, F (g) = BB (BB is a constant) Further, when the absolute difference value g between pixels is larger than the threshold th12 = PNDP / b12 and smaller than the threshold th11 = BEP / b11, F (g) = (BB−AA) · (g−th11) / (th12 -Th11) + AA. The thresholds th11 and th12 only need to use the PNDP tendency. For example, the threshold th11 = (PNDP) ² / b11, th12 = (PNDP) ² / b12, or the threshold th11 = (√PNDP) / b11, th12 = (√PNDP) / b12 may be used.

  In the above, as shown in FIG. 1, after the natural image prediction unit 131 and the artificial image prediction unit 132 perform the respective prediction processes, they are converted into the artificial image Art obtained by the natural image artificial image determination unit 114. Although an example of combining based on the above has been described, a natural image and an artificial image may be separated in advance and supplied to the natural image prediction unit 131 and the artificial image prediction unit 132, respectively.

  FIG. 61 illustrates a configuration example of the image conversion apparatus 101 in which a natural image and an artificial image are separated in advance and supplied to the natural image prediction unit 131 and the artificial image prediction unit 132, respectively. Note that the same components as those of the image conversion apparatus 101 in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The image conversion apparatus 101 in FIG. 61 differs from the image conversion apparatus 101 in FIG. 1 in that a separation unit 1201 is provided at the subsequent stage of the output phase conversion unit 112, and the natural image prediction unit 131 and the artificial image prediction unit 132 The difference is that an adder 1202 is provided in the subsequent stage.

  The separation unit 1201 separates the HD image supplied from the output phase conversion unit 112 on a pixel-by-pixel basis based on the artificial image degree Art from the natural-image / artificial-image determination unit 114, and the natural-image prediction unit 131 and the artificial-image prediction unit 132.

  The adding unit 1202 adds the pixels supplied from the natural image prediction unit 131 and the artificial image prediction unit 132, that is, rearranges the separated and high-quality pixels to the positions before the separation. Make an image.

  Next, image conversion processing by the image conversion apparatus 101 in FIG. 61 will be described with reference to the flowchart in FIG. Note that the processing of steps S1111 to S1114, S1116, and S1117 in the flowchart of FIG. 62 is the same as the processing of steps S1 to S3, S6, S4, and S5 in the flowchart of FIG.

  In step S1115, the separation unit 1201 separates the HD image supplied from the output phase conversion unit 112 in units of pixels based on the artificial image degree Art supplied from the natural image / artificial image determination unit 114, and performs natural image prediction. To the unit 131 and the artificial image prediction unit 132, respectively. More specifically, in the pixel unit, the separation unit 1201 supplies the pixel to the artificial image prediction unit 132 when the artificial art degree Art is 1, while if the artificial art degree Art is 0, the separation unit 1201 The image is supplied to the image prediction unit 131. Further, when the artificial picture degree Art is larger than 0 and smaller than 1, the pixel is supplied to both the natural picture prediction unit 131 and the artificial picture prediction unit 132 together with information on the artificial picture degree. At this time, the separation unit 1201 also supplies the input image to the natural image prediction unit 131 and the artificial image prediction unit 132 for the natural image prediction process and the artificial image prediction process together with the separated pixel information.

  After the natural image prediction process and the artificial image prediction process are performed by the processes in steps S1116 and S1117, the addition unit 1202 is supplied from the natural image prediction unit 131 and the artificial image prediction unit 132 in step S1118. By combining the pixels, an image with high image quality is generated and output. At this time, when the image is separated by the separation unit 1201, the artificial image degree Art is added to each of the natural image prediction unit 131 and the artificial image prediction unit 132 when the artificial image degree Art is larger than 0 and smaller than 1. For the separated pixels, the artificial image is multiplied by the artificial art degree Art, and the natural image is multiplied by the value obtained by subtracting the artificial art degree Art from 1 (1−Art), and then the pixel is added. Generate.

  With the above processing, it is possible to obtain the same effect as that of the image conversion apparatus 101 in FIG. In the image conversion apparatus 101 in FIG. 61, the pixels supplied to the natural image prediction unit 131 and the artificial image prediction unit 132 are distributed according to the artificial image degree Art, so that the processing load can be reduced.

  In the above description, as shown in FIG. 42, the example in which the wide-area artificial image boundary line L1 is present above the wide-area natural image boundary line L2 has been described, but this positional relationship is a feature adopted as BEP and BFP. Depending on the amount, it may be reversed.

  That is, as the BEP, the difference dynamic range of the reference pixel, the pixel value dynamic range using the reference pixel, or the difference absolute value of the pixel value between adjacent pixels of the reference pixel are rearranged in ascending order, and the upper difference absolute value is As a BFP, the difference value between adjacent pixels of a reference pixel is compared with a predetermined threshold th as a BFP, and for a value larger than the threshold th, a predetermined value is set by a function. When using the summation, if the vertical axis is BFP and the horizontal axis is BEP, the wide-area artificial image boundary line is positioned above the wide-area natural image boundary line. Also, as BEP, the difference dynamic range of the reference pixel, the pixel value dynamic range using the reference pixel, or the difference absolute value of the pixel value between adjacent pixels of the reference pixel are rearranged in ascending order, and the upper difference absolute value is If the absolute value of the difference between adjacent pixels of the reference pixel is rearranged in ascending order as BFP, and the higher value is used, BFP on the vertical axis and BEP on the horizontal axis, The artificial image boundary line is located below the wide area natural image boundary line.

Further, broad-range artificial image boundary L1 and the broad-range natural image boundary L2 is, for example, parameter for the image conversion processing, for example, the resolution axis value, the noise-axis value, by setting such a zoom magnification, broad-range degree-of-artificiality Art _b The value of can be increased or decreased.

That is, the wide-area artificial image boundary line L1 and the wide-area natural image boundary line L2 shown in the middle part of FIG. 63 are the same as those in FIG. 42. For example, when the resolution in the image conversion process is changed, the resolution is increased. 63, the wide-area artificial image boundary line L1 ′ and the wide-area natural image boundary line L2 ′ are lower than the wide-area artificial image boundary line L1 and the wide-area natural image boundary line L2, respectively, as shown in the upper part of FIG. For this reason, if the same wide area feature amount is set, the wide area artificial art degree Art _b is likely to take a high value. On the other hand, when the resolution is lowered, as shown in the lower part of FIG. 63, the wide-area artificial image boundary line L1 ″ and the wide-area natural image boundary line L2 ″ are at high positions. For example, the wide-area artificial image degree Art _b is likely to take a low value.

  Similarly, in the above description, as shown in FIG. 48, an example in which the narrow natural image boundary line L11 is present above the narrow artificial image boundary line L12 has been described, but this positional relationship is based on PNDP and SNDP. May be reversed depending on the feature quantity adopted.

  That is, when the pixel value dynamic range of the reference pixel is used as the PNDP, and when the weighted difference absolute value sum by the path between the target pixel and each reference pixel is used as the SNDP, the vertical axis indicates the PNDP and the horizontal When SNDP is taken as an axis, the narrow-area artificial image boundary line L12 is positioned above the narrow-area natural image boundary line L11. In addition, when a long tap pixel value dynamic range is used as the PNDP, and when a short tap pixel value dynamic range is used as the SNDP, the vertical axis is PNDP and the horizontal axis is SNDP. The area artificial image boundary line L12 is located below the narrow area natural image boundary line L11.

In addition, the narrow natural image boundary line L11 and the narrow artificial image boundary line L12 are, for example, a narrow artificial image by setting parameters such as a resolution axis value, a noise axis value, and a zoom magnification when performing image conversion processing. It is possible to increase or decrease the value of Art _n .

That is, the narrow natural image boundary line L11 and the narrow artificial image boundary line L12 shown in the middle stage of FIG. 64 are the same as those in FIG. 48. For example, when the noise axis value in the image conversion processing is changed. When the noise axis value is increased, as shown in the upper part of FIG. 64, the narrow natural image boundary line L11 ′ and the narrow artificial image boundary line L12 ′ are respectively converted into the narrow natural image boundary line L11 and the narrow regional artificial line. Since the position is lower than the image boundary line L12, the narrow-area artificial image degree Art _n is likely to take a high value with the same narrow-area feature amount. On the other hand, when the noise axis value is lowered, as shown in the lower part of FIG. 64, the narrow natural image boundary line L11 ″ and the narrow artificial image boundary line L12 ″ become the narrow natural image boundary line L11 and Since the position is higher than the narrow-area artificial image boundary line L12, the narrow-area artificial image degree Art _n is likely to take a low value if the same narrow-area feature amount.

  In the above, examples have been described in which BEP and BFP are used as the wide area feature quantity, and PNDP and SNDP are used as the narrow area feature quantity, but for the wide area feature quantity, if it is a parameter that can express an edge or a flat part, The feature amount is not limited to BEP and BFP, and other feature amounts may be used. Similarly, the narrow area feature amount is not limited to the above-described PNDP and SNDP, but may be any feature amount that can express a narrow line, an edge, a point, a flat portion near the edge, or a gradation. An amount may be used. Further, in the above description, examples of using two types of feature amounts for each of the wide-area feature amount and the narrow-area feature amount have been described. However, more types of feature amounts may be used. The boundary line between the artificial image and the natural image is not two-dimensional as shown in FIGS. 42 and 48, but is a boundary line or boundary surface expressed in n-dimensions according to the number n of feature quantities. However, the basic processing is the same, such as determining the existence area of the feature quantity in the n-th order space and obtaining the ratio of the distance between the boundary lines or the boundary surfaces in the mixed area.

  According to the above, it is possible to identify a natural image or an artificial image in units of pixels and perform natural image prediction processing and artificial image prediction processing, and appropriately perform prediction processing for each pixel. It is possible to accurately improve the overall quality.

  The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 65 shows the configuration of an embodiment of a personal computer when the electrical internal configuration of the image conversion apparatus 101 of FIGS. 1 and 61 is realized by software. A CPU 2001 of the personal computer controls the overall operation of the personal computer. Further, when a command is input from the input unit 2006 such as a keyboard or a mouse from the user via the bus 2004 and the input / output interface 2005, the CPU 2001 is stored in a ROM (Read Only Memory) 2002 correspondingly. Run the program. Alternatively, the CPU 2001 reads a program read from a removable disk 2011 including a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory connected to the drive 2010 and installed in the storage unit 2008 into a RAM (Random Access Memory). It is loaded into 2003 and executed. As a result, the above-described functions of the image conversion apparatus 101 in FIGS. 1 and 61 are realized by software. Further, the CPU 2001 controls the communication unit 2009 to communicate with the outside, and exchange data.

  As shown in FIG. 65, the recording medium on which the program is recorded is distributed to provide the program to the user separately from the computer. The magnetic disk (including the flexible disk) on which the program is recorded and the optical disk are distributed. (Including CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disk)), magneto-optical disk (MD (Mini-Disc) included), or removable media 2011 including semiconductor memory And a ROM 2002 on which a program is recorded and a hard disk included in the storage unit 2008, which is provided to the user in a state of being incorporated in the computer in advance.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

It is a block diagram which shows the structure of one Embodiment of the image converter to which this invention is applied. 3 is a flowchart for explaining image conversion processing executed by the image conversion apparatus in FIG. 1. It is a block diagram which shows the detailed structural example of a natural picture prediction part. It is a figure which shows the tap structure of a class tap. It is a figure which shows the tap structure of a prediction tap. It is a flowchart explaining the detail of a natural image prediction process. It is a block diagram which shows the structural example of a learning apparatus. It is a figure explaining the positional relationship of the pixel of a teacher image and a student image. It is a flowchart explaining a learning process. It is a block diagram which shows the structural example of the artificial-image prediction part. It is a block diagram which shows the detailed structural example of a class classification | category part. It is a figure which shows the tap structure of another class tap. It is a block diagram which shows the detailed structural example of a prediction part. It is a figure which shows the tap structure of another prediction tap. It is a flowchart explaining an artificial image prediction process. It is a flowchart explaining a class classification process. It is a block diagram which shows the structural example of another learning apparatus. It is a block diagram which shows the detailed structural example of a production | generation part. It is a flowchart explaining another learning process. It is a block diagram which shows the other detailed structural example of a class classification | category part. It is a figure which shows the tap structure of another class tap. It is a flowchart explaining another class classification process. It is a block diagram which shows the further detailed structural example of a class classification | category part. It is a block diagram which shows the detailed structural example of a classifier. It is a figure which shows the tap structure of another class tap. It is a block diagram which shows the detailed structural example of a class selection part. It is a flow chart explaining other class classification processing. It is a flowchart explaining the class classification process of step S781 of FIG. It is a flowchart explaining a class selection process. It is a block diagram which shows the structural example of the natural image artificial image determination part. It is a block diagram which shows the structural example of the BEP extraction part of FIG. It is a block diagram which shows the structural example of the BFP extraction part of FIG. It is a block diagram which shows the structural example of the PNDP extraction part of FIG. It is a block diagram which shows the structural example of the SNDP extraction part of FIG. It is a flowchart explaining the natural image artificial image determination process by the natural image artificial image determination part of FIG. It is a flowchart explaining the BEP extraction process by the BEP extraction part of FIG. It is a figure explaining a reference pixel. It is a flowchart explaining the BFP extraction process by the BFP extraction part of FIG. It is a figure explaining the relationship of the reference pixel which calculates | requires the difference value between adjacent pixels. It is a figure explaining conversion function f. It is a flowchart explaining a wide-area artificial image degree calculation process. It is a figure explaining a wide area boundary line. It is a flowchart explaining the PNDP extraction process by the PNDP extraction part of FIG. It is a figure explaining a long tap. It is a flowchart explaining the SNDP extraction process by the SNDP extraction part of FIG. It is a figure explaining a short tap. It is a flowchart explaining a narrow area artificial image degree calculation process. It is a figure explaining a narrow region boundary line. It is a block diagram which shows the other structural example of a BEP extraction part. It is a flowchart explaining the BEP extraction process by the BEP extraction part of FIG. It is a figure explaining the BEP extraction process by the BEP extraction part of FIG. It is a block diagram which shows the other structural example of a BFP extraction part. 53 is a flowchart for describing BFP extraction processing by a BFP extraction unit in FIG. 52. FIG. 53 is a diagram illustrating a BFP extraction process by a BFP extraction unit in FIG. 52. It is a block diagram which shows the other structural example of a PNDP extraction part. 56 is a flowchart for describing PNDP extraction processing by a PNDP extraction unit in FIG. 55. It is a block diagram which shows the other structural example of a SNDP extraction part. 58 is a flowchart for describing SNDP extraction processing by the SNDP extraction unit in FIG. 57. 6 is a diagram for explaining a conversion function F. It is a figure explaining the pixel produced | generated by interpolation. It is a figure explaining the other structural example of an image converter. FIG. 62 is a flowchart for describing image conversion processing according to FIG. 61. FIG. It is a figure explaining a wide area | region artificial image boundary line and a wide area | region natural image boundary line. It is a figure explaining a narrow area artificial picture boundary line and a narrow area natural picture boundary line. It is a figure explaining a medium.

Explanation of symbols

  132 Artificial image prediction unit, 651 class classification unit, 652 coefficient seed memory, 653 prediction coefficient generation unit, 654 prediction coefficient memory, 655 prediction unit, 811 learning device, 822 class classification unit, 823 generation unit, 824 coefficient generation unit, 825 Normal equation generator, 826 coefficient seed determiner, 827 coefficient seed memory, 831 prediction tap extractor, 832 normal equation generator, 840 class classifier, 841 class determiner, 861 class classifier, 862 class classifier, 863 class Select part

Claims (6)

In an image processing apparatus for obtaining an output image with a high quality artificial image, which is an artificial image with low gradation and clear edges, from the input image,
First class tap extraction means for extracting, from the input image, a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the output image into a first class;
A difference calculating means for calculating a difference between pixel values of adjacent pixels of the first class tap;
First classification means for classifying the target pixel corresponding to the difference into the first class based on the difference;
Second class tap extraction means for extracting a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class from the input image;
Second classification means for classifying a target pixel corresponding to the second class tap into the second class based on a pixel value of each pixel of the second class tap;
Based on the first class, said first class of the pixel of interest, either one of the second class of the pixel of interest, and selecting means for selecting as a class of the pixel of interest,
Storage means for storing a prediction coefficient for each of the first classes and a prediction coefficient for each of the second classes acquired by learning using a plurality of artificial images;
Image processing comprising: an arithmetic unit that calculates the output image from the input image by calculating using the input image and the prediction coefficient of the first or second class selected as the class of the target pixel apparatus. The image processing apparatus according to claim 1, wherein the first classifying unit classifies the target pixel corresponding to the difference into the first class based on the absolute value of the difference. The image processing apparatus according to claim 1, wherein the first classifying unit classifies the target pixel corresponding to the difference into the first class based on a ratio of absolute values of the differences. In an image processing method for obtaining an output image with a high quality artificial image, which is an artificial image with few gradations and clear edges, from the input image,
Extracting from the input image a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the output image into a first class;
Obtaining a difference between pixel values of adjacent pixels of the first class tap;
Based on the difference, classify the target pixel corresponding to the difference into the first class,
Extracting a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class from the input image;
Based on the pixel value of each pixel of the second class tap, classify the target pixel corresponding to the second class tap into the second class,
Based on the first class, said first class of the pixel of interest, either one of the second class of the pixel of interest is selected as class of the pixel of interest,
Of the input image and the prediction coefficient for each first class and the prediction coefficient for each second class acquired by learning using a plurality of artificial images, the first selected as the class of the pixel of interest An image processing method including a step of obtaining the output image from the input image by calculating using a prediction coefficient of one or a second class. A program for causing a computer to execute a process for obtaining an output image with a high quality artificial image, which is an artificial image with a small gradation and a clear edge, from the input image,
Extracting from the input image a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the output image into a first class;
Obtaining a difference between pixel values of adjacent pixels of the first class tap;
Based on the difference, classify the target pixel corresponding to the difference into the first class,
Extracting a second class tap composed of a plurality of pixels used for classifying the pixel of interest into a second class from the input image;
Based on the pixel value of each pixel of the second class tap, classify the target pixel corresponding to the second class tap into the second class,
Based on the first class, said first class of the pixel of interest, either one of the second class of the pixel of interest is selected as class of the pixel of interest,
Of the input image and the prediction coefficient for each first class and the prediction coefficient for each second class acquired by learning using a plurality of artificial images, the first selected as the class of the pixel of interest A program including a step of obtaining the output image from the input image by calculating using a prediction coefficient of the first or second class. A learning device that learns a prediction coefficient used when performing a conversion process for converting an artificial image, which is an artificial image with small gradation and clear edges, into a high-quality artificial image,
An acquisition means for acquiring a teacher image to be a target artificial image after the conversion process and a student image corresponding to the artificial image before the conversion process;
First class tap extraction means for extracting, from the student image, a first class tap composed of a plurality of pixels used for classifying a target pixel as a pixel of the teacher image into a first class;
A difference calculating means for obtaining a difference between pixel values of adjacent pixels of the extracted first class tap;
First classification means for classifying the target pixel corresponding to the difference into the first class based on the difference;
Second class tap extraction means for extracting, from the student image, a second class tap composed of a plurality of pixels used for classifying the target pixel into a second class;
Second classification means for classifying a target pixel corresponding to the second class tap into the second class based on a pixel value of each pixel of the second class tap;
Based on the first class, the first class of the pixel of interest, either one of the second class of the pixel of interest, and selecting means for selecting as a class of the pixel of interest,
A prediction tap extracting means for extracting a prediction tap composed of a plurality of pixels used for obtaining the target pixel from the student image;
Based on the extracted prediction tap, for each of the first or second class selected as the class of the target pixel corresponding to the prediction tap, the prediction error of the target pixel obtained using the prediction coefficient is calculated. A learning device comprising: a calculation unit that calculates the prediction coefficient that is statistically minimized.

Images