JP3470403B2 - Digital image signal processing unit - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image signal processing apparatus for receiving a sub-sampling signal, interpolating a thinned pixel, and correcting a transmission pixel.

[0002]

2. Description of the Related Art As one method for band compression or information amount reduction when recording or transmitting a digital image signal, there is a method of reducing the amount of transmission data by thinning out pixels by subsampling. One example is the multiple sub-Nyquist sampling encoding method in the MUSE method. In this system, it is necessary to interpolate non-transmitted pixels that have been decimated on the receiving side.

Offset subsampling is known as an example of subsampling. FIG. 12 is an example of the offset sub-sampling circuit, in which a digital video signal is supplied to the input terminal 61 and is supplied to the sub-sampling circuit 63 via the pre-filter 62. A sampling pulse having a predetermined frequency is supplied to the sub-sampling circuit 63 from the input terminal 64.

FIG. 13 shows an example of two-dimensional offset subsampling performed by the subsampling circuit 63. The sampling interval (Tx, Ty) in the horizontal direction (x direction) and the vertical direction (y direction) is set to twice the pixel interval (Hx, Hy) in the original signal, and thinning is performed every other pixel (thinning pixel is × Together with the transmission pixels (indicated by ◯) vertically adjacent to each other in the vertical direction by half the sampling interval (Tx
It is offset by / 2). The transmission band resulting from such offset subsampling is
The spatial frequency component in the horizontal or vertical direction can be widened with respect to the spatial frequency in the diagonal direction.

The output signal of the sub-sampling circuit 63 is taken out to the output terminal 66 via the post filter 65. The pre-filter 62 limits the band of the image signal to be sampled, and the post-filter removes unnecessary or harmful signal components. The amount of data transmitted by the sub-sampling can be reduced, and the digital video signal can be transmitted through a relatively low speed transmission line. When the received offset sub-sampled image signal is displayed on the monitor or printed out, the thinned pixels are interpolated using the adjacent pixels.

By the way, the offset sub-sampling as described above is a very effective method when the pre-filter before the sampling is correctly performing the filtering process. However, if the pre-filter cannot be applied sufficiently or the pre-filter is not applied sufficiently to widen the transmission band, the problem of image quality deterioration due to aliasing distortion occurs.

In order to reduce the occurrence of the above-mentioned folding distortion,
Adaptive interpolation methods have been proposed. This is a method in which the optimum interpolation method is determined in advance during subsampling, and the determination result is transmitted or recorded as auxiliary information. For example, which one of the horizontal half average value interpolation and the vertical half average value interpolation is closer to the true value is detected at the time of sub-sampling and transmitted as 1 bit of auxiliary information per pixel. However, at the time of interpolation, interpolation processing is performed according to this auxiliary information.

In the above-mentioned adaptive interpolation method using the auxiliary information, it is necessary to transmit the auxiliary information in addition to the transmission pixels, which causes a problem that the compression rate of the data amount is lowered. Further, when an error occurs in the auxiliary information in the process of transmission or recording / reproduction, there is a drawback that the reproduced image is likely to be deteriorated because incorrect interpolation is performed.

As one method for solving this problem, Japanese Patent Laid-Open No. 63-48088 proposed by the applicant of the present application describes the value of a pixel of interest by a linear linear combination of pixels around it and a coefficient, and It has been proposed to determine the value of this coefficient by the method of least squares using the actual value of the pixel of interest so that the sum of squares of is minimized. Here, the coefficient of the linear linear combination is previously determined by learning, and the coefficient of determination is stored in the memory. Furthermore, when interpolating the pixel of interest,
The average value of the surrounding reference pixels is calculated, and each pixel is represented by 1 bit according to the magnitude relationship between the average value and the value of each pixel,
The classification is performed according to the pattern of (the number of reference pixels × 1 bit), and the interpolated value reflecting the local feature of the image including the target pixel is formed. This method can interpolate thinned pixels well without the need for auxiliary information.

[0010]

On the other hand, considering a transmission pixel, the pre-filter 62 and the post-filter 6 are considered.
Since the signal is transmitted through the signal 5, the high frequency component is lost, resulting in the problem that the signal waveform is rounded. That is, the filtering process required for subsampling also adversely affects the transmission pixels. Therefore,
It is preferable to correct not only the thinned-out pixels but also the transmitted pixels by the class classification adaptive processing, similarly to the thinned-out pixels.

Further, if the transmission pixel used for interpolating the thinned pixel has also been subjected to the above-mentioned filtering processing, the interpolation value formed by the class classification adaptive processing is also affected. That is, if the transmission pixel has lost the high frequency component, the interpolation value using it may be insufficiently compensated in the high frequency range.

Therefore, an object of the present invention is to use a class classification adaptive process when decoding a sub-sampling signal so as to not only interpolate a thinned pixel but also compensate a band lost by a filter or the like for a transmission pixel. Another object of the present invention is to provide a digital image signal processing device capable of performing the above.

Another object of the present invention is to provide a digital image signal processing apparatus capable of favorably performing interpolation of thinned pixels by interpolating thinned pixels using compensated transmission pixels.

[0014]

According to a first aspect of the present invention, a digital image signal that has passed through a prefilter is sampled, a signal whose number of pixels is reduced by sampling is received, and pixels thinned by sampling are received. In a digital image signal processing device adapted to interpolate a first pixel for determining a class of a target transmission pixel using a plurality of transmission pixels around a target transmission pixel existing in a received digital image signal. Of the plurality of transmission pixels included in the input digital image signal and spatially and / or temporally close to the transmission pixel of interest, and a linear linear combination of the first coefficient and the transmission pixel of interest When the value of is created, the first coefficient for generating the first coefficient for each class that minimizes the error between the created value and the true value of the transmission pixel of interest. A linearly-linear combination of the first coefficient and the values of a plurality of transmission pixels spatially and / or temporally adjacent to the transmission pixel of interest to generate a corrected value of the transmission pixel of interest. And a correction transmission pixel from the first calculation circuit, and using a plurality of correction transmission pixels around the target thinning pixel,
A second class classification circuit for determining the class of the thinned pixel of interest, and a linear linear combination of the values of a plurality of correction transmission pixels spatially and / or temporally adjacent to the thinned pixel of interest and a second coefficient. A second coefficient generation circuit for generating a second coefficient for each class so as to minimize an error between the created value and the true value of the target thinned pixel when the value of the target thinned pixel is created. And a second linear coefficient combination of the second coefficient and the values of a plurality of correction transmission pixels spatially and / or temporally close to the target thinning pixel to generate an interpolated value of the target thinning pixel. A digital image signal processing device comprising an arithmetic circuit.

According to a third aspect of the present invention, a first class for determining a class of a target transfer pixel using a plurality of transfer pixels around the target transfer pixel existing in the received digital image signal. The classifying circuit and the first representative value acquired by learning in advance are stored for each class, and the first representative value corresponding to the class determined by the first class classifying circuit is output as the value of the target transmission pixel. A first memory circuit for receiving a correction transmission pixel from the first memory circuit, and using a plurality of correction transmission pixels around the target thinning pixel to determine a class of the target thinning pixel. And the second representative value acquired by learning in advance are stored for each class, and the second representative value corresponding to the class determined by the second class classification circuit is used as the target thinning pixel A processing apparatus in a digital image signal, comprising the second memory circuit for outputting a.

[0016]

With respect to the transmission pixel, the predicted value of the transmission pixel can be formed by the class classification adaptive processing.
The class classification adaptation process forms a correction value of a transmission pixel by linearly combining the coefficient obtained by learning in advance and the values of surrounding transmission pixels, or a representative value or a normalized representative value obtained by learning in advance. Is output as the value of the transmission pixel. The coefficient or the representative value (or the normalized representative value) is determined for each class corresponding to the characteristic of the partial small area including the transmission pixel. After correcting the transmission pixels in this way, the thinned pixels are interpolated by the class classification adaptive processing using the corrected transmission pixels. Since the corrected transmission pixel is used, the interpolation is performed well.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a subsampling signal interpolating device will be described below. Figure 1
In the figure, 1 is an input terminal of the offset sub-sampled digital video signal. Specifically, a transmission signal by broadcasting or the like, a reproduction signal from a VTR or the like is supplied to the input terminal 1. The value of the transmission pixel is represented by an 8-bit code. Reference numeral 2 is a time-series conversion circuit for converting an input signal arriving in a television raster order into a block order.

The output signal of the time series conversion circuit 2 is supplied to the first class classification circuit 3. The class classification circuit 3 determines the class of the target transmission pixel to be corrected, and the class code indicating the class is supplied to the first memory 4 as an address. The prediction coefficient read from the memory 4 is supplied to the first correction value generation circuit 5.

As will be described later, the memory 4 stores the first prediction coefficient acquired by learning in advance. This coefficient is needed to predict the correction value of the transmitted pixel. The correction value generation circuit 5 is supplied from the time series conversion circuit 2 with the values of a plurality of pixels around the target transmission pixel.
The correction value generation circuit 5 stores the correction value of the transmission pixel of interest in the memory 4
It is generated by a linear first-order combination of the coefficient from x and the value of the surrounding transmission pixel.

The output signal of the correction value generation circuit 5 is supplied to the second class classification circuit 6. The class classification circuit 6 determines the class of the target thinned pixel to be interpolated, and the class code indicating the class is supplied to the second memory 7 as an address. The prediction coefficient read from the memory 7 is supplied to the interpolation value generation circuit 8.

Similar to the memory 4, the memory 7 stores prediction coefficients previously acquired by learning. This coefficient is needed to predict the interpolated value of the thinned pixels. Both the interpolation value and the correction value are prediction values, but the prediction value for the thinned pixel is called an interpolation value, and the prediction value for the transmission pixel is called a correction value. The correction value generating circuit 5 supplies the values of a plurality of pixels around the target thinned pixel to the interpolation value generating circuit 8. Then, the interpolation value generation circuit 8 generates the prediction value of the thinned pixel of interest by linear linear combination of the coefficient from the memory 7 and the values of the surrounding transmission pixels.

The generated correction value and interpolation value are supplied to the synthesizing circuit 9, the thinned pixels are interpolated at the output terminal 10, and the digital video signal in which the frequency component lost by the filtering process is compensated is output. It Since the correction value is generated before the interpolation value, a delay circuit for time adjustment is used as necessary. Although not shown, a time series conversion circuit is connected to the output terminal 10 to form a digital video signal converted from the block order to the raster scan order.

The class classification circuit 6 uses the values of the transmission pixels in the vicinity of the target thinned pixel to determine the class of the target thinned pixel. As shown in FIG. 2A, the pattern of the level distribution of the transmission pixels (A, B, C, D) on the upper, lower, left and right sides of the target thinning pixel is determined as a class. These transmission pixels have been corrected by the adaptive processing. As an example, the average value Av of the four referenced pixels is calculated, and the surrounding pixels are selected from 8 bits to 1 according to the magnitude relationship with the average value Av.
Compress to bits. That is, as shown in an example in FIG. 3, when the value is larger than the average value Av, "1" is assigned, and when the value is smaller than the average value Av, "0" is assigned. In the example of FIG. 3, the class code (1010) is generated from the class classification circuit 6.

The class classification circuit 3 determines the class of the transmission pixel of interest. As shown in FIG. 2B, the class of the target transmission pixel is determined based on the pattern of the level distribution of the target transmission pixel (whose value is y) and the surrounding pixels a to d. This is the same as the above-described thinned pixel, in which an average value of a total of 5 pixels of the transmission pixel of interest and the surrounding 4 pixels is calculated, and according to the comparison result with respect to the average value, `0` corresponding to each pixel is calculated.
Alternatively, it is a process of forming a bit of "1". in this case,
A compressed version of the value of the transmission pixel of interest may be included in the class information.

In the above classification, the value of the pixel referred to for classification is compared with the average value and compressed to 1 bit, but it may be compressed by 1-bit or several-bit ADRC. That is, the ADRC detects the dynamic range DR and the minimum value MIN of a plurality of pixels, subtracts the minimum value MIN from the value of each pixel, divides the value obtained by subtracting the minimum value by the dynamic range DR, and calculates the quotient as an integer. It is the process of converting.

The case where 1-bit ADRC is applied to the classification of the thinned pixels will be described. For classification, the maximum value MAX and the minimum value MIN among the four pixels A to D are detected and the dynamic value is determined. Range DR
(= MAX-MIN) is calculated. The minimum value MIN is subtracted from the value of each pixel A to D, and the value after the minimum value is removed is divided by the dynamic range DR. The quotient of this division is 0.
It is compared with 5, and when it is 0.5 or more, it is set to "1", and when the quotient is less than 0.5, it is set to "0". 1-bit AD
RC gives substantially the same result as comparing the above average value with the value of each pixel.

Some other examples of classifying the transmission pixel of interest will be described. As shown in FIG. 2B, not only a to d but also transmission pixels (e,
f, g, h) are also included in the referenced pixels. The classification is performed on the basis of the absolute value of the difference between the values a to h of the peripheral transmission pixels and the value y of the transmission pixel of interest and the values of the pixels aligned in the same direction. This classification will be described with reference to FIG.

FIG. 4A shows a case where a difference in value between pixels vertically aligned is calculated. That is, 4 shown below
The absolute value of the difference is calculated. Here, the value of 0 to 3 represents the value of 2-bit direction data for distinguishing each difference. 0: | h-g | 1: | a-y | 2 :: | y-c | 3: | e-f |

The minimum value among the absolute values of these four differences is detected, and 2-bit direction data regarding the vertical direction, which indicates the minimum value, is determined. For example | ay
When the value of | is the minimum value, the direction data of 1 (that is, 01) is determined. FIG. 4B shows a case where a difference in value between pixels arranged in a diagonally upward right direction is calculated. That is, the absolute value of the difference of 0: | h-d | 1: | e-y | 2 :: y-g | 3 :: b-f | is calculated. Then, the minimum value among them is detected, and 2-bit direction data regarding the diagonal direction, which indicates the minimum value, is determined.

FIG. 4C shows the absolute value of the difference between the pixels arranged in the diagonally upward left direction, that is, 0: | d-g | 1: | h-y | 2 :: y-f | 3 :: e The case where -b | is calculated is shown. The minimum value of these values is detected, and 2-bit direction data regarding the diagonal direction, which indicates the minimum value, is determined.

Further, FIG. 4D shows the absolute value of the difference between pixels aligned in the horizontal direction, that is, 0: | he-e | 1: | d-y | 2 :: y-b | 3 :: g- The case where f | is calculated is shown. The minimum value of these values is detected, and 2-bit direction data regarding the horizontal direction indicating the minimum value is determined.

A total of 8 bits of direction data for the above four directions is determined as a class code indicating the class of the target transmission pixel. Here, in addition to the minimum value of the absolute value of the difference, the subtraction formula may be fixed and 1 bit indicating the polarity of the obtained difference may be added. For example (a-
When the absolute value of the difference obtained by the equation y) is the minimum, the polarity of the difference may be represented by 1 bit, and the code signal of 3 bits may be formed by combining with the 2 bits of the direction data. In this case, the class code indicating the class is 12 bits.

As class information, in addition to the direction data,
A value obtained by converting the value y of the transmission pixel of interest into a smaller number of bits by requantization or ADRC (encoding adapted to the dynamic range) may be used. For example, when the value of the transmission pixel of interest is quantized into 3 bits, 8 bits + 3 bits = 11 bits are the class code.

As described above, the direction data is formed by detecting the one having the smallest difference value in the same direction in the small area centered on the transmission pixel of interest, and further, the direction data concerning a plurality of directions is formed. By performing the class division consisting of, it is possible to determine the class of the transmission pixel of interest with a small number of bits, in other words, with a small number of classes. Since the quantized value of the value of the transmission pixel of interest can be further added to the direction data to be used as class information, it is possible to classify the class with higher accuracy. If 8-bit data of peripheral transmission pixels is used as it is, the number of classes becomes enormous, and the memory capacity and the scale of hardware such as a memory control circuit become too large. Peripheral transmission pixels a-
Even if each h is compressed to 2 bits, the total number of bits becomes 16 bits, and the number of classes is too large. The above-described classification method can solve such a problem.

The correction value generation circuit 5 generates a correction value by linear linear combination of the values of the transmission pixels around the prediction coefficient from the memory 4. For this prediction, my own value y
Not used. In addition, as the number of pixels for prediction, eight pixels a to h or a number of surrounding transmission pixels larger than this are used. The interpolation value generation circuit 8 uses the linear primary combination of the prediction coefficient from the memory 7 and the corrected value of the peripheral transmission pixel to
Generate interpolated values. As shown in FIG. 2A, four pixels are used for classification, but more pixel values (uncompressed) are used for prediction.

Another method of classifying the target transmission pixel will be described. In the pixel array shown in FIG. 2B, the following absolute value of the difference value is formed between the surrounding transmission pixels a to h and the target transmission pixel y.

[0037]   D1 = | a−y | D2 = | by−y | D3 = | c−y |   D4 = | d−y | D5 = | e−y | D6 = | f−y |   D7 = | g−y | D8 = | h−y |

The minimum value among these eight difference values D1 to D8 is detected, and 3-bit direction data indicating the direction having the minimum value is formed. Further, the polarity of the minimum difference value is represented by 1 bit. Therefore, the class code indicating the class is 4 bits. The direction data indicates the direction in which the correlation of the images in the small area as shown in FIG. 2B is strong. Note that the polarity bit is not always necessary. Further, the value of the transmission pixel of interest itself is compressed from 8 bits to 3 bits to form a class code of 4 bits + 3 bits = 7 bits.

Next, another example of classifying thinned pixels and still another example will be described. The range of pixels referred to for classification is wider than in the case of FIG. 2A,
As shown in FIG. 5A. That is, by using the thinned pixel of interest and the transmission pixels A, B, C and D on the upper, lower, left and right sides,
Classify. Further, these transmission pixels A to
The second classification is performed using the average value of D and the transmission pixels E to P around them. Then, the first and second classifications are integrated into the class of the target thinned pixel.

Generation of an average value for the second classification will be described. As shown in FIG. 5B, by using a transmission pixel A, a pixel O above it, and pixels E and H diagonally above it,
An average value A ′ (= (1/4) · (A + E + H + O)) is generated. The average value B is obtained by the transmission pixel B, the transmission pixel M on the right side of the transmission pixel B, the pixel I diagonally above the pixel I, and the pixel J diagonally below the pixel I.
'(= (1/4). (B + I + J + M)) is generated. Similarly, an average value C ′ (= (1/4) · (C + F + G + P)) is generated by the transmission pixel C and the transmission pixels F, G, P in the periphery thereof, and the transmission pixel D and the transmission in the periphery thereof are generated. Pixels K, L,
Depending on N, the average value D '(= (1/4). (D + K + L +
N)) is generated.

As shown in FIG. 5C, the above-mentioned average values A'to D'are estimated values of the thinned pixels adjacent to the peripheral pixels A to D of the target thinned pixel. This average value A '~
A second classification is performed using D '. In the first classification, as in the method shown in FIG. 3, the average value Av of four transmission pixels A to D on the upper, lower, left and right sides is calculated, and the magnitude relationship between each pixel A to D and this average value Av is calculated. Generate class code accordingly. Similarly, the average value Av ′ of A ′ to D ′ as the estimated value is calculated. The average value Av ′ and the average value A ′
A 4-bit second class code is generated according to the magnitude relation of ~ D '. Eight bits, which is a combination of both the first and second class codes, is adopted as the class code of the target thinned pixel.

As described above, in the small area centered on the target thinned pixel, the average value A is added to the peripheral transmission pixels a to d.
By performing classification using'-D ',
It is possible to determine the class of the thinned pixel of interest by reflecting the characteristics of a wide area and by using a small number of bits, in other words, a small number of classes.

FIG. 6 shows another example of classification of thinned pixels. The class of the target thinned pixel is determined based on the respective directivities of the transmission pixels B and D on the left and right sides of the thinned pixel. FIG. 6A shows a state where the directionality is checked centering on the transmission pixel D on the left side of the target thinning pixel, and FIG. 6B shows a state where the directionality is checked centering on the transmission pixel B on the right side of the target thinning pixel.

That is, as shown in FIG. 6A, the transmission pixel D on the left side and the eight transmission pixels (A, B, C,
G, H, K, L, N) and the respective differences are calculated. The one with the smallest difference value is detected. To indicate 8 directions, 3 direction bits are needed, and the direction bits indicate the direction that produces the smallest difference value. Similarly, as shown in FIG. 6B, the transmission pixel B on the right side and the surrounding 8 pixels
The minimum value of the difference from each of the transmission pixels (A, C, D, E, F, I, J, M) is detected, and 3 direction bits are formed according to the detection result. Six bits, which is a combination of these two direction bits, is adopted as the class code. If necessary, the code bit indicating the polarity of the smallest difference can be included in the class code.

The present invention is not limited to the above method of classifying transmission pixels or thinned pixels. A feature of the present invention is that the transmission pixels themselves are corrected by the class classification adaptive processing, and the thinned pixels are interpolated by the class classification adaptive processing using the corrected transmission pixels.

As described above, each of the memories 4 and 7 stores the prediction coefficient, representative value or normalized representative value obtained by learning in advance. FIG. 7 shows a configuration at the time of learning for determining a prediction coefficient for correcting a transmission pixel. The learning process is similar to the process of forming the digital video signal supplied to the input terminal 1 of FIG. 1 from the original digital video signal. Through learning, a coefficient that minimizes the sum of squares of the error of the predicted value with respect to the true value of the transmission pixel of interest is determined by the least square method.

In FIG. 7, the original digital video signal is supplied to the input terminal indicated by 11. A prefilter 12, a subsampling circuit 13, and a postfilter 15 are connected to the input terminal 11. The sub-sampling circuit 13 is supplied from the input terminal 14 with a sampling pulse having a predetermined frequency for performing offset sub-sampling. Therefore, at the output of the post filter 15, the offset sub-sampled digital video signal is obtained.

A time series conversion circuit 16 is connected to the post filter 15, and the video data converted from the raster scan order to the block order is supplied to the class classification circuit 17. Similar to the class classification circuit 3 described above, the class classification circuit 17 determines the class of the transmission pixel of interest using the transmission pixels around the transmission pixel of interest. The class code from the class classification circuit 17 is supplied to the coefficient determination circuit 18.

The coefficient determining circuit 18 determines a prediction coefficient that minimizes the sum of squares of the error between the predicted value y'generated by linear linear combination and its true value y. The original data supplied to the input terminal 11 is supplied to the time series conversion circuit 19, and the true value of the transmission pixel of interest is supplied from this circuit 19 to the coefficient determination circuit 18. The transmission pixels used for prediction are supplied from the time series conversion circuit 16 to the coefficient determination circuit 18.

The coefficient determining circuit 18 determines the best prediction coefficient by the method of least squares. The determined prediction coefficient is stored in the memory 20. The storage address is designated by the class code from the class classification circuit 17. After the coefficients for the transmission pixels are thus determined, the coefficients for the interpolation of the thinned pixels are determined. FIG. 8 shows an example of a configuration when learning coefficients for thinned pixels.

The prefilter 1 is connected to the input terminal 11.
2, the sub-sampling circuit 13 and the post filter 15 are connected to perform offset sub-sampling processing. The output signal of the time series conversion circuit 16 is the class classification circuit 1
7 and classifies the transmission pixels. Then, the class code from the class classification circuit 17 is supplied to the memory 20 as an address.

The memory 20 stores coefficients for correcting the transmission pixels in advance by the learning shown in FIG. The coefficient read from the memory 20 and the output signal of the time series conversion circuit 16 are supplied to the correction value generation circuit 21.
The correction value generation circuit 21 generates a correction value by linearly combining the coefficient and the values of a plurality of transmission pixels in the periphery. The thus-corrected transmission pixel is used to generate a prediction coefficient for the thinned pixel.

That is, the output signal of the correction value generation circuit 21 is supplied to the class classification circuit 22, and the thinned-out pixels of interest are classified into classes using the correction transmission pixels. Similar to the class classification circuit 6 described above, the class classification circuit 22 determines the class of the target thinned pixel using the thinned pixels around the target thinned pixel. The class code from the class classification circuit 22 is supplied to the coefficient determination circuit 23.

The coefficient determination circuit 23 determines a prediction coefficient that minimizes the sum of squares of the error between the predicted value generated by linear linear combination and its true value. The original data supplied to the input terminal 11 is supplied to the time series conversion circuit 24, and this circuit 2
The true value of the thinned pixel of interest is supplied from 4 to the coefficient determining circuit 23. Further, the correction value generation circuit 21 supplies the transmission pixels used for prediction to the coefficient determination circuit 23. The coefficient determination circuit 23 determines the best prediction coefficient by the least square method. The determined prediction coefficient is the memory 2
Stored in 5. The storage address is the class classification circuit 22.
Instructed by the class code from.

As an example, regarding the operation of performing the coefficient determination processing relating to the correction value of the transmission pixel by software processing,
This will be described with reference to FIG. The determination of the coefficient related to the interpolated value of the thinned pixel is also performed by the same process as in FIG. First, control of the process is started from step 41, and in the learning data formation of step 42, learning data corresponding to a known image is formed. At the end of the data in step 43,
If the processing of all the input data, for example, the data of one frame has been completed, the control proceeds to the prediction coefficient determination of step 46, and if not completed, the control proceeds to the class determination of step 44.

As described above, the class determination in step 44 is a step in which the class is determined in correspondence with the level distribution pattern of the value of the transmission pixel of interest and the values of the peripheral pixels. In the normal equation generation in the next step 45, a normal equation described later is created.

After the processing of all the data is completed from the end of the data in step 43, the control proceeds to step 46, and step 4
In the determination of the prediction coefficient of 6, the coefficient is determined by solving the equation (8) described later using the matrix solution method. The prediction coefficient is stored in the memory 20 by the prediction coefficient store in step 47, and the control of the learning process ends in step 48.

Step 45 in FIG. 9 (normal equation generation)
The process of step 46 (determination of prediction coefficient) will be described in more detail. At the time of learning, the true value y of the transmission pixel of interest is known. When the correction value of the transmission pixel of interest is y ′ and the values of the surrounding pixels are x ₁ to x _n , the coefficients w ₁ to w for each class are given.
_An n-tap linear primary combination by n y ′ = w ₁ x ₁ + w ₂ x ₂ + ... + w _n x _n (1) is set. Before learning, w _i is an undetermined coefficient.

As described above, learning is performed for each class,
When the number of data is m, according to the equation (1), y _j ′ = w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (2) (where j = 1, 2, ...

When m> n, w _{1 to} w _n are not uniquely determined, so the elements of the error vector E are e _j = y _j − (w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (3) (However, j = 1, 2, ..., M) is defined, and a coefficient that minimizes the following expression (4) is obtained.

[0061]

[Equation 1]

This is the so-called least squares method. Here, the partial differential coefficient by w _i of the equation (4) is obtained.

[0063]

[Equation 2]

Since each w _i may be determined so that the equation (5) becomes 0,

[0065]

[Equation 3]

If a matrix is used as

[0067]

[Equation 4]

It becomes This equation is generally called a normal equation. If this equation is solved for w _i using a general matrix solution method such as a sweeping method, the prediction coefficient w _i is obtained, and this prediction coefficient w _i is stored in the memory 20 using the class code as an address.

Although FIG. 9 shows a software structure for learning, learning can be performed by a hardware structure or a combination of software and hardware. Further, the interpolation value and the correction value are respectively formed not only by linear linear combination using the prediction coefficient,
The values themselves of these data may be created in advance by learning, and these values may be respectively adopted as the interpolation value and the correction value.

FIG. 10 is a flow chart for explaining learning for creating the data value itself in advance. The control start step 51, the learning data formation step 52, the data end step 53, and the class determination step 54 are the same as the steps 41, 42, 43, and 44 in the learning for determining the prediction coefficient described above. Is the step of performing.

The representative value determining step 55 is a step of obtaining an average value of true values for each class and determining the average value as a representative value. That is, the representative value is obtained by dividing the cumulative value of the true value obtained in the learning process by the cumulative frequency. As described above, the process of using the average value of the true values as the representative value is called the centroid method. Further, when the data values themselves are accumulated, the accumulated data amount increases, so a value normalized by the reference value in the block and the dynamic range DR of the block may be obtained as the representative value.

That is, when the reference value of the block is B (for example, the minimum value of the pixels in the block) and the dynamic range is represented by DR, the normalized representative value G is G = (y−B) / DR Stipulated. In step 56, the determined representative value is stored in the memory, and the learning ends.

When the thus normalized value is obtained by learning, the configuration shown in FIG. 11 is used for generating the interpolation value or the correction value. FIG. 11 shows only a configuration for generating a correction value for simplicity. The output signal of the time series conversion circuit 2 is the class classification circuit 3 and the detection circuit 27.
Is supplied to. The normalized representative value is read from the address of the memory 4 designated by the class code from the class classification circuit 3. Further, the detection circuit 27 detects the dynamic range DR and the minimum value MIN of a plurality of transmission pixels used for prediction.

The normalized representative value from the memory 4 is the multiplication circuit 2
8 and the normalized representative value is multiplied by the detected dynamic range DR. The output of the multiplication circuit 28 is supplied to the addition circuit 29 and is added to the detected minimum value MIN. The output signal of the adder circuit 29 is the correction value.

For class classification or prediction calculation according to the present invention, the values of the pixels surrounding the pixel of interest are not limited to those spatially used, but a pixel close to the pixel of interest in the time direction (for example, the same pixel in the previous frame). Pixels) can also be used.

Further, the interpolated value and the correction value are not limited to be predicted by the same prediction method, and the above-described prediction formula (linear 1
It is also possible to combine prediction by (second combination), prediction using a representative value, and prediction using a normalized representative value.

[0077]

According to the present invention, not only the pixels thinned out by sampling but also the values of transmission pixels are corrected, so that the high frequency component lost by the filtering process for sampling can be compensated. . Therefore, the distortion of the waveform of the decoded signal can be compensated, and the quality of the decoded image can be improved.

Further, according to the present invention, since the thinned pixels are interpolated by using the corrected transmission pixels, the interpolation can be favorably performed.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram showing positions of pixels to be referred to for classification.

FIG. 3 is a schematic diagram for explaining an example of a method of classifying thinned pixels.

FIG. 4 is a schematic diagram for explaining an example of classification of transmission pixels.

FIG. 5 is a schematic diagram for explaining another example of classification of thinned pixels.

FIG. 6 is a schematic diagram for explaining still another example of classification of thinned pixels.

FIG. 7 is a block diagram of an example of a configuration for obtaining a prediction coefficient regarding a transmission pixel by learning.

FIG. 8 is a block diagram of an example of a configuration for obtaining a prediction coefficient regarding a thinned pixel by learning.

FIG. 9 is a flowchart when learning for obtaining a prediction coefficient is performed by software processing.

FIG. 10 is a flowchart when learning for obtaining a representative value is performed by software processing.

FIG. 11 is a block diagram illustrating a process for generating a prediction value using a normalized representative value.

FIG. 12 is a block diagram of an example of a configuration for offset subsampling.

FIG. 13 is a schematic diagram showing a structure of two-dimensional offset subsampling.

[Explanation of symbols]

3, 6 class classification circuit 4,7 Prediction coefficient memory 5 Correction value generation circuit 8 Interpolation value generation circuit

─────────────────────────────────────────────────── ─── Continuation of front page (56) References JP-A-5-328185 (JP, A) JP-A-6-178277 (JP, A) JP-A-7-212752 (JP, A) JP-A-7- 240903 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N ^7/ 00-7/088 H04N ^7/ 24-7/68

Claims (5)

(57) [Claims] 1. A digital image signal processing for sampling a digital image signal that has passed through a pre-filter, receiving a signal in which the number of pixels is reduced by the sampling, and interpolating pixels thinned by the sampling. A plurality of transmission pixels around a transmission pixel of interest existing in the received digital image signal in the apparatus, a first classifying means for determining a class of the transmission pixel of interest, and the input digital image When the value of the transmission pixel of interest is created by a linear linear combination of the values of a plurality of transmission pixels spatially and / or temporally adjacent to the transmission pixel of interest included in the signal, and a first coefficient, A first coefficient for generating the first coefficient for each class so as to minimize the error between the created value and the true value of the transmission pixel of interest. A corrected linear value of the transmission pixel of interest is generated by a generating means and a linear linear combination of the first coefficient and the values of a plurality of transmission pixels spatially and / or temporally adjacent to the transmission pixel of interest. For receiving a correction transmission pixel from the first calculation means, and using a plurality of correction transmission pixels around the target thinning pixel,
Second class classification means for determining the class of the target thinning pixel, and linear first-order values of a plurality of correction transmission pixels spatially and / or temporally adjacent to the target thinning pixel and a second coefficient When the value of the above-mentioned thinned pixel of interest is created by combining,
Second coefficient generating means for generating a second coefficient for each class so as to minimize an error between the created value and the true value of the target thinned pixel, the second coefficient and the above Spatial and / or thinned pixel of interest
Or a second calculation means for generating an interpolated value of the thinned-out pixel of interest by linear linear combination with the values of a plurality of correction transmission pixels that are temporally close to each other. Processing equipment. 2. The digital image signal processing device according to claim 1, wherein the first and second coefficient generating means respectively determine the first and second coefficients by a least square method. Characteristic digital image signal processing device. 3. A digital image signal processing for sampling a digital image signal that has passed through a pre-filter, receiving a signal in which the number of pixels is reduced by the sampling, and interpolating pixels thinned out by the sampling. In the apparatus, a first class classification means for determining the class of the transmission pixel of interest by using a plurality of transmission pixels around the transmission pixel of interest existing in the received digital image signal; The stored first representative value is stored for each class, and the first representative value corresponding to the class determined by the first class classification means is output as a value of the transmission pixel of interest. One memory means and receiving the correction transmission pixel from the first memory means,
A second method for determining the class of the thinning-out pixel of interest using a plurality of correction transmission pixels around the thinning-out pixel of interest.
And the second representative value acquired by learning in advance is stored for each class, and the second representative value corresponding to the class determined by the second class classifying means is the above-mentioned 2. A digital image signal processing device, comprising: a second memory means for outputting as a value of a thinned pixel. 4. The digital image signal processing device according to claim 3, wherein the representative values stored in each of the first and second memory means are a target transmission pixel and a target thinning pixel given at the time of learning. A digital image signal processing device characterized by averaging true values. 5. The digital image signal processing device according to claim 3, wherein the representative value stored in each of the first and second memory means is within a block including a target transmission pixel and a target thinning pixel, respectively. Processing of a digital image signal, which is formed from values obtained by normalizing true values of the transmission pixel and the thinned-out pixel of interest, respectively, according to the reference values of the plurality of pixels and the dynamic range of the block. apparatus.