JP2003230147A - Apparatus and method for coding image signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method for coding image data in which deteriorations of colors of an edge and a macro block including an edge of an image are suppressed. <P>SOLUTION: In the coding apparatus (EN) for compression-coding an image signal (Dv) to be input, an edge detector (1006) detects the edge at each frame of the image signal (Dc) to generate edge information (Ie) representing a strength of the edge. An edge class information generator (1007) generates edge class information (Iec) representing an importance of the macro block (MB) based on the edge information (Ie). An amount-of-quantization deciding unit (1008) reduces a quantizing step (VQc) for the macro block (MB) having a high importance based on the edge class information (Iec) and increases the step (VQc) for the macro block (MB) having a low importance. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-efficiency coding method for video signals representing moving images, and more particularly to a coding method suitable for storage media such as optical disks and hard disks. The present invention relates to a coding method for controlling a quantizer of each macroblock based on edge class information generated based on edge information at a level and edge information at a color difference level.

[0002]

2. Description of the Related Art In recent years, there has been an increasing demand for storing and storing video data in personal homes using personal computers. However, since video data is much larger than conventional digital data, a data accumulator having an enormous storage capacity is required for its storage. In addition, the capacity of the data storage device has been increasing in recent years, but the storage capacity is still insufficient as compared with the size of each video data to be stored. Further, even when the capacity of the data storage device is drastically increased, the data storage device and its storage medium are expensive, so that it cannot be used as easily as a conventional video tape.

Further, in the personal computer, digital data other than video data is also stored in the data storage and storage medium. Therefore, even when the data storage device and storage medium become cheaper,
A technology for efficiently compressing and storing video data as well as other digital data is extremely important. MPEG is generally used as a video data compression technique. Also in MPEG, fixed rate coding and variable rate coding are selectively used according to the form in which compressed data is provided to the user.

For example, when data is distributed via a general transmission line, fixed rate coding is generally adopted.
On the other hand, in the case of being provided by being stored in a package medium such as a DVD having a limited storage capacity, variable rate coding that makes the coding rate variable according to the scene is adopted. Variable rate coding effectively utilizes the property that a video generally includes a large amount of information and a non-information-rich scene.

Below, FIG. 15, FIG. 16, FIG. 17, and FIG.
A conventional MPEG encoding device and a video encoding method thereof will be described with reference to FIGS. FIG. 17 shows the configuration of an MPEG video encoding device (hereinafter, abbreviated as “encoding device”) that compresses data using a motion-compensated DCT method, which is generally used in the past. The motion-compensated DCT method compresses one frame selected periodically in the input video data by using only the data in the frame (intra-frame compression), and the remaining frames are processed in the previous frame. This is a method of compressing (compressing between frames) the difference between and and transmitting.

For the intra-frame compression and the inter-frame compression, a discrete cosine transform, which is a kind of orthogonal basis transform, is typically used. In addition, when calculating the difference between frames, the motion vector of the video with respect to the previous frame is detected, and the difference is calculated after matching the motions, thereby significantly improving the compression rate.

As shown in FIG. 17, an encoder ENc is used.
Is an input terminal 1800, a reorder unit 1801, a selector 1802, a first encoder 1803, a code generation amount recorder 1
804, target code amount calculator 1805, second encoder 18
08, and an output terminal 1809. Note that the overall operation of the encoding device ENc is a code that determines encoding information such as the type of input video data Dv, an encoding target portion, and an encoding method based on an encoding instruction from the user. It is controlled by an activation controller (not shown). Then, the individual components of the encoding device ENc perform the processing described below in order to execute the operation determined and controlled by the encoding controller.

The second encoder 1808 outputs the video data Dv
Is provided to encode encoded video data Denc that is actually stored in the storage device or storage medium. On the other hand, the first encoder 1803, the code generation amount recorder 1804, and the target code amount calculator 1805 include the second encoder 1808 so that the second encoder 1808 can efficiently encode the video data Dv. It is provided to determine the target code amount of the video data Dv before the video data Dv is encoded. The second encoder 1808 actually encodes the video data Dv on the basis of the determined target code amount to obtain the encoded data Den.
produces c.

Video data Dv is input to the reorder unit 1801 from an external video data source (not shown) via the input terminal 1800. Reorder device 1801
Generates the coding order video data Dc by rearranging the videos input in the display order of the video data Dv in the coding order.
From the viewpoint of the order of images, the image data Dv is also referred to as display order image data Dv.

FIG. 15 shows a state in which the videos arranged in the video data Dv in the display order are rearranged in the coding order to generate the coding order video data Dc. In the figure, “I” is an intra picture frame (hereinafter referred to as “I picture”) to be intra-frame coded, and “P” is a predictive picture frame to be forward predictive coded (hereinafter “P picture”). ")," B "is a bidirectional predictive coding that predictively codes from the front and back.
A picture frame (hereinafter referred to as "B picture") is shown. In this example, the display order video data Dv is displayed in the display order Ovw in the frames I0, B1, B2, P3, B4, B.
5 and P6. The frame P6 and the subsequent frames are omitted for simplification. In the encoding order video data Dc, the video is I0, P3, B1, B2, P6, B4,
And B5 are arranged in the encoding order Oen.

Returning to FIG. 17, the selector 1802 reorders the target encoder in order to calculate the target code amount before the second encoder 1808 encodes the encoded video data Dc and generates the encoded data Denc. The encoding order video data Dc input from the encoder 1801 is converted into the first encoder 18
Output to 03. The first encoder 1803 and the code generation amount recorder 180 are based on the encoded video data Dc.
4, and as a result of the processing by the target code amount calculator 1805, the target code amount is determined. Next, the second encoder 18
In order to perform encoding based on the target code amount calculated by 08, the selector 1802 determines that the encoding order video data D
c is output to the second encoder 1808. Note that the encoding of the encoding-order video data Dc by the second encoder 1808 is repeatedly performed until the encoding instructed by the user is completed.

That is, the selector 1802, based on the path switching signal Sw supplied from the above-mentioned encoding controller, performs the encoding sequence video data Dc during the first encoding and the second and subsequent encoding processes. Switch the output destination of. In addition,
For convenience of explanation, the first encoder 180 is used in the first encoding.
The encoded sequence video data Dc output to 3 is referred to as first-time encoded video data Dc1 and is referred to as the second encoded data after the second encoding.
Encoding order video data Dc output to the encoder 1808
Is referred to as encoded video data Dc2 from the first second time onward,
Each shall be identified.

First, at the time of the first encoding, the input image (first-time encoded image data Dc1) is processed by the first encoder 18
The first encoded video data Dc1e encoded by
Is generated. The code generation amount recorder 1804 receives the first encoded video data D input from the first encoder 1803.
Generated code amount D generated by encoding based on c1e
1 is detected and recorded in frame (F) units or macroblock (MB) units, and the detected generated code amount D1 is notified to the target code amount calculator 1805.

In the target code amount calculator 1805, based on the generated code amount D1 notified from the code generation amount recorder 1804, the generated code amount D1 (F) of each frame and the generated code amount D1 (S) of the entire scene. Then, the target code amount DT of each frame is calculated and notified to the second encoder 1808.

In FIG. 16, the solid line Ld1 shows the variation of the generated code amount of each frame in the first encoder 1803, and the solid line Ldt shows the variation of the target code amount DT (t) of each frame. The horizontal axis represents the frame t, the vertical axis D1 (t) represents the generated code amount for each frame, and DT
(T) represents the target code amount DT (t) of the frame. The total generated code amount is represented by ΣD1 (t), and the target total code amount, which is the total amount of the target code amount DT in each frame, is ΣDT.
It is represented by (t).

The target code amount calculator 1805 calculates the target total code amount ΣDT (t) and the generated code amount D1 of the t-th frame.
(T) and the total generated code amount ΣD1 (t),
The target code amount DT (t) of each frame is calculated using the following Expression 1. DT (t) = D1 (t) * (ΣDT (t) / ΣDT1 (t)) (Equation 1)

The second encoder 1808, based on the target code amount DT (t) for each frame notified from the target code amount calculator 1805, for the first and subsequent encoded video data Dc2. The second and subsequent encodings are performed. In this way, after the first encoding is completed and the target code amount DT (t) of each frame is calculated, the second and subsequent encodings are performed.

In the second and subsequent encodings, the selector 180
2 outputs the encoded video data Dc to the second encoder 1808, which causes the second encoder 1808 to encode the encoded video data Dc2 for the first and subsequent times. The second encoder 1808 performs encoding so that the target code amount DT (t) of each frame calculated by the target code amount calculator 1805 is obtained, and the encoded data Denc is generated.

Next, FIG. 18 shows the configuration of the first encoder 1803. The first encoder 1803 has an input terminal 190.
0, subtractor 1901, discrete cosine transform (DCT) device 1
902, quantizer 1903, variable length coding (VLC) device 1904, inverse quantizer 1905, inverse DCT (IDCT)
1906, adder 1907, refresh controller 19
08, refresh switch 1909, refresh switch 1910, frame memory 1911, motion compensator 1912, output terminal 1913, and motion detector 191.
Including 4.

The input terminal 1900 is connected to the selector 1802 and receives the first-time encoded video data Dc1. One of the input ports of the subtractor 1901 has an input terminal 1
900, and the other input port is refresh switch 1909, refresh switch 1910,
Motion compensator 1912, frame memory 1911, adder 1907, inverse DCT unit 1906, inverse quantizer 1905,
Quantizer 1903 and discrete cosine transformer 1902
Connected to its own output port via the first encoded video data Dc1t of the current frame and the first frame before the predetermined number of frames (d) restored as a result of various processes described later.
The first-time encoded video difference ΔDc1 that is the difference from the motion-compensated video data α of is calculated.

As will be described later in detail, the first motion-compensated video data α is generated by motion-compensating the video data Dc1rd in the first delay frame, and the first delay frame. And the data generated by motion compensation of the inter-picture data (ΔDc1r + α) d. However, the first-time encoded video difference ΔDc1 includes only the first-time encoded video data Dc1 and the first motion-compensated video data α of the first delayed inter-frame video data (ΔDc1r + α) d. Calculated by subtraction.

The refresh controller 1908 controls on / off of the refresh switch 1909 and the refresh switch 1910 according to the type of picture of the first-coded video data Dc1 to be processed. That is, when the picture to be processed is an I picture, the refresh controller 1908 refreshes the refresh switch 1909 to perform intraframe compression.
And 1910 by turning off subtractor 19
The first encoded video difference ΔDc1 of 01 is not generated. Regarding the operation of the refresh controller 1908, an inverse DCT unit 1906, an adder 1907, an
The relationship between the frame memory 1911 and the operation between the frame memories 1911 will be described in detail.

As described above, the first-time encoded video data Dc1 input from the input terminal 1900 is subtracted from the subtracter 190.
It may be output to the discrete cosine converter 1902 without being subjected to the subtraction processing by 1, or the first-time encoded video difference ΔDc1 generated by the subtraction processing by the subtractor 1901 may be output. The first-coded video data Dc1 or the first-coded video difference ΔDc1 is discrete cosine transformed by a discrete cosine transformer (DCT) 1902. The discrete cosine transform is performed in two dimensions.
Discrete cosine transform is performed for each block defined by 8 pixels in the vertical direction and 8 pixels in the horizontal direction (hereinafter, abbreviated as “8 × 8” block), and 8 × 8 = 64 coefficients are obtained as a result of the transform. In addition, these 64 coefficients are converted into the first coefficient K1.
Collectively. The discrete cosine transformed data is originally a continuous quantity, but since it is calculated using a digital circuit, each of the 64 coefficients forming the first coefficient K1 has a digital value of a predetermined bit width. Obtained as.

This data (first coefficient K1) is optimally quantized by the quantizer 1903 for each frequency component. Usually, the low-frequency component is important for forming an image, so the quantization step is made fine. Then, the high-frequency components are not so important in composing the image, so that the quantization step is roughened. At this point, the quantizer is not controlled and a fixed quantization Q set from the outside is set.
Quantization with S produces a quantized first coefficient K1Q. Generally, by using the fixed quantization QS, the quantization error becomes approximately the same in each frame, so that the image quality becomes the same in each frame. The difference (ΔAD1e) in the generated code amount (AD1e) of each frame at this time can be defined as the so-called compression difficulty.

That is, when encoding with fixed quantization QS,
It is said that the larger the generated code amount (AD1e), the higher the degree of compression difficulty. The variable length encoder 1904 variable length encodes the quantized first coefficient K1Q output from the quantizer 1903 to generate first encoded video data Dc1e. Variable length coding is a method of assigning a shorter code length to data having a statistically higher occurrence probability. By this method, the statistically redundant component of the data is removed.

On the other hand, the quantized first coefficient K1Q output from the quantizer 1903 is dequantized by the inverse quantizer 1905. That is, the inverse quantizer 1905
In contrast to the quantization, the first coefficient K1 is restored by returning the amplitude of each frequency component to the original amplitude. However, the inverse quantizer 1905 cannot completely restore the first coefficient K1 due to uncertain factors such as noise during quantization by the quantizer 1903. That is, the inverse quantizer 1905 has a first coefficient K1 that is substantially the same as the first coefficient K1.
The coefficient K1r of is generated (restored). The first coefficient K1r is referred to as a restored first coefficient K1r and is discriminated from the first coefficient K1.

The inverse DCT device 1906 is an inverse quantizer 190.
Inverse discrete cosine transform (IDCT) processing is further applied to the restored first coefficient K1r output by No. 5, and the first encoded video data Dc1 or 1 which is the original video data.
The first-coded video data Dc1r or the first-coded video difference ΔD that is almost the same as the first-coded video difference ΔDc1
Generate c1r. These first encoded video data Dc
The 1r and first encoded video difference ΔDc1r are referred to as the restored first encoded video data Dc1r and the restored first encoded video difference ΔDc1r, respectively.

When the output from the inverse DCT unit 1906 is the restored first encoded video data Dc1r, that is, the intra-frame video data, the refresh controller 1908 turns off the refresh switch 1910 and the adder 1907 as described above. Do not execute the addition process. on the other hand,
When the output from the inverse DCT unit 1906 is the restored first encoded video difference ΔDc1r, that is, the inter-frame video data, the refresh controller 1908 turns on the refresh switch 1910, and the adder 1907 outputs the first
The addition processing of the motion-compensated video data α is executed. Therefore, the adder 1907 outputs the restored first-time encoded video data Dc1r or the restored first-time encoded video data Dc.
Either 1r + first motion-compensated video data α is output. After that, the restored first encoded video difference ΔDc1r + first
The motion-compensated video data α is referred to as first refresh data (ΔDc1r + α).

After that, the frame memory 1911 stores the restored video data (the restored first-time encoded video data Dc1).
r or some other first refresh data (ΔDc1r
+ Α)) is delayed by a predetermined number (d) of frames to generate frame-delayed video data. That is, the frame memory 1911 stores the restored first encoded video data Dc1r.
Is delayed by d frames to generate the first delayed frame video data Dc1rd, and the first refresh data (ΔDc1r + α) is delayed by d frames to generate the first delayed interframe video data (ΔDc1r +
α) d is generated.

The motion detector 1914 has an input terminal 1900.
On the basis of the first-time encoded video data Dc1 input from A
Detect m.

The motion compensator 1912 is a motion detector 191.
Based on the motion amount Am detected in step 4, the position of the image represented by the first delayed frame image data Dc1rd or the first delayed interframe image data (ΔDc1r + α) d output from the frame memory 1911 is currently determined. Of 1
First motion-compensated video data α is generated, which is motion-compensated so as to have the same position as the video represented by the second-time encoded video data Dc1.

When this first motion-compensated video data α is output to the subtractor 1901, it is used to calculate the difference (ΔDc1) from the next video data (first-time encoded video data Dc1). Be seen. However, since the first motion-compensated video data α generated by motion-compensating the first delayed intra-frame video data Dc1rd is substantially the same as the current first-time encoded video data Dc1, the subtractor 1901 is used.
Therefore, it is meaningless to use it to generate the first encoded video difference ΔDc1.

That is, the video data of several frames following the I picture (first-time encoded video data Dc1) is used to compress the difference (ΔDc1) from the video data of the previous frame. Therefore, it is necessary to prevent the motion-compensated video data generated by motion-compensating the video data Dc1rd in the first delayed frame from being used for the first-time encoded video difference ΔDc1 generation by the subtractor 1901. is there.

Therefore, the refresh controller 1908 is
If the picture to be processed is a P picture or a B picture, the refresh switches 1909 and 1910 are turned on to restore the first refresh data (ΔDc1r + α) from the adder 1907, not the first encoded video data Dc1r. ) Is the frame memory 1911
The frame memory 1911 outputs the first delayed interframe video data (ΔDc1r + α) d to the motion compensator 1912, and the motion compensator 1912 outputs the first delayed interframe video data (ΔDc1r + α) d to the motion compensator 1912.
Of the delayed inter-frame video data (ΔDc1r + α) d of the first motion-compensated video data α of the adder 19
07 and the subtractor 1901 to output the inter-frame compression.

The refresh switch 1909 is turned on when calculating the difference between frames, and the subtractor 190
Used to run 1. Refresh switch 1
The refresh switch 910 repeatedly turns on and off in the same cycle as the refresh switch 1909. Refresh switch 1
When 910 is on, the adder 1907 operates to operate the inter-frame difference data (restoration first-time encoded video difference ΔDc1).
r) and the previous frame data (first motion-compensated video data α1) are added to restore the frame (first refresh data (ΔDc1r + α)). The variable length encoder 1904 uses the inter-frame compressed data (quantized first coefficient K1Q; first-time encoded video difference ΔDc1).
Variable length coding is also performed for.

Here, from I picture to the next I picture is called 1 GOP (group of pictures), and is usually composed of video signals of about 15 frames (about 0.5 seconds).

The first encoded video data Dc1e encoded by the variable length encoder 1904 is output from the output terminal 1913.

Next, FIG. 19 shows the configuration of the second encoder 1808. The second encoder 1808 has an input terminal 200.
0, subtractor 2001, discrete cosine transform (DCT) device 2
002, quantizer 2003, variable length coding (VLC) device 2004, inverse quantizer 2005, inverse DCT (IDCT)
Device 2006, adder 2007, refresh controller 20
08, refresh switch 2009, refresh switch 2010, frame memory 2011, motion compensator 2012, output terminal 2013, motion detector 2014, buffer 2015, and quantization determiner 2016.

Input terminal 2000 and subtractor 200
1, a discrete cosine transformer 2002, a quantizer 2003,
Variable length encoder 2004, inverse quantizer 2005, inverse DC
T unit 2006, adder 2007, refresh controller 2
008, refresh switch 2009, refresh switch 2010, frame memory 2011, motion compensator 2012, output terminal 2013, motion detector 2014
Are input terminal 1900, subtractor 1901, discrete cosine transformer 1902, quantizer 1903, variable length encoder 1904, dequantizer 1905, and inverse quantizer 1905 in the first encoder 1803 shown in FIG. 18, respectively. DCT device 190
6, adder 1907, refresh controller 1908, refresh switch 1909, refresh switch 1
910, frame memory 1911, motion compensator 191
2, the output terminal 1913, and the motion detector 1914.

In the second encoder 1808, the buffer 2 is provided between the variable length encoder 2004 and the output terminal 2013.
015 is newly provided, and a quantization determiner 2016 is further newly provided between the buffer 2015 and the quantizer 2003. That is, the quantization decision unit 201
Reference numeral 6 determines the quantization amount VQ based on the buffer occupation amount information Ib indicating the state of the buffer 2015 and the target code amount DT output from the target code amount calculator 1805, and controls the quantizer 2003. .

The input terminal 2000 is connected to the selector 1802, and the first and second encoded video data Dc2
Is entered. The subtractor 2001 has one input port connected to the input terminal 2000, and the other input port having a refresh switch 2009, a refresh switch 2010, a frame memory 2011, and a motion compensator 201.
2, the adder 2007, the inverse DCT unit 2006, the inverse quantizer 2005, the quantizer 2003, and the discrete cosine transformer 2002, and is connected to its own output port, and the first second time of the current frame. After that, encoded video data D
The difference between c2 (t) and the second motion-compensated video data β before the restored predetermined number of frames (d) is obtained as the first second-time encoded video difference ΔDc2.

As will be described in detail later, the second motion-compensated video data β includes the motion-compensated second delayed intra-frame video data Dc2rd and the second delayed inter-frame video data (Δ). Dc2r + β) motion compensated. However, the first second encoded video difference ΔDc2 is the encoded video data Dc2 and the second delayed inter-frame video data (ΔDc2r) after the first and second encoded video differences.
+ Β) d of the motion-compensated second motion-compensated video data β alone.

The refresh controller 2008 has a first
The refresh switch 2009 according to the type of the picture of the encoded video data Dc2 to be processed from the second time onward.
Also, it controls on / off of the refresh switch 2010. That is, if the picture to be processed is an I picture, the refresh controller 2008 may refresh the switch 20 to perform intra-frame compression.
By turning off 09 and 2010, the subtractor 2001 does not generate the first second-coded video difference ΔDc2.

As described above, the first and second and subsequent encoded video data Dc2 input from the input terminal 2000 are output to the discrete cosine converter 2002 without being subjected to the subtraction processing by the subtractor 2001. Subtractor 200
The first second-time coded video difference ΔDc2 generated by receiving the subtraction process at 1 may be output to the discrete cosine converter 2002. These first and subsequent encoded video data Dc2 or the first and second encoded video difference ΔDc
2 is subjected to discrete cosine transform by the discrete cosine transformer 2002, and 8 × 8 = 64 coefficients (second coefficient K2)
Is obtained.

The second coefficient K2 is optimally distributed by the quantizer 2003 to each frequency component,
The quantized coefficient K2Q is generated. In the second encoder 1808, unlike the case of the first encoder 1803, the quantization determiner 2016 controls the quantization step QS by the quantizer 2003 at this point. In other words, the quantizer determiner 2016 uses the target code amount DT and the buffer 201 provided for each frame.
Based on the state of 5, the quantization step QS (quantization amount VQ) of each macroblock MB is calculated, and the calculated quantization step QS (quantization amount VQ) is quantized by the quantizer 2
Give to 003

The control of the quantization step QS based on the state of the buffer 2015 is basically performed by the buffer 2
This is the case where the possibility that 015 will fail is detected. In the normal state, the control of the quantization step QS based on the state of the buffer 2015 (buffer occupancy information Ib) is not performed. When calculating the target code amount DT, the buffer state is simulated and the code amount is assigned so that the buffer does not fail. Therefore, if the prediction accuracy is guaranteed to some extent, the normal buffer 2015 may fail. There is no.

The second quantized coefficient K2Q output from the quantizer 2003 is variable-length coded by the variable-length coder 2004 and the second coded video data Dc2.
e is generated. On the other hand, the second quantized coefficient KQ2 is
The inverse quantizer 2005 restores the original quantization to generate the restored second coefficient K2r.

The restored second coefficient K2r restored to the original amplitude by the inverse quantization is the original image data by the inverse DCT unit 2006, which is the encoded image data Dc2r or the second encoded image data after the second restoration. Restoration of 1 is restored to the encoded video difference ΔDc2r after the second time. If the restored video data is the intra-frame video data (the second and subsequent encoded video data Dc2r), the adder 2007
Does not work.

When the output from the inverse DCT device 2006 is the second and subsequent encoded video data Dc2r, that is, the intra-frame video data, the refresh controller 2008 turns off the refresh switch 2010 as described above. Therefore, the addition processing by the adder 2007 is not executed. On the other hand, when the output from the inverse DCT unit 2006 is the encoded video difference ΔDc2r, that is, the inter-frame video data for the first and second restoration, the refresh controller 200
8 turns on the refresh switch 2010 to cause the adder 2007 to execute the addition process of the second motion-compensated video data β. Therefore, the adder 2007 outputs one of the first and second encoded video data Dc2 or the first restored second and later encoded video difference ΔDc2r + second motion-compensated video data β. In addition, after the first restoration, the encoded video difference ΔDc2r + the second motion-compensated video data β is changed to the second refresh data (ΔD).
c2r + β).

After that, the frame memory 2011 stores the restored video data (the second restored second and subsequent encoded video data Dc2r or the second refresh data (ΔD
c2r + β) is delayed by a predetermined number (d) of frames to generate frame-delayed video data. That is, the frame memory 2011 delays the encoded video data Dc2r for the second and subsequent second restorations by d frames to generate the second delayed intra-frame video data Dc2rd, and
The second refresh data (ΔDc2r + β) is delayed by d frames to generate second delayed inter-frame video data (ΔDc2r + β) d.

The motion detector 2014 has an input terminal 2000.
The second and subsequent encoded video data Dc input from
Based on 2, the motion amount Am is detected by obtaining the motion vector between the frames of the video.

The motion compensator 2012 is a motion detector 301.
Based on the motion amount Am detected in step 4, the position of the image represented by the second delayed intra-frame video data Dc2rd or the second delayed inter-frame video data (ΔDc2r + β) d output from the frame memory 2011 is currently determined. The second motion-compensated video data β that is motion-compensated to be the same as the position of the video represented by the encoded video data Dc2 is generated from the first second time.

When the second motion-compensated video data β is output to the subtractor 2001, the first 2 which is the difference from the next video data (the first and subsequent encoded video data Dc2).
It is used to calculate the time-coded video difference ΔDc2.
However, the second delayed frame video data Dc2
The second motion-compensated video data β generated by motion-compensating rd is substantially the same as the current first and subsequent encoded video data Dc2, so that the subtractor 2001 performs the first second-time video data β. It is meaningless to use to generate the encoded video difference ΔDc2.

Therefore, the refresh controller 2008 is
If the picture to be processed is a P or B picture, the refresh switches 2009 and 2010 are turned on. As a result, from the adder 2007 to the second
Restoration second time and thereafter, not the encoded video data Dc2r but the restoration difference ΔDc2r + β is output to the frame memory 2011, and the second delayed inter-frame video data (ΔDc2r + β) d is output from the frame memory 2011.
12 is output to the adder 2007 and the subtractor 2001 from the motion compensator 2012, which is the second motion-compensated video data β obtained by motion-compensating the second delayed interframe video data (ΔDc2r + β) d. Performs interframe compression.

The refresh switch 2009 is turned on when the difference between frames is calculated, and the subtracter 200 is turned on.
Used to run 1. Refresh switch 2
010 repeatedly turns on and off in the same cycle as the refresh switch 2009. Refresh switch 2
When 010 is on, the adder 2007 is operated to add the inter-frame difference data (ΔDc2) and the previous frame data (encoded video data Dc2 for the first and second times) to restore the frame. Be seen. Variable length encoder 200
4 also performs variable length coding on the inter-frame compressed data (ΔDc2).

The second coded video data Dc2e coded by the variable length coder (VLC) 2004 is output from the output terminal 2013.

As described above, the second encoder 180 is used in accordance with the degree of difficulty of encoding (target code amount DT) for each section detected by the first encoding by the first encoder 1803.
8 assigns a code amount to each frame to achieve compression coding suitable for a storage medium.

[0058]

As described above, in the conventional encoding device ENc, the feature of the image is extracted by the first encoding, and the result is reflected in the second and subsequent passes, thereby making it variable. This is a method that allows rate control to efficiently record video data on storage media such as optical disks. However, the coding within each area of the screen is generally controlled based on the activity of the area of interest. This utilizes the property that deterioration of a high activity portion, that is, a region containing a large amount of high frequency components is generally difficult to understand in terms of visual characteristics. However,
Even if the activity is high, the deterioration is remarkable in the edge part. Therefore, it is an object of the present invention to provide a video data coding apparatus and a method thereof in which color deterioration of a macro block including an edge portion and an edge in a video is suppressed.

[0059]

A first aspect of the present invention is an encoding method for compression-encoding an input video signal, which detects an edge for each frame of the video signal and detects the edge. An edge detection step of generating edge information indicating strength, an edge class information generation step of generating edge class information indicating macroblock importance based on the edge information, and an edge class information for each macroblock based on the edge class information. A quantization amount determination step for determining a quantization step is provided, and it is possible to reduce the quantization step for macroblocks with high importance and increase the quantization step for macroblocks with low importance. Characterize.

As described above, according to the first aspect of the present invention, the deterioration of the edge portion due to the coding of the video signal can be prevented by adjusting the quantization step of the macro block having the edge for each frame type.

A second aspect of the present invention is characterized in that, in the first aspect, in the edge detecting step, different intensities are set for edges of the same intensity for each frame type of the video signal.

The third invention is characterized in that, in the first invention, in the edge class information generating step, different importance levels are set for edges of the same intensity for each frame type of the video signal. To do.

In a fourth aspect based on the third aspect, in the edge class information generating step, the importance level is lower in the order of intra-coded frame, forward predictive-coded frame and bidirectional predictive-coded frame. It is characterized by being set.

In a fifth aspect based on the first aspect, the edge class information generating step includes a luminance level edge detecting step of detecting an edge of a luminance level in macroblock units in an intra-coded frame, and a macroblock unit. A color difference level edge detection step of detecting a color difference level edge, and a level edge information generation step of integrating the luminance level edge information and the color difference level edge information to generate level edge information.

In a sixth aspect based on the fifth aspect, the level edge information generating step includes a visual characteristic information generating step of generating visual characteristic information, and an edge information correcting step of correcting the level edge information based on the visual characteristic information. And a level edge information mapping step of mapping the corrected level edge information to edge class information.

In a seventh aspect based on the sixth aspect, the luminance level edge detecting step includes an average luminance level calculating step for calculating an average luminance level in macroblock units, and the color difference level edge detecting step is in macroblock units. It is characterized in that the visual characteristic information is generated based on the average luminance level information and the average color difference level information in the visual characteristic information generation step.

In an eighth aspect based on the sixth aspect, the edge information correcting step includes a luminance level edge weighting information setting step and a color difference level edge weighting information setting step, and the level edge information mapping step includes luminance The level edge information is mapped based on the level edge weighting information and the color difference level edge weighting information.

A ninth aspect of the present invention is an encoding apparatus for compression-encoding an input video signal, which detects an edge for each frame of the video signal and generates edge information representing the strength of the edge. And an edge class information generator that generates edge class information indicating the importance of a macroblock based on the edge information, and a quantization amount determination that determines a quantization step for each macroblock based on the edge class information And a small quantization step for macroblocks of high importance, and a large quantization step for macroblocks of low importance.

As described above, in the ninth aspect of the present invention, by adjusting the quantization step of the macro block having an edge for each frame type, it is possible to prevent the deterioration of the edge portion due to the encoding of the video signal.

The tenth invention is characterized in that, in the ninth invention, the edge detector sets different intensities for edges having the same intensity for each frame type of the video signal.

In an eleventh aspect based on the ninth aspect, the edge class information generator has
A feature is that different importance levels are set for each frame type of a video signal.

The twelfth invention is the eleventh invention, wherein
The edge class information generator is an intra-coded frame,
It is characterized in that the degree of importance is set in the order of the forward predictive coding frame and the bidirectional predictive coding frame.

In a thirteenth aspect based on the ninth aspect, the edge class information generator comprises a luminance level edge detector for detecting an edge of a luminance level in macroblock units in an intra-coded frame, and a macroblock unit. In addition, a color difference level edge detector that detects an edge of a color difference level, and a level edge information generator that generates level edge information by integrating luminance level edge information and color difference level edge information are provided.

A fourteenth invention is the thirteenth invention, wherein
The level edge information generator is a visual characteristic information generator that generates visual characteristic information, an edge information corrector that corrects level edge information based on the visual characteristic information, and the corrected level edge information is mapped to edge class information. And a level edge information mapper.

A fifteenth invention is the fourteenth invention, wherein
The luminance level edge detector includes an average luminance level calculator that calculates an average luminance level in macroblock units, and the color difference level edge detector includes an average color difference level calculator that calculates an average color difference level in macroblock units, The visual characteristic information generator is characterized by generating visual characteristic information based on the average luminance level information and the average color difference level information.

A sixteenth invention is the fourteenth invention, wherein
The edge information corrector includes a luminance level edge weighting information setting device and a color difference level edge weighting information setting device.In the level edge information mapping device, the level is calculated based on the luminance level edge weighting information and the color difference level edge weighting information. It is characterized by mapping edge information.

A seventeenth invention is a recording medium on which coded data generated by using the coding method of the first invention and the coding apparatus of the ninth invention is recorded.

[0078]

BEST MODE FOR CARRYING OUT THE INVENTION First, the basic concept of video signal encoding according to an embodiment of the present invention will be described. According to the present invention, the coding in each area on the screen is not controlled simply based on the activity of the target area as in the related art, but the edge coding is performed according to the frame type of the video signal to be coded. By appropriately setting the protection strength, it is possible to suppress the color deterioration of the edge portion and the macroblock including the edge in the encoded image. An encoding device EN according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 13. After that, a modification of the embodiment of the present invention will be described with reference to FIG.

First, an encoding device according to the first embodiment of the present invention will be described with reference to FIG. The encoding apparatus EN in this example includes an input terminal 1000, a reorder unit 1001, a selector 1002, a first encoder 1003,
Code generation amount recorder 1004, target code amount calculator 100
5, edge detector 1006, edge class information generator 1
007, second encoder 1008, and output terminal 100
Including 9. The input terminal 1000 and the reorder device 100
1, code generation amount recorder 1004, target code amount calculator 10
05 and the output terminal 1009 substantially correspond to the input terminal 1800, the reorder unit 1801, the code generation amount recording unit 1804, the target code amount calculation unit 1805, and the output terminal 1809 which form the above-described conventional encoding device ENc. Is the same.

The second encoder 1008 uses the video data Dv
Is provided to encode encoded video data Denc that is actually stored in the storage device or storage medium. On the other hand, the first encoder 1003, the code generation amount recorder 1004, and the target code amount calculator 1005 include the second encoder 1008 so that the second encoder 1008 can efficiently encode the video data Dv. It is provided to determine the target code amount of the video data Dv before the encoding of the video data Dv is executed.

The second encoder 1008 effectively protects the edges while the video data Dv (encoding order video data Dc,
An edge detector 1006 and an edge class information generator 1007 are provided to encode the first and second encoded video data Dc2). Note that the first encoder 1003 is very similar to the first encoder 1803, but unlike the first encoder 1803, the first motion-compensated video data for the edge detector 1006. It has an output port for outputting α.

The overall operation of the encoding device EN is based on the user's encoding instruction and the input video data D
It is controlled by a coding controller (not shown) that determines coding information and a scheme such as a type of v, a coding target portion, and a coding method. Then, the individual components of the encoding device EN perform the processing described below in order to execute the operation determined and controlled by the encoding controller.

Display order video data Dv is input to the reorder unit 1001 from an external video data source (not shown) via the input terminal 1000. Reorder unit 1
As described with reference to FIG. 15, 001 rearranges the videos input in the display order of the video data Dv in the coding order to generate the coding order video data Dc. As described above, the selector 1002, based on the path switching signal Sw supplied from the coding controller, performs the coding order video data Dc during the first coding and the second and subsequent coding processes.
Switch the output destination of.

Also in this example, the first encoding is performed at the first encoding.
The encoding order video data Dc output to the encoder 1003 and the edge detector 1006 is referred to as first-time encoded video data Dc1, and the second encoder 1 is used during the second and subsequent encoding.
The encoded sequence video data Dc output to 008
The second and subsequent times will be referred to as encoded video data Dc2 to identify each. The first encoder 1003 and later operate at the first encoding, and the second encoder 1002 and later at the second encoding.
The encoder 1008 and later operate.

The selector 1002 is the selector 18 described above.
In the same manner as 02, prior to the encoding of the second and subsequent encoded video data Dc2 and the generation of the encoded data Denc by the second encoder 1008, first, the reorder unit 1001 calculates the target code amount. The input encoded video data Dc is output to both the first encoder 1003 and the edge detector 1006 as the first encoded video data Dc1. A first encoder 1003 and a code generation amount recorder 100 based on the first-time encoded video data Dc1.
4, and as a result of the processing by the target code amount calculator 1005, the target code amount DT is determined.

The edge detector 1006 is the selector 100.
2 based on the first-time encoded video data Dc1 output from the first encoder 1003 and the first motion-compensated video data α output from the first encoder 1003, the edge is detected in macroblock units to obtain edge information Ie. To generate. The edge class information generator 1007 generates edge class information Iec indicating the degree of importance of the macroblock, based on the detected edge information Ie.

Next, the selector 1002 outputs the coded video data Dc2 for the first and subsequent times to the second encoder 1008, and the second encoder 1008 outputs the coded image data Dc2 based on the target code amount DT and the edge class information Iec. Then, the first and subsequent encoded video data Dc2 are encoded. This will be described later in detail with reference to FIGS. 3 to 12.
The encoding of the encoded video data Dc2 by the second encoder 1008 from the first time onward is repeated any number of times until the encoding instructed by the user is completed.

First, at the time of the first encoding, the input video (first-time encoded video data Dc1) is processed by the first encoder 10
Then, the first motion-compensated video data α and the first coded video data Dc1e are generated. The code generation amount recorder 1004 uses the first encoded video data D.
Generated code amount D generated by encoding based on c1e
1 is detected and recorded in frame (F) units or macroblock (MB) units, and the detected generated code amount D1 is notified to the target code amount calculator 1005.

In the target code amount calculator 1005, the generated code amount D1 (F) of each frame is calculated based on the generated code amount D1.
The target code amount DT of each frame is calculated from the generated code amount D1 (S) of the entire scene, and the second encoder 100
Notify 8. The target code amount calculator 1005 calculates the target code amount DT (t) based on the above-mentioned formula 1.

On the other hand, the edge detector 1006 generates edge information Ie based on the first-time encoded video data Dc1 and the first motion-compensated video data α. And
The edge class information generator 1007 generates edge class information Iec. Then, the first encoded video data D
The encoding of c1 ends.

As described above, the first-time encoded video data D
After the target code amount DT and the edge class information Iec are generated as a result of the encoding of c1, the encoding of the encoded video data Dc2 for the first and subsequent times is started. That is, the second encoder 1008 determines the macroblock based on the target code amount DT (t) for each frame given from the target code amount calculator 1005 and the edge class information Iec given from the edge class information generator 1007. While controlling quantization, the first code amount DT (t) is set to the first value.
After the second time, the encoded video data Dc2 is encoded to generate the encoded data Denc. The generated encoded data Denc is output to the recording device via the output terminal 1009 and recorded on the recording medium M. Since the recording device and the recording medium M used for recording the encoded data Denc may be ordinary ones, their illustration and description will be omitted. The operations of the first encoder 1003 and the second encoder 1008 will be described below with reference to FIGS. 2 and 3.

As shown in FIG. 2, the first encoder 1003
Is an input terminal 2000, a subtractor 2001, a discrete cosine (DCT) converter 2002, a quantizer 2003, a variable length coding (VLC) 2004, an inverse quantizer 2005, an inverse DCT (IDCT) 2006, and an adder. 2007, refresh controller 2008, refresh switch 200
9, refresh switch 2010, frame memory 2
011 includes a motion compensator 2012, an output terminal 2013, and a motion detector 2014. The input terminal 200
0, subtractor 2001, discrete cosine transformer 2002, quantizer 2003, variable length encoder 2004, dequantizer 2005, inverse DCT device 2006, adder 2007, refresh controller 2008, refresh switch 200.
9, refresh switch 2010, frame memory 2
011, the motion compensator 2012, the output terminal 2013, and the motion detector 2014 include the input terminal 1900 and the subtractor 190 which configure the first encoder 1803 shown in FIG.
1, a discrete cosine transformer 1902, a quantizer 1903,
Variable length encoder 1904, inverse quantizer 1905, inverse DC
T unit 1906, adder 1907, refresh controller 1
908, refresh switch 1909, refresh switch 1910, frame memory 1911, motion compensator 1912, output terminal 1913, and motion detector 19
Substantially the same as each of the fourteen.

Therefore, as described with reference to FIG. 15, the first-time encoded video data Dc1 input from the selector 1002 via the input terminal 2000 is shown in FIG. 18 inside the first encoder 1003. Various processes are performed in the same manner as described with reference to FIG. As a result, the first encoded video data Dc1e is generated and output from the output terminal 2013 to the code generation amount recorder 1004. However, the first motion-compensated video data α generated by the motion compensator 2012
Is different from the first encoder 1803 in that it is also directly output to the edge detector 1006.

As shown in FIG. 3, the second encoder 1008 is provided.
Is an input terminal 3000, a subtractor 3001, a discrete cosine transform (DCT) device 3002, a quantizer 3003, a variable length coding (VLC) device 3004, an inverse quantizer 3005, an inverse DCT (IDCT) device 3006, an adder 3007, refresh controller 3008, refresh switch 300
9, refresh switch 3010, frame memory 3
011, a motion compensator 3012, an output terminal 3013, a motion detector 3014, a buffer 3015, and a quantization decision unit 3016. Input terminal 3000, subtractor 300
1, a discrete cosine transformer 3002, a quantizer 3003,
Variable length encoder 3004, inverse quantizer 3005, inverse DC
T unit 3006, adder 3007, refresh controller 3
008, refresh switch 3009, refresh switch 3010, frame memory 3011, motion compensator 3012, output terminal 3013, motion detector 3014,
And the buffer 3015 corresponds to the input terminal 2 shown in FIG.
000, subtractor 2001, discrete cosine converter 200
2. Quantizer 2003, variable length encoder 2004, inverse quantizer 2005, inverse DCT device 2006, adder 200
7, refresh controller 2008, refresh switch 2009, refresh switch 2010, frame memory 2011, motion compensator 2012, output terminal 201
3, the motion detector 2014, and the buffer 2015, respectively.

The quantizer determiner 3016 uses the target code amount DT.
Based on the buffer occupancy information Ib, the quantization amount VQ is determined for each macroblock MB, and the quantizer 3003
The part for controlling is the same as that of the quantization decision unit 2016. However, the quantizer determiner 3016 differs from the quantizer decider 2016 in that
Is a quantization amount VQc based on the edge code information Iec in addition to the target code amount DT and the buffer occupation amount information Ib.
Is determined and the quantizer 3003 is controlled. In other words, the quantization amount VQc is the quantization amount VQ corrected according to the importance of the edge portion to be encoded. Focusing on this point, the operation of the second encoder 1008 will be described below.

The input terminal 3000 is connected to the selector 1002, and the first and second and subsequent encoded video data Dc2 are connected.
Is entered. The subtractor 3001 has one input port connected to the input terminal 3000, and the other input port having a refresh switch 3009, a refresh switch 3010, a motion compensator 3012, and a frame memory 301.
1, the adder 3007, the inverse DCT unit 3006, the inverse quantizer 3005, the quantizer 3003, and the discrete cosine transformer 3002, and is connected to its own output port, and the first second time of the current frame. After that, encoded video data D
A third difference ΔDc2c, which is the difference between c2 (t) and the third motion-compensated video data βc before the predetermined number of frames (d) to be restored, is calculated.

The quantization amount VQ by the quantization determiner 2016 is the target code amount DT and the buffer occupation amount information I.
Quantization decider 3 whereas decision is made based on b only
The third motion-compensated video is that the quantization amount VQc by 016 is determined so that the converted edge is protected based on the target code amount DT, the buffer occupation information Ib, and the edge class information Iec. The data βc is the second
It has a feature different from the motion-compensated video data β of. This will be described in detail below.

In the third motion-compensated video data βc, data generated by motion-compensating the third delayed intra-frame video data Dc2rcd and the third delayed inter-frame video data (ΔDc2rc + βc) are motion-compensated. And the generated data. Then, the third difference ΔDc2
c is the encoded video data Dc2 and the third encoded video data from the first second time onward.
Of the delayed inter-frame video data (ΔDc2rc + βc) is motion-compensated to generate only the third motion-compensated video data βc.

The refresh controller 3008 has a first
The refresh switch 3009 according to the type of picture to be processed in the encoded video data Dc2 from the second time onward.
And controls on / off of the refresh switch 3010. That is, the refresh controller 3008
By turning off refresh switches 3009 and 3010 for intra-frame compression if the picture to be processed is an I-picture,
The subtracter 3001 does not generate the third difference ΔDc2c.

As described above, the first and second and subsequent encoded video data Dc2 input from the input terminal 3000 are output to the discrete cosine converter 3002 without being subjected to the subtraction processing by the subtractor 3001. Subtractor 300
Third difference ΔDc2c generated by subtraction processing at 1
May be output to the discrete cosine transformer 3002. The coded video data Dc2 or the third difference ΔDc2c from the first second time onward is calculated by the discrete cosine converter 2
The discrete cosine transform is performed by 002 to obtain 8 × 8 = 64 coefficients. This coefficient is called the third coefficient K2c. This third coefficient K2c also differs from the above-mentioned second coefficient K2 in that it includes a coefficient generated by performing a discrete cosine transform of a third difference ΔDc2c different from the first second-time encoded video difference ΔDc2. different.

The third coefficient K2c is optimally bit-distributed for each frequency component by the quantizer 3003 to generate a third quantized coefficient K2Qc. Since the third coefficient K2c is different from the second coefficient K2, the third quantized coefficient K2Qc is a coefficient having characteristics different from the second quantized coefficient K2Q.

In the second encoder 1008,
Unlike the case in the first encoder 1003, at this point, the quantization determiner 3016 controls the quantization step QS by the quantizer 3003. That is, the quantization determiner 3016 calculates the quantization step QS (quantization amount VQc) based on the target code amount DT, the buffer occupation amount information Ib, and the edge class information Iec given for each frame, and The calculated quantization step QS (quantization amount VQc) is given to the quantizer 3003.

The quantization step QS is controlled on the basis of the state of the buffer 3014 (buffer occupancy information Ib) basically when the possibility that the buffer 3014 will fail is detected. Under normal conditions, buffer 3
The control of the quantization step QS based on the state of 014 is not performed. When calculating the target code amount, the buffer state is simulated and the code amount is assigned so that the buffer does not fail. Therefore, if the prediction accuracy is guaranteed to some extent, the normal buffer 3014 will not fail. .

The third quantized coefficient K2Qc output from the quantizer 3003 is subjected to variable length coding by the variable length coder 2004, and third coded video data Dc is obtained.
2 ec is generated. The third quantized coefficient K2Qc is
Since the edge is protected as compared with the second quantized coefficient K2Q, the edge of the third coded video data Dc2ec is also protected as compared with the second coded video data Dc2e described above. On the other hand, the third quantized coefficient K2Qc is
The inverse quantizer 3005 restores the original quantization, and the third
The coefficient K2rc of is generated.

The third coefficient K2rc returned to the original amplitude by the inverse quantization is the original image data by the inverse DCT unit 3006, that is, the second and subsequent encoded image data Dc2rc or the second encoded image data Dc2rc. Is restored to the encoded video difference ΔDc2rc. Note that both the second restored second and subsequent encoded video data Dc2rc and the second restored second and subsequent encoded video difference ΔDc2rc are both edges protected based on the edge class information Iec by the quantization determiner 3016. Each of the quantized points is the second restoration second and subsequent encoded video data Dc2r.
Alternatively, the coded video difference ΔDc2 from the second restoration onward
Different from r.

When the output from the inverse DCT unit 3006 is the encoded image data Dc2rc from the second restoration onward, that is, the in-frame image data, the refresh controller 3008 causes the refresh switch 3010 to operate as described above.
Is turned off, and the addition processing by the adder 3007 is not executed. On the other hand, when the output from the inverse DCT unit 3006 is the encoded image difference ΔDc2rc, that is, the inter-frame image data after the second restoration, the refresh controller 3
008 turns on the refresh switch 3010,
Third motion-compensated video data βc by the adder 3007
Is executed. Therefore, from the adder 3007, the encoded video data Dc2rc after the second restoration
Alternatively, the third refresh data (ΔDc2rc + β
Either c) is output.

After that, the frame memory 3011 stores the restored video data (the coded video data Dc2rc or the second refresh data after the second restoration or the third refresh data (ΔDc).
2rc + βc)) is delayed by a predetermined number (d) of frames by the frame memory 3011 to generate frame-delayed video data. That is, the frame memory 3011
Is the second restored second and subsequent encoded video data Dc2rc
Is delayed by d frames to generate the third delayed frame video data Dc2rcd, and the third refresh data (ΔDc2rc + βc) is delayed by d frames to generate the third delayed interframe video data (ΔDc
2rc + βc) d is generated.

The motion detector 3014 has an input terminal 3000.
The second and subsequent encoded video data Dc input from
Based on 2, the motion amount Am is detected by obtaining the motion vector between the frames of the video.

The motion compensator 3012 is the motion detector 301.
Based on the amount of motion Am detected in step 4, the position of the image represented by the third delayed frame video data Dc2rcd or the third delayed interframe video data (ΔDc2rc + βc) d output from the frame memory 3011 is currently determined. 3rd motion-compensated video data βc that has been motion-compensated to be the same as the position of the video represented by the encoded video data Dc2 from the first second time onward.

The third motion-compensated video data βc is
The difference (third difference ΔDc) from the next video data (first and subsequent encoded video data Dc2) is calculated by the subtractor 3001.
It is used to calculate 2c). However, the third motion-compensated video data βc generated by motion-compensating the third delayed intra-frame video data Dc2rcd is substantially the same as the current first and subsequent encoded video data Dc2. It is meaningless to use the subtractor 3001 to generate the third difference ΔDc2c.

Therefore, the refresh controller 3008 is
If the picture to be processed is a P-picture or a B-picture, the refresh switches 3009 and 3010 are turned on so that the adder 3007 returns to the third and not the second encoded video data Dc2rc after the second restoration.
Refresh data (ΔDc2rc + βc) of the frame memory 3011 is output to the frame memory 3011.
From the third delayed inter-frame video data (ΔDc2rc
+ Βc) d is output to the motion compensator 3012, and from the motion compensator 3012, the third delayed interframe video data (Δ
The inter-frame compression is performed by outputting the motion-compensated third motion-compensated video data βc of Dc2rc + βc) d to the adder 3007 and the subtractor 3001.

The refresh switch 3009 is turned on when calculating the difference between frames, and the subtractor 300
Used to run 1. Refresh switch 3
010 repeatedly turns on and off in the same cycle as the refresh switch 3009. Refresh switch 3
When 010 is on, the adder 3007 is operated, and the inter-frame difference data (second restored second and subsequent encoded video difference ΔDc2rc) and the previous frame data (second restored second and subsequent encoded video data Dc2rc ) Is added and used to restore the frame. Variable length encoder 3004
Performs variable length coding also on the inter-frame compressed data (first second-coded video difference ΔDc2c).

The third encoded video data Dc2ec encoded by the variable length encoder 3004 is output to the output terminal 3013.
Is output by.

As described above, based on the target code amount DT and the edge class information Ie for each macroblock, which indicates the degree of difficulty of encoding for each section detected by the first encoding by the first encoder 1003. , The second encoder 1008
Controls the quantization of the black block so as to protect the edges, and encodes the encoded video data Dc2 for the first and subsequent times.

A method for compression-encoding a video signal according to the embodiment of the present invention will be described with reference to the flowcharts shown in FIGS. 4 and 6. As described above, the encoding flow of the video signal according to the present invention is performed by the first encoder 100.
3, code generation amount recorder 1004, target code amount calculator 10
05, the edge detector 1006, and the edge class information generator 1007 perform the preliminary encoding flow (FIG. 4) executed on the first-time encoded video data Dc1, and the edge class information Iec obtained by the preliminary encoding flow. The second encoder 1008 first 2 based on the target code amount DT
The actual encoding flow (FIG. 6) executed for the encoded video data Dc22 after the first time is roughly divided into two.

As shown in FIG. 4, when the preliminary encoding flow is started, first, in step S40001, the video data (display order video data) Dv is input terminal 1000.
Via the reorder device 1001 via the frame. The reorder unit 1001 rearranges the fetched display order video data Dv from the display order to the coding order to generate the coding order video data Dc and outputs it to the selector 1002. Then, the process proceeds to the next step S40002.

In step S40002, it is determined for each frame of the encoded video data Dc whether the picture type is I, that is, the intra frame. No
In the case of, the process proceeds to step S40003. on the other hand,
If Yes, that is, if it is determined to be an intra frame, the process proceeds to step S40012.

In step S40003, the motion detector 2014 of the first encoder 1003 detects the motion of the first-time encoded video data Dc1 (Dc) and generates the motion amount Am. Then, the process is step S40.
Proceed to 004.

In step S40004, the motion compensator 2012 makes a first decision based on the first delayed intra-frame video data Dc1rd input from the frame memory 2011 and the motion amount Am input from the motion detector 2014.
Motion-compensated video data α is generated. Then, the process proceeds to step S40005.

In step S40005, the edge detector 1006 detects an edge in the first motion-compensated video data α input from the motion compensator 2012, and generates edge information Ie. The edge information Ie is referred to as non-intra edge information Ien. Then, the process proceeds to step S40006.

On the other hand, in step S40012,
The edge detector 1006 detects an edge in the first-time encoded video data Dc1 input from the selector 1002, and generates edge information Ie. This edge information Ie
Is referred to as intra-edge information Iei. The above-mentioned non-intra edge information Ien and intra edge information Iei are collectively referred to as edge information Ie. Then, the process proceeds to step S40006. It should be noted that the edge detection and the generation of the intra edge information Iei in this step will be described later in detail with reference to FIGS. 7, 8, 9 and 10.

In step S40006, the edge class information generator 1007 has the non-intra edge information Ie.
n and the intra-edge information Iei, respectively, based on the non-intra-edge class information Iecn and the intra-edge class information I, which are macroblock importance information.
generate eci. The non-intra edge class information Iecn and the intra edge class information Ieci are
Collectively referred to as edge class information Iec.

In the present invention, step S40012.
The intra-frame edge detection intensity in step S4005 is made higher than the non-intra-frame edge detection intensity in step S4005. That is, even when the intra edge information Iei and the non-intra edge information Ien represent edges of the same strength, the intra edge class information Iec
i is set so as to have a higher degree of importance than the non-intra edge class information Iecn. In this way, increase the edge class of the intra frame (increased importance)
By protecting the edges.

In this embodiment, step S4
The edge detection intensities in 0006 and S40012 are set to be different. However, it goes without saying that step S4006 may be set such that the value of the edge class information Iec is different depending on whether or not the target frame is an intra frame for the same edge strength. Then, the process proceeds to the next step S4000.
Proceed to 7.

In step S40007, the quantizer 2003 uses the fixed QS to calculate the discrete cosine transformer 20.
The first coefficient K1 input from 02 is encoded to generate a quantized first coefficient K1Q. Then, the quantized first coefficient K1Q is further variable-length coded by the variable-length encoder 2004 and output from the output terminal 2013 as the first encoded video data Dc1e. Then, the process proceeds to the next step S40008.

In step S40008, the code generation amount recorder 1004 counts the generated code amount D1 and the generated total code amount ΣD1 of each frame of the first encoded video data Dc1e input from the first encoder 1003. To do.
Then, the process proceeds to the next step S40009.

In step S40009, it is determined whether the processing of all input frames of the first-time encoded video data Dc1 has been completed. If all the input frames have not been processed, the determination is No and the process returns to step S40001. Then, in this step, the above-described step S40 is performed until it is determined Yes.
The processing from 01 to S40009 and S40012 is repeated. Then, after the processing of all the input frames of the first-time encoded video data Dc1 is completed, Yes at this step.
Then, the process proceeds to step S40010.

In step S40010, the first-time encoded video data Dc1 input through the above-described processing is input.
The target code amount DT detected for each of all the frames is saved in the file. When the second and subsequent encodings are performed without terminating the application, it is not necessary to save the calculated target code amount D of each frame in a file.

Next, the actual encoding flow will be described with reference to FIG. As shown in FIG. 5, when the actual encoding flow is started, first, in step S50001, the second
The target code amount DT of the frame is read into the quantization determiner 3016 of the encoder 1808. Then, the process proceeds to the next step S50002.

In step S50002, the edge class information Iec is read into the quantization determiner 3016 from the edge class information generator 1007 for each frame.
Then, the process proceeds to the next step S50003.

In step S50003, video data (display order video data) Dv is fetched into the reorder unit 1001 in frame units via the input terminal 1000. The reorder unit 1001 rearranges the fetched display order video data Dv from the display order to the coding order to generate the coding order video data Dc and outputs it to the selector 1002.
Then, the process proceeds to the next step S50004.

In step S50004, it is determined whether the picture type of the frame of the encoded video data Dc is I, that is, the intra frame. In No, a process progresses to step S50005.

In step S50005, the motion detector 3014 of the second encoder 1008 causes the first 2
The motion of the encoded video data Dc2 (Dc) is detected after the first time, and the motion amount Am is generated. Then, the process proceeds to step S50006.

In step S50006, the motion compensator 3012 uses the second delayed intra-frame video data Dc2rd input from the frame memory 3011 and the motion amount Am input from the motion detector 3014 to determine the third value.
Motion-compensated video data βc is generated. Then, the process proceeds to step S50007.

On the other hand, in step S50004 described above, if Yes, that is, if it is determined that the frame is not an intra frame, the process skips steps S50005 and S50006 described above, and proceeds to step S50007.

In step S50007, the quantization determiner 3016 determines the quantization amount VQc based on the target code amount DT read in step S50001 and the edge class information Iec read in step S50002.
To decide. Then, the process proceeds to the next step S50008.
Proceed to.

In step S50008, the quantizer 3
003 variable-length-encodes the third coefficient K2c input from the discrete cosine transformer 3002 based on the quantization amount VQc to generate a third quantized coefficient K2Qc. In this way, encoding is performed according to the target code amount DT read in step S50002. Note that the above step S
In the case of an I picture determined to be Yes in 5004, the third
Coefficient K2c of the first and second encoded video data Dc
2 is the result of discrete cosine transformation. Then, in the case of the non-I picture determined as No in step S5004, the third coefficient K2c is the difference between the first and second encoded video data Dc2 and the third motion-compensated video data βc. The second-time coded video difference ΔDc2c is subjected to discrete cosine transform. Then, the process proceeds to the next step S50009.

In step S50009, the first 2
It is determined whether or not the processing of all input frames of the encoded video data Dc2 has been completed after the first time. If all the input frames have not been processed, the determination is No and the process returns to step S50001. Then, in this step, the above-described steps S5001 to S40008 are repeated until it is determined Yes. Then, after the processing of all the input frames of the encoded video data Dc2 has been completed from the first second time onward, Ye is performed in this step.
It is determined to be s, and the process ends.

It goes without saying that the compression coding in the coding device EN according to the above-described embodiment of the present invention can be realized mainly by software.

Hereinafter, the edge detection and the generation of the intra-edge information Iei in step S40012 will be described in detail with reference to FIGS. 6, 7 and 8. First, as shown in FIG. 6, step S40012.
Are roughly classified into a "macroblock unit edge detection" subroutine of step # 40002 and a step # 400.
No. 03 "edge information generation" subroutine.

The "macroblock unit edge detection" subroutine of step # 4002 further includes step # 60.
A "luminance edge detection" subroutine of 001 and a "color difference edge detection" subroutine of step # 60002 are included.

Further, as shown in FIG. 7, step # 6
The “luminance edge detection” subroutine of 0001 is “macroblock luminance data reading” processing of step S70001, “block activity calculation” processing of step S70002, “macroblock variance calculation” processing of step S70003, and “step 700004”. Luminance edge determination ”processing.

Further, as shown in FIG. 8, step # 60
The "color-difference edge detection" subroutine 02 is performed in step S
80001 "Read macro block color difference data" processing,
This includes the "block activity calculation" process of step S80002 and the "color difference edge determination" process of step S80003.

The overall operation of step S40012 will be described below while describing the operation of each step. As shown in FIG. 7, in the "luminance edge detection" subroutine # 60001 of the "macroblock unit edge detection" subroutine of step # 40002, an edge for the luminance data is detected and luminance edge information is generated. Subsequently, in "color difference edge detection" subroutine # 60002, an edge for the color difference data is detected, and color difference edge information is generated.

As shown in FIG. 7, in step S7001 of the "luminance edge detection" subroutine # 60001, the luminance data is read in macroblock MB units. The macro block MB has four sub blocks BLK0, BLK1, and BLK as shown in FIGS.
2 and BLK3.

In step S70002, four sub-blocks BLK0 to BLK3 in the macro block MB.
Activity ACT (BLK) is calculated. The activity ACT (BLK) is the pixel level X.
And based on the average value avgX of the pixel levels in the sub-block, for example, according to the following Expression 2.

ACT (BLK) = 1/64 * Σ (X-avgX) ² (Equation 2)

In step S70003, the variance of the activity of each sub-block is calculated. The variance value VA of the activity ACT (BLK) of the sub block
R (MB) is calculated based on the average value avgACT (BLK) of the activity of the subblocks in the macroblock, for example, according to the following Expression 3.

VAR (MB) = 1/4 * Σ (ACT (BLK) −avgACT (BLK)) ² (Equation 3)

In step S70004, edge determination is performed based on the variance value VAR (MB) obtained in step S70003. The edge determination is performed by, for example, the threshold value (TH) determination of the variance value VAR (MB).

As shown in FIG. 8, in step S80001 of the "color difference edge detection" subroutine # 60002, color difference (U and V) data is read in macroblock units. For color difference data, the macroblock is 1
It consists of two sub-blocks.

In step S80002, the activity ACT of one subblock in the macroblock
(BLK) is calculated, for example, according to the above equation 2.

In step S80003, a color difference edge determination is made based on the activity ACT (BLK). For example, the edge determination is performed by the threshold (TH) determination of the activity ACT (BLK) obtained according to the above expression 2.

As shown in FIG. 6, step # 60003.
At, the edge information Ie of each macroblock calculated in step S40002 is corrected by the visual characteristics. Note that, as shown in FIG. 11, step # 60
003 further includes step S100000 for determining whether the average luminance level of the macroblock (MB) is equal to or higher than the threshold value TH, and step S100001 for clearing the edge information Ie from Edge.

If it is determined in step S100000 that the average brightness level of the macroblock is equal to or higher than the threshold value TH, the edge of the edge information Ie is cleared in step S100001. This is based on the characteristic that the visual sensitivity is generally lower as the brightness level is higher. That is, when the input level is 0 to 255 (8 bits), the threshold information TH is set to about 200, whereby the edge information Ie of the macroblock whose deterioration is visually difficult to be recognized can be corrected. With this correction, the code amount can be assigned to the macroblock whose deterioration is more noticeable in terms of visual characteristics.

In step # 60004, based on the weighting information of the luminance and the color difference, step # 6000.
Luminance edge information detected in 1 and step # 6000
Each of the color difference edge information detected in 2 is weighted to generate edge information Ie. The edge information Ie is luminance edge information IeY, color difference (U) edge information IeU, color difference (V) edge information IeV, luminance weighting WtY, color difference (U) weighting WtU (however, WtY + WtU + WtV).
= 1.0), it can be calculated according to the following equation 4.

[0157]     Ie = IeY * WtY + IeU * WtU + IeV * WtV (Formula 4)

In Step # 60005, Step #
The edge information Ie calculated in 60004 is mapped to the edge class information Iec. In FIG. 12, edge information I
An example of linearly mapping e and edge class information Iec will be shown. FIG. 13 shows an example in which the edge information Ie and the edge class information Iec are non-linearly mapped. 12 and 13, the horizontal axis indicates Edge and the vertical axis indicates edge class.

Referring to the flow chart shown in FIG. 14,
A modified example of the compression encoding method in the encoding device EN according to the embodiment of the present invention will be described. In the figure,
Similar to FIG. 4, a preliminary encoding flow executed by the first encoder 1003 to the edge class information generator 1007 for the first-time encoded video data Dc1 is shown.

As shown in FIG. 14, when the preliminary coding flow is started, first, in step S40101,
Video data (display order video data) Dv is input terminal 100
It is fetched into the reorder unit 1001 in frame units via 0. The reorder unit 1001 rearranges the fetched display order video data Dv from the display order to the coding order to generate the coding order video data Dc and outputs it to the selector 1002. Then, the process proceeds to the next step S40102.

In step S40102, the edge detector 1006 detects the motion compensator 2 of the first encoder 1003.
An edge in the first motion-compensated video data α input from 012 is detected to generate edge information Ie.
Then, the process proceeds to step S40103.

In step S40103, it is determined whether the picture type of the encoded video data Dc is I, that is, the intra frame. In No, a process progresses to step S40104. On the other hand, if Yes,
The process proceeds to step S40112.

In step S40104, the motion detector 2014 of the first encoder 1003 detects the motion of the first-time encoded video data Dc1 (Dc), and generates the motion amount Am. Then, the process is step S40.
Proceed to 105.

In step S40105, the motion compensator 2012 makes a first decision based on the first delayed intra-frame video data Dc1rd input from the frame memory 2011 and the motion amount Am input from the motion detector 2014.
Motion-compensated video data α is generated. Then, the process proceeds to step S40106.

In step S40106, the edge class information generator 1007 determines, based on the edge information Ie generated in step S40102 described above, that the first encoded image is a non-intra frame (No in S40103). The non-intra edge class information Iecn of the data Dc1 is generated. Then, the process proceeds to step S40107.

In step S40112, the edge class information generator 1007, based on the edge information Ie detected in step S40102 described above, performs the first encoding which is determined to be an intra frame (Yes in step S40103). The intra edge information Iei of the video data Dc1 is generated. Then, the process proceeds to step S40107. In the present invention, the edge is protected by making the class value assigned by the intra-edge information Iei larger than the class value assigned by the non-intra-edge information Ien (increasing the importance).

In step S40107, the quantizer 2003 uses the fixed QS and the discrete cosine transformer 20
The first coefficient K1 input from 02 is encoded to generate a quantized first coefficient K1Q. Then, the quantized first coefficient K1Q is further variable-length coded by the variable-length encoder 2004 and output from the output terminal 2013 as the first encoded video data Dc1e. Then, the process proceeds to the next step S40108.

In step S40108, the code generation amount recorder 1004 counts the generated code amount D1 and the generated total code amount ΣD1 of each frame of the first encoded video data Dc1e input from the first encoder 1003. To do.
Then, the process proceeds to the next step S40109.

In step S40109, it is determined whether all the input frames of the first-time encoded video data Dc1 have been processed. If all the input frames have not been processed, the determination is No, and the process returns to step S40101. Then, in this step, the above-described step S40 is performed until it is determined Yes.
The processing from 11 to S40109 and S40112 is repeated. Then, after the processing of all the input frames of the first-time encoded video data Dc1 is completed, Yes at this step.
Then, the process proceeds to step S40110.

In step S40110, the first-time encoded video data Dc1 input through the above-described processing is input.
The target code amount DT detected for each of all the frames is saved in the file. It should be noted that the second and subsequent encoding steps S are performed without terminating the application.
Is required, the calculated target code amount D of each frame is required.
It is not necessary to save T in the file.

As described above, according to the present invention, the coding is performed by using the edge class information for each macroblock in consideration of the visual characteristic calculated at the time of the first coding, so that only the edges in which visually deteriorated are conspicuous. Can be protected.

[Brief description of drawings]

FIG. 1 is a block diagram schematically showing a configuration of an encoding device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a first encoder of the encoding device shown in FIG.

FIG. 3 is a block diagram showing a configuration of a second encoder of the encoding device shown in FIG.

FIG. 4 is a flowchart showing a preliminary encoding process of the encoding device shown in FIG.

5 is a flowchart showing an actual encoding process of the encoding device shown in FIG.

FIG. 6 is a flowchart showing details of the edge detection step shown in FIG.

FIG. 7 is a flowchart showing details of a luminance edge detection subroutine shown in FIG.

8 is a flowchart showing details of a color difference edge detection subroutine shown in FIG.

FIG. 9 is a schematic diagram showing an example of an edge on a macroblock composed of four sub-blocks.

10 is a schematic diagram showing an example different from an edge on a macroblock composed of four sub-blocks shown in FIG.

11 is a flowchart showing details of a correction subroutine based on the visual characteristic information shown in FIG.

FIG. 12 is a schematic diagram showing an example of linearly mapping edge information and edge class information.

FIG. 13 is a schematic diagram showing an example of non-linear mapping of edge information and edge class information.

14 is a flowchart showing a modified example of preliminary coding processing of the coding apparatus shown in FIG.

FIG. 15 is an explanatory diagram of generation of encoded sequence video data by the reorder unit shown in FIG. 1.

16 is a schematic diagram showing a relationship between a generated code amount in the first encoder shown in FIG. 1 and a target code amount calculated by a target code amount calculator.

FIG. 17 is a block diagram showing a configuration of a conventional encoding device.

18 is a block diagram showing a configuration of a first encoder of the encoding device shown in FIG.

19 is a block diagram showing a configuration of a second encoder of the encoding device shown in FIG.

[Explanation of symbols]

EN, ENc Encoding device 1000, 1800, 1900, 2000, 3000
Input terminal 1001, 1801 Reorder unit 1002, 1802 Selector 1003, 1803 First encoder 1004, 1804 Code generation amount recorder 1005, 1805 Target code amount calculator 1006, 1806 Edge detector 1007, 1807 Edge class information generator 1008 , 1808 Second encoder 1009, 1809, 1913, 2013, 3013
Output terminals 1901, 2001, 3001 Subtractors 1902, 2002, 3002 Discrete cosine transformers 1903, 2003, 3003 Quantizers 1904, 2004, 3004 Variable length encoders 1905, 2005, 3005 Dequantizers 1906, 2006, 3006 Inverse DCT units 1907, 2007, 3007 Adders 1908, 2008, 3008 Refresh controllers 1909, 1910, 2009, 2010, 3009,
3010 refresh switch 1911, 2011, 3011 frame memory 1912, 2012, 3012 motion compensator 1914, 2014, 3014 motion detector 2015, 3015 buffer 2016, 3016 quantization determiner MB macroblock BLK1, BLK2, BLK3, BLK4 sub-block

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C053 FA23 GA11 GB23 GB26 GB30                       GB33 KA01                 5C057 AA09 AA10 EA02 EA07 EC02                       EC03 ED07 ED09 ED10 EK04                       EM04 EM09 EM13 EM16 FB03                 5C059 KK01 KK27 MA05 MA14 MA23                       MC11 ME01 PP05 PP06 PP07                       PP16 SS20 TA46 TB07 TC10                       TC38 TD09 UA02

Claims (17)

[Claims] 1. An encoding method for compressing and encoding an input video signal, comprising: an edge detection step of detecting an edge for each frame of the video signal and generating edge information representing the intensity of the edge. An edge class information generation step of generating edge class information indicating importance of a macroblock based on the edge information, and a quantization amount determination step of determining a quantization step for each macroblock based on the edge class information. And a step of increasing the quantization step for macroblocks of high importance, and increasing the quantization step for macroblocks of low importance. 2. The encoding method according to claim 1, wherein in the edge detecting step, different intensities are set for edges of the same intensity for each frame type of the video signal. 3. The method according to claim 1, wherein, in the edge class information generating step, different degrees of importance are set for edges of the same intensity for each frame type of the video signal. . 4. In the edge class information generation step, the importance is set to be low in the order of an intra-coded frame, a forward predictive coded frame, and a bidirectional predictive coded frame. The encoding method according to claim 3. 5. The edge class information generation step includes a luminance level edge detection step of detecting a luminance level edge in macroblock units, and a color difference detection of a color difference level edge in macroblock units in an intra-coded frame. The encoding method according to claim 1, further comprising: a level edge detecting step; and a level edge information generating step of integrating the luminance level edge information and the color difference level edge information to generate level edge information. 6. The level edge information generating step, a visual characteristic information generating step of generating visual characteristic information, an edge information correcting step of correcting the level edge information based on the visual characteristic information, and the corrected The level edge information mapping step of mapping level edge information to the edge class information, the encoding method according to claim 5. 7. The brightness level edge detection step comprises:
An average luminance level calculation step for calculating an average luminance level in macroblock units; the color difference level edge detection step includes an average color difference level calculation step for calculating an average color difference level in macroblock units; and the visual characteristic information generating step. The encoding method according to claim 6, wherein the visual characteristic information is generated based on the average luminance level information and the average color difference level information. 8. The edge information correcting step includes a luminance level edge weighting information setting step and a color difference level edge weighting information setting step, and in the level edge information mapping step,
The encoding method according to claim 6, wherein the level edge information is mapped based on the luminance level edge weighting information and the color difference level edge weighting information. 9. An encoding device for compressing and encoding an input video signal, comprising: edge detection means for detecting an edge for each frame of the video signal and generating edge information representing the intensity of the edge. Edge class information generating means for generating edge class information indicating the importance of the macroblock based on the edge information, and quantization amount determination for determining a quantization step for each macroblock based on the edge class information. And a means for reducing the quantization step for macroblocks having high importance, and increasing the quantization step for macroblocks having low importance. 10. The encoding apparatus according to claim 9, wherein the edge detection unit sets different intensities for edges having the same intensity for each frame type of the video signal. 11. The apparatus according to claim 9, wherein the edge class information generating means sets different importance levels for edges of the same intensity for each frame type of the video signal. 12. The edge class information generating means sets the importance to an intra coded frame, a forward predictive coded frame, and a bidirectional predictive coded frame in the order of importance. Encoding device. 13. The edge class information generating means detects a luminance level edge detecting means for each macroblock in luminance level edge detecting means for an intra-coded frame, and a color difference detecting a color difference level edge for each macroblock. 10. The encoding device according to claim 9, further comprising level edge detection means, and level edge information generation means for generating level edge information by integrating the luminance level edge information and the color difference level edge information. 14. The level edge information generating means, visual characteristic information generating means for generating visual characteristic information, edge information correcting means for correcting the level edge information based on the visual characteristic information, and the corrected The coding device according to claim 13, further comprising level edge information mapping means for mapping level edge information to the edge class information. 15. The brightness level edge detection means comprises an average brightness level calculation means for calculating an average brightness level in macroblock units, and the color difference level edge detection means is an average for calculating an average color difference level in macroblock units. The encoding device according to claim 14, further comprising color difference level calculating means, wherein the visual characteristic information generating means generates the visual characteristic information based on the average luminance level information and the average color difference level information. . 16. The edge information correction means includes luminance level edge weighting information setting means and color difference level edge weighting information setting means, and in the level edge information mapping means, the luminance level edge weighting information and color difference level. The encoding device according to claim 14, wherein the level edge information is mapped based on edge weighting information. 17. A recording medium on which encoded data generated by using the encoding method according to claim 1 and the encoding method according to claim 9 is recorded.