JP6775339B2 - Image coding device and its control method - Google Patents

Description

The present invention relates to a coding technique for an image captured by an imaging device.

An image pickup device such as a digital camera or a digital camcorder employs a CCD sensor or a CMOS sensor as an image pickup element. These sensors are mainly composed of a single plate type, and one color component (for example, one of R, G, and B) is used for one pixel by a color filter array (hereinafter referred to as CFA) on the surface of the sensor. Get the pixel value of. By using CFA, for example, image data of a Bayer array arranged in a periodic pattern of R (red), G1 (green), B (blue), and G2 (green) as shown in FIG. 2 (hereinafter, RAW). (Called data) is obtained. Human vision has high sensitivity to luminance components. Therefore, when focusing on the 2 × 2 pixels of the Bayer array, the red (R) component and the blue (B) component are each one pixel, while the green pixel containing a large amount of the luminance component is as shown in FIG. It has 2 pixels (double) of G1 and G2. As described above, the RAW data has only one color component information per pixel. On the other hand, one pixel of a normal color image has a plurality of components such as R, G, and B. Therefore, by applying a process called demosaic to the RAW data, image data having three component values of red, blue, and green per pixel is generated. Generally, the image data of the RGB signal obtained by demosaic or the YUV signal obtained by converting from the RGB signal is encoded and recorded. However, since one pixel of the image data obtained by the demosaic process is composed of three color components, the amount of data is three times that of the RAW data of one pixel and one component. Therefore, a method of encoding and recording RAW data before demosaicing has been proposed (see, for example, Patent Document 1).

For example, Patent Document 1 discloses a method of classifying RAW data into signal components (R, G1, B, G2), organizing them into four components (planes), and then encoding the data.

Japanese Unexamined Patent Publication No. 2003-125209

However, as in Patent Document 1, when the RAW data is separated for each signal component to form a plane, the pixel value is extracted from the RAW data by using subsampling of one pixel thinning. Since frequency wrapping occurs in each plane due to subsampling, it becomes difficult to properly separate the low frequency component and the high frequency component in the frequency conversion in the coding processing process. Therefore, when the quantization step of the high frequency band component having a small influence on the image quality is increased, the low frequency band component having a large influence on the image quality is also strongly quantized. Therefore, when the RAW data is encoded at a high compression rate, the image quality deteriorates significantly, and the problem is that the compression efficiency is poor.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for encoding image data of a Bayer array with a higher compression rate than before while suppressing deterioration in image quality.

ことを特徴とする。 In order to solve this problem, for example, the image coding apparatus of the present invention has the following configuration. That is,
An image coding device that encodes image data in a Bayer array.
A separation means for separating the image data of the Bayer array into four color planes: an R plane showing the R component, a B plane showing the B component, a G1 plane showing the G1 component, and a G2 plane showing the G2 component.
A conversion means for converting the four color planes obtained by the separation means into a Y plane indicating the luminance Y and a luminance color difference plane composed of the C1 plane, the C2 plane, and the C3 plane showing the luminance C1, C2, and C3. When,
Have a coding means for coding the luminance and chrominance planes,
The conversion means converts the color plane into a luminance color difference plane according to the following equation.

It is characterized by that .

According to the present invention, it is possible to encode the image data of the Bayer array with a higher compression rate than before while suppressing deterioration of image quality.

The block block diagram of the image processing apparatus of 1st Embodiment. The figure which shows the Bayer array. The figure which shows the table related to the setting of the coding information. It is a figure which showed the detail of the coding part which comprises the parallel coding part 109. It is a figure which shows the example of the subband division. The figure which shows the flowchart which shows the image code processing procedure of 1st Embodiment. The figure which shows the flowchart which shows the processing of the coding part in the parallel coding part 109. The figure which shows the flowchart in the quantization setting for each line. The block block diagram of the image processing apparatus which concerns on 2nd Embodiment. The flowchart which shows the procedure of the image coding processing of 2nd Embodiment. The figure which shows the example of the code amount allocation for each subband.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the following description, an example in which an image coding device is mounted on an image pickup device (digital video camera or the like) will be described. However, as long as the image data to be encoded has a Bayer array, the source of the image data to be encoded does not matter. For example, it can be applied to a case where image data to be encoded is acquired from a storage medium such as a memory card or a network and encoded. It should be understood that the image pickup device is used as an example for the sake of easy understanding.

[First Embodiment]
FIG. 1 is a block configuration diagram of an imaging device (digital video camera) to which the first embodiment is applied. Hereinafter, each component will be described with reference to the figure.

The image pickup device 100 includes an operation unit 101, a main control unit 102, an image pickup unit 104, a signal processing unit 105, an image coding device 114, a coding storm generation unit 110, a primary storage unit 111, a memory I / O unit 112, and a storage medium. It has 113 and a bus 150 for connecting the above configurations.

The image coding device 114 includes a coding control unit 103, a color separation unit 106, a plane conversion unit 107, a plane selection unit 108, and a parallel coding unit 109.

The image pickup apparatus in the first embodiment compresses the image data of the Bayer array shown in FIG. 2 (hereinafter referred to as RAW image data) without performing development processing. After the compression mode is set by the operation unit 101, the image pickup apparatus 100 according to the first embodiment receives an input of an instruction to start shooting a subject and performs imaging, coding, and recording. The compression mode is provided with a plurality of modes such as a low compression mode for post production, a medium compression mode that is assumed to be used regularly, and a high compression mode that requires recording for a long time at the expense of image quality. The selection of these modes will be set by the user via the operation unit 101. The compression mode is used when the main control unit 102 sets the compression rate at the time of shooting. The operation unit 101 may be a shutter switch (recording switch), a push button such as an operation key, or a touch panel, or a combination thereof.

The main control unit 102 includes a CPU and a memory for storing a control program executed by the CPU, and the CPU reads a control program from the memory and controls the image pickup apparatus 100 in response to a command from the operation unit 101. Further, the main control unit 102 supplies the photographing information to the coding control unit 103. The shooting information in this embodiment is composed of, for example, the number of pixels of image data, the bit depth per pixel, and the compression rate, but shutter speed, ISO sensitivity, and the like may also be used. The compression rate is set by the operation unit 101, and a predetermined value is set according to the compression mode sent to the main control unit 102. For example, it can be set to 1/2 in the low compression mode, 1/5 in the medium compression mode, and 1/8 in the high compression mode.

The coding control unit 103 sets the coding conditions based on the shooting information sent by the main control unit 102. The coding conditions in the present embodiment are the attributes of the plane to be coded and the target code amount for each plane. Here, the attributes of the plane refer to the color plane and the luminance color difference plane described later.

Further, the target code amount for each plane is set by adding the weighting of the code amount of each plane from the total target code amount. The total target code amount can be obtained by, for example, calculating "the number of pixels of image data x the bit depth per pixel x the compression rate". The target code amount for each plane is, for example, if the total target code amount is 45 MB and the weighting of each plane consisting of P1 to P4 is P1: P2: P3: P4 = 2: 1: 1: 1, the P1 plane is 18 MB. , P2 to P4 planes are obtained as 9 MB each.

The attributes of the planes and the weighting of the code amount for each plane are set by subtracting the table from the table as shown in FIG. 3 according to the compression ratio. The example in Fig. 3 assumes a low compression rate for post-production editing and processing with a compression rate of 1/3 as a delimiter, and an application for viewing recorded moving images in the same way as a conventional consumer digital video camera. Two patterns are defined in the case of a high compression ratio.

When editing and processing compressed RAW image data in post-production, it is expected that image processing such as sharpness and noise removal will be strongly applied or color correction will be performed. Therefore, the luminance color difference plane conversion is not performed, the weights of the code amounts between the color planes are made the same, and the coding is performed with low compression so that the balance of each color originally possessed by the RAW image data is not impaired.

For viewing purposes, it is important to maintain image quality while increasing the compression rate to extend the recording time of moving images. Therefore, in the case of a high compression ratio, the luminance color difference plane is used as the coding target plane, and the code amount of the Y plane corresponding to the luminance component having a large visual influence is used as C1 to C3 corresponding to the luminance component having a small visual influence. The code amount is weighted so that it is larger than the code amount of the plane. The compression rate is increased by strongly compressing the color difference component, and the image quality can be maintained by suppressing the deterioration of the luminance component. A table may be prepared in a pattern different from that shown in FIG. 3. The coding control unit 103 supplies the attributes of the plane acquired from the table to the plane selection unit 108, and similarly acquires the plane from the table. The target code amount for each plane calculated according to the weighting of the code amount for each plane is supplied to the parallel coding unit 109.

After the coding control unit 103 sets the coding conditions, the main control unit 102 drives the imaging unit 104 to acquire the captured image data of the subject. The image pickup unit 104 includes an image pickup lens, an image pickup sensor having a Bayer array color filter shown in FIG. 2, and an A / D converter. Then, after the optical image of the subject is imaged on the sensor by the image pickup lens, the imaged optical image is converted into digital RAW image data by the A / D converter. The image sensor is composed of CMOS or CCD. Further, the imaging device according to the embodiment shall capture a moving image at, for example, 30 frames / second.

One frame of RAW image data sent from the imaging unit 104 is supplied to the signal processing unit 105. The signal processing unit 105 repairs the values of missing pixels and unreliable pixels in the image sensor of the imaging unit 104 in the input RAW image data by using the peripheral pixel values, or sets a predetermined offset value. A process such as subtraction is performed, and the process result is supplied to the image coding device 114.

The color separation unit 106 in the image coding device 114 separates the RAW image data repaired by the signal processing unit 105 into the four planes shown below, and supplies the RAW image data to the plane conversion unit 107 and the plane selection unit 108.
-R plane: A plane composed of pixels containing only the R component-B plane: A plane composed of pixels containing only the B component-G1 plane: A plane composed of pixels containing only the G1 component-G2 plane: Only the G2 component Plane composed of pixels

The plane conversion unit 107 converts the above four planes into a Y plane showing the luminance Y, a C1 plane showing the color differences C1, C2, and C3, a C2 plane, and a C3 plane based on the following equation (1). In the following formula, + and-indicate the calculation of the value of the pixel at the same position in each plane, and assume that the calculation result is an integer (rounded down to the nearest whole number).

When the number of pixels in the horizontal direction of the RAW image data is expressed as W and the number of pixels in the vertical direction is expressed as H, the sizes of the above R, B, G1, G2, Y, C1, C2, and C3 planes are the same. There is, W / 2 × H / 2. That is, when the RAW image data is separated into 4 planes, the total number of pixels of the 4 planes after the separation is the same as the number of pixels of the RAW image data.

The plane selection unit 108 selects either the color plane group of {R plane, B plane, G1 plane, G2 plane} or the luminance color difference plane group of {Y plane, C1 plane, C2 plane, C3 plane}. Then, it is supplied to the parallel coding unit 109. The selection of the plane group to be encoded is determined by the main control unit 101 from the mode selected by the operator for the operation unit 101, and is set in the coding control unit 103. Specifically, as shown in FIG. 3, when the operator has a compression ratio of 1/1 to 1/3 of the code amount (range of 100% to 33% of the original information amount), the color plane Is the coding target, and when the compression ratio exceeds 1/3, the luminance color difference plane is the coding target. It should be noted that the compression ratio of plane switching shown here is an example.

As shown in FIG. 4, the parallel coding unit 109 is composed of four coding units 109a to 109d, each of which compression-encodes one plane constituting the supplied plane group. The coding units 109a to 109d in the embodiment perform compression coding independently of each other.

The coding units 109a to 109d in the parallel coding unit 109 have the same configuration. Therefore, the coding unit 109a will be described below. FIG. 4 is a block configuration diagram of the coding unit 109a. Note that the other coding units 109b to d are the same.

The coding unit 109a includes a discrete wavelet transform unit 401 and a quantization control unit 402. It has a quantization unit 403 and an entropy coding unit 404.

The discrete wavelet unit 401 performs frequency separation processing on the input coded plane to generate a plurality of subbands. FIG. 5 shows an example of the subband generated by the frequency separation process by the discrete wavelet unit 401. The figure shows the sub-band when the wavelet transform is executed twice.

When the wavelet transform is performed once, four subbands of LL, HL, LH, and HH are generated. Here, L represents a low frequency band and H represents a high frequency band. The numbers before L and H represent the decomposition level. For example, 1HL represents a subband image having a decomposition level = 1 in which the horizontal direction is a high frequency component and the vertical direction is a low frequency component. There is no limit to the number of wavelet transforms. Further, the second and subsequent wavelet transforms are recursively performed with respect to the subband LL obtained by the immediately preceding wavelet transform. Therefore, there is only one subband LL when the wavelet transform is performed a predetermined number of times. The illustration shows an example in which the wavelet transform is performed twice, so that seven subbands are generated.

The quantization control unit 402 sets the code amount for each subband and the quantization step size for each line constituting each subband. The quantization control unit 402 allocates the code amount for each subband as in the example shown in FIG. 11 based on the target code amount for each plane sent from the coded control unit 101. FIG. 11 shows an example in which a target code amount of 18 MB set on the plane is assigned to the subband generated by executing the wavelet transform once. In the example of FIG. 11, the code amount ratio between the subbands has a relationship of "LL: 1HL: 1LH: 1HH = 4: 2: 2: 1". Based on this, 8 MB is allocated to the sub band LL, 4 MB to each of the sub bands 1HL and 1 LH, and 2 MB to the sub band 1HH band. Since the sub-band LL corresponds to a low-frequency component having a large effect on the image quality in FIG. 11, the sub-band LL is given preferential treatment and a large amount of code is assigned. Subsequently, the 1HL and 1LH bands containing the low frequency components in one horizontal or vertical direction are set to allocate a larger amount of code than the 1HH band having a slight influence on the image quality. Further, FIG. 11 shows the code amount when the wavelet transform is performed once, but the target code amount for each subband is also set when the wavelet transform is performed twice or more. Simply, the number of wavelet transforms and the target code amount of each subband may be tabulated.

Further, the quantization control unit 402 refers to the coding result sent from the entropy coding unit 404, which will be described later, and sets the quantization step size in units of coefficient lines. The details of setting the quantization step size will be described later in the description of the processing flow of the parallel coding unit 109.

The quantization unit 403 targets one line composed of the coefficient data in the subband of interest sent from the discrete wavelet unit 401 as the quantization target, and quantizes using the quantization step size set by the quantization control unit 402. Quantization.

The entropy coding unit 404 compresses and encodes a line of coefficients quantized by the quantization unit 403, outputs coded data in line units, and outputs information on the amount of code generated in line units to the quantization control unit 402. send. Here, DPCM, Huffman code, or the like can be used as the compression coding used in the entropy coding unit 404.

The coded stream generation unit 110 collects the coded data (coded data of all planes and all subbands) for one frame generated by the parallel code unit 109, and adds a header indicating the coding conditions and the like. , Generate a decodable stream. Further, the coded stream generation unit 110 saves the code sent from the parallel coding unit 109 to the temporary storage unit 111 until the codes for all subbands and all planes are aligned. The temporary storage unit 111 can be composed of a storage element such as a DRAM. The coded stream generation unit 110 reads out the code saved in the temporary storage unit 111 when the codes for all subbands and all planes are aligned, and generates a stream.

The coded stream generation unit 110 writes the generated stream to the recording medium 113 via the memory I / O unit 112. The recording medium 113 can be composed of a magnetic disk, a semiconductor memory, or the like, but the type thereof does not matter.

The processing of the configuration of the image pickup apparatus 100 according to the first embodiment has been described above. Subsequently, the details of the coding process in the digital video camera equipped with the image coding device according to the first embodiment will be described with reference to the flowchart shown in FIG.

First, in response to the issuance of a shooting recording start command from the operation unit 101 to the main control unit 102, the main control unit 102 sends shooting information to the coding control unit 103 (S601). The coding control unit 103 sets the coding conditions according to this photographing information (S602). This coding condition also includes information on whether the plane selection unit 108 selects a color plane or a luminance color difference plane. After that, the RAW image data is acquired by the imaging unit 104, and the signal processing unit 105 repairs the RAW image data (S603). The color separation unit 106 inputs the RAW image data after the restoration process, and separates the RAW image data into four color planes {R plane, B plane, G1 plane, G2 plane} (S604). The plane conversion unit 107 converts the four color planes into luminance color difference planes {Y plane, C1 plane, C2 plane, C3 plane} (S605). The plane selection unit 108 selects one of the color plane and the luminance color difference plane as the coding target plane according to the coding conditions (S606), and supplies the parallel coding unit 109 (S607). Each of the coding units 109a to 109d inside the parallel coding unit 109 performs the coding process of each of the coding target planes in parallel, and outputs the generated coded data to the coded stream generation unit 110. The coded stream generation unit 110 stores the coded data output from the parallel coded unit 109 in the temporary storage unit 111, and determines whether or not the coded data of all the subbands has been completed (S608). .. If it is not finished, the processing state is maintained as S608. When the coding of all planes and all subbands is completed, the coded stream generation unit 110 reads all the coded data from the temporary storage unit 111 and generates a coded stream of a predetermined data format ( S609). Then, the coded stream generation unit 110 writes the generated coded stream to the storage medium 113 via the memory I / O unit 112 (S610). The main control unit 102 determines whether or not the operator has instructed the end of shooting via the operation unit 101 (S611). If no, the main control unit 102 assumes that the shooting and recording of the moving image is continued, and returns the process to step S603. Further, when it is determined that the end instruction has been input, the main control unit 102 ends this process.

Next, the details of the processing of the coding units 109a to 109d constituting the parallel coding unit 109 in the embodiment will be described with reference to the flowchart shown in FIG. As described above, the coding units 109a to 109d have the same configuration, and here, the coding units 109a (see FIG. 4) will be described as a representative.

The discrete wavelet unit 401 executes wavelet transform on the input plane of interest, generates a plurality of subbands (S701), sets the subband to be encoded and the target code amount of the subband (S702). Each subband is composed of a plurality of coefficient data. The quantization control unit 402 determines whether or not the coefficient data for one line input to the quantization unit 403 is the first line of the subband of interest (S703). In the case of the first line, the quantization control unit 402 sets an initial predetermined quantization step size (S704), and in the case of not the first line, the quantization control unit 402 sets the coding result of the previous line. The quantization step size is calculated and set with reference to (S705). As the predetermined quantization step size set in the coding of the first line, a value defined in advance by, for example, a statistical value can be used. Here, as a method of calculating the quantization step size of S705, for example, there is a method shown in FIG. 8 described later.

After that, the quantization unit 403 quantizes the coefficient data for one line according to the quantization step size set by the quantization control unit 402 (S706), and the coefficient data for the line after quantization is entropy coding unit. Supply to 404. The entropy coding unit 404 entropy-encodes the input quantization coefficient data for one line, generates the coded data, and outputs the coded data (S707). At this time, the entropy coding unit 404 supplies information indicating the amount of coded data of the line of interest to the quantization control unit 402.

After that, the coding unit 109 determines whether or not the final line of the subband of interest has been processed (S708), and returns to S703 if it is not the final line. When the final line is processed, it is subsequently determined whether the processing of all subbands is completed (S709), if it is not completed, the process returns to S702, and if it is completed, the coding is terminated.

Subsequently, the calculation process of the quantization step size by the quantization control unit 402 of S705 will be described with reference to the flowchart shown in FIG.

Lines adjacent to each other in one subband are highly correlated. Therefore, when determining the quantization step size of the line to be encoded, it can be estimated that the coding efficiency will be high if the coding result of the immediately preceding (upper) line is used. Therefore, in the present embodiment, the quantization control unit 402 divides the target code amount of the focus subband into equal parts by the number of lines and equalizes the target code amount T per line prior to coding the focus subband. (S801). Then, when the quantization control unit 402 encodes the line of interest to be encoded, it compares the actual generated code amount (R) of the immediately preceding line with T and determines whether R> T + α. Is determined (S802). Here, α is a predetermined positive tolerance value used when comparing R and T, and can be obtained by calculation such as T × 0.01 or set by a numerical value such as 50 KB. When R> T + α, it is predicted that the code amount of the coded target line will be larger than the target code amount if the quantization step size set in the immediately preceding line is used. Therefore, the quantization control unit 402 sets the quantization step size of the line of interest to be encoded as a value obtained by adding a predetermined value (positive value) to the quantization step size of the immediately preceding line (S803). On the other hand, if R> T + α does not apply, the quantization control unit 402 determines whether or not R <T—α is satisfied (S804). When R <T-α, it means that the code amount of the coded target line is predicted to be less than the target code amount when the quantization step size set in the immediately preceding line is used. Therefore, the quantization control unit 402 sets the quantization step size of the line of interest to a value obtained by subtracting a predetermined value from the quantization step size of the immediately preceding line (S805). If none of the above applies, that is, if T−α ≦ R ≦ T + α, it is predicted that the code amount of the coded line will be generated as the target if the quantization step size set in the immediately preceding line is used. , The quantization step size of the immediately preceding line is set as it is (S806).

As described above, the image coding device mounted on the digital video camera of this embodiment generates a plane image of each color by separating each color component from the image data of the Bayer array, and then from each color plane. Separate into a luminance plane and a color difference plane, and encode the luminance color difference plane. The luminance plane is an average component obtained by extracting the correlation of R, G1, G2, and B, and has a great visual influence. On the contrary, the color difference plane is a difference component from which the correlation of R, G1, G2, and B is removed, and has a smaller visual influence than the luminance plane. Therefore, by giving preferential treatment to the code amount of the luminance plane and reducing the code amount of the color difference plane in performing the quantization, it is possible to compress the image data of the Bayer array while maintaining the image quality.

In this embodiment, a digital video camera that compresses image data in a Bayer array is taken as an example, but the present invention is not limited thereto, and may be used for, for example, a digital still camera or a smartphone.

[Second Embodiment]
FIG. 9 is a block configuration diagram of the imaging device according to the second embodiment. The difference from the first embodiment is that the approximate plane conversion unit 901 is used instead of the plane conversion unit 107 of the first embodiment. Hereinafter, the operation in the second embodiment will be described with reference to FIG.

In the calculation formula 1 used in the image coding apparatus 114 in the first embodiment described above, it is assumed that the calculation result of each plane is an integer value. Therefore, when the calculation result of division is not an integer value. Will cause a rounding error. Information loss occurs in the rounding process of division, and it is not possible to perform reversible conversion from the luminance color difference plane to the original color plane. Therefore, in order to exclude the information loss due to the rounding process, a calculation formula that approximates the calculation formula of the plane conversion unit 107 is applied to the approximation plane conversion unit 901.

The approximate plane conversion unit 901 in the second embodiment converts the color plane into a luminance color difference plane composed of a Y plane, a C1 plane, a C2 plane, and a C3 plane based on the following equation (2).

近似プレーン変換部９０１に用いられる計算式は、第１の実施例のプレーン変換部１０７で用いられている計算式を近似したものである。近似プレーン変換部９０１に用いられる式（２）は、復号側で次式（３）に基づいて色プレーンに逆変換することで、完全に元に戻せることが特徴である。

The calculation formula used in the approximate plane conversion unit 901 is an approximation of the calculation formula used in the plane conversion unit 107 of the first embodiment. The equation (2) used in the approximate plane conversion unit 901 is characterized in that it can be completely restored by reverse conversion to a color plane based on the following equation (3) on the decoding side.

It should be noted that the equation (3) is an inverse conversion equation obtained by modifying each equation of the equation (2) by using only the transposition.

Subsequently, the details of the operation of the image pickup apparatus 900 equipped with the image coding apparatus 902 of the second embodiment will be described with reference to the flowchart shown in FIG. The same processing as in the first embodiment is designated by the same reference numerals, and the description thereof will be simplified.

First, as in the first embodiment, the coding conditions are set after receiving the photographing command. The color separation unit 106 acquires RAW image data and separates it into color planes (S601-S604). After that, the approximate plane conversion unit 901 generates a luminance color difference plane according to the equation (2) based on the color plane (S1001). Next, the plane to be encoded is selected and encoded to generate an encoded stream (S606-S611).

As described above, the image coding apparatus mounted on the image pickup apparatus of the second embodiment uses a calculation formula that approximates the calculation formula used in the first embodiment when converting from each color plane to the luminance color difference plane. Use. As a result, although the quantization error due to the quantization process by the coding process occurs, the calculation error due to the color conversion is eliminated, and it is possible to generate the coded data with less loss as compared with the first embodiment. ..

As described above, according to the first and second embodiments, the RAW data can be separated into a luminance having a large influence on the image quality and a color difference having a small influence on the image quality. Unlike the method using only subsampling described in Patent Document 1, it is possible to separate a component having a large effect on image quality and a component having a small effect on image quality, so that the color difference is quantized in order to achieve a high compressibility. Can be made larger. Therefore, the compression efficiency can be improved by suppressing the deterioration of the image quality when encoding the RAW data with a high compression rate.

In the above embodiment, the coding units 109a to 109d have been described as being executed in parallel, but one coding unit may encode a plurality of planes in order.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101 ... operation unit, 102 ... main control unit, 103 ... coding control unit, 104 ... imaging unit, 105 ... signal processing unit, 106 ... color separation unit, 107 ... plane conversion unit, 108 ... plane selection unit, 109 ... parallel Coding unit, 110 ... Coded stream generation unit, 111 ... Temporary storage unit, 112 ... Memory I / O unit, 113 ... Storage medium

Claims (8)

An image coding device that encodes image data in a Bayer array.
A separation means for separating the image data of the Bayer array into four color planes: an R plane showing the R component, a B plane showing the B component, a G1 plane showing the G1 component, and a G2 plane showing the G2 component.
A conversion means for converting the four color planes obtained by the separation means into a Y plane indicating the luminance Y and a luminance color difference plane composed of the C1 plane, the C2 plane, and the C3 plane showing the luminance C1, C2, and C3. When,
Have a coding means for coding the luminance and chrominance planes,
The conversion means converts the color plane into a luminance color difference plane according to the following equation.

An image coding device characterized in that. The coding means is
A wavelet transform means that generates multiple subbands by wavelet transforming the plane of interest,
A quantization means that quantizes the coefficient data of each subband obtained by the wavelet transform means using a quantization step set for each line, and a quantization means.
An entropy coding means that entropy encodes the coefficient data quantized by the quantization means in line units, and
The image code according to claim 1 , further comprising a coding control means for determining a quantization step for the next line based on the code amount of the coded data generated by the entropy coding means. Device. The coding control means is
The target code amount T per line is obtained by dividing the target code amount of the focus subband by the number of lines included in the focus subband.
Let R be the amount of code obtained when coding the line immediately before the line of interest in the quantization target in the subband of interest, and let α be the positive tolerance.
When R> T + α, a value obtained by adding a predetermined value to the quantization step size used in the immediately preceding line is determined as the quantization step size of the line of interest.
In the case of R <T-alpha determines the value obtained by subtracting a predetermined value from the quantization step size used in the immediately preceding line as the quantization step size of the intended line,
The image coding according to claim 2 , wherein when T−α ≦ R ≦ T + α, the quantization step size used in the immediately preceding line is determined as the quantization step size of the line of interest. apparatus. The image coding according to claim 3 , wherein the relationship between the target code amounts in the subbands LL, HL, LH, and HH generated by the wavelet transform means is LL> HL = LH> HH. apparatus. Further, claim 1 is characterized in that it has a selection means for selecting either one of the color plane separated by the separation means and the luminance color difference plane obtained by the conversion means and supplying the coding means. The image coding apparatus according to any one of 3 to 3 . When the color plane is used as a coding target, the target code amount of each plane is the same.
The image coding apparatus according to claim 5 , wherein the target code amount of the luminance plane when encoding the luminance color difference plane is larger than that of any of the other three luminance planes. A control method for an image coding device that encodes image data in a Bayer array.
The separation step separates the image data of the Bayer array into four color planes: an R plane showing the R component, a B plane showing the B component, a G1 plane showing the G1 component, and a G2 plane showing the G2 component.
The conversion means changes from the four color planes obtained in the separation step to a luminance color difference plane composed of a Y plane showing the luminance Y and a C1 plane, a C2 plane, and a C3 plane showing the luminance C1, C2, and C3. The conversion process to convert and
Encoding means, it possesses an encoding step of encoding the luminance and chrominance planes,
In the conversion step, the color plane is converted into a luminance color difference plane according to the following equation.

A control method for an image coding device. A program for causing the computer to function as each means included in the image coding apparatus according to any one of claims 1 to 6 , when the computer reads and executes the image.

Images