JP2007312399A - Image encoding device and image decoding device, and image display device and method utilizing the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that computational complexity in encoding is increased by scale-up processing in priority encoding to a region of interest (ROI) in an image and an image quality difference can not be provided among a plurality of ROIs. <P>SOLUTION: A wavelet transforming unit 10 performs wavelet transformation on a source image, and a quantization unit 12 quantizes a wavelet transform coefficient. An ROI selection unit 18 selects a plurality of ROIs on the source image, and an ROI priority setting unit 19 sets priorities among the plurality of ROIs. A low-order bit zero replacing unit 23 replaces with zero low-order bits of wavelet transform coefficients corresponding to a region of non-interest and the plurality of ROIs in accordance with the priorities. An entropy encoding unit 14 performs entropy encoding on the replaced wavelet transform coefficients in order from a high-order bit plane, and an encoded data generating unit 16 outputs the encoded data in a stream. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image encoding device, an image decoding device, an image display device, and an image display method.

  In ISO / ITU-T, standardization of JPEG2000 using discrete wavelet transform (DWT) is performed as a successor of JPEG (Joint Photographic Expert Group), which is a standard technology for compression coding of still images. JPEG2000 can encode a wide range of image quality from low bit rate encoding to lossless compression with high performance, and it is easy to realize a scalability function that gradually increases image quality. In addition, JPEG2000 provides various functions not found in the conventional JPEG standard.

  As one of the functions of JPEG2000, ROI encoding that encodes and transmits a region of interest (ROI) of an image with priority over other regions is standardized. With ROI encoding, when there is an upper limit on the encoding rate, the reproduction image quality of the attention area can be preferentially made high quality, and when the encoded stream is sequentially decoded, the attention area is quickly improved in quality. Will be able to play.

  As ROI encoding, the bit plane of a wavelet transform coefficient (hereinafter referred to as ROI transform coefficient) corresponding to a region of interest of an image is the largest bit plane of a wavelet transform coefficient (hereinafter referred to as non-ROI transform coefficient) corresponding to a non-target region. There is a Max Shift method that scales up by the number of bits. According to this method, all bit planes of ROI transform coefficients are encoded before bit planes of any non-ROI transform coefficients. There is also known a method in which a part of the upper bits of the ROI transform coefficient is encoded with priority over the non-ROI transform coefficient by scaling up the bit plane of the ROI transform coefficient by a predetermined number of bits. Patent Document 1 proposes an improved technique for these ROI encoding methods.

  FIG. 1 is a configuration diagram of a conventional image encoding device 100. The image encoding apparatus 100 performs ROI encoding by the max shift method. The wavelet transform unit 10 performs wavelet transform on the input original image and outputs wavelet transform coefficients. The quantization unit 12 quantizes the wavelet transform coefficient of the original image. FIG. 2A shows the wavelet transform coefficient 50 after quantization, and includes 5 bit planes from the most significant bit (MSB) to the least significant bit (LSB).

  The ROI selection unit 18 selects a region of interest on the original image, and the ROI mask generation unit 20 generates a wavelet transform coefficient corresponding to the selected region of interest, that is, an ROI mask for specifying the ROI transform coefficient. The ROI transform coefficient is indicated by diagonal lines in the wavelet transform coefficient 50 of FIG.

  The ROI scale-up unit 22 refers to the ROI mask and scales up the quantized ROI transform coefficient by S bits. That is, the value of the ROI conversion coefficient is shifted left by S bits. Here, the scale-up amount S is a natural number larger than the number of bits of the maximum value of the quantized value of the wavelet transform coefficient corresponding to the non-target region, that is, the non-ROI transform coefficient. FIG. 2B shows the wavelet transform coefficient 52 in a state where the ROI transform coefficient is scaled up by 5 bits. In the wavelet transform coefficient 52 after the scale-up, a zero value is assigned to the digit newly generated by the scale-up.

  The entropy encoding unit 14 performs entropy encoding while sequentially scanning the quantized value of the scaled-up wavelet transform coefficient 52 from the upper bit brain as indicated by the arrow in FIG. The encoded data generation unit 16 streams the entropy-encoded data together with encoding parameters such as a quantization width, ROI position information indicating the position of the region of interest, and a scale-up amount, and outputs the data as an encoded image. .

  FIG. 3 is a configuration diagram of a conventional image decoding device 110. The image decoding device 110 decodes an ROI-encoded image by the max shift method. The encoded data extraction unit 30 extracts encoded data, various encoding parameters, ROI position information, and a scale-up amount from the input encoded image. The entropy decoding unit 32 decodes the encoded data for each bit plane and stores the obtained quantized value of the wavelet transform coefficient in the memory. FIG. 4A shows the wavelet transform coefficient 54 after entropy decoding. Corresponding to the examples of FIGS. 2A to 2C, the scale-up amount S is 5 bits, and the wavelet transform coefficient 54 after entropy decoding includes a 10-bit plane as a whole.

  The ROI mask generation unit 38 generates a mask for specifying the ROI conversion coefficient based on the ROI position information. The ROI scale down unit 40 refers to the ROI mask and scales down the ROI transform coefficient by S bits among the quantized wavelet transform coefficients. FIG. 4B shows the wavelet transform coefficient 56 in a state where the ROI transform coefficient is scaled down by 5 bits. The upper 5 bits of the non-ROI transform coefficients left by the scale-down are discarded, and a quantized wavelet transform coefficient 56 of a 5-bit plane as a whole is obtained.

The inverse quantization unit 34 inversely quantizes the quantized value of the wavelet transform coefficient including the scaled down ROI transform coefficient, and the wavelet inverse transform unit 36 performs the wavelet inverse transform on the inversely quantized wavelet transform coefficient. The decoded image obtained is output.
JP 2001-45484 A

  In the conventional ROI encoding described above, when there are a plurality of attention areas, there is a problem that a difference in image quality cannot be provided between the attention areas.

  The present invention has been made in view of these problems, and an object of the present invention is to provide an image encoding technique, an image decoding technique, and an image that can encode, decode, and display an image including a plurality of regions having different image quality. To provide display technology.

One embodiment of the present invention relates to an image encoding method. In this method, information for specifying a region of interest defined on a moving image is explicitly described in an encoded stream including encoded data of the moving image.
Another aspect of the present invention also relates to an image encoding method. In this method, information for specifying a plurality of regions of interest defined on an image is explicitly described in an encoded stream including encoded data of the image.
One embodiment of the present invention relates to an image encoding device. The apparatus includes: a selection unit that selects a region of interest on a moving image; an encoding unit that encodes the moving image; and an encoded stream that includes the encoded moving image and information for specifying the region of interest. And a generating unit that generates
Another embodiment of the present invention also relates to an image encoding device. The apparatus includes a selection unit that selects a plurality of regions of interest on an image, an encoding unit that encodes the images, and an encoded image and information for identifying the plurality of regions of interest. A generation unit that generates a stream.

One embodiment of the present invention relates to an image display apparatus. This apparatus includes a display unit that displays moving images having different image quality for each of a plurality of areas set for the screen.
The image processing apparatus further includes a designation unit that designates at least one of the attention area and the non-attention area for the screen, and the plurality of areas are set for the screen by designation of at least one of the attention area and the non-attention area. May be. For example, when one attention area is designated for the screen, two areas of the attention area and other areas are set for the screen. Also, by designating one non-attention area for the screen, two areas of the non-attention area and other areas are set for the screen. A plurality of attention areas or non-attention areas may be specified, and a plurality of areas such as attention areas, non-attention areas, and other areas are set by combining the attention area and the non-attention area. May be.

  The attention area and the non-attention area are in a relative relationship such that when one area is determined as attention, the other area becomes non-attention. It may be an area having a relative difference in request level. The attention area is an area that has a higher priority than other areas, particularly high image quality requirements, and the non-attention area is an area that has a lower priority than other areas, especially low image quality requirements. May be.

  Here, the “display unit to be displayed on the screen” includes an interface connected to a display device in addition to a form including both a screen such as a display and a display control unit for controlling output of an image signal supplied to the screen. In addition, it is intended to include the form of only the display control unit that does not include the display device, and any of these forms is included in the technical scope.

  Another embodiment of the present invention also relates to an image display device. This apparatus includes a display unit that has a plurality of areas with different image quality and displays moving images that overlap the areas on the screen.

Still another embodiment of the present invention also relates to an image display device. This apparatus includes a display unit that displays a moving image having a plurality of regions with different image quality while maintaining visibility. Here, “changing the image quality while maintaining visibility” means adjusting the image quality within a range that can be recognized as an image of the object, or lowering the image quality within a range where the object can be recognized, or conversely This includes making the region of interest have higher image quality than other regions.
In any one of the image display devices described above, a decoding block that receives position information of the plurality of areas, decodes an encoded image with reference to the position information, and generates a decoded image having different image quality for each of the plurality of areas May further be included. The display unit may display the decoded image generated by the decoding block on the screen.
In any one of the above image display devices, encoding that receives position information of the plurality of areas, encodes an original image with reference to the position information, and generates encoded images having different image quality for each of the plurality of areas A block may further be included. The display unit may display an image decoded based on the encoded image on the screen.

  Still another embodiment of the present invention also relates to an image display device. This apparatus is configured to select at least one of an attention area and a non-attention area on a screen on which a moving image is displayed, and set on the screen by selecting at least one of the attention area and the non-attention area. A display unit that displays the moving images having different image quality for each of the plurality of regions. According to this configuration, it is possible to intentionally change the image quality of a plurality of areas for display.

  Here, the plurality of areas may be selected by the user specifying areas on the screen. In addition, the plurality of regions may be selected by extracting a region where a specific object exists by some method. For example, an area in which characters are shown, an area in which a person is shown, and other areas may be extracted as a plurality of areas by an image recognition method. In addition, specific locations such as the center of the screen, its periphery, and the outer periphery may be set in advance, and the set locations may be selected as a plurality of areas.

  The selection unit may set a priority for each region, and the display unit may display the plurality of regions with image quality corresponding to the priority. When there is an overlap between the plurality of regions, the image quality may be determined based on the higher priority in the overlapping portion. The priority may be specified when the user specifies an area for the screen. Also, the priority may be determined in advance, for example, by increasing the priority of the center of the screen or by increasing the priority of the area where the characters are shown.

  Still another embodiment of the present invention also relates to an image display device. The apparatus includes a selection unit that selects a region of interest for a screen on which a moving image is displayed, and the state where the region of interest is not selected, the entire screen is displayed with a predetermined image quality, and the region of interest is selected. The display unit displays the image quality of the attention area higher than the image quality of the other areas. According to this configuration, at the time of normal playback of a moving image, simple playback with, for example, low image quality can be performed, and when a region of interest is selected, it can be reproduced with a higher image quality of that region of interest.

  The display unit may display the attention areas with different image quality when a plurality of the attention areas are selected. The selection unit may select an area having a time change as the attention area. When there are a plurality of regions that have changed over time, the display unit may change the image quality according to the degree of change over time.

  Still another embodiment of the present invention relates to an image display method. In this method, moving images are displayed on the screen with different image quality for each of a plurality of areas set for the screen. This moving image may be read from the storage device or input via a network.

  Yet another embodiment of the present invention relates to an image encoding device. The apparatus scales up the bit sequence of the transform coefficient corresponding to each of the plurality of regions by the number of bits corresponding to the priority of each region, by transforming the image into a spatial frequency domain to generate transform coefficients. And a coding unit that sequentially compresses and codes the transform coefficients of the entire image in order from the upper bit after the scale-up by the scale-up unit. You may further include the mask production | generation part which produces | generates the mask for specifying the said conversion coefficient corresponding to each of the some area | region on the said image.

  Still another embodiment of the present invention also relates to an image encoding device. The apparatus scales up the bit sequence of the wavelet transform coefficient corresponding to each of the plurality of regions by the number of bits corresponding to the priority of each region, and a transform unit that generates wavelet transform coefficients by wavelet transforming the image. A scale-up unit; and a coding unit that sequentially compresses and codes the wavelet transform coefficients of the entire image from the upper bit plane after the scale-up by the scale-up unit. You may further include the mask production | generation part which produces | generates the mask for pinpointing the said wavelet transformation coefficient corresponding to each of the several area | region on the said image.

  Yet another embodiment of the present invention relates to an image encoding method. This method specifies the wavelet transform coefficient corresponding to a plurality of regions on the image in the wavelet transform coefficient of the image, and sets a bit string of the wavelet transform coefficient corresponding to the specified plurality of regions to a priority of each region Are scaled up by the number of bits corresponding to the above, and then the bit strings of the wavelet transform coefficients of the entire image are collectively compressed and encoded sequentially from the upper bit plane.

  Still another embodiment of the present invention also relates to an image encoding device. The apparatus includes: an area selection unit that selects a plurality of areas having different priorities on an image; and a code that includes a plurality of areas having different image quality by compressing and encoding the image according to the priority of each of the plurality of areas. And an encoding unit that generates an encoded image. A transform unit that transforms an image into a spatial frequency domain and generates transform coefficients; and a mask generator that generates a mask for specifying the transform coefficients corresponding to the plurality of regions selected by the region selector. In addition, the encoding unit may compress and encode the image provided with the plurality of regions with reference to the mask.

  Yet another embodiment of the present invention relates to an image decoding device. The apparatus includes a region selection unit that selects a plurality of regions having different priorities on an image, and inversely transforms the compression-encoded image according to the priority of each of the plurality of regions, and a plurality of regions having different image quality. And an inverse transform unit for restoring an image including the region. A decoding unit that decodes a compression-coded image to obtain transform coefficients in a spatial frequency domain, and generates a mask for specifying the transform coefficients corresponding to the plurality of areas selected by the area selection unit And a mask generation unit, wherein the inverse conversion unit may inverse convert the compression-coded image provided with the plurality of regions with reference to the mask.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a computer program, a recording medium, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to display an image by changing the image quality of the area set for the screen.

Embodiment 1
FIG. 5 is a configuration diagram of the image coding apparatus 200 according to Embodiment 1. The configuration of the image encoding device 200 can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and is realized in software by a program having an encoding function loaded in a memory. However, here, functional blocks that are realized by their cooperation are depicted. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The image encoding apparatus 200 compresses and encodes the input original image as an example using the JPEG2000 method. The original image input to the image encoding device 200 may be a moving image frame. The image encoding device 200 can generate a moving image encoded stream by continuously encoding each frame of a moving image using the JPEG2000 method.

  The wavelet transform unit 10 divides the input original image into subbands, calculates wavelet transform coefficients of each subband image, and generates layered wavelet transform coefficients.

  The wavelet transform unit 10 applies a low-pass filter and a high-pass filter in the x and y directions of the original image, and divides it into four frequency subbands for wavelet transform. These subbands are LL subbands having low frequency components in both x and y directions, and HL subbands having low frequency components in either one of x and y directions and high frequency components in the other direction. And an HH subband having a high frequency component in both the x and y directions. The number of vertical and horizontal pixels in each subband is ½ that of the image before processing, and a subband image having a resolution, that is, a quarter of the image size, can be obtained by one filtering.

  Of the subbands thus obtained, the wavelet transform unit 10 performs the filtering process again on the LL subband, further divides it into four subbands LL, HL, LH, and HH, and performs wavelet transform. . The wavelet transform unit 10 performs this filtering a predetermined number of times, hierarchizes the original image into subband images, and outputs wavelet transform coefficients for each subband. The quantization unit 12 quantizes the wavelet transform coefficient output from the wavelet transform unit 10 with a predetermined quantization width.

  The ROI selection unit 18 selects a region of interest on the original image and provides ROI position information indicating the position of the region of interest to the ROI mask generation unit 20. When the attention area is selected as a rectangle, the ROI position information is given by the coordinate value of the pixel at the upper left corner of the rectangular area and the number of pixels in the vertical and horizontal directions of the rectangular area.

  The attention area may be selected by the user specifying a specific area on the original image, or a predetermined area such as a central area of the original image may be selected. Also, an important area such as an area in which a person or a character is shown may be automatically extracted as the attention area. When a frame of a moving image is continuously input to the image encoding device 200, a region of interest may be automatically selected by tracking the movement of a specific region on the image frame.

  The ROI mask generation unit 20 generates a ROI mask for specifying the wavelet transform coefficient corresponding to the region of interest, that is, the ROI transform coefficient, based on the ROI position information.

  FIGS. 6A to 6C are diagrams illustrating the ROI mask generated by the ROI mask generation unit 20. As shown in FIG. 6A, it is assumed that the attention area 90 is selected on the original image 80 by the ROI selection unit 18. The ROI mask generation unit 20 specifies a wavelet transform coefficient necessary for restoring the attention area 90 selected on the original image 80 in each subband.

  FIG. 6B shows a converted image 82 of the first layer obtained by performing wavelet transform on the original image 80 only once. The converted image 82 of the first hierarchy is composed of four subbands LL1, HL1, LH1, and HH1 of the first level. The ROI mask generation unit 20 uses the wavelet transform coefficients on the first-level transformed image 82 necessary for restoring the attention area 90 of the original image 80, that is, the ROI transform coefficients 91 to 94, to the first-level subbands LL1. , HL1, LH1, and HH1.

  FIG. 6C shows a second-layer converted image 84 obtained by further wavelet transforming the subband LL1 of the lowest frequency component of the converted image 82 of FIG. 6B. As shown in the drawing, the second-layer converted image 84 includes four subbands LL2, HL2, LH2, and HH2 at the second level in addition to the three subbands HL1, LH1, and HH1 at the first level. The ROI mask generation unit 20 uses the wavelet transform coefficients on the transformed image 84 of the second hierarchy necessary for restoring the ROI transform coefficients 91 in the subband LL1 of the transformed image 82 of the first hierarchy, that is, ROI transform coefficients 95 to 98. Is specified in each of the second level subbands LL2, HL2, LH2, and HH2.

  Similarly, the ROI transform coefficient corresponding to the attention area 90 is recursively specified in each layer by the number of wavelet transforms, so that the ROI necessary for restoring the attention area 90 in the converted image of the final hierarchy is obtained. All conversion coefficients can be specified. The ROI mask generation unit 20 generates an ROI mask for specifying the position of the finally specified ROI conversion coefficient on the converted image of the final hierarchy. For example, when the wavelet transform is performed only twice, an ROI mask that can specify the positions of the seven ROI transform coefficients 92 to 98 indicated by diagonal lines in FIG. 6C is generated.

  The lower bit zero replacement unit 24 in FIG. 5 adjusts the number of lower bits to be replaced with a zero value in the bit string of the wavelet transform coefficient corresponding to the non-target region according to the relative priority of the target region with respect to the non-target region. With reference to the ROI mask generated by the ROI mask generation unit 20, only the S bits counted from the least significant bit in the bit string of the non-ROI transform coefficient not masked by the ROI mask are replaced with zero. Here, the zero substitution bit number S corresponds to the relative priority of the attention area with respect to the non-attention area, and is an arbitrary natural number with the maximum number of bits of the quantization value in the non-attention area as an upper limit. By changing the number of zero replacement bits S, it is possible to continuously adjust the degree of deterioration in the reproduction image quality of the non-target region with respect to the target region.

  FIGS. 7A to 7C are diagrams for explaining how the lower bits of the wavelet transform coefficient 60 of the original image are zero-substituted by the lower-bit zero substitution unit 24. FIG. FIG. 7A shows the wavelet transform coefficient 60 after quantization by the quantizing unit 12, includes a 5-bit plane, and the ROI transform coefficient is indicated by diagonal lines.

  As shown in FIG. 7B, the low-order bit zero replacement unit 24 replaces the S bit on the LSB side of the non-ROI transform coefficient not masked by the ROI mask with zero. In this example, S = 2 and a wavelet transform coefficient 62 is obtained in which 2 bits on the LSB side of the non-ROI transform coefficient are replaced with zero as indicated by reference numeral 64.

  The entropy encoding unit 14 in FIG. 5 performs entropy while sequentially scanning the wavelet transform coefficient 62 including the ROI transform coefficient and the non-ROI transform coefficient zero-replaced from the upper bit plane, as indicated by the arrow in FIG. Encode.

  FIGS. 8A to 8C are diagrams for explaining how the lower bits of the wavelet transform coefficient are replaced with zero when there is no region of interest on the original image. FIG. 8A shows a wavelet transform coefficient 70 of a 5-bit plane consisting only of non-ROI transform coefficients because no attention area is set in the original image. When the zero substitution bit number S is 2, the lower bit zero permutation unit 24, as shown in FIG. 8 (b), wavelet transform coefficient 72 obtained by substituting the lower 2 bit plane on the LSB side with zero among the 5 bit planes. Is generated.

  As shown in FIG. 8C, the entropy encoding unit 14 entropy encodes the upper 3 bit planes of the wavelet transform coefficient 72 after the zero substitution in order from the top. In this case, the low-order 2-bit plane subjected to zero substitution is not encoded. Note that the lower 2-bit plane may be simply discarded instead of replacing the lower 2-bit plane with zero.

  The encoded data generation unit 16 streams the entropy-encoded data together with an encoding parameter such as a quantization width and outputs it as an encoded image.

  In general, when an upper limit is set for the data size of the final encoded image due to storage capacity or transmission rate limitations, the entropy encoding unit 14 encodes the quantized wavelet transform coefficients in order from the upper bit plane. In order to maintain the upper limit of the data size, encoding may be interrupted in the bit plane on the way. Alternatively, when the encoded data generation unit 16 outputs the encoded data streamed in order from the upper bit plane, the stream output may be interrupted on the intermediate bit plane in order to keep the transmission rate restriction.

  Even when there is a restriction on the data size of the encoded image in this way, in the present embodiment, in the lower bit plane, the wavelet transform coefficient corresponding to the non-target region is zero-substituted, and thus corresponds to the target region. Since only wavelet transform coefficients are subject to encoding as significant information, the compression efficiency of the lower bit plane is high, and the data size does not increase greatly even when encoding is performed up to the lowest bit plane.

  FIG. 9 is a configuration diagram of the image decoding apparatus 210 according to Embodiment 1. Since the image in which ROI is preferentially encoded by image encoding apparatus 200 according to Embodiment 1 is a normal encoded image in which the ROI transform coefficient is not scaled up, image decoding apparatus 210 simply performs encoding. The decoded image can be decoded by a normal JPEG2000 decoding method.

  The encoded data extraction unit 30 extracts encoded data from the input encoded image. As an example, data obtained by encoding a 5-bit plane shown in FIG. 7C in order from the top is acquired. Here, it should be noted that the encoded data extraction unit 30 does not particularly need to extract ROI position information that gives the position of the region of interest. The entropy decoding unit 32 decodes the encoded data for each bit plane. In the example of FIG. 7C, the 5-bit plane is decoded and the quantized data is stored in the memory.

  The inverse quantization unit 34 inversely quantizes the decoded quantized data, and the wavelet inverse transform unit 36 inversely transforms the inversely quantized wavelet transform coefficients, and outputs the obtained decoded image.

  As described above, the image coding apparatus 200 according to the present embodiment performs priority replacement of ROI transform coefficients by performing zero substitution on non-ROI transform coefficients, and preferentially codes a region of interest. To do. Since the scale-up processing of the ROI transform coefficient is not performed, the encoding operation can be performed efficiently. Further, since the number of bit planes to be encoded does not increase, it is not necessary to provide an extra storage area, and hardware costs can be reduced.

  Further, since the scale-down process at the time of decoding is unnecessary, it is not necessary to add the ROI position information and the scale-up amount to the encoded data. Furthermore, since the ROI-encoded image by the image encoding apparatus 200 according to the present embodiment is not distinguished from the normal encoded image in terms of format, the image is decoded by the same process as the normal encoded image decoding process. And compatibility of the decoding process can be maintained.

Embodiment 2
FIG. 10 is a configuration diagram of the image decoding device 220 according to the second embodiment. In the present embodiment, the encoded image input to the image decoding device 220 is a normal encoded image that has not been subjected to ROI encoding. The image decoding apparatus 220 according to the present embodiment designates a region of interest at the time of decoding and preferentially decodes the region of interest.

  The encoded image input to the image decoding device 220 may be an encoded frame of a moving image. A moving image can be reproduced by continuously decoding each encoded frame of a moving image input as an encoded stream.

  The encoded data extraction unit 30 extracts encoded data from the input encoded image, and the entropy decoding unit 32 decodes the encoded data for each bit plane, and a quantized wavelet transform obtained as a result of decoding Store the coefficients in memory.

  The ROI selection unit 18 selects a region of interest on the image, and provides ROI position information indicating the position of the region of interest to the ROI mask generation unit 20. The attention area may be selected by specifying the position of the image, such as the center area of the image, or selected by automatically extracting an important area, such as an area in which a person or character is shown, or by the user. May be. In the case where encoded frames of moving images are continuously input to the image decoding device 220, a region of interest is designated or extracted from the decoded image frame output from the image decoding device 220, and the region of interest designated or extracted is You may select as an attention area | region of the following encoding frame.

  The ROI mask generation unit 20 generates a ROI mask for specifying the wavelet transform coefficient corresponding to the region of interest, that is, the ROI transform coefficient, based on the ROI position information. The low-order bit zero replacement unit 24 adjusts the number of low-order bits to be replaced with a zero value in the bit string of the wavelet transform coefficient corresponding to the non-target region according to the relative priority of the target region with respect to the non-target region. Yes, with reference to the ROI mask, a process of substituting a predetermined number of bits into zero from the LSB side of the non-ROI transform coefficient among the wavelet transform coefficients decoded by the entropy decoding unit 32 is performed.

  FIGS. 11A to 11C show a state in which the lower bits of the wavelet transform coefficient after decoding of the encoded image are zero-substituted by the lower bit zero substitution unit 24. FIG. 11A shows a wavelet transform coefficient 74 of an entropy-decoded image and includes a 5-bit plane. In FIG. 11B, the ROI conversion coefficient corresponding to the region of interest specified by the ROI selection unit 18 is indicated by hatching. As shown in FIG. 11C, the lower bit zero replacement unit 24 generates a wavelet transform coefficient 76 in which the lower two bits of the non-ROI transform coefficient are replaced with zero.

  The inverse quantization unit 34 inversely quantizes the wavelet transform coefficient including the ROI transform coefficient and the non-ROI transform coefficient in which the lower bits are zero-substituted, and the wavelet inverse transform unit 36 inversely transforms the inversely quantized wavelet transform coefficient. The obtained decoded image is output.

  Note that the ROI selection unit 18 may select a non-attention area instead of selecting an attention area. For example, when it is desired to blur an area where personal information such as a person's face or car license plate is shown, the area is selected as a non-attention area. In this case, the ROI mask generation unit 20 can generate a mask for specifying the ROI conversion coefficient by inverting the mask for specifying the non-ROI conversion coefficient. Alternatively, the ROI mask generation unit 20 may provide a mask for specifying non-ROI transform coefficients to the lower bit zero replacement unit 24.

  When encoded frames of moving images are continuously input to the image decoding device 220, the image decoding device 220 can be operated as follows. In order to reduce the processing load, the image decoding apparatus 220 performs simple reproduction in which the lower bit planes of the wavelet transform coefficients are appropriately discarded and reproduced. As a result, even when the processing performance of the image decoding apparatus 220 is limited, since the lower bit plane is discarded, simple reproduction can be performed at, for example, 30 frames / second.

  When the attention area on the image is selected during the simple reproduction, the image decoding apparatus 220 applies the highest value to the wavelet transform coefficient in a state where the lower bits of the non-target area are zero-replaced by the lower bit zero substitution unit 24. Decodes to lower bit plane and reproduces image. At this time, the processing load increases, so that the frame may be dropped at 15 frames / second or the like, or the slow reproduction state may be entered, but the attention area can be reproduced with high image quality.

  In this way, when the attention area is selected, only the attention area can be reproduced with higher quality while the non-attention area has the same quality as the simple reproduction. This is useful when high quality is not required at normal times and the point of interest is to be reproduced with high quality only at the time of abnormality, such as surveillance video. Also, when a moving image is played back on a mobile terminal, from the viewpoint of battery life, it is possible to use a method of playing back a moving image with low quality in the power saving mode and playing back only the attention area with high image quality as necessary.

  According to the image decoding apparatus 220 of the present embodiment, relatively low-order bits of the wavelet transform coefficient corresponding to the non-target region are replaced with zero with respect to a normal encoded image that has not been ROI encoded. The image quality of the attention area can be decoded higher than that of the non-attention area. In order to reproduce the entire image with high quality, the amount of calculation increases. However, in the present embodiment, since only the region of interest is preferentially decoded, an increase in amount of calculation can be suppressed.

Embodiment 3
FIG. 12 is a configuration diagram of an image encoding device 200 according to Embodiment 3. In the image coding apparatus 200 according to the first embodiment, only one region of interest is selected. However, the image coding apparatus 200 according to the present embodiment selects a plurality of areas having different priorities on the image, and The image is compression-encoded according to the priority of the region, and an encoded image including a plurality of regions having different image quality is generated. The same components as those in the first embodiment are denoted by the same reference numerals, and different configurations and operations from those in the first embodiment will be described.

  The ROI selection unit 18 selects a plurality of attention regions on the original image, and gives ROI position information indicating the position of each attention region to the ROI mask generation unit 20. A plurality of attention areas may overlap, and a non-attention area may be included inside the attention area. The ROI mask generation unit 20 generates a ROI mask for specifying the wavelet transform coefficient corresponding to each region of interest, that is, the ROI transform coefficient, based on the ROI position information.

  The ROI mask generation unit 20 performs the process of specifying the ROI conversion coefficient described in the first embodiment for each of the plurality of regions of interest selected by the ROI selection unit 18, and generates an ROI mask for each region of interest. .

  The ROI priority setting unit 19 sets priorities among a plurality of attention areas. For example, when the center part of the image and the periphery of the center part are selected as a plurality of attention areas, and the other outer peripheral part is set as a non-attention area, priority is given to the center part of the image being reproduced with high image quality. The degree of priority is set high, and the priority is set low so that the periphery of the center is reproduced with standard image quality. As another example, when a region where characters are reflected and a region where a person's face is reflected are selected as a plurality of attention regions, the priority is set so that the character region has the highest image quality, The priority is set next to the face area so that the image quality is high, and the other areas are set as non-attention areas so that the standard image quality is obtained. For the purpose of protecting privacy, a low priority may be set or a non-attention area may be set so that an area where a person's face is captured has low image quality.

  FIG. 13 is a diagram for explaining an example of setting priorities when a plurality of attention areas are provided in the original image 80. As shown in the figure, when two attention areas 81 and 83 are set in the original image 80, the ROI priority setting unit 19 sets, for example, a first attention area 81 (hereinafter referred to as ROI1), a second Priorities are set so that the priorities become lower in the order of the attention area 83 (hereinafter referred to as ROI2) and the other non-attention areas (hereinafter referred to as non-ROI).

  Based on the set priority, the ROI priority setting unit 19 sets the wavelet transform coefficient corresponding to the non-target region, that is, the lower-order bit number S0 to be replaced with zero in the bit string of the non-ROI transform coefficient, and each of the plurality of target regions. A lower-order bit number Si (i = 1,..., N; N is the number of regions of interest) to be zero-substituted in the corresponding wavelet transform coefficient, that is, the bit string of the ROI transform coefficient is determined.

  In the example of FIG. 13, the ROI priority setting unit 19 sets the zero replacement bit number S1 to 0 for the first priority region of interest ROI1 and the second priority when the wavelet transform coefficient of the original image is a 7-bit plane, for example. The zero replacement bit number S2 is set to 2 for the attention area ROI2 and the zero replacement bit number S0 is set to 4 for the non-attention area. That is, the lower the priority, the larger the number of zero replacement bits.

  12 refers to the ROI mask for each region of interest generated by the ROI mask generation unit 20, and counts S0 from the least significant bit in the bit string of the non-ROI transform coefficients not masked by the ROI mask. In addition to replacing only the bits with zero, only the Si bits counted from the least significant bit are replaced with zero in the bit string of the ROI transform coefficient masked by the ROI mask.

  Here, the zero permutation bit number S0 in the non-ROI transform coefficient and the zero permutation bit number Si in the ROI transform coefficient are based on the relative priority between the non-target region and the plurality of target regions by the ROI priority setting unit 19, respectively. Is an arbitrary natural number up to the number of bit planes of the wavelet transform coefficient, and satisfies S0> Si. The ROI priority setting unit 19 continuously changes the degree of deterioration of the reproduction image quality of the non-attention area relative to the attention area and the superiority or inferiority of the reproduction image quality among the plurality of attention areas by changing the number of zero substitution bits S0 and Si. Can be adjusted. The plurality of attention areas are reproduced while maintaining an image quality that allows the object to be visually recognized.

  FIGS. 14A to 14C are diagrams for explaining how the lower bits of the wavelet transform coefficient 60 of the original image are zero-substituted by the lower-bit zero substitution unit 24. FIG. FIG. 14A shows the wavelet transform coefficient 60 after quantization by the quantizing unit 12, which includes a 7-bit plane, and the ROI transform coefficient is indicated by diagonal lines. This figure illustrates a bit string of wavelet transform coefficients corresponding to pixels on the line P1-P2 in the example of the original image 80 including the two regions of interest ROI1 and ROI2 in FIG.

  As shown in FIG. 14B, the low-order bit zero replacement unit 24 replaces the S0 bit on the LSB side of the non-ROI transform coefficient not masked by the ROI mask with zero. In this example, S0 = 4, and as indicated by reference numeral 64, the 4 bits on the LSB side of the non-ROI transform coefficient are replaced with zero. Further, the lower bit zero replacement unit 24 replaces the Si bit on the LSB side of the ROI transform coefficient masked by the ROI mask with zero. In this example, two regions of interest ROI1 and ROI2 are set, and the zero substitution bit numbers S1 and S2 are S1 = 0 and S2 = 2, respectively, and as indicated by reference numeral 66, the ROI corresponding to ROI2 is set. Two bits on the LSB side of the conversion coefficient are replaced with zero. In this way, the wavelet transform coefficient 62 that has been zero-substituted by the low-order bit zero replacing unit 24 is obtained.

  The entropy coding unit 14 in FIG. 12 performs entropy while sequentially scanning the wavelet transform coefficient 62 including the ROI transform coefficient and the non-ROI transform coefficient zero-substituted from the upper bit plane, as indicated by the arrow in FIG. Encode.

  Note that, when there is no region of interest on the original image, the process of coding by substituting the lower bits of the wavelet transform coefficient with zeros as a whole is the same as in the first embodiment.

  The encoded data generation unit 16 streams the entropy-encoded data together with an encoding parameter such as a quantization width and outputs it as an encoded image. This encoded image includes a plurality of regions having different image quality at the time of reproduction, is supplied to a storage device, a network, or the like through an output unit, is decoded by an image display device including a decoding unit, and is reproduced on the screen. .

  As described in the first embodiment, when an upper limit is set for the data size of the final encoded image due to storage capacity, transmission rate limitations, and the like, the entropy encoding unit 14 performs encoding sequentially from the upper bit plane. When encoding, the encoding is stopped at the middle bit plane, or when the encoded data generation unit 16 outputs the encoded data streamed in order from the upper bit plane, the stream output is stopped at the intermediate bit plane. There is.

  In this embodiment, even when there is a restriction on the data size of the encoded image, the wavelet transform coefficients corresponding to the non-target region and the low-priority region of interest are zero-substituted in the lower bit plane. Since only the wavelet transform coefficient corresponding to the attention area with high priority is subject to encoding as significant information, the compression efficiency of the lower bit plane is high, and data is encoded even if the lowest bit plane is encoded. Size does not increase greatly.

  FIG. 15 is a diagram for explaining an example of setting priorities when there are overlaps in a plurality of attention areas provided on the original image 80. As shown in the figure, when two attention areas 81 and 83 are set in the original image 80, the ROI priority setting unit 19 sets the zero of the first attention area 81 (ROI1) as in the example of FIG. The number of replacement bits S1, the number of zero replacement bits S2 of the second region of interest 83 (ROI2), and the number of zero replacement bits S0 of the other non-region of interest (non-ROI) are set.

  The low-order bit zero replacement unit 24 replaces the S0 bit with zero from the LSB side in the non-ROI transform coefficient, and replaces the Si bit with zero from the LSB side in each ROI transform coefficient corresponding to the two regions of interest ROI1 and ROI2. However, for the region where the two regions of interest ROI1 and ROI2 overlap, the zero replacement bit number Si of the region of interest with the higher priority is preferentially applied.

  FIG. 16 is a diagram for explaining an example of setting priorities when a plurality of attention areas provided on the original image 80 are overlapped and a non-attention area is included in the attention area. As shown in the figure, when the first attention area 81 (ROI1), the second attention area 83 (ROI2), and the third attention area 85 (ROI3) are set in the original image 80, the priority decreases in this order. To do. ROI1 is in ROI2 and ROI2 is in ROI3. A part of ROI3 includes a non-attention area (non-ROI). The outside of ROI 3 is also non-ROI.

  In this case, as in the case of FIG. 15, the lower-bit zero replacement unit 24 applies the higher-priority zero replacement bit number to the overlapping region, so that ROI1 has the zero replacement bit number of ROI1. The S1 bit is preferentially applied, the ROI2 (excluding ROI1) is preferentially applied with the ROI2 zero substitution bit number S2, and the ROI3 (excluding ROI2) is preferentially applied with the ROI3 zero substitution bit number S3. . However, the non-ROI zero replacement bit number S0 is exceptionally applied to the non-ROI inside ROI3. This is because when a non-attention area is set in the attention area, there is a purpose to blur only that area even in the attention area because personal information is included in the non-attention area. .

  FIGS. 17A and 17B are diagrams for explaining how the lower bits of the wavelet transform coefficient 60 of the original image are zero-substituted by the lower-bit zero substitution unit 24 in the example of FIG. FIG. 17A shows the wavelet transform coefficient 60 after quantization by the quantization unit 12, includes a 7-bit plane, and the ROI transform coefficient is indicated by hatching. This figure illustrates a bit string of wavelet transform coefficients corresponding to pixels on the line P1-P2 in the example of the original image 80 including the three regions of interest ROI1 to ROI1 of FIG.

  FIG. 17B shows the wavelet transform coefficient 62 that has been zero-substituted by the low-order bit zero replacing unit 24. The S0 bit on the LSB side of the non-ROI transform coefficient corresponding to the non-ROI outside and inside ROI3 is replaced with zero. In this example, S0 = 6. Further, the Si bit on the LSB side of the ROI conversion coefficient corresponding to ROI 1 to 3 is replaced with zero. In this example, S1 = 0, S2 = 2, and S3 = 4.

  At the time of decoding, an image decoding apparatus having the same configuration as that of the image decoding apparatus 210 according to the first embodiment decodes an image encoded by the image encoding apparatus 200 according to the present embodiment according to a normal JPEG2000 decoding method. The decoded image includes a plurality of regions having different image quality, is supplied to an image display device or the like through an output unit, and is reproduced on the screen.

  As described above, image coding apparatus 200 according to the present embodiment performs zero substitution on non-ROI transform coefficients and also performs zero substitution on ROI transform coefficients corresponding to a plurality of regions of interest by the number of bits corresponding to the priority. Thus, encoding is performed with priority given to a plurality of attention areas. In addition, since encoding is performed with a difference in priority between a plurality of attention areas, a reproduction image quality difference can be provided between the plurality of attention areas.

Embodiment 4
FIG. 18 is a configuration diagram of the image decoding device 220 according to the fourth embodiment. In the present embodiment, the encoded image input to the image decoding device 220 is a normal encoded image that has not been subjected to ROI encoding. In the image decoding device 220 according to the second embodiment, only one region of interest is designated at the time of decoding. However, the image decoding device 220 according to the present embodiment designates a plurality of regions of interest at the time of decoding. Are decoded according to the priority. The same components as those in the second embodiment are denoted by the same reference numerals, and different configurations and operations from those in the second embodiment will be described.

  The ROI selection unit 18 selects a plurality of regions of interest on the image and gives ROI position information indicating the position of each region of interest to the ROI mask generation unit 20.

  In the case where encoded frames of moving images are continuously input to the image decoding device 220, a region of interest is designated or extracted from the decoded image frame output from the image decoding device 220, and the region of interest designated or extracted is You may select as an attention area | region of the following encoding frame. In that case, a plurality of regions that have changed with time may be selected according to the degree of change. For example, the region with the largest change, the region with the next largest change, and the other regions are selected as the first priority attention region, the second priority attention region, and the non- attention region, respectively.

  The ROI mask generation unit 20 generates a ROI mask for specifying the wavelet transform coefficient corresponding to each region of interest, that is, the ROI transform coefficient, based on the ROI position information. The ROI priority setting unit 19 sets the priority of each attention area, and determines the zero replacement bit number S0 of the non-attention area and the zero replacement bit number Si of each attention area based on the priority. The low-order bit zero replacement unit 24 refers to the ROI mask, replaces the S0 bit portion from the LSB side of the non-ROI conversion coefficient among the wavelet conversion coefficients decoded by the entropy decoding unit 32, and also converts the ROI conversion coefficient to zero. A process of replacing the Si bit portion with zero from the LSB side of is performed.

  FIGS. 19A to 19C show a state in which the lower bits of the wavelet transform coefficients after decoding of the encoded image are zero-substituted by the lower-bit zero substitution unit 24. FIG. 19A shows a wavelet transform coefficient 74 of an entropy-encoded image and includes a 7-bit plane. In FIG. 19B, ROI conversion coefficients corresponding to a plurality of regions of interest designated by the ROI selection unit 18 are indicated by diagonal lines. In this example, two attention areas ROI1 and ROI2 are provided, and it is assumed that ROI1 has a higher priority. FIG. 19C shows the wavelet transform coefficient 76 after zero substitution by the lower bit zero substitution unit 24. In this example, the lower 4 bits of the non-ROI transform coefficient are replaced with zero, the ROI transform coefficient of ROI1 is left as it is, and the lower 2 bits of the ROI transform coefficient of ROI2 are replaced with zero.

  When a plurality of attention areas on the image are selected during the simple reproduction, the image decoding apparatus 220 causes the lower bit zero replacement unit 24 to set the lower bits of the non-attention area and at least one lower bit of the plurality of attention areas to zero. The wavelet transform coefficient in the replaced state is decoded up to the lowest bit plane to reproduce the image. At this time, because the processing load becomes high, the frame may be dropped at 15 frames / second or the like, or it may be in a slow reproduction state, but a plurality of attention areas should be reproduced with high image quality according to priority. Can do.

  In this way, when a plurality of attention areas are selected, the plurality of attention areas can be reproduced with high quality according to the priority, while the non-attention areas have the same quality as the simple reproduction.

  According to the image decoding device 220 of the present embodiment, a low-order bit of a wavelet transform coefficient corresponding to a non-target region is zero-substituted for a normal encoded image that is not ROI encoded, and a plurality of target regions By substituting the low-order bits of the wavelet transform coefficient corresponding to 1 with zero according to the priority, it is possible to perform decoding with the image quality of the plurality of regions of interest being relatively higher than that of the non-region of interest according to the priority. In order to reproduce the entire image with high quality, the amount of computation increases. However, in this embodiment, since a plurality of attention areas are decoded according to priority, an increase in the amount of computation can be suppressed.

Embodiment 5
FIG. 20 is a configuration diagram of an image encoding device 300 according to Embodiment 5. In the present embodiment, the ROI conversion coefficient corresponding to a plurality of regions of interest is scaled up according to the priority. A configuration and operation different from the third embodiment will be described.

  The ROI priority setting unit 19 sets priorities of a plurality of attention regions selected by the ROI selection unit 18, and the scale-up amount Si of the ROI conversion coefficient corresponding to each of the plurality of attention regions based on the priorities. To decide. It is set so that the ROI conversion coefficient scale-up amount Si increases as the attention area has a higher priority.

  The ROI scale-up unit 22 scales up the ROI conversion coefficient corresponding to each region of interest based on the scale-up amount Si determined by the ROI priority setting unit 19.

  FIGS. 21A and 21B are diagrams for explaining how the wavelet transform coefficient 320 of the original image is scaled up by the ROI scale-up unit 22. FIG. 21A shows the wavelet transform coefficient 320 after quantization by the quantizing unit 12, includes a 7-bit plane, and the ROI transform coefficient is indicated by diagonal lines. In this example, three attention regions ROI1, ROI2, and ROI3 are provided, and the priority is assumed to decrease in this order. Based on this priority, the ROI priority setting unit 19 determines the scale-up amounts S1, S2, and S3 of ROI1, ROI2, and ROI3 as S1 = 7, S2 = 5, and S3 = 3, respectively.

  FIG. 21B shows the wavelet transform coefficient 322 scaled up by the ROI scale-up unit 22. The ROI scale-up unit 22 scales up the ROI conversion coefficient of ROI1 by 7 bits, scales up the ROI conversion coefficient of ROI2 by 5 bits, and scales up the ROI conversion coefficient of ROI3 by 3 bits. In the wavelet transform coefficient 322 after the scale-up, a zero value is assigned to the digit newly generated by the scale-up. As a result, a 14-bit plane wavelet transform coefficient 322 is obtained as a whole.

  FIG. 22 is a configuration diagram of the image decoding apparatus 310 according to Embodiment 5. The image decoding apparatus 310 according to the present embodiment decodes an image that has been ROI encoded by the image encoding apparatus 300.

  The encoded data extraction unit 30 extracts encoded data, various encoding parameters, ROI position information, and a scale-up amount from the input encoded image. The entropy decoding unit 32 decodes the encoded data for each bit plane and stores the obtained quantized value of the wavelet transform coefficient in the memory.

  FIG. 23A shows the wavelet transform coefficient 324 after entropy decoding. Corresponding to the example of FIG. 21B, the wavelet transform coefficient 324 after entropy decoding includes a 14-bit plane as a whole. The ROI scale-down unit 40 refers to the ROI mask of each region of interest, and based on the scale-up amount Si of each region of interest extracted by the encoded data extraction unit 30, the ROI conversion coefficient of each region of interest is expressed by only Si bits. Scale down. In this example, the scale-up amounts of the attention regions ROI1, ROI2, and ROI3 are S1 = 7, S2 = 5, and S3 = 3, respectively, and the ROI conversion coefficients of ROI1, ROI2, and ROI3 are as shown in FIG. Each of them is scaled down by 7 bits, 5 bits, and 3 bits, and the digits left by the scale down are discarded, and a quantized wavelet transform coefficient 326 of a 7-bit plane is obtained as a whole.

Embodiment 6
FIG. 24 is a configuration diagram of an image display apparatus 400 according to the sixth embodiment. The image display device 400 displays a moving image on a display or the like. As an example, the image display device 400 is a display control unit such as a DVD (digital video disk) player, a television receiver, or a surveillance camera.

  The encoding block 420 encodes the input original image and stores the encoded image in the storage unit 440. The original image input to the encoding block 420 is a moving image frame, and the moving image frame is continuously encoded and stored in the storage unit 440.

  The encoding block 420 has the configuration of the image encoding device 200 according to any one of the first, third, and fifth embodiments, receives information on the attention area set on the screen from the area selection unit 450, and prioritizes the attention area. The original image is compression-encoded according to the degree, and encoded images with different image quality of the attention area are generated.

  The decoding block 430 reads the encoded image from the storage unit 440 from the storage unit 440, decodes it, and provides it to the display unit 410. The encoded image read from the storage unit 440 is a moving image encoded frame, and the encoded frame is continuously decoded and provided to the display unit 410. The decoding block 430 may acquire an encoded image via a communication interface of a wireless or wired network, or may acquire an encoded image via a reception block that receives broadcast radio waves.

  The decoding block 430 has the configuration of any one of the image decoding devices 220 according to Embodiments 1 to 5, and decodes the encoded image stored in the storage unit 440. Here, when the attention area is already selected at the encoding stage and the image quality of the attention area is adjusted, the decoding is performed as it is. However, when the attention area is not selected at the encoding stage, the decoding block is selected. Reference numeral 430 receives information on the attention area set on the screen from the area selection unit 450, preferentially decodes the attention area, and generates decoded images with different image quality of the attention area.

  The display unit 410 receives the decoded image from the decoding block 430 and outputs it to the display. The image frames decoded by the decoding block 430 are continuously displayed on the screen of the display, and a moving image is reproduced.

  The region selection unit 450 selects a region of interest on the display screen. The attention area may be freely specified by the user, or may be set in advance at the center or top of the screen. The area selection unit 450 may be provided with a designation unit for the user to designate an attention area on the screen. For example, an area is specified using a pointing device such as a mouse, or an area is specified using a touch-type display device such as a touch panel. The area selecting unit 450 receives attention area designation information from the user, or receives attention area designation information set in advance from the display unit 410, and encodes the position information of the attention area into an encoding block 420 and a decoding block. 430.

  The position information of the attention area supplied from the area selection section 450 to the encoding block 420 and the decoding block 430 is given to the ROI selection section 18 of each block, and thereafter, the attention area is preferentially encoded in each block. Or it is decrypted. Note that a non-attention area may be selected instead of the attention area by the area selection unit 450. In this case, in the encoding block 420 and the decoding block 430, areas other than the non-attention area are preferentially used as the attention area. Encoded or decoded.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. . Such a modification is shown below.

  In the above embodiment, the lower bits of the non-ROI transform coefficients are merely replaced with zero, and the ROI transform coefficients are not scaled up at all. However, the ROI transform coefficients are scaled up and the lower bits of the non-ROI transform coefficients are zeroed. Combinations of substitutions may be performed.

  In any of the embodiments, the wavelet transform has been described as spatial filtering for image coding, but other spatial frequency transforms may be used. For example, even in the case of the discrete cosine transform used in the JPEG standard, the compression efficiency of the entire image is increased by sacrificing the image quality of the non-target region by replacing the lower bits of the transform coefficient of the non-target region with zero in the same manner. At the same time, the image quality of the attention area can be relatively improved.

  By incorporating the image encoding device and the image decoding device of the above-described embodiment into the signal processing unit of the surveillance camera, the region of interest to be monitored can be preferentially encoded or decoded. The image data captured and encoded by the surveillance camera may be transmitted via the network. In this case, the image decoding device connected to the network decodes and reproduces the encoded image data received from the network.

  In the above description, the image is divided into a non-attention area and a plurality of attention areas, but the image may be divided into a plurality of areas according to priority without distinguishing between the attention area and the non-attention area. In the above embodiment, since priority is given to a non-attention area and a plurality of attention areas, it can be considered that there is substantially only a difference in priority between the non-attention area and the attention area. The same processing can be performed even when the non-attention area and the attention area are not distinguished and the areas are divided by priority.

It is a block diagram of the conventional image coding apparatus. It is a figure explaining a mode that the wavelet transformation coefficient of an original image is scaled up. It is a block diagram of the conventional image decoding apparatus. It is a figure explaining a mode that the wavelet transform coefficient after decoding of an encoding image is scaled down. 1 is a configuration diagram of an image encoding device according to Embodiment 1. FIG. It is a figure explaining the mask for specifying the wavelet transform coefficient corresponding to the attention area | region of an original image. It is a figure explaining a mode that the low-order bit of the wavelet transform coefficient of an original image is zero-substituted. It is a figure explaining a mode that the low-order bit of a wavelet transform coefficient is zero-substituted when an attention area does not exist on an original image. 1 is a configuration diagram of an image decoding device according to Embodiment 1. FIG. 6 is a configuration diagram of an image decoding device according to Embodiment 2. FIG. It is a figure explaining a mode that the low-order bit of the wavelet transform coefficient after decoding of an encoding image is zero-substituted. FIG. 6 is a configuration diagram of an image encoding device according to Embodiment 3. It is a figure explaining the example of a setting of the priority in case the some attention area is provided in the original image. It is a figure explaining a mode that the low-order bit of the wavelet transform coefficient of an original image is zero-substituted. It is a figure explaining the example of a setting of the priority when there exists duplication in the some attention area provided on the original image. It is a figure explaining the example of a setting of the priority in case the some attention area provided on the original image has overlap, and a non-attention area is included in an attention area. In the example of FIG. 16, it is a figure explaining a mode that the low-order bit of the wavelet transform coefficient of an original image is substituted by zero. FIG. 10 is a configuration diagram of an image decoding device according to a fourth embodiment. It is a figure explaining a mode that the low-order bit of the wavelet transform coefficient after decoding of an encoding image is zero-substituted. FIG. 10 is a configuration diagram of an image encoding device according to a fifth embodiment. It is a figure explaining a mode that the wavelet transformation coefficient of an original image is scaled up. FIG. 10 is a configuration diagram of an image decoding device according to a fifth embodiment. It is a figure explaining a mode that the wavelet transform coefficient after decoding of an encoding image is scaled down. FIG. 10 is a configuration diagram of an image display device according to a sixth embodiment.

Explanation of symbols

  10 wavelet transform unit, 12 quantization unit, 14 entropy coding unit, 16 encoded data generation unit, 18 ROI selection unit, 19 ROI priority setting unit, 20 ROI mask generation unit, 22 ROI scale-up unit, 24 lower bits Zero substitution unit, 30 encoded data extraction unit, 32 entropy decoding unit, 34 inverse quantization unit, 36 wavelet inverse transformation unit, 38 ROI mask generation unit, 40 ROI scale down unit, 200, 300 image encoding device, 210, 220, 310 Image decoding apparatus.

Claims (16)

  An image encoding method, wherein information for specifying a region of interest defined on a moving image is explicitly described in an encoded stream including encoded data of the moving image.   The image coding according to claim 1, wherein when a plurality of attention areas are defined on the moving image, the information for specifying the attention area includes a priority of the attention area. Method.   The image encoding method according to claim 1, wherein the image quality of a region of interest defined on the moving image is encoded differently from other regions. When a plurality of attention areas are defined on the moving image, the information for specifying the attention area includes the priority of the attention area,
The image encoding method according to claim 3, wherein encoding is performed by changing the image quality of the plurality of regions of interest according to priority.   An image encoding method, wherein information for specifying a plurality of regions of interest defined on an image is explicitly described in an encoded stream including encoded data of the image.   6. The image encoding method according to claim 5, wherein the information includes a priority of each attention area.   6. The image encoding method according to claim 5, wherein encoding is performed with different image quality of the plurality of regions of interest.   The image encoding method according to claim 6, wherein encoding is performed by changing the image quality of the plurality of regions of interest according to priority. A selection unit for selecting a region of interest on the moving image;
An encoding unit for encoding the moving image;
A generating unit that generates an encoded stream including an encoded moving image and information for specifying the region of interest;
An image encoding device comprising:   The image coding apparatus according to claim 9, wherein when a plurality of attention areas are defined on the moving image, the priority is included in the information for specifying the attention area.   The image encoding device according to claim 9, wherein the encoding unit encodes the image quality of a region of interest selected on the moving image different from that of other regions. When a plurality of attention areas are defined on the moving image, the information for specifying the attention area includes the priority of the attention area,
The image encoding device according to claim 11, wherein the encoding unit performs encoding by changing the image quality of the plurality of regions of interest according to priority. A selection unit for selecting a plurality of attention areas on the image;
An encoding unit for encoding the image;
A generating unit that generates an encoded stream including an encoded image and information for specifying the plurality of regions of interest;
An image encoding device comprising:   The image encoding device according to claim 13, wherein the encoded stream generation unit generates an encoded stream including the priority of each region of interest.   The image encoding device according to claim 13, wherein the encoding unit encodes the plurality of regions of interest with different image quality.   The image encoding device according to claim 14, wherein the encoding unit performs encoding by changing the image quality of the plurality of regions of interest according to priority.

Images