JPH10149431A - Picture processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate processing speed by deciding the setting of a color area in chromakey synthesis through the use of a color coordinate distance in a color coordinate space by means of two color difference signals and lightness. SOLUTION: A color having chromaticity and lightness in a first color area in the prescribed color coordinate is set to be a first key color for chromakey synthesis, and a boundary between a first picture part and a second picture part except for the part is corrected. The first picture part is substituted for the corresponding different picture. A third picture part where the color having chromaticity and lightness in the second color area other than the first color area is the second key color is mixed with the corresponding different picture at a mixing ratio corresponding to the color coordinate distance from the first color area. Then, the third picture part is substituted so as to obtain a synthesized picture. Thus, a picture processing which reduces aliasing at the time of chromakey synthesis superior in operability and high in processing speed can be obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for synthesizing an image and an image, and more particularly to an image processing apparatus for eliminating an aliasing phenomenon occurring at a boundary of a synthesized image and for controlling a gradation of the boundary. The present invention relates to an image processing method for making the image processing more natural.

[0002]

2. Description of the Related Art FIG. 3 shows an example of a block configuration of an image processing apparatus for performing image processing such as image synthesis. In FIG. 3, reference numeral 1 denotes a computer that executes image processing, and includes a main storage device and an input / output device such as a keyboard (not shown). A monitor 2 connected to the computer 1 displays various controls, displays images before image processing, images processed, and the like. Reference numeral 3 denotes an auxiliary storage device connected to the computer 1 for storing a large amount of data such as image data and transmitting and receiving the data to and from the computer 1.

In general, when performing image processing for synthesizing an image and another image, a synthesis method called chroma key synthesis is often used. The chroma key combination means, for example, that the computer 1 reads the image 100 shown in FIG. 11 stored in the auxiliary storage device 3 from the auxiliary storage device 3. The computer 1 sets the color or color area as a key color, using an image in which a black background portion of the image 100, that is, a portion to be replaced with a corresponding portion of another image is a predetermined substantially single color. Then, when a key color image portion is detected, the detected key color portion is combined so as to replace another corresponding image 101 that is also read, and the combined image 102 is The generated image is written in the auxiliary storage device 3, and the auxiliary storage device 3 stores the written image 102.

Here, a basic operation procedure of chroma key synthesis will be described with reference to FIG. In FIG. 6, in step 601, a color area range of a predetermined key color and a color area range of a soft key color in color coordinates are set. In step 602, one frame of the image to be combined is read from the auxiliary storage device. And
In step 603, the images are chromakey-combined. In step 604, the combined image is output to the auxiliary storage device 3 and stored.

At the time of this synthesis, for example, FIG. 10 is an enlarged view of a part of the image 100 displayed on the monitor 2.
As shown in the figure, when a figure having a border of a relatively simple linear shape such as a character is combined with a jagged figure, if the combining is performed by simply replacing the image as it is, the combined As shown in FIG. 13, the jagged figure of the boundary line (jaggy) remains in the image.

Therefore, as one of the prior arts, before synthesizing the image 100 of FIG. 11, the computer 1 performs an anti-aliasing process as an image process for making the jaggedness inconspicuous, and performs the anti-aliasing process. There is an image in which aliasing is reduced by obtaining a composite image by performing image replacement on an image subjected to.

Here, the anti-aliasing processing is, for example, as shown in FIG. 12 (a), when a figure which is generated when there is a slanted line figure and has a conspicuous jagged line is formed at a contour boundary portion. This is a process of making the jaggedness inconspicuous as if an image portion having an intermediate color between the graphic color and the background color is newly generated with a predetermined arrangement.

However, when the anti-aliasing process is performed, an intermediate color portion mixed with a color component of a background color, which is a key color of a chroma key, appears in a synthesized image, and therefore, as shown in FIG. The color component of the color is displayed as noise.

For this reason, when using an image subjected to the anti-aliasing processing as described above, the computer 1 uses a method called a soft key in the vicinity of a key color color area (first color area) at a boundary portion. In another color area, that is, a soft key color area (second color area), the image is mixed with the image by changing the mixing ratio adaptively according to the degree of color change from the key color. Some methods use a method that replaces the image of the boundary portion with the mixed image so that the boundary portion becomes a more natural image.

In the method using this soft key, conventionally, (1) to (6) shown in FIGS.
Any of the above-described six key color color area (first color area) and soft key color color area (second color area) determination methods have been used.

That is, (1) The mixing ratio k of the blue-back image is determined according to the degree d of blue. (2) A mixing ratio is determined according to the degree of hue. (3) The mixture ratio is determined according to the degree of saturation. (4) A mixing ratio is determined according to the degree of brightness (luminance). (5) The mixing ratio is determined according to the hue and the degree of lightness. (6) A mixing ratio is determined according to the degree of hue, saturation, and lightness. The mixing ratio k of the blue-back image is set to 0 in the darkly filled portion (or the solid region in the solid line) in the figure, and the mixture ratio k in the lightly filled portion (or the solid region in the dotted line and the solid region in the solid line) is set. In the (enclosed portion), the mixture ratio k is changed from 0 to 1 according to the color coordinate distance from the first color area.

In this manner, in the soft key color region, the images are mixed so as to reduce the color component of the background color which does not originally exist, so that a visually appealing composite image with less noticeable noise is obtained. be able to.

[0013]

In the above-mentioned prior art,
When combining jagged figures with boundary lines, if the image is synthesized by simply replacing the image as it is,
As shown in FIG. 13, a jagged figure (jaggy) of the boundary line remains as it is, and the appearance is not beautiful.

[0014] Further, if the anti-aliasing process makes the jaggies of the outline inconspicuous, but if the image is synthesized only by replacing the image, the intermediate color portion mixed with the color component of the background color which is the key color of the chroma key is obtained after the synthesis. 14, the color component of the background color which does not exist originally is displayed as noise as shown in FIG.

Further, using the image subjected to the anti-aliasing processing, the above-described six key color areas (first color area) and soft key color areas (1) to (6) are used. The method using the soft keys according to any one of the methods for determining the (second color area) has the following disadvantages.

That is, in (1) to (4), the processing time involved in the image synthesizing process is relatively short, and the number of parameters set by the operator can be relatively small, such as two to three. Although it is advantageous, the color area of the key color (first color area) and the color area of the soft key color (second color area) cannot be set to arbitrary sizes and positions in the color coordinates. There is a disadvantage that the usable background color or the combination of the background color and the color other than the background is limited.

Further, in (5) and (6), although the restriction on the background color or the combination of the background color and the color other than the background is relatively small, the number of setting parameters set by the operator is advantageous. However, there are drawbacks in that the operability is poor due to the relatively large number of images and the processing time involved in the image synthesis processing is relatively long.

Further, when the image is not subjected to the anti-aliasing process, the method using the soft keys does not provide the effect of making the jagged figure look smoother.

A first object of the present invention is to solve the above-mentioned problems by using a method using soft keys, in which the processing speed is relatively low, the number of parameter settings is small, and those parameters can be easily set. An object of the present invention is to provide a method that can be set and that can further reduce unnecessary color components in order to make a gradation of a boundary portion look more natural.

Further, a second object of the present invention is to solve the above-mentioned problem, when an image not subjected to anti-aliasing processing is synthesized, without using a method using soft keys. By detecting the boundary portion between the image portion where the image is replaced and the image portion where the image is not replaced, each boundary portion of the image to be synthesized is efficiently filtered, and the aliasing can be significantly reduced. It was done.

[0021]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an image processing method used in an image processing apparatus for generating at least two images by chroma key synthesis with computer assistance to generate a synthesized image. A chromaticity in a first color area at a predetermined color coordinate or a color having chromaticity and lightness (luminance) is set as a first key color of chroma key synthesis, and a first key color in any one of the above images is selected. In order to make the aliasing phenomenon occurring at the boundary between the first image portion, which is the key color, and the second image portion excluding the first image portion from the image less noticeable, the images at the boundary portions are respectively Correcting to a predetermined color, replacing the first image portion in the image with a corresponding another image, and converting the color region including the first color region at the predetermined color coordinates to the first color region Excluding The chromaticity in the second color area or a color having chromaticity and lightness is set as a second key color for chroma key synthesis, and a third image portion, which is a second key color in the image, is used as the second key color. The third image portion is mixed with the corresponding other images at a mixing ratio corresponding to the color coordinate distance from the first color region at the predetermined color coordinates of the color of the third image portion. , A composite image is obtained by performing image replacement with the corresponding corresponding mixed image.

According to an embodiment of the present invention, a color having a chromaticity within a circular area having a first radius centered on a predetermined coordinate on a color coordinate plane including two color difference signal axes is used for chroma key synthesis. A boundary between a first image portion, which is a first key color in any one of the images, and a second image portion obtained by removing the first image portion from the image; In order to make the aliasing phenomenon that occurs in
The image of the boundary portion is corrected, and the first image in the image is corrected.
Is replaced with a corresponding different image, respectively, and the area within the circle obtained by removing the circle area of the first radius from the circle area of the second radius larger than the first radius with the predetermined coordinates as the center. A color having a chromaticity of 2 as a second key color for chroma key synthesis, and a different ratio of the image portion having the second key color from the corresponding another image with a mixing ratio corresponding to a color coordinate distance from the predetermined coordinates. Mixing the third image portion, which is the second key color in the image, with a mixing ratio according to the distance of the color of the third image portion from the predetermined coordinates in the predetermined color coordinates, The third image portion is mixed with the corresponding other image, and the third image portion is replaced with the mixed corresponding image.

Also, as an embodiment of the present invention, the chromaticity and lightness in a sphere region of a first radius centered on a predetermined coordinate in a color coordinate space composed of two color difference signal axes and a lightness axis are provided. The color is a first key color of chroma key synthesis, and a first image portion that is a first key color in any of the images, and a second image portion obtained by removing the first image portion from the image. In order to make the aliasing phenomenon occurring at the boundary with the image part inconspicuous, each image at the boundary is corrected, and the first image part in the image is replaced with another corresponding image. With the predetermined coordinates as the center, the first
A color having chromaticity and lightness in an area excluding the sphere area of the first radius from the sphere area of the second radius larger than the radius of the second key color is used as a second key color for chroma keying. Is mixed with the corresponding another image at a mixing ratio according to a color coordinate distance from the predetermined coordinate of the image portion having the third image portion, which is a second key color in the image. 3
Are mixed with the corresponding other images at a mixing ratio according to the distance from the predetermined coordinates in the predetermined color coordinates of the color of the image portion, and the third image portion is mixed with the mixed image. The image is replaced with a new image.

According to the present invention, in order to achieve the above object, a chromaticity in a first color area at a predetermined color coordinate or a color having chromaticity and lightness is set as a key color of chroma key combination, The mask data corresponding to the first image portion which is the key color in any one of the images.
In the case of the image portion, the value of the mask data becomes a first value corresponding to the image having the maximum value, and in the case of the second image portion obtained by removing the first image portion from the image, the value becomes the image of the minimum value Corresponding to a second value, generating an image in which the second image portion is replaced with a minimum value image while leaving the first image portion in the image without replacing the image, and generating the mask In order to detect a boundary portion between the image of the first value and the image of the second value in the data, respectively, and to make an aliasing phenomenon occurring at the detected boundary portion in the replaced image inconspicuous. , Correcting the image of the boundary portion to a predetermined value,
In order to make the aliasing phenomenon occurring at the detected boundary portion in the set masking data inconspicuous, the image of the boundary portion is corrected in a predetermined manner, and the corrected masking data is converted from the first value. Each of the subtraction generates inverted masking data, multiplies the inverted masking data by another corresponding image and generates a normalized image, and generates an image obtained by multiplying and normalizing the image. And a predetermined filtered and corrected image are added to obtain a composite image.

Also, as an embodiment of the present invention, a color having a chromaticity within a circular area of a predetermined radius centered on a predetermined coordinate on a color coordinate plane composed of two color difference signal axes is used as a key color for chroma key synthesis. It is assumed that.

Also, as an embodiment of the present invention, a color having chromaticity and lightness in a sphere region of a predetermined radius centered on a predetermined coordinate in a color coordinate space consisting of two color difference signal axes and a lightness axis. Is used as a key color for chroma key synthesis.

According to another aspect of the present invention, a color having a chromaticity within a circular area having a predetermined radius centered on a predetermined coordinate on a color coordinate plane including two color difference signal axes is chromakey-combined. , And the image portion which is the key color is replaced with another corresponding image.

In order to achieve the above object, the present invention provides a chromaticity and lightness value in a sphere region having a predetermined radius centered on a predetermined coordinate in a color coordinate space comprising two color difference signal axes and a lightness signal axis. Is a key color for chroma key synthesis, and the image portion that is the key color is image-replaced with another corresponding image.

[0029]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image processing method according to the present invention will be described with reference to the drawings. Note that an example of a block configuration of the image processing apparatus according to the present invention is the same as that of FIG. 3 described in the related art, and a description thereof will be omitted.

First, as a first embodiment of the image processing method of the present invention, an image processing method using a soft key which is effective when using an image subjected to anti-aliasing processing and which is improved from the conventional technique is used. Will be described.

FIG. 7 is a flowchart showing the procedure of chroma key synthesis using soft keys. The procedure from step 701 to step 711 of this flowchart is as follows:
This corresponds to step 603 in FIG. In FIG. 7, in step 702, a key color area and a soft key color area are set. The present invention provides a new method for this setting method, and the two new methods will be described below with reference to FIGS.

FIG. 2 shows a method of determining the setting of a color area by using the color coordinate distance (plane distance) on two color difference signal planes. FIG. 2 (5) of FIG. The “method of determining based on hue and saturation” is such that parameter setting is simplified. According to the method of FIG. 2, since the number of parameters set by the operator is at most two, it is possible to provide a very easy setting method, and the processing time involved in the image processing is relatively short. Shorter.

FIG. 1 shows a method of determining the setting of a color area using a color coordinate distance (space distance) in a color coordinate space based on two color difference signals and lightness (luminance). )
In, the hue, saturation and brightness have the same area setting freedom as the color area determined by three,
Therefore, according to this method, it is possible to arbitrarily select a key color to be used as a background color, and to greatly reduce restrictions on a combination of a background color and a color other than the background. Also, the number of parameter settings is only three, and in (6) of FIG.
Operability is greatly improved. Further, the processing time related to the image processing can be relatively short, and is substantially equal to the processing time in (5) of FIG. 16 described above.

The procedure after step 703 in FIG. 7 is the same as that used in the conventional technique. Briefly, in step 703, the color of the pixel of the image (A) having the key color as the background color is set. Color coordinates are calculated. Step 7
At 04, it is determined whether the calculated coordinates are within the key color area. If it is within the key color area, the image is replaced with another corresponding image (B) in step 705. If it is not within the key color area, the further calculated coordinates are used in step 706.
It is determined whether or not it is within the soft key color area. If the image is within the soft key color area, a mixture ratio corresponding to the color coordinate distance is obtained in step 707, and then another image (B) corresponding to the image (A) is obtained using the mixture ratio obtained in step 709. Are mixed. Then, the image is replaced with the mixed image. If it is not within the soft key color area, the image (A) is left at step 708 and the image is not replaced. In step 710 following steps 705, 708, and 709, it is determined whether all the pixels in one frame have been processed. If all the pixels have been processed, the process proceeds to step 711. If not, the process proceeds to step 703. Return.

As described above, by performing chroma key synthesis using soft keys, as shown in FIG. 4 showing a display example of a synthesized image, jaggies are improved and become inconspicuous, and background colors are reduced. A high-quality composite image with a reduced color component display amount can be obtained with a simple operation and a fast processing time.

Next, an image processing method according to the present invention which is used when an image having a key color as a background color and has not been subjected to anti-aliasing processing will be described with reference to FIGS. 8 and 9.
This will be described with reference to the flowchart of FIG. 8 and 9
9 is a flowchart illustrating an example of a procedure at the time of chroma key synthesis using mask filtering according to the present invention. Here, the mask filtering is a technique for performing filtering correction for smoothing a boundary portion of mask data generated based on a key color background image, that is, anti-aliasing.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS. Step 8 of this flowchart
01 to step 821 through step 813 (C).
The procedure up to is equivalent to step 603 in FIG. In FIG. 8, step 802 is the same as step 702 in FIG. Step 803 to Step 80
6 generates mask data. Steps 807 to 808 convert the signal of the background color portion of the image (A) having the key color as the background color into the minimum value. Step 809
Steps 812 to 812 detect a boundary portion of the mask data.

In FIG. 9, steps 814 to 817 perform filtering correction on the image (A) and the mask data, respectively. Steps 818 to 820 generate a composite image by inverting the mask data and adding the result obtained by multiplying the image (B) and the result obtained by correcting the image (A).

As described above, aliasing can be significantly reduced by adding the corrected values.

[0040]

According to the present invention, it is possible to provide an image processing method which is excellent in operability and has a high processing speed, while reducing aliasing in chroma key synthesis.

[Brief description of the drawings]

FIG. 1 is a diagram schematically illustrating a setting state of a key color area and a soft key color area for performing chroma key synthesis using soft keys according to the present invention.

FIG. 2 is a diagram schematically illustrating a setting state of a key color area and a soft key color area for performing chroma key synthesis using soft keys according to the present invention.

FIG. 3 is a diagram showing an example of a block configuration of an image processing apparatus according to the present invention.

FIG. 4 is a diagram showing a display example of a combined image at the time of chroma key combination using soft keys according to the present invention.

FIG. 5 is a diagram showing a display example of a synthesized image at the time of chroma key synthesis using mask filtering according to the present invention.

FIG. 6 is a flowchart illustrating an example of a chroma key synthesis procedure according to the present invention.

FIG. 7 is a flowchart showing an example of a procedure for synthesizing a chroma key per frame using a soft key according to the present invention.

FIG.

FIG. 9 is a flowchart illustrating an example of a procedure at the time of chroma key synthesis using mask filtering according to the present invention.

FIG. 10 is a diagram schematically illustrating a display example of an image in which jaggies have occurred.

FIG. 11 is a view for explaining how a combined image is generated by chroma-key combining two images.

FIG. 12 is a view for explaining image display states before and after anti-aliasing processing.

FIG. 13 is a diagram showing a display example of a combined image at the time of chroma key combination by pixel replacement according to the related art.

FIG. 14 is a view showing a display example of a combined image at the time of chroma key combination by pixel replacement according to the related art.

FIG.

FIG. 16 is a diagram schematically illustrating a setting state of a key color area and a soft key color area for performing chroma key synthesis using soft keys according to a conventional technique.

[Explanation of symbols]

1 computer, 2 monitor, 3 auxiliary storage device.

Claims (8)

[Claims] 1. An image processing method for use in an image processing apparatus for generating a synthesized image by combining at least two images by chroma key synthesis with the aid of a computer, wherein the chromaticity in a first color area at a predetermined color coordinate or A color having chromaticity and lightness (luminance) is defined as a first key color of the chroma key synthesis, and a first image portion that is a first key color in any one of the images and a first key color from the image. In order to make the aliasing phenomenon occurring at the boundary portion between the second image portion and the second image portion less noticeable, the images at the boundary portions are each corrected in a predetermined manner, and the first image portion in the image is Chromaticity or chromaticity in a second color area obtained by excluding the first color area from the color area including the first color area at the predetermined color coordinates by replacing each image with a corresponding different image. And have lightness The color is a second key color of chroma key combination, and a third image portion, which is a second key color in the image, is the first color at the predetermined color coordinates of the color of the third image portion. The combined image is mixed with the corresponding other image with a mixing ratio corresponding to the color coordinate distance from the region, and the third image portion is replaced with the mixed corresponding image, respectively, to form a composite image. An image processing method characterized by obtaining. 2. The image processing method according to claim 1, wherein a color having a chromaticity within a circular region having a first radius centered on a predetermined coordinate on a color coordinate plane including two color difference signal axes is chroma-keyed. A first image portion which is a first key color of the combination and is a first key color in any one of the images, and a second image portion obtained by removing the first image portion from the image; In order to make the aliasing phenomenon occurring at the boundary portion of the image inconspicuous, the image at the boundary portion is corrected, the first image portion in the image is replaced with a corresponding image, and the predetermined coordinates are replaced. A color having a chromaticity within an area obtained by excluding the circular area having the first radius from the circular area having the second radius larger than the first radius is defined as a second key color of chroma key synthesis, and A color coordinate of the image portion having the key color of 2 from the predetermined coordinates The third image portion, which is a second key color in the image, is mixed with the corresponding another image at a mixing ratio corresponding to the target distance, and the predetermined color of the color of the third image portion is used. Mixing with the corresponding another image with a mixing ratio according to the distance from the predetermined coordinate in the coordinates, and performing image replacement of the third image portion with the mixed corresponding image, respectively. Image processing method. 3. The image processing method according to claim 1, wherein the chromaticity and the chromaticity within a spherical area of a first radius centered on a predetermined coordinate in a color coordinate space including two color difference signal axes and a lightness axis are set. A color having lightness is defined as a first key color of chroma key synthesis, and a first image portion which is a first key color in any of the images and a first image portion is removed from the image. In order to make the aliasing phenomenon occurring at the boundary with the second image part inconspicuous, the images at the boundary part are respectively corrected, and the first image part in the image is converted into a corresponding different image. The color having the chromaticity and lightness in an area excluding the sphere area of the first radius from the sphere area of the second radius larger than the first radius with the predetermined coordinates as the center is substituted for the color of chroma key synthesis. The second key color and the second
The image portion having the key color is mixed with another corresponding image at a mixing ratio corresponding to the color coordinate distance from the predetermined coordinate, and the third image portion as the second key color in the image is mixed. Mixing the third image portion with the corresponding other image at a mixing ratio corresponding to the distance from the predetermined coordinate in the predetermined color coordinate of the color of the third image portion; An image processing method, wherein image replacement is performed for each of corresponding mixed images. 4. An image processing method for use in an image processing apparatus for generating a synthesized image by synthesizing at least two images by chroma key synthesis with computer assistance, comprising: a chromaticity in a first color area at a predetermined color coordinate; A color having chromaticity and lightness is defined as a key color of chroma key combination, and a matrix position of each pixel of the mask data is determined according to a first image portion which is the key color in any of the images. In the case of corresponding to the first image portion, the value of the pixel of the mask data is set to a first value corresponding to the image having the maximum value, and the value of the second image portion obtained by removing the first image portion from the image is set. In this case, the second value is set to a second value corresponding to the image having the minimum value, and the first image portion is replaced with the image having the minimum value, while the second image portion in the image is not replaced. Generate images Detecting a boundary portion between the image of the first value and the image of the second value in the mask data, and observing an aliasing phenomenon occurring at the detected boundary portion in the replaced image. In order to reduce the aliasing phenomenon that occurs at the detected boundary portion in the set masking data, the boundary portion of the mask data is corrected to a predetermined value. Generating the inverted masking data by subtracting the corrected masking data from the first value, generating an image obtained by normalizing the inverted masking data and multiplying the inverted masking data by another corresponding image, An image processing unit for obtaining a composite image by adding the image generated by multiplying and normalizing the image and the image corrected in a predetermined manner. Method. 5. The image processing method according to claim 4, wherein a color having a chromaticity within a circular region having a predetermined radius around a predetermined coordinate on a color coordinate plane including two color difference signal axes is chromakey-combined. An image processing method characterized by using the following key colors. 6. The image processing method according to claim 4, wherein the chromaticity and lightness in a sphere region having a predetermined radius centered on a predetermined coordinate in a color coordinate space including two color difference signal axes and a lightness axis. An image processing method characterized in that a color having the following is used as a key color for chroma key synthesis. 7. An image processing method for use in an image processing apparatus for generating a synthesized image by combining at least two images by chroma key synthesis with the assistance of a computer, wherein a predetermined coordinate on a color coordinate plane including two color difference signal axes is centered. An image processing method characterized in that a color having chromaticity within a circular area having a predetermined radius is used as a key color for chroma key synthesis, and an image portion that is the key color is replaced with another corresponding image. 8. An image processing method for use in an image processing apparatus for generating a composite image by synthesizing at least two images by chroma key synthesis with the assistance of a computer, wherein the color coordinate space includes two color difference signal axes and a lightness signal axis. Image processing characterized in that chromaticity and lightness within a predetermined radius sphere region centered on predetermined coordinates are used as key colors for chroma key synthesis, and the image portion which is the key color is replaced with another corresponding image. Method.