JP2003248825A - Image processing apparatus, image processing method, image processing program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method for image processing which can perform more precise image correction. <P>SOLUTION: A body/area recognition part 1 recognizes objects in an inputted image and a main/sub object determination part 2 determines a main object and a sub object among the recognized objects. According to the relation between the determined main object and sub object, a correcting method determination part 3 determines a correcting method for correcting a target object by using feature quantity of the object. A correction quantity calculation part 4 determines a correction quantity for the correction and a correction execution part 5 actually performs determined correction processing for the respective objects or the whole image. Consequently, different optimum corrections can be made for an image of a 'person' according to the scene, e.g. whether the background is the sea or mountains and the sky. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image taken by a digital camera or the like, a digital image obtained by inputting a negative film or the like with a film scanner or the like, a digital image obtained by inputting a photograph or the like with a reflection type scanner, a PDL (Page).
The present invention relates to a technique for determining a correction method to be performed in order to preferably reproduce an image described in Description Language (page description language). The present invention also relates to a technique for determining classification information for classifying images using the same technique.

[0002]

2. Description of the Related Art When a general digital camera or the like is used for shooting, the photometric system provided in the digital camera automatically sets shooting conditions such as exposure conditions when shooting. In many cases, shooting is done according to the conditions. The automatically set shooting conditions are based on the assumption that the subject is a very average condition.
A desired image may not be obtained under a condition different from the average condition such as the illumination condition at the time of shooting, the degree of light hitting the subject, and the backlight condition. For example, under the backlight condition, the main scene (for example, the face of a person) may appear dark because the entire scene is bright. Also, the difference in the light source that illuminates the subject (sunlight, fluorescent light, etc.) and the conditions of the sun outdoors (sunset, cloudy, etc.)
As a result, there may be a problem that the color balance is remarkably shifted and reflected. Of course, the same applies to film cameras, etc., and when shooting under the automatically set shooting conditions, similar problems occur in images on film, and in digital images read by film scanners etc. Is also the same.

For such a problem, it is possible to obtain a desired image by correcting the digital image. However, it is very difficult for a general unfamiliar user to adjust the parameters and perform the correction process while selecting various correction processes. Therefore, it is considered to automatically correct the image.

In order to automatically perform image correction processing,
It is necessary to recognize what kind of scene the image was taken and perform a correction process according to the taken scene. In order to realize the correction processing for the image according to such a shooting scene, some methods have been conventionally proposed. As one of them, for example, JP-A-5-4183
In the device described in Japanese Patent No. 0, the exposure setting is switched depending on whether the subject is a person or not, and correction is performed at the time of image acquisition. Further, for example, in the device disclosed in Japanese Patent Laid-Open No. 5-336535, the contour correction parameters are switched between the person in the image and the other portions.
Further, for example, in the device disclosed in Japanese Patent Laid-Open No. 2000-123164, an image is divided into blocks, an attribute is discriminated for each block, and saturation conversion is performed according to the attribute.

FIG. 14 is a block diagram showing an example of a conventional image processing apparatus. In the figure, 11 is an input image, 12 is a main object / region recognition unit, 13 is a correction amount calculation unit, 1
Reference numeral 4 is a correction execution unit, and 15 is an output image. The above-mentioned conventional techniques have the configuration shown in FIG. That is, the main object / area recognition unit 12 detects the presence or absence of a main object such as a person in the input image 11 and the area of the main object,
The correction amount calculation unit 13 determines a correction method and a correction parameter for distinguishing the main object from other objects, and the correction execution unit 14 executes the correction process using the correction method and the correction parameter to output the output image 15. It has gained.

With such a conventional technique, a correction process focusing on a main object can be performed, and a good image can be obtained for the main object. However, there is a problem that the presence or absence of the main object and the correction are uniquely determined for each object. It is effective to make corrections centering on the main objects that humans pay most attention to, but if you focus on and correct only the main objects, only the main objects will be emphasized, and the overall balance will be lost and you will feel uncomfortable. It may become a certain image.

For example, when a "person" is photographed under a backlit condition, the "person" appears dark. To correct this, if the entire image is corrected so that the "person" becomes bright, the background color such as the sky will be lost. For example, even when determining a correction method for each object, a uniform correction method such as "person" is bright and "sky" is emphasized in saturation is suitable for a scene "person + sky". In some cases, it was not a correction.

In the above-mentioned example, the conventional technique has been explained for the case of correcting an image. For example, when an image is saved, classification information indicating the contents of the image is automatically added and accumulated. There is. Even in such a case, conventionally, a main object is extracted from the image and the classification information indicating the object is added. However, for example, even if the classification information “person” is added to an image in which a person is shown in the center, it is difficult to accurately grasp the content of the image. For example, I didn't even know if it was taken at the sea or in the mountains. In addition, for example, by simply listing the attribute information indicating an object,
For example, there is a problem that the classification information is not useful, such as "Sky" is always present in landscape photographs.

[0009]

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an image processing apparatus and an image processing method capable of performing more preferable correction on an image depending on a scene. An object of the present invention is to provide an image processing device and an image processing method capable of adding more preferable classification information to an image. Another object of the present invention is to provide an image processing program that causes a computer to execute such an image processing method, and a storage medium storing such an image processing program.

[0010]

SUMMARY OF THE INVENTION The present invention provides an image processing apparatus and an image processing method for correcting an image, in which a plurality of objects are recognized from the image and the objects are recognized based on the recognition result of the scene of the image. It is characterized in that a correction method is determined. Particularly, after recognizing a plurality of objects from the image, a main object and a sub-object are determined from the recognized plurality of objects, and an image correction method is determined based on the determined relationship between the main object and the sub-object. It is characterized by that. In this way, the recognition result of the scene of the image, in particular, the correction method is determined from the relationship using the sub-object together with the main object in the image, so compared to the case of correcting based on the recognition result of only the main object, The correction process can be performed according to the surrounding conditions. Therefore, it is possible to perform well-balanced correction processing on the image, and it is possible to realize more preferable correction processing.

As the relationship between the main object and the sub-object, it is possible to use a combination of the types of the main object and the type of the sub-object, or at least one of the position, size and weight of the two objects. The correction method can be determined for the entire image or for each object. Alternatively, the correction method is determined based on at least one of the feature amount of the main object, the sub object, the non-recognition target area, and the feature amount of the entire image, or the relationship between the attribute information of the main object and the attribute information of the sub object. can do. Further, the correction method may include correction of color tone and definition.

The object to be recognized can be a region, an object, or an aggregate thereof, and when determining a main object from among them, characteristics such as size, position, orientation, and number of objects, It is possible to use a weighting factor determined for each object or to determine it by a user's instruction. The image to be processed may be an image described in PDL, for example, other than the bitmap image.

Further, according to the present invention, in an image processing apparatus and an image processing method for classifying images, a plurality of objects are recognized from the image, a main object and a sub-object are determined from the recognized plurality of objects, and the determined main object It is characterized in that the classification information of the image is determined from the relationship between the attribute information and the attribute information of the sub-object. In this way, not only the attribute information of the main object but also the attribute information of the sub-object is considered, and the classification information is determined from the relationship between them.
Even if it is an image showing, was it taken in the "mountain"?
Whether or not the image was taken in the "sea" can be extracted as the classification information, and the classification information indicating the content of the image can be accurately added and stored in the image. Note that the method of recognizing an object and determining the main object as described above can also be used in the present invention. The image to be processed may be an image described in PDL as well as a bitmap.

Further, the present invention provides an image processing program for causing a computer to execute these image processing methods, and a storage medium storing such an image processing program. For example, a personal computer or a workstation, Furthermore, it can be realized in a server or the like that provides such image processing.

[0015]

FIG. 1 is a block diagram showing an embodiment of the present invention. In the figure, 1 is an object / region recognition unit,
2 is a main / sub object determination unit, 3 is a correction method determination unit, 4
Is a correction amount calculation unit, 5 is a correction execution unit, and 6 is a relational information database. The object / region recognition unit 1 recognizes a plurality of objects from the input image. The recognized object can be a region, an object, or an aggregate of them. In the object recognition, for example, in the object recognition example of "sky region", the frequency structure has a characteristic that is lower than the low frequency, and the position is at the top of the image.
As the color information, a closed area included in the blue hue can be captured and recognized. In addition, recognition of objects such as humans and human faces can be performed by detecting skin color areas and face structures.
It can be performed using a known method. The object recognition method is arbitrary, and various known methods can be used.

The main / sub object determining unit 2 determines a main object and a sub object from the plurality of objects recognized by the object / region recognizing unit 1. As a method of determining the main object, for example, the size and position of the recognized object in the image, the weight of the predetermined object, and the like can be determined. Alternatively, the image may be displayed and the user may instruct. Recognized objects other than the primary object are considered secondary objects. However, when the number of sub-objects is large, it is not necessary to make all the sub-objects.

The correction method determination unit 3 basically recognizes a scene of an image and determines a correction method for each object based on the recognition result. As an example thereof, here, the main object and the sub object determined in the main / sub object bottom part 2 are used.
First, the relationship between the main object and other sub-objects is analyzed. At the time of analysis, for example, by referring to the relationship information database 6, information such as a correction point, a feature amount calculation method, a correction processing method, and the like can be acquired from the relationship between the main object and the sub object. As the relationship between the main object and the sub-object, a combination of the type of the main object and the type of the sub-object is used.
The relationship of the position, size, weight, etc. of one object, the relationship of each attribute information, etc. can be used. Further, the correction location to be determined can be the entire image or a main object, sub-object, etc., and the correction processing method can be determined for each correction location. Note that the correction processing method is to obtain the feature amounts of the main object, the sub-object non-recognition target area, the entire image, etc. according to the feature amount calculating method acquired from the relationship between the main object and the sub object described above, You can decide.

The correction amount calculation unit 4 calculates how much correction is to be performed according to the correction method determined by the correction method determination unit 3. For example, the feature amount is calculated based on the feature amount calculation method acquired by the correction method determination unit 3 (or the feature amount calculated by the correction method determination unit 3 is used), and the correction amount corresponding to each correction method is calculated. You can Unlike the conventional uniform calculation method, the characteristic amount calculation method is defined by which object characteristic amount is used to calculate the correction amount from the pattern of the combination of the main object and the sub-object. And, based on this definition,
The characteristic amount that matches the pattern of each combination is obtained, and the correction amount is calculated.

The correction execution unit 5 performs the correction processing determined by the correction method determination unit 3 on the input image,
The correction process is actually performed by the correction amount calculated by the correction amount calculation unit 4. As the correction processing, for example, color tone correction or definition correction can be performed.

With such a configuration, the user who views the image is most interested in the image by deciding the main object, as compared with the conventional technique for performing uniform correction processing for each object. It is possible to make a correction in consideration of the object to be processed. Furthermore, by determining not only the main object but also the sub-object, it is possible to perform an optimum correction process on the relationship between the main object and the sub-object for each object, the non-recognition target area, or the entire image. Therefore, it becomes possible to avoid the inconsistency with the surroundings due to the correction of only the main object as in the conventional case, and to perform the best correction process considering the characteristics of the scene as a whole. Considering the correction of the skin color of a person as an example, conventionally, only the target color can be corrected for the skin color of the person, but by determining the correction method from the relationship between the main object and the sub-object, In the case of a portrait having a sub-object of “”, the correction can be changed depending on the recognized sub-object, such as changing the correction target color to a healthy wheat color.

The details of the operation of each unit will be described below with reference to specific examples. In the following specific examples of images, hatching is used to express colors, and when the actual color correction (for example, vividness correction) is performed, for example, the hatching intervals are different. I have shown.

FIG. 2 is an explanatory diagram of an example of an object recognized by the object / region recognition unit 1. As described above, in the object / region recognition unit 1, for example, a face of a person, a certain target article, or the like can be recognized and defined as an object, or a certain closed region or the like can be recognized and defined.
As an example, in a landscape image including a person as shown in FIG. 2, objects such as “person”, “sky area”, “mountain area” can be recognized. For example, "empty area"
The object has a low frequency as a frequency structure,
The position information is at the top, and the color information can capture the closed areas contained in the blue hue. In addition, recognition as an object such as a person or a face of a person may be performed by using a known method. It should be noted that, in this example, areas other than the recognized object are non-recognition target areas.

Next, the main / sub object determining unit 2 determines a main object and a sub object from the objects recognized by the object / region recognizing unit 1. Figure 3
It is explanatory drawing of an example of the determination method of a main object. FIG. 3 shows an example of a landscape image of mountains and the sky, and shows a process in which a main object is determined by the size of the area. For example, as a method of determining the main object, the larger object can be the main object and the others can be the sub-objects. In this case, in the example shown in FIG. 3A, the proportion of the mountain objects is large,
The "mountain area" object becomes the main object,
Other objects such as "empty areas" are sub-objects. On the other hand, in the example shown in FIG. 3B, a wide sky is conversely widened, so that the “sky region” object is the main object and other objects such as “mountain region” are the sub-objects.

FIG. 4 is an explanatory diagram of another example of the method of determining the main object. Here, an example is shown in which the main object is determined based on the size, position, orientation, weighting factor, etc. of each object. The input image is shown in FIG. 4A, and a to e are added to the recognized objects.
The symbol is attached. That is, the a object is the “sky region”, the b object is the “mountain region”, and the c to e objects are the “person”, and the larger the person on the right side, the larger the person. The size, position, and orientation of each of these objects is checked and weighted. The result is shown in FIG. As a position score,
It is closer to the center and the foreground is more heavily weighted. Therefore, the weight of the d object, which is the central person, is the largest. Further, as the size score, the larger the weight, the larger the weighting. In this example, the b object indicating the “mountain region” occupies the largest area, and thus the largest weight is given. The orientation score is weighted depending on whether a person or the like is facing the front or facing another direction. In this example, a large weighting is performed on the assumption that the c object indicating the “person” on the right side is facing the front. The “sky region” and the “mountain region” have no orientation and are set to 0.

Further, in this example, a weighting coefficient determined for each object is set. For example, when a person exists, it is generally more dominant than a mountain or sky object. Therefore, also in this case, a larger weighting coefficient is assigned to the c-e objects of the “person” than the a object of the “sky region” and the b object of the “mountain region”.

Then, the total score is calculated by integrating the position score, the size score, the orientation score, and the weighting coefficient respectively set in this way. In this example, the total score is calculated by (position score + size score + direction score) × weighting coefficient. The object having the largest total score may be the main object. In this example, the c-object, which is the “person” on the right side, is determined to be the main object. In addition, other objects become sub-objects. The number of sub-objects can be limited to some. For example, selection can be made arbitrarily, such as selection in descending order of total score, or selection of different types of objects.

In the example shown in FIG. 4B, the size, position, and orientation of the object are digitized and used, but various other features can be used. Of course, the method of calculating the total score is also arbitrary.

Here, two methods are shown as the method of determining the main object, but various methods other than this can be applied. For example, it is possible to manually select a main object while displaying an image on a monitor screen of a personal computer or the like used by the user.

Next, the correction method determination unit 3 determines the correction method to be applied to the input image based on the relationship between the main object and the sub object determined by the main / sub object determination unit 2 as described above. . As the relationship between the main object and the sub-object used at this time, a combination of the type of the main object and the type of the sub-object can be used. FIG. 5 is an explanatory diagram of an example of determination of a correction method based on the relationship between the object types of the main object and the sub-object. As described with reference to FIG. 3, in FIG. 5A, the “mountain region” is the main object and the “sky region” is the sub-object “mountain” scene. Such a combination of the types of the main object and the sub-object is used, and in the case of this combination, a correction method such as making the "sky region" slightly brighter and making the "mountain region" brighter is associated. be able to. This makes it possible to increase the contrast between "sky" and "mountain" and correct the image in which "mountain" is emphasized. On the contrary, in FIG. 5B, the “sky region” is the main object and the “mountain region” is the sub object, that is, the “sky” scene. In such cases,
It is possible to associate a correction method such as making the "sky region" vivid. As a result, the color of the "sky region", which is the main object, is changed to the "mountain" existence,
It can be corrected to be more vivid.

As described above, the combination of the types of the main object and the sub-object represents the scene of the image, and the more preferable correction method is applied by determining the correction method according to the combination of the types of the main object and the sub-object. It is possible. For example, even when the same "sky region" and "mountain region" objects exist as in the above example, a more preferable correction method can be applied to each image due to the relationship of the main / sub combination. .

When deciding the correction method in this way, which of the main objects and the sub-objects is to be combined, which part is to be corrected, and what kind of characteristics are to be used and how to make corrections are determined. It is necessary to decide for each combination. Which is to be corrected means which one of the main object and the sub-object, or which object is to be corrected, or whether to correct the entire image. In addition, what kind of characteristics means what kind of characteristic quantity such as hue, shade, and specific color is used for which part of the main object, sub-object, non-recognition target area, and entire image. is there. Further, it is necessary to determine what kind of correction processing should be performed, such as color tone correction, definition correction, and emphasis correction, using such characteristics.

In this example, the relational information database 6 is used to determine what kind of characteristics are used and how to correct them. Figure 6
It is explanatory drawing of an example of the content of a related information database. In the example of the relationship information database shown in FIG. 6, the object combination table shown in FIG. 6A, the main object table shown in FIG. 6B, the sub-object order table shown in FIG. It is composed of the feature amount calculation data shown in (D) and the correction data shown in FIG. 6 (E).

The object combination table shown in FIG. 6A is a table for referring to the main object table from the combination of objects existing in the image, and in this example, "person", "sky", "sea". ",
An example in which an object "mountain" is defined is shown. Here, as shown in FIG. 2, "person", "sky",
An arrow indicating a pointer to the main object table is shown as if an image with "mountains" was input.

The main object table shown in FIG. 6B is a table for referring to the sub-object rank table depending on which of the objects in the input image is the main object. In this example, the "person" object is the main object, and an arrow line indicating the pointer to the sub-object ranking table is shown.

The sub-object order table shown in FIG. 6C is a table for selecting the feature amount calculation data and the correction data according to the priority order between the sub-objects. By the reference so far, depending on what objects exist in the image, what the main object is among those objects, and the priority order of the sub-objects, the feature amount calculation data and the correction data are Will be selected. In the example shown in FIG. 6C, the case where "sky" is prioritized over "mountain" is shown.

The feature amount calculation data shown in FIG. 6D indicates which feature amount from which object in the priority order of the object, the main object and the sub object existing in the image determined by the above-mentioned tables. It defines whether to extract. In this example, it is indicated that the shade information is obtained from the "person" object, the average hue information and the shade information are obtained from the "sky" object, and the average hue information is obtained from the "mountain" object.

Further, the correction data shown in FIG. 6 (E) is based on what kind of feature amount is given to which object in the priority order of the object, the main object and the sub object existing in the image determined by the above-mentioned tables. Is used to define what kind of correction processing is to be performed. The feature amount to be used is defined in the feature amount calculation data shown in FIG. 6D, but the object for extracting the feature amount and the object for which the correction process is performed using the feature amount do not necessarily match. On the contrary, it is possible to perform the correction processing from the characteristics of other objects. For example, in FIG.
In the example shown in (E), when the main object is “person” and the sub-objects are “sky” and “mountain” in the order of priority, the “person” has a human skin based on the gray-scale information of “sky”. It is defined to make a correction to brighten. Also "empty"
Defines that the average hue of "sky" is corrected based on the average hue information of "mountain". Further, it is defined that the correction process is not performed on the "mountain" and the entire image. By defining the correction processing by using the characteristics of other objects in this way, the correction processing according to the surrounding situation becomes possible, and more preferable correction can be performed.

FIG. 7 is an explanatory diagram of an example of applying the determined correction method to an image. For example, an example of the image shown in FIG. 2 is input, and the feature amount calculation data shown in FIG. 6D and the feature amount calculation data shown in FIG. 6E are obtained by searching the relational information database 6 shown in FIG. When the correction data is obtained, the light and shade information of “sky” is reflected on the “person”, especially the human skin part such as the face as shown by the arrow in FIG. 7A according to the definition, and the average hue of the “mountain” is also obtained. The information is reflected in the "sky" hue. As a result, for example, the correction is performed as shown in FIG. In this way, since the correction is performed for each object according to the scene of the image, it is possible to perform a more harmonious correction process as compared with the case where the correction process is performed for each object alone.

How to correct which object and how to use which characteristic amount of which object can be changed by the definition of the correction data. Figure 8
FIG. 6 is an explanatory diagram of a correction example in which the correction target is different. Figure 8
The input images in both (A) and (B) are the same,
"Sky" is the main object, and "mountain" is the sub-object. In such a case, the main object may be corrected, or the sub-object may be corrected. For example, as shown in FIG. 8A, the sky can be made to look preferable by adjusting the color of the main object, "sky". In addition, as shown in FIG. 8B, there is also a correction method in which the color of the sub object “mountain” is adjusted without adjusting the “sky” to make the main object “sky” look preferable. . Of course, the entire image may be corrected. Which correction method is applied depends on the definition of the correction data. Alternatively, it is possible to use various other information such as the type, size, and attribute of an object described later, or to use a configuration in which the correction data itself is switched according to the feature amount of each object extracted according to the feature amount calculation data. Is also possible.

As described above, by using the relational information database 4, the objects recognized by the object / area recognition unit 1, the main object and the sub-objects determined by the main / sub-object determination unit 2 (at this time, the order of the sub-objects is also determined. From the above), it is possible to determine the feature amount to be calculated and the correction method using the feature amount.

In the above example, the correction method is determined only from the combination of the main object and the sub-object, but the correction method can be determined in consideration of other various information. FIG. 9 is an explanatory diagram of an example of determining a correction method based on the relationship between the types and sizes of the main object and the sub-object. In Figure 9,
An example in which a size element is added together with a combination of a main object and a sub-object is shown.
In the two examples shown in FIG. 9, the objects existing in the image are “sky”, “mountain”, and a plurality of “clouds”. Since the "sky" has a large area here, the "sky" object is the main object, and the "mountain" and "cloud" are the main objects.
The object is a sub-object. However, FIG.
In (A), the presence of "clouds" is large. In such a case, the "sky" of the main object should be kept slightly bright. Also, as shown in FIG. 9 (B), if only "clouds" are small, those "clouds"
Can be ignored and the "sky" can be adjusted more vividly.

As described above, by determining the correction method in consideration of the size of the existing object, the scene of the image can be accurately grasped without being influenced by the existence of the small object. A preferable correction can be made as a whole.

FIG. 10 is an explanatory diagram of an example of determining a correction method based on the relationship between the attributes of the main object and the sub-object. In this example, the finer adjustment is possible by using the attribute of the object. For example, if the combination of objects such as "sky" and "mountain" is "sky" is a clear sky,
It is possible to change the reproduction method depending on whether the sky is the sunset sky or not. In the example shown in FIG. 10A, a case is shown in which "clear" is added as an attribute value to the "sky" object which is the main object. In such a clear scene, for example, the sky is vividly reproduced, so that the “sky” can be vividly reproduced by using the feature amount of the “mountain” object.

Further, the example shown in FIG. 10B shows a case where "sky" is added as an attribute value to the "sky" object which is the main object. In the case of such a sunset scene, in order to emphasize the color of the sunset, the features of the “sky” object are used to emphasize the silhouette of the mountain at the expense of detail in the mountain, and “mountain” is used.
Can be corrected to reproduce darkly.

As described above, by utilizing the attributes of the object, it becomes possible to perform the correction processing according to each attribute, and the finer correction processing can be performed. In addition, for the switching of the correction method based on such an attribute value, for example, an attribute value table or the like may be provided in the relationship information database 6 so that the feature amount calculation data and the correction data based on the attribute value can be switched.

When the correction method determination unit 3 determines the correction method as described above, information about the correction method is passed to the correction amount calculation unit 4, and the correction amount calculation unit 4 determines the correction amount. For example, abstract expressions such as “bright”, “dark”, “bright”, etc. are also used in each of the above examples, but it is corrected how much “bright”, “dark”, and “bright” The amount is calculated by the amount calculation unit 4. At this time, based on the feature amount calculation data and the correction data obtained by searching the relational information database 6 by the correction method determination unit 3, according to the definition in the correction data, the feature amount of the designated object Calculate the correction amount. The characteristic amount may be extracted by the correction method determination unit 3 and passed to the correction amount calculation unit 4, or the characteristic amount calculation data may be passed from the correction method determination unit 3 to the correction amount calculation unit 4 and then corrected by the correction amount calculation unit 4. The feature amount may be extracted.

The correction method determination unit 3 and the correction amount calculation unit 4 determine what kind of correction process is performed for which object (or the entire image) and to what extent. Based on these, the correction execution unit 5 performs a correction process on the input image and outputs it. As the correction processing performed by the correction execution unit 5, it is possible to perform correction processing of image elements such as brightness, saturation, contrast, memory color, and sharpness. Of course, various other correction processes may be performed.

Although the input image is not particularly limited in the above description, for example, it is photographed by a digital camera, input by a reflection type scanner, or a film photographed by a film camera is scanned and input by a film scanner or the like. A so-called bitmap image is assumed. However, the present invention is not limited to this, and may be image data described in PDL, for example. In this case, the object recognized by the object / region recognition unit 1 is 1
Objects in a broader sense that exist on the page, such as character objects and photographs (images)
Objects etc. are recognized. Then, the correction method for each object can be determined by recognizing the scene of the image of one page. For example, the main / sub object determination unit 2
In, the main object and the sub-object are determined, and the correction method can be determined from the relationship between the main object and the sub-object.

FIG. 11 is an explanatory diagram of an application example to image data described in PDL. In the example shown in FIG. 11 (A), a large photograph is arranged together with the caption characters. In this image, the main object is “photograph” and the sub-object is “character”. Such an image is, for example, an image found in a pamphlet or the like, and in many cases, it is desired to reproduce the photograph preferably. Therefore, the "photograph" may be vividly and preferably corrected by utilizing the characteristic amount of the "character".

In the example shown in FIG. 11B, "photograph"
And the "letter" that describes the photo are displayed. Such documents are often found in, for example, product description materials, and it is often desired to make the main objects in a photograph particularly clear. Therefore, it is preferable to perform a correction process in which the characteristic amount of the “character” is used to blur the background in the “photograph” to highlight the main object.

Even in the case of image data described in PDL in this way, the scene of an image in one page, for example, the layout of "photograph" which is the main object, or "characters".
By switching the correction method according to the layout of the sub-object, it is possible to apply a more preferable correction method to the one-page document (each object in the document).

FIG. 12 is a block diagram showing another embodiment of the present invention. In the figure, the same parts as those in FIG. Reference numeral 7 is a classification information determination unit, and 8 is an image database. In the above-described embodiment, an example is shown in which the object recognition, the determination of the main object and the sub-object, and their relation are used for image correction. However, the relationship between the main object and the sub-object can be used for other purposes.
The example shown in FIG. 12 shows an example in which the present invention is applied to an image database system that classifies and stores input images.

The classification information determining unit 7 generates the classification information of the input image based on the relationship between the main object and the sub object determined by the main / sub object determining unit 2. Then, the classification information is registered and stored in the image database 8 together with the input image. As a result, it is possible to classify the image in consideration of the material, and it is possible to retrieve the image from the image database 8 based on the relationship between the main object and the sub object, for example.

FIG. 13 is an explanatory diagram of an example of the relationship between the image and the classification information. For example, the input image is shown in FIG.
As shown in, when a person is photographed in the mountains, the main object is “person” and the sub-objects are “mountain” or “sky”. FIG. 13B shows a photograph of the same person, but it is photographed at the sea. In this case, the main object is "person" and the sub-objects are "sea" and "sky". The relationship between such a main object and a sub-object can be stored in the image database 8 as classification information. Therefore, FIG.
It is possible to classify the scenes of the image accurately, such as a photograph of a person taken in the mountain in (A) and a photograph of a person taken in the sea in FIG. 13 (B).

Conventionally, since the images are classified only by the main object, all the images shown in FIG. 13 are "persons" as the classification information, and it is impossible to search them separately. However, in the present invention, the relationship between the main object and the sub-object is stored in the image database 8 as the classification information.
Although it is a photograph of, it is possible to search for a photograph taken in the "sea".

[0056]

As is apparent from the above description, according to the present invention, the recognition result of the scene of the image, for example, the main object and the peripheral sub-objects is determined without uniformly determining the correction method depending on the object in the image. Since the correction method can be determined from the relationship of 1), there is an effect that more preferable correction processing can be realized according to the scene of the image.

Further, by using such a relationship between the main object and the sub-object as the classification information, more preferable classification information can be added to the image, and the classification and the image content that accurately indicate the entire image can be added. An accurate search by can be realized.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an example of an object recognized by an object / region recognition unit 1.

FIG. 3 is an explanatory diagram of an example of a method of determining a main object.

FIG. 4 is an explanatory diagram of another example of a method of determining a main object.

FIG. 5 is an explanatory diagram of an example of determination of a correction method based on a relationship between object types of a main object and a sub-object.

FIG. 6 is an explanatory diagram of an example of contents of a relation information database.

FIG. 7 is an explanatory diagram of an example of applying the determined correction method to an image.

FIG. 8 is an explanatory diagram of a correction example in which the correction target is changed.

FIG. 9 is an explanatory diagram of an example of determining a correction method based on the relationship between the types and sizes of the main object and the sub-object.

FIG. 10 is an explanatory diagram of an example of determination of a correction method based on a relationship between attributes of main objects and sub-objects.

FIG. 11 is an explanatory diagram of an application example to image data described in PDL.

FIG. 12 is a block diagram showing another embodiment of the present invention.

FIG. 13 is an explanatory diagram of an example of a relationship between an image and classification information.

FIG. 14 is a block diagram showing an example of a conventional image processing apparatus.

[Explanation of symbols]

1 ... Object / region recognition unit, 2 ... Main / Sub object determination unit,
3 ... Correction method determination unit, 4 ... Correction amount calculation unit, 5 ... Correction execution unit, 6 ... Relationship information database, 7 ... Classification information determination unit,
8 ... Image database, 11 ... Input image, 12 ... Main object / area recognition unit, 13 ... Correction amount calculation unit, 14 ...
Correction execution unit, 15 ... Output image.

Claims (30)

[Claims] 1. An image processing apparatus for correcting an image,
Image processing comprising: a recognition unit that recognizes a plurality of objects from the image; and a correction method determination unit that recognizes a scene of the image and determines a correction method for each of the objects based on the recognition result. apparatus. 2. An image processing apparatus for correcting an image,
A recognition unit that recognizes a plurality of objects from the image, an object determination unit that determines a main object and a sub-object from the plurality of objects, and a correction method for the image based on the relationship between the main object and the sub-object. An image processing apparatus comprising a correction method determining unit. 3. The correction method determining means uses a combination of a type of the main object and a type of the sub object as a relationship between the main object and the sub object. Image processing device. 4. The correction method determining means determines the relationship between the main object and the sub-object,
The image processing apparatus according to claim 2, wherein at least one of the relationship between the position, size, and weight of one object is used. 5. The image processing apparatus according to claim 2, wherein the correction method determination unit determines a correction method to be performed for the entire image or for each object. 6. The correction method determining means determines the correction method based on at least one feature amount of the main object, the sub-object, the non-recognition target region, and the feature amount of the entire image. Claims 2 to 4
The image processing device according to any one of 1. 7. The correction method determining means determines the correction method from the relationship between the attribute information of the main object and the attribute information of the sub-object. The image processing device according to item 1. 8. The image processing apparatus according to claim 2, wherein the correction method determination unit determines a correction method including a color tone and a definition. 9. An image processing apparatus for classifying images,
The image based on the relationship between a recognition unit that recognizes a plurality of objects from the image, an object determination unit that determines a main object and a sub-object from the plurality of objects, and a relationship between the attribute information of the main object and the attribute information of the sub-object. An image processing apparatus having a classification information determining means for determining the classification information of. 10. The image processing apparatus according to claim 2, wherein the object is a region, an object, or an aggregate thereof. 11. The method according to claim 2, wherein the object determining means determines the main object by using at least one characteristic among the size, position, orientation, and number of the objects. The image processing apparatus according to any one of items. 12. The image according to claim 2, wherein the object determining means determines the main object by using a weighting coefficient determined for each object. Processing equipment. 13. The image processing apparatus according to claim 2, wherein the object determining unit determines the main object according to a user's instruction. 14. The image according to claim 1, wherein the image is data described in PDL, and the recognizing means recognizes a plurality of objects from the PDL.
The image processing device according to any one of 3 above. 15. An image processing method for correcting an image, recognizing a plurality of objects from the image, recognizing a scene of the image, and determining a correction method for each of the objects based on a recognition result of the scene. An image processing method characterized by the above. 16. An image processing method for correcting an image, recognizing a plurality of objects from the image, determining a main object and a sub-object from the plurality of objects, and determining the main object and the sub-object from the relationship between the main object and the sub-object. An image processing method characterized by determining a method for correcting an image. 17. The image processing method according to claim 16, wherein a combination of the type of the main object and the type of the sub object is used as a relationship between the main object and the sub object. 18. The image processing method according to claim 16, wherein at least one of a position, a size, and a weight of the two objects is used as a relationship between the main object and the sub object. 19. The image processing method according to claim 16, wherein the correction method is determined for the entire image or for each object. 20. When determining the correction method, the main object, the sub-object, the non-recognition target area,
19. The image processing method according to claim 16, wherein the correction method is determined based on at least one of the feature amounts of the entire image. 21. When determining the correction method, the correction method is determined based on the relationship between the attribute information of the main object and the attribute information of the sub-object. The image processing method according to item 1. 22. The correction method according to claim 16, further comprising correction of color tone and definition.
2. The image processing method according to any one of 1. 23. An image processing method for classifying images, recognizing a plurality of objects from the image, determining a main object and a sub-object from the plurality of objects, attribute information of the main object and attributes of the sub-object. An image processing method, characterized in that the classification information of the image is determined from the relationship with the information. 24. The image processing method according to claim 16, wherein the object is a region, an object, or an aggregate thereof. 25. The main object is determined by using at least one characteristic among the size, position, orientation, and number of the objects. The image processing method described in the item. 26. The image processing method according to claim 16, wherein the main object is determined by using a weighting coefficient determined for each object. 27. The image processing method according to claim 16, wherein the main object is determined according to a user's instruction. 28. The image according to claim 15, wherein the image is data described in PDL, and a plurality of objects are recognized from the PDL.
The image processing method described in the item. 29. An image processing program that causes a computer to execute the image processing method according to claim 15. Description: 30. A computer-readable storage medium storing a program for causing a computer to execute the image processing method according to any one of claims 15 to 28.