JP2001126070A - Noticing area extracting device and automatic composition deciding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a noticing area adjusted to human subjectively and to automatically device well-balanced composition. SOLUTION: A noticing area extracting device and an automatic composition deciding device 10 include an image forming device 14, which generates the original image of a panoramic image from video photographed by a camera 12. A noticing area extracting device 20 extracts a noticing area from the original image given from the device 14. Namely, evaluation matched with human subjectivity is given in accordance with the physical feature of the original image and the noticing area is extracted in accordance with the evaluated result. A composition cutting off device 22 cuts out the extracted noticing area and an adjacent image area from the original image by referring to data on paintings painted by painters and photographs taken by photographers stored in a memory 24. Namely, the data can be cut out by the same composition as the case with painted images or photographed images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an attention area extracting apparatus and an automatic composition determining apparatus using the same, and more particularly, for example, extracting an attention area from an original image, and extracting the extracted attention area by an expert such as a painter or a photographer. 1. Field of the Invention The present invention relates to an attention area extracting apparatus that cuts an original image so as to fit in a composition of a produced picture and an automatic composition determination apparatus using the same.

[0002] Here, the region of interest refers to a region of interest in a picture or video.

[0003]

2. Description of the Related Art As a conventional technique of this kind, various methods have been proposed for a method of extracting an attention area from an image by an observer. (1) Milanese, Itti and Koch et al. Assume a discontinuous portion in an image as a region of interest, generate a feature map (shade image) corresponding to a plurality of physical features obtained from the image, and then generate Discontinuous portions of the feature map are obtained, and a combination thereof is extracted as a region of interest.

(2) Milanese et al. Disclose a plurality of difference-of-ofie fixed-size discontinuities in each feature map.
The feature map was filtered using the nted-Gaussians filter, and the filtering result with the maximum output was selected and obtained. Itti, Koch et al. Normalized each feature map by the square error from each mean, and integrated all feature maps by linear combination. Then, the integrated feature map is recursively filtered by the Difference of Gaussian filter, and a local peak of a finally obtained filtering result is extracted as a region of interest.

As described above, Milanese and Itti et al. Extracted a region of interest by pixel-level processing such as filtering and relaxation.

(3) Martin and Takeuchi et al. Evaluated luminance information obtained from an image based on Shannon's information theory, and determined a portion having a large amount of information obtained as a result as a region of interest. In this method, a region where the variance of the luminance value is large, that is, a region that looks complicated or a bright region is mainly extracted.

In the conventional camera, a human has manually determined the subject and the composition of the subject.

[0008]

However, in the methods (1) and (2), since the size of the attention area differs depending on the image, the attention area is accurately extracted using a fixed-size filter. It was difficult. Further, in the method (3), for example, as in the case of a picture of a black vase placed in front of a wall with a complicated pattern, a region that looks complicated or a bright region does not necessarily match the region of interest. In such a case, it was difficult to accurately extract the attention area. Furthermore, in such proposals, there are few examples in which the subjectivity of the observer (human) is compared with the extraction result, and it has been questioned whether a region of interest that fits the human subjectivity can be actually extracted.

[0009] Further, when a general person who does not have a sense for a photograph determines a composition with a conventional camera, it is not always possible to take a well-balanced photograph.

[0010] Therefore, a main object of the present invention is to provide a region-of-interest extraction apparatus capable of accurately extracting a region-of-interest adapted to human subjectivity.

It is another object of the present invention to provide an automatic composition determining apparatus capable of automatically determining a well-balanced composition.

[0012]

A first aspect of the present invention is an attention area extracting apparatus for extracting an attention area from an original image, comprising: an evaluation means for evaluating an attraction degree (conspicuous degree) based on physical characteristics; An attention area extraction device includes an extraction means for extracting an attention area according to an evaluation result of the means.

According to a second aspect of the present invention, there is provided an automatic composition determining apparatus using the attention area extracting apparatus according to the first aspect of the present invention, wherein the storage means retains data relating to a reference image having a reference composition, and a reference composition. The automatic composition determination apparatus further includes a cutout unit that cuts out an image of a region of interest from an original image.

[0014]

In the attention area extracting apparatus according to the first invention, the evaluation means evaluates the degree of attraction according to the physical characteristics of the original image. Here, the degree of attraction means a parameter suitable for human subjectivity. The extracting unit extracts a region that stands out from the evaluation result as a region of interest. In other words, since the evaluation means performs evaluation according to the human subjectivity according to the physical characteristics, it is possible to extract the attention area suitable for the human subjectivity.

For example, when the physical characteristics include the degree of heterogeneity of colors, the degree of attraction can be evaluated based on the difference in color of each region.

The physical characteristics further include, in addition to color heterogeneity, shape heterogeneity, area heterogeneity, and texture (pattern) heterogeneity, so that at least one of the four heterogeneities is different. If you evaluate the degree of interest based on the degree,
The degree of attraction can be accurately evaluated according to the characteristics of the original image.

If three color components (hue, saturation, and lightness) are also evaluated, a region close to a human-subjective color (red) can be evaluated as the most prominent region.

Further, if the spatial frequency and the area of each region in the original image are also evaluated, the evaluation of the most prominent region can be determined more accurately.

Further, an original image can be generated by photographing a desired image by the photographing means and combining an image corresponding to the photographed image, for example.

For example, if the position and height of the camera included in the photographing means are fixed so that the image can be photographed by rotating by 360 °, the joining means joins the images for each frame, and An original panoramic image can be generated within the range.

In the automatic composition determining apparatus according to the second invention, the holding means uses, for example, a painting image and a photographic image corresponding to a painting drawn by a painter or a photograph taken by a photographer as a reference image, and stores data relating to the reference image. Hold. The cutout unit cuts out the image of the attention area from the original image with reference to the data on the reference image, so that a well-balanced composition can be automatically determined.

Further, if data relating to a plurality of reference images is held, the selecting means can select a composition suitable for the image of the region of interest, so that a well-balanced composition can be determined for every image.

The data relating to the reference image as described above is
Since at least the image data corresponding to the reference image, the shape data of the subject, and the position data of the subject are included, a composition suitable for the attention area can be selected, and a composition with good balance can be automatically determined. That is, it is possible to generate a photograph as if the subject was photographed in the composition.

[0024]

According to the present invention, a region of interest is extracted according to the evaluation result of a physical feature that matches the subjectivity of a human, so that a region of interest suitable for the subjectivity of a human can be extracted.

According to another aspect of the present invention, the region of interest is cut out from the original image using the reference composition of the reference image, so that a well-balanced composition can be automatically determined.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

[0027]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG. 1, an automatic composition determination apparatus 10 of this embodiment includes a video camera (hereinafter, simply referred to as "camera") 12. The camera 12 has a fixed shooting position (position and height) using, for example, a tripod, and has a 360-degree position.
° Rotate to shoot. A video captured by the camera 12 is input to the image generation device 14, and a panoramic image as an original image is generated from the captured video. Since the camera 12 can be rotated by 360 °, the image generating device 14 can be rotated by 360 °.
A panoramic image can be generated within the range.

The image generation device 14 is used for the video capture 1
6 and an image joining device 18 to generate a panoramic image according to the flowchart shown in FIG. That is, the camera 12
, The image generation device 14 starts processing, and in step S1, captures the first video frame (first frame) using the video capture 16,
After generating a composite image from the image corresponding to the first frame, the image joining device 18 is used to generate a grayscale image from the composite image.

In the following step S3, the next frame is captured, and a gray scale image is generated as in the first frame. In step S5, a search template is cut out from the grayscale image of the target frame as shown in FIG. 3 in order to determine from which position the synthetic image corresponding to the current frame (target frame) should be synthesized.
The width of this search template is 50 pixels, and its height is the same as the frame image. The search plate is cut out from the origin of the target frame.

Subsequently, in step S7, a search range is set from the grayscale image of the composite image, and an image having the same size as the search template is cut out from the search range. That is, the width of the search range is 100 pixels, and the height is the same as the frame image. When the upper left of the frame image is the origin ((x, y) = (1, 1)), the x coordinate of the search plate is a position obtained by subtracting 100 from the width of the frame image, and the y coordinate thereof is 1. is there.

Next, in step S9, an image having the same size as the search template is cut out from the search range, and the absolute value of the difference between the cut-out image and the pixel value corresponding to the search plate is calculated. In a succeeding step S11, it is determined whether or not the difference between the pixel values is the minimum (0). If "YES" in the step S11, it is determined that the clipped image is the same size as the search plate, and a step S13 is performed.
Proceed to. On the other hand, if “NO” in the step S11, it is determined that the size of the clipped image and the search plate are different, and the search range is shifted by one pixel in the positive direction of the x-axis in a step S15. Return. Thus, 1
The processing is repeated while shifting the pixel by pixel until the clipped image and the search template have the same size.

In step S13, the X coordinate when the absolute value of the difference is minimum (0) is calculated. Subsequent step S1
In step 7, the composite image and the target frame image are superimposed on the calculated X coordinate to generate a new composite image. And
In step S19, a grayscale image is generated from the generated composite image, and it is determined in step S21 whether a panoramic image has been generated. “NO” in step S21
, That is, if no panoramic image has been generated,
It returns to step S3. On the other hand, in step S21, "YE
If "S", it is determined that a panoramic image (original image) as shown in FIG. 4A has been generated, and the original image is output to the subsequent region of interest extraction device 20 (FIG. 1) in step S23.
The process ends.

Returning to FIG. 1, the attention area extraction device 20
The most prominent area (attention area) is extracted from the given original image. Specifically, the attention area extraction device 20 performs processing according to the flowchart shown in FIG. That is, when the original image is provided from the image generation device 14, the attention area extraction device 2
0 starts the process, and divides the original image into regions in step S31. Specifically, as shown in FIG. 4B, the original image is divided into a drawing area and a picture area. The method of this area division
In 1997 IEEE, WYMa and BSManjunath et al. Described "Edge Flow: A Framework of Boundary Detection and
A boundary detection method based on “edge flow” described in “Image Segmentation” is applied. In simple terms, this method determines the direction of color or pattern change at each location in an image, and determines the edge flow consisting of the direction of change and the magnitude of change (large or small).
Determine the vector. Then, edge flo
The w vector is propagated in the direction of each vector, and a location where the final vectors collide with each other is defined as a boundary line of each area.

Therefore, in step S33, FIG.
The divided figure area as shown in (C) is extracted, and in step S35, the degree of attraction of the figure area is evaluated. That is, the attraction parameter of the figure area is obtained. Here, according to the subjective evaluation experiment performed by the inventor, the physical characteristics necessary for the evaluation of the degree of attraction are color heterogeneity, texture heterogeneity, shape heterogeneity and area heterogeneity, color, spatial frequency, and the like. It turned out to be an area. In general, the relationship between the subjective evaluation result of a human and the physical characteristics is often represented by an S-shaped curve, and therefore, the one-time function shown in Expression 1 is used for the evaluation of the attraction.

[0035]

(Equation 1)

Using this beta function, an attractiveness evaluation function as shown in Expression 2 is defined.

[0037]

(Equation 2)

The degree of attraction HET for each of the four heterogeneities in each region shown in Expression 2 is defined by Expression 3.

[0039]

(Equation 3)

Further, the heterogeneity H of each physical feature is obtained by calculating the difference between the feature value and the average feature value of all regions as d and the average value of the difference d as d.
When the standard deviation of m and the difference d is std, it is calculated according to Equation 4.

[0041]

(Equation 4)

That is, the degree of heterogeneity HC of the color of each area is CI
E L * a * b * The color difference between the average color of the area and the average color of the entire area using the color difference equation in the perceived uniform color space,
The standard deviation of the color difference is calculated, and the calculated result is substituted into Equation 4 for calculation. The color difference equation is described in detail in the 1994 “Color Science Handbook”. Specifically, the color information (R, G, B) of each pixel of the image is stored in the uniform perceived color space L
* a * b *, and the Euclidean distance in that space is used as the color difference. That is, the color difference equation is expressed as in Equation 5.

[0043]

(Equation 5)

Next, the heterogeneity HT of the texture will be described. Texture is BS in 1996 IEEE
Manjunath and WYMa et al.
wsing and Retrieval of Image Data ”, and the difference between textures is represented by the Euclidean distance between texture feature vectors.

Here, the texture feature vector is represented by a vector whose elements are responses when an image is filtered by a Gabor filter bank composed of a plurality of Gabor filters having different sizes and directions. However, when filtering is performed by a plurality of Gabor filters, each response does not have orthogonality, and thus the filtering result may include redundant information. Therefore, the parameters of each filter in the Gabor filter bank are determined by the method described in the above-mentioned 1996 IEEE. Specifically, θ, a, σu, and σv of the Gabor filter expressed by Expression 6 are obtained by Expression 7.
In this method, as shown in FIG. 6, the scale (size) and the azimuth parameter of the filter are determined so that adjacent filters are in contact at Half-Peak. (4 scales, 6 directions) are used.

[0046]

(Equation 6)

[0047]

(Equation 7)

Expression 6 represents a texture feature vector. Therefore, the heterogeneity HT of the texture of each area is calculated by substituting the distance from the average vector of all the areas, the average value of the distance, and the standard deviation of the distance into Equation 4.

Further, the heterogeneity degree HS of the area of each area is calculated by substituting the difference from the average area of all the areas, the average value of the difference, and the standard deviation of the difference into Equation 4.

Furthermore, the heterogeneity HSh of the shape of each region
Is obtained by integrating the heterogeneity of the outline of the region and the heterogeneity of the hole included in the region as shown in Expression 8.

[0051]

(Equation 8)

Here, regarding the difference in shape, it is necessary to consider both the difference in shape itself and the difference in appearance due to the rotation of the figure. Therefore, 1984/3
The shape can be described using a P-type Fourier descriptor described in "New Fourier descriptor applicable to open curves" by Yoshinori Uesaka in "Transactions of the Institute of Telecommunications".
In this case, if the power for each frequency after the P-type Fourier transform is detected, the powers match when the shapes of the two figures are the same. Also, if a Fourier descriptor is used, the Fourier descriptor matches only when the shape and the rotation angle match. Therefore, it is possible to evaluate how much the shape and appearance of the two figures match with the Fourier descriptor and the power for each frequency.

As described above, the features of the contour can be represented by a vector composed of Fourier coefficients and power, and the difference of the contours can be represented by using the Euclidean distance between the feature vectors. Therefore, the heterogeneity degree HSh of the outer shape of each area is calculated by substituting the distance from the average vector of all the areas, the average value of the distance, and the standard deviation of the distance into Equation 4.

Next, the heterogeneity Hho of the hole will be described. In order to express the feature of the hole included in the region, it is necessary to consider not only the shape of the hole but also the number and position of the hole. The difference in hole position can be defined by obtaining the first moment of the area. The first moment is 19
92 "Image Analysis Handbook". Therefore, the feature of the hole was represented by a vector as shown in Expression 9. In addition, the order of the holes is set in the order close to the origin of the image.

[0055]

(Equation 9)

Therefore, the degree of heterogeneity Hho of the holes in each region is
Is calculated by substituting the distance from the average vector of all regions, the average value of the distance, and the standard deviation of the distance into Equation 4.

The feature attractiveness FP shown in Expression 2 is given by Expression 1.
0 can be defined.

[0058]

(Equation 10)

In equation (10), the color attractiveness PC will be described first.

In previous studies, it has been reported that 1) warm colors are more noticeable than cold colors, 2) higher saturation is more noticeable, and 3) higher lightness is more noticeable. Regarding 1), an experiment conducted by God making reports that red is a highly attractive color independent of the background color. Based on this result, in this embodiment, red (R, G, B = 255, 0,
0) is the hue with the highest attractiveness, and it is assumed that the attractiveness is higher as the hue is closer to red. Here, the hue is set to the value of the above 19
HS described in detail in 92 "Image Analysis Handbook"
Using the I-bihexagonal pyramid color model, the red (R,
The hue of G, B = 255, 0, 0) is 0. That is,
The HSI dihedral pyramid color model has BK (black) as the origin, and a point opposite thereto is W (white). And B
R (red), M (magenta), B (blue), and approximately the center of a straight line connecting K and W
A hexagon having C (cyan), G (green), and Y (yellow) as vertices is formed. The color space of the HSI bihexagonal pyramid color model formed in this manner is used. Note that the color space refers to a space in an orthogonal three-dimensional coordinate system used to display a perceived color as one point in the space. That is, in this embodiment, since the hue is obtained using the HSI bi-hexagonal pyramid color model, the hue attractiveness He increases as the hue approaches zero. Therefore, the hue attractiveness He is calculated by the equation shown in Expression 11.

[0061]

[Equation 11]

Regarding 2), there is a linear relationship between saturation and eye-catching degree. Therefore, in this embodiment, the saturation itself in the HSI bi-hexagonal pyramid color model is used as the saturation attraction.

Regarding 3), in a study on the relationship between the amount of human sensation and lightness, Semmelroth found that
It has been shown that the relationship shown by 2 holds. Note that the relationship shown in Equation 12 is shown by Oyama, Imai, Waki, et al. In the 1996 "Handbook of Sensation and Perception Psychology".

[0064]

(Equation 12)

In equation 12, k = 0.65, m
= 0.4, n = 0.2, it was shown that it fits well with human sensation. Therefore, in this example, Semm
The expression shown by elroth was taken as the lightness attraction.

The attractiveness for the three color components (hue, saturation, and lightness) is linearly combined using Equation 13 to define the attractiveness PC of the color.

[0067]

(Equation 13)

Next, the attraction degree PT of texture (spatial frequency) will be described.

It has been clarified that the human visual system has the property of a bandpasser that maximizes the sensitivity at a specific spatial frequency. So far, Kubota and Nishizawa et al. Formulated the spatial frequency characteristics of vision as shown in Equation 14. The expression of the spatial frequency characteristic is described in "Third-Dimensional Noise Evaluation Function of Television System and Its Application to High-Definition Television" by Kubota et al. In Equation 14, the unit of the spatial frequency is converted to cpd (the number of cycles per 1 degree of visual perception).

[0070]

[Equation 14]

Based on the visual response V shown in Expression 14, the attractiveness PT of the texture (spatial frequency) is defined.

Next, the area attraction PS will be described.

It is generally said that immediately after an image is presented, objects closer to the center of the image stand out.
It is also said that when the point of interest is moved, the degree of attraction of the object closer to the moved point of interest is higher. This is defined as the degree of attraction of the place. As described above, in order to model that the degree of attraction of each point becomes gradually smaller as the distance from the point centered on a certain point (gaze point),
The two-dimensional Gaussian function shown in FIG. However, it is assumed that the center of the Gaussian function changes according to the time at which the gazing point moves.

[0074]

(Equation 15)

Here, the spread coefficient σ of the Gaussian function depends on the distance d from the viewpoint to the image. That is, as the distance d increases, the range that can be observed at a time increases. In general, the human visual field is between 20 and 30 degrees. Therefore,
The spread coefficient σ when the screen height of the presented image is H and the viewing distance is d · H can be defined as in Expression 16. Pix is the number of pixels in the vertical direction, and θ is 20 degrees to 30 degrees (0.176 <tan θ / 2 <0.26
8).

[0076]

(Equation 16)

Here, considering that the region is a set of pixels, the attractiveness PS of the area of the region can be represented by the sum of the attractiveness of the field of the region. Therefore, the attractiveness PS of the area is defined as in Expression 17.

[0078]

[Equation 17]

As described above, the attractiveness of each figure area is evaluated in step S37 using the defined attractiveness evaluation function, and in step S39, the figure area having the maximum attractiveness is determined. That is, the most noticeable area is determined as the attention area. Therefore, an attention area as shown in FIG. 4D can be extracted.

In this embodiment, each coefficient of the evaluation function of the degree of attraction is represented by [wh1, wh2, wh3, wh4] =
[0.039, 0.010, 0.027, 0.02
0], [wf1, wf2, wf3] = [0.132,
0.005, 0.100], [m, n] = [1.35
8, 4.250], θ in Equation 14 was set to 20 degrees, the viewing distance d was set to 1 m, and the P-type Fourier coefficient was set to the tenth order.

Further, in this embodiment, the degree of eye-catching is evaluated using the above eight physical characteristics. However, this is for adapting to a plurality of images having all characteristics. Does not necessarily need to be evaluated.

Subsequently, in step S41, a figure area adjacent to the determined attention area is determined, and a color difference between the attention area and the adjacent figure area and a Euclidean distance of a texture feature vector are determined. The texture feature vector whose Euclidean distance is within 0.3 is extracted together with the attention area, and the process is terminated. In addition,
As described above, the color difference is represented by CIE L as shown in Expression 5.
* a * b * Determined by a color difference equation in a perceived uniform color space. Further, the texture feature vector is obtained according to Equation 6.

Returning to FIG. 1, the attention area extracted by the attention area extraction device 20 is given to the composition cutout device 22. A memory 24 is connected to the composition cutting device 22,
The memory 24 stores a plurality of data relating to a picture drawn by a painter and a photograph taken by a photographer. The composition clipping device 22 refers to the data stored in the memory 24 and cuts a region of interest from the original image in accordance with the composition of the painter or photographer. Specifically, the composition clipping device 22 clips the region of interest according to the flowchart shown in FIG. That is, when the attention area and the adjacent figure area are extracted by the attention area extraction apparatus 20, the composition cutout apparatus 22 starts processing, and in step S51, outer peripheral pixels of the attention area are obtained.
That is, the edge of the extracted region of interest is determined. In the following step S53, by the method of the paper described by Uesaka described above,
P-type Fourier coefficients up to the tenth order are obtained, and the region of interest is set as a shape vector.

Here, the data stored in the memory 24 is image data corresponding to a reference image such as a painting image or a photographic image, a subject is extracted from the painting image or the photographic image, and the periphery of the subject is extracted by a P-type Fourier coefficient. It is shape vector data describing the shape, and position data corresponding to the position information of the subject as shown in FIG. That is, the position data of the subject is data of the reference composition of the reference image,
The length of the horizontal side of the circumscribed rectangle surrounding the object is a, the length of the vertical side is b, the position of the origin of the circumscribed rectangle (vertex closest to the upper left vertex of the image) is (w1, h1), If the position of the end point (the vertex closest to the lower right vertex of the image) (the position based on the lower right vertex of the image) is (w2, h2), Equation 18
Is represented by a vector like

[0085]

## EQU18 ## Position data [kx1, ky1, kx2, ky]
2] = [w1 / a, h1 / b, w2 / a, h2 / b] Then, in step S55, the shape vector obtained in step S53 and the shape vector of the painting image or the photographic image stored in the memory 24 are compared. The U grid distance is obtained, that is, matching with the data of the reference composition is executed, and step S5 is performed.
In step 7, data with the most similar area shape is obtained. In other words, the position data of the subject of the painting image having the smallest Euclidean distance is acquired. That is, image data optimal for the attention area and the adjacent figure area is selected from the plurality of pieces of image data. Then, in step S59, a circumscribed rectangle of the attention area is obtained, and a part of the original image is cut out from the origin and end point of the circumscribed rectangle according to Expression 19. In Equation 19, *
Means multiplication.

[0086]

X1 = x1-W * kx1 Y1 = y1-H * ky1 X2 = x2 + W * kx2 Y2 = y2 + W * ky2 Here, the width of the circumscribed rectangle of the attention area is W, the height is H, and the origin of the circumscribed rectangle is (X1, y1) and the end point are (x2, y2). When X1 <1, X1 = 1, when Y1 <1, Y1 = 1, X2> when the width of the original image is X2 = the width of the original image,
When Y2> the height of the original image, let Y2 = the height of the original image.
At this time, the coordinates of the vertices of the rectangle (image) to be cut are shown as in Expression 20.

[0087]

[Equation 20] The vertex coordinates of the image to be cut = [(X1, Y
1), (X1, Y2), (X2, Y1), (X2, Y
2)] Subsequently, the result (image) cut out in step S61 is output, and the process ends.

Therefore, the attention area and the adjacent figure area as shown in FIG. 9B can be cut out according to the composition of the painting image as shown in FIG. 9A.

According to this embodiment, the most noticeable area (attention area) from the original image according to the physical characteristics as described above.
Is extracted, it is possible to extract an attention area suitable for human subjectivity.

Further, since the extracted region of interest is cut out in accordance with the composition of the picture drawn by the painter or the composition of the photograph taken by the photographer, it is possible to generate a photograph as if the subject were photographed in that composition. That is, a well-balanced composition can be automatically determined.

Since the attention area extracting apparatus can determine the degree of attraction of an image area in conformity with human subjectivity, for example, in the quality evaluation of a digitally compressed image / video, the image or video to be evaluated is evaluated. The present invention can be applied to an apparatus that performs objective evaluation such as weighting a region according to the degree of prominence (attraction) of each region.

Further, in the printing field, in automation of quality control, the present invention can be applied to an apparatus which can automatically determine an area in which a problem such as a printing error can be neglected and an area in which it is not so, according to the degree of attraction.

Further, in the design field, particularly in the production of posters used for advertisements, etc., the present invention can be applied to an apparatus for objectively evaluating whether or not a part which a company wants to appeal most is conspicuous.

If the user wears noticeable clothing, the camera can automatically follow the user by controlling the rotation, tilt, and zoom of the camera using the attention area extraction device. Therefore, for example, if the shutter is released at a predetermined timing, a snapshot of the user can be taken. In addition, the photographed image (original image)
By automatically determining the composition using the composition determination device, a well-balanced snapshot of the user can be created.

[Brief description of the drawings]

FIG. 1 is an illustrative view showing one embodiment of the present invention;

FIG. 2 is a flowchart showing a part of the processing of the image generating apparatus shown in FIG. 1 embodiment.

FIG. 3 is an illustrative view showing a method of synthesizing an original image by the image joining device of the embodiment in FIG. 1;

FIG. 4 is a flowchart showing a part of the process of the attention area extracting apparatus shown in FIG. 1 embodiment;

FIG. 5 is an illustrative view showing a method of extracting a region of interest according to the flowchart shown in FIG. 4;

FIG. 6 is an illustrative view showing a Gabor filter;

FIG. 7 is a flowchart showing a part of the processing of the composition cutting apparatus shown in FIG. 1 embodiment.

FIG. 8 is an illustrative view showing position information when an image is cut out according to the flowchart shown in FIG. 7;

9 is an illustrative view showing a painting image of a painter to be referred to when the image is cut out according to the flow chart shown in FIG. 7, and an image cut out by reference;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Automatic composition determination apparatus using attention area extraction apparatus 12 ... Camera 14 ... Image generation apparatus 16 ... Video capture 18 ... Image joining apparatus 20 ... Attention degree extraction apparatus 22 ... Composition cutting apparatus 24 ... Memory

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/66 470A 15/70 330Q 460Z 460B (72) Inventor Yuichi Iwatate 5 Mihiraya F-Term Co., Ltd. Intelligent Telecommunications Imaging Communications Laboratory F-term (reference) 5B057 BA15 CA01 CB01 CE09 CE10 CH09 DA08 DB02 DB06 DC04 DC09 DC25 5C022 AA01 AB62 AC27 5C023 AA06 AA16 BA02 BA12 BA13 CA03 DA04 5C076 AA02 AA19 BA03 CA02 CA10 5L096 CA04 FA02 FA15 FA23 FA33 FA46 FA59 FA70 GA38 GA55 JA11

Claims (10)

[Claims] 1. An attention area extraction device for extracting an attention area from an original image, comprising: an evaluation unit for evaluating an attraction degree based on physical characteristics; and extracting the attention area in accordance with an evaluation result of the evaluation unit. A region-of-interest extraction device comprising an extraction unit. 2. The attention area extracting apparatus according to claim 1, wherein said physical characteristics include a degree of heterogeneity of color. 3. The physical feature further includes a shape heterogeneity, an area heterogeneity, and a texture heterogeneity, and the estimating unit determines the attraction degree based on at least one of the four heterogeneities. 3. The attention area extraction device according to claim 2, wherein 4. The attention area extracting apparatus according to claim 1, wherein said physical feature further includes a color. 5. The attention area extracting apparatus according to claim 4, wherein the physical characteristics further include an area of the area in the original image and a spatial frequency. 6. The attention area extracting apparatus according to claim 1, further comprising a photographing unit for photographing a desired video, and an image generating unit for generating the original image based on the video. 7. The camera according to claim 1, wherein the photographing unit includes a camera, and a driving unit that rotationally drives the camera, and the image generating unit includes a joining unit that joins the images captured by the camera frame by frame. 6. The attention area extraction device according to item 6. 8. An automatic composition determining apparatus using the attention area extracting apparatus according to claim 1, wherein the storage means retains data relating to a reference image having a reference composition, and the reference composition includes: An automatic composition determination device including a cutout unit that cuts out an image of a region of interest from an original image by referring to the image. 9. The automatic composition determining apparatus according to claim 8, wherein said holding means holds data relating to a plurality of said reference images, and further comprises a selection means for selecting said data suitable for the image of said attention area. 10. The automatic composition determining apparatus according to claim 8, wherein the data includes at least image data corresponding to the reference image, shape data of a subject, and position data of the subject.