JP2007293438A - Device for acquiring characteristic quantity - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for acquiring a variety of characteristic quantities from an image, at a high speed. <P>SOLUTION: The device for acquiring characteristic quantity that acquires characteristic quantity at a target point from an inputted image, acquires a Haar characteristic quantity, expressed as a difference in the pixel value between a first region and a second region from each of the sampling points, arranged around the target point according to a retina structure, and acquires a combination of the plurality of the Haar characteristic quantities as the characteristic quantity at the target point. A variety of characteristic quantities can be acquired by arbitrarily designing a pattern of the characteristic region of the Haar characteristic quantities at each sampling point. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for acquiring a feature amount from an image.

  In image processing using a computer, information about an image is digitized and used as a feature value so that it can be used by the computer. Conventionally, as such a feature quantity, a feature quantity called a Gabor-Wavelet feature quantity has been widely used. The Gabor-Wavelet feature amount is a convolution of the image with the resolution (k) and direction (θ) of the Gabor filter expressed by Equation 1 changed.

  According to the Gabor-Wavelet feature quantity, it is possible to acquire the periodicity and directionality of shading change around a certain point (gaze point) on the image as the feature quantity. This Gabor-Wavelet feature amount is known to be an effective feature amount particularly in the field of face authentication and the like because it has features similar to human visual characteristics (Non-patent Document 1).

Further, it is known that a feature quantity more suitable for face authentication or the like can be obtained by combining a technique called Gabor-Wavelet feature quantity and Retina sampling (Non-patent Documents 2 and 3). Retina sampling is a technique for acquiring feature quantities discretely from around a gazing point, and using (a combination of) feature quantities at these sampling points as feature quantities at the gazing point. In the retina sampling, the sampling points are generally arranged so that the closer to the gazing point, the denser the sampling point, and the farther away from the gazing point, the sparser the arrangement. In this way, by combining the Gabor-Wavelet feature quantity and the retina sampling, it is possible to acquire a local feature in the vicinity of the gazing point and a global frequency and directionality in the vicinity thereof in a balanced manner. Thereby, the shape of an image, texture information, etc. can be obtained efficiently.
Chengjun Liu and Harry Wegsler. “Gabor Feature Based Classification Using the Fisher Linear Discriminant Model for Face Recognition Enhanced”, IEEE Trans. Image Processing, vol. 11, no. 4, pp. 467-476, 2002. Smalaldi and J.M. Bigun, “Facial feature detection by Saccadic Exploration of the Gabor Decomposition”, Proceedings of the 1998 International Conferencing on Image Processing. 3, pp. 163-167, October 4-7, 1998. Erina Takikawa, Kiyoshi Hosoi, “Automatic gender and age estimation using face images”, OMRON TECHNICS, Vol. 43, no. 1, 2003.

  However, in the case of the prior art as described above, the following problems have occurred. That is, there is a problem that it takes a very long time to calculate the Gabor-Wavelet feature amount. Although the calculation time can be reduced by using the Fourier transform compared to the case of performing the convolution operation directly, it is necessary to perform the Fourier transform and the inverse Fourier transform on the entire target area from which the feature amount is acquired in advance before the feature amount extraction. is there. Therefore, a large calculation time is required even in a calculation method using Fourier transform.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of acquiring various feature amounts from an image at high speed.

  In order to achieve the above object, in the present invention, a feature amount is acquired from an image by the following means or processing.

  The present invention is a feature amount acquisition apparatus that acquires a feature amount at a gazing point from an input image. The feature amount acquisition apparatus according to the present invention acquires a feature amount at each sampling point from a plurality of sampling points that are discretely arranged with reference to the gazing point, and combines a combination of feature amounts at the plurality of sampling points at the gazing point. Acquired as a feature value.

  Next, a feature amount extraction process at each sampling point will be described. First, the first region and the second region are defined for each sampling point, and the difference between the pixel values of the first region and the second region is acquired as a feature amount at the sampling point. The The pixel value in the area can be calculated as the sum of the pixel values of the pixels in the area.

  These first and second regions can have any shape. Further, these areas do not need to be continuous areas. That is, each of the first and second regions can be configured as a set of a plurality of subregions. The first region and the second region are preferably regions that do not overlap each other. In addition, the first region and the second region are preferably adjacent regions, but are not necessarily adjacent.

  The patterns of the first region and the second region at the sampling point (hereinafter referred to as region patterns or simply patterns) can all be the same.

  In addition, the pattern of the defined area may be different for each sampling point. That is, it is not necessary to use the same region pattern at all sampling points, and the region pattern at at least one sampling point may be different from the region patterns at other sampling points. By using different region patterns for each sampling point, various feature amounts can be extracted.

  Further, the first area and the second area are preferably rectangular areas (or a set of a plurality of rectangular areas). By making these areas rectangular, it is possible to obtain pixel values in these areas at high speed using an integral image.

According to the present invention configured as described above, first, the feature amount at the sampling point is calculated as the difference between the pixel values, so that high-speed processing is possible. In addition, since feature quantities are acquired from sampling points discretely arranged around the gazing point, local features near the gazing point and global features around the gazing point can be acquired in a well-balanced manner. Also, various feature extractions are possible by changing the pattern of the feature region for each sampling point. According to the Gabor-Wavelet feature quantity, only a feature quantity having a single frequency component can be acquired at the same time. However, according to the present invention, it is possible to simultaneously acquire feature quantities having different frequency components.

  Note that the present invention can also be understood as a feature amount acquisition method including at least a part of the above-described means, or a program for realizing the method. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  For example, a feature amount acquisition method according to an aspect of the present invention is a feature amount acquisition method for acquiring a feature amount at a gazing point from an input image, and the information processing device is arranged discretely based on the gazing point The difference between the pixel values of the first region and the second region determined for each sampling point is obtained from each of the plurality of sampling points, and the difference between the pixel values at the plurality of sampling points is calculated as described above. Acquired as a feature value at a gazing point.

  Further, for example, a feature amount acquisition program as one aspect of the present invention is a feature amount acquisition program for acquiring a feature amount at a gazing point from an input image. The difference between the pixel values of the first region and the second region determined for each sampling point is obtained from each of the plurality of discretely arranged sampling points, and the pixel values at these sampling points are obtained. The difference is acquired as a feature amount at the gazing point.

  According to the present invention, it is possible to acquire various feature amounts from an image at high speed.

  Hereinafter, a feature point detection device that acquires feature amounts from a face image and detects the position of an eye will be described. This feature point detection device includes a feature amount acquisition device as a feature amount acquisition unit, and detects the position of an eye from a face image based on the feature amount acquired by the feature amount acquisition unit.

<Device configuration>
The feature point detection device 1 is configured to include a CPU (Central Processing Unit), a main storage device (RAM), an auxiliary storage device, and the like connected via a bus in terms of hardware. In this case, the feature point detection apparatus 1 is realized by the program being executed by the CPU. The auxiliary storage device here is configured using a nonvolatile storage device. A nonvolatile storage device is a memory that can retain the stored contents even when power supply is stopped. It is a so-called ROM (Read-Only Memory: EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only). Memory), mask ROM, etc.), FeRAM (Ferro-electric RAM), hard disk and the like.

  FIG. 1 is a diagram illustrating an example of functional blocks of the feature point detection apparatus 1. The feature point detection device 1 loads various programs (OS, applications, etc.) stored in the auxiliary storage device into the main storage device and is executed by the CPU, whereby the image input unit 2, the face detection unit 3, the feature amount It functions as the acquisition unit 4 and the identification unit 5. Further, all or part of the feature point detection apparatus 1 may be configured as a dedicated chip. Next, each functional unit included in the feature point detection apparatus 1 will be described.

  The image input unit 2 functions as an interface for inputting face image data to the feature point detection apparatus 1. The image input unit 2 inputs face image data to the feature point detection apparatus 1. The image input unit 2 may be configured using any existing technique for inputting face image data to the feature point detection apparatus 1.

For example, face image data may be input to the feature point detection apparatus 1 via a network (such as a LAN or the Internet). Further, a face image may be input to the feature point detection device 1 from a digital camera, a scanner, a recording device (for example, a hard disk drive), or the like. In addition, the feature point detection device 1 is included in various devices (such as a mobile phone and a PDA (Personal Digital Assistant)) including an imaging device such as a digital camera or an imaging device such as a digital camera. It may be input to the dispensing device 1.

  The face detection unit 3 detects a human face from the face image input via the image input unit 2. The face detection unit 3 may be configured to detect a face by template matching using a reference template corresponding to the outline of the entire face, for example. The face detection unit 3 may be configured to detect a face by template matching based on face components (eg, eyes, nose, ears). The face detection unit 3 may be configured to detect a region close to the skin color and detect the region as a face. Further, the face detection unit 3 may be configured to perform learning by a teacher signal using a neural network and detect a face-like region as a face. The face detection unit 3 may be realized by applying any other existing technique.

  The face detection unit 3 identifies the position of the detected face and passes the position to the feature amount acquisition unit 4. At this time, the face detection unit 3 may be configured to acquire the tilt of the face and the size of the face, and further pass the values to the feature amount acquisition unit 4.

  The feature amount acquisition unit 4 acquires a feature amount from an arbitrary point (gaze point) in the image. The feature amount at the gazing point is calculated as a vector in which the luminance value differences (brightness and darkness differences) at the sampling points discretely arranged around the gazing point are arranged in a line. Details of the feature amount acquisition processing performed by the feature amount acquisition unit 4 will be described later.

  The identification unit 5 determines whether the gaze point is an eye (feature point) using the identification function acquired in advance by the learning process and the feature amount acquired by the feature amount acquisition unit 4. Note that the identification unit 5 may calculate a probability (score) that the point is the eye for a plurality of gazing points, and obtain the position of the point having the highest probability as the eye position.

<Feature amount acquisition processing>
The feature amount acquisition process performed by the feature amount acquisition unit 4 will be described in detail. The feature amount acquisition unit 4 acquires a feature amount called a Haar feature amount from a plurality of sampling points discretely arranged around the gazing point, and a combination of the Haar feature amounts at the plurality of sampling points is a feature at the gazing point. Amount. That is, the feature amount at the gazing point is acquired as a vector amount whose elements are the Haar feature amounts at a plurality of sampling points. In the following description, feature quantities acquired at individual sampling points are referred to as Haar feature quantities, and feature quantities at the point of interest (vector quantities composed of a plurality of Haar feature quantities) are referred to as sampling feature quantities.

[Area pattern]
First, the Haar feature amount acquired at each sampling point will be described. The Haar feature amount is obtained as a difference between luminance values of two areas around the sampling point. That is, a value obtained by subtracting the sum of the luminance values of the pixels in the second region from the sum of the luminance values of the pixels in the first region located around the sampling point is calculated as the Haar feature amount at the sampling point. Is done.

The pattern of the area | region used in the Haar feature-value extraction process in a sampling point is shown in FIG. As shown in the drawing, various patterns can be used for the combination of the first region 11 and the second region 12 (hereinafter referred to as “first feature region” and “second feature region”). Since the Haar feature amount is calculated as the sum of the luminance values in the first feature region minus the sum of the luminance values in the second feature region, FIGS. 2 (a) and 2 (b) show the horizontal direction and the vertical direction, respectively. It is a pattern that reacts strongly to the luminance change in the direction. Strong reaction means that the value of the Haar feature value calculated as a difference in luminance value is large. The patterns shown in FIGS. 2C and 2D are highly responsive to changes in luminance in the horizontal and vertical directions and having a specific frequency. Moreover, FIG.2 (e) reacts strongly with respect to the line of a diagonal direction. Further, FIG. 2F shows a pattern that strongly reacts to a feature whose center is dark.

  Further, the first feature region and the second feature region do not need to be defined as a single region as shown in FIGS. 2A and 2B, but as shown in FIGS. 2C, 2D, and 2E. May be defined as a set of a plurality of regions.

  Thus, the Haar feature amount can capture various features by combining the first feature region and the second feature region. The direction of the luminance change that reacts can be changed depending on the combination of regions, and the frequency of the luminance change that reacts can be changed depending on the size of each region. In particular, the patterns of FIGS. 2C and 2D can be regarded as a simplified version of the Gabor-Wavelet feature with respect to the horizontal and vertical patterns, and have reaction characteristics similar to those of the Gabor-Wavelet feature. . 2C and 2D, when the size of each region is increased, the feature amount responds to a low frequency, and when the size of each region is decreased, the feature amount responds to a high frequency.

Further, the first feature region and the second feature region are not limited to rectangular regions, and can take any shape. However, by making these areas rectangular, it is possible to calculate the sum of the luminance values in the areas at high speed. That is, as shown in FIG. 3A, for all points in the image, the total of the luminance values in the rectangular area having the origin (the upper left corner of the image) and the diagonal as the point is obtained in advance. . When the sum of the luminance value at the point A and L _A, the sum L of the luminance values in the rectangular area ABCD shown in FIG. 3 (b) can be expressed as in equation 2, it is possible to determine at high speed .

[Retina sampling]
Next, the arrangement of sampling points will be described. The sampling of the feature amount is performed from sampling points according to the retina structure around the gazing point. The retina structure is a structure of sampling points arranged discretely and radially around the gazing point. A specific example is an arrangement as shown in FIG. In FIG. 4, a black circle is a gazing point, and a white circle is a sampling point arranged around the point. It should be noted that the Haar feature amount is acquired also from the gazing point, and is used to calculate the sampling feature amount at the gazing point.

  The arrangement of the sampling points may be such that the sampling points become sparse as they move away from the gazing point as shown in FIG. 4 (a), and are arranged at equal distances around the gazing point as shown in FIG. 4 (b). It may be a thing. In FIG. 4, the sampling points are arranged so as to be point-symmetric with respect to the gazing point. However, the sampling points are not limited to such a symmetrical arrangement, and any arrangement can be taken.

  By performing sampling with such a retina structure, it is possible to efficiently sample information around the gazing point in a low dimension.

[Feature acquisition]
At each sampling point, different Haar feature values can be acquired. That is, a Haar feature amount different from that of other sampling points can be used for a certain sampling point. A specific example of the pattern of combinations of Haar feature values is shown in FIG. FIG. 5A shows an example in which the gazing point and six surrounding points are set as sampling points, and the same Haar feature is applied to all sampling points. Assuming that the difference in luminance value between feature areas at each sampling point (points a to g) is f _i (i = a to g), the sampling feature amount p at the point of gaze is 7 dimensions as shown in Equation 3. It will be expressed as a vector of

  Note that with the feature region pattern (a combination of Haar feature values) as shown in FIG. 5A, it is possible to obtain sampling feature values that react strongly to luminance changes in the horizontal direction. This sampling feature amount can take a good balance between local information in the vicinity of the gazing point and global information in the vicinity.

  Further, in the feature region pattern as shown in FIG. 5B, it is possible to acquire a sampling feature amount in which the gazing point is darker than the surroundings and the left side is more responsive to the brighter pattern near the gazing point. In this way, by applying patterns of different feature areas to sampling points, it is possible to extract feature amounts of complex patterns adapted to the object.

  Further, as shown in FIG. 5C, the feature region at the sampling point other than the gazing point is made smaller than the feature region at the gazing point, thereby reacting to a low frequency component in the vicinity of the gazing point, A sampling feature amount that reacts to a high-frequency component can be extracted. In this way, by changing the size of the feature region, it is possible to acquire sampling feature amounts that respond simultaneously to different frequencies.

  As described above, by combining the Haar feature amount and the retina sampling and appropriately designing the feature amount region at each sampling point, it is possible to acquire the sampling feature amount that reacts to various features. The sampling points in FIG. 5 are all equidistant from the gazing point (that is, the structure of FIG. 4B), but as shown in FIG. 4A, they are at different distances from the gazing point. Sampling points may be arranged.

[Flow of feature acquisition processing]
An operation example of the feature amount acquisition process described above will be described with reference to the flowchart of FIG. First, an image is acquired via the image input unit 2 (S10). Next, the feature quantity acquisition unit 4 creates an integrated image from the input image (S11). The integral image is obtained by calculating the sum of pixel values in a rectangular area having a straight line connecting the origin and the point as a diagonal line for each point in the input image. Next, sampling points according to the retina structure are arranged around the gazing point (S12), and Haar feature quantities are acquired from these sampling points (S13). The acquisition of the Haar feature amount is obtained by using the integrated image created in S12. The Haar feature value for each sampling point obtained in this way is vectorized, and the feature value at the gazing point can be acquired (S14).

<Learning process>
Next, a learning process necessary for detecting an eye from an image will be described with reference to a flowchart of FIG. This learning process is a process that needs to be performed in advance in order for the identification unit 5 to detect eyes from the face image.

  In the learning process, first, a plurality of learning face images are prepared, and eye positions (coordinates) in the face image are determined. Then, this face image is read (S20). It is preferable that the position of the eye in the face image is determined by a human in order to improve learning accuracy. The combination of the face image and the eye position is called correct answer data, and the eye position in the correct data is called a correct answer point.

  Next, normalization of the face image is performed (S21). That is, the size and inclination of the face image are normalized according to the size and angle of the face. For example, the image is converted so that the face width (pixel width) becomes a predetermined size and a straight line connecting both eyes becomes horizontal. It is also desirable to normalize the luminance value of the image. For normalization of the luminance value, for example, it is desirable to correct the luminance value so that the average luminance is 0 and the variance is 1.

  Next, the sampling feature amount described above is acquired with the correct point in the face image as the gazing point (S22). These acquired sampling feature values are added to the correct learning data set (S23). Assuming that the correct answer learning data set is P, P is expressed as in Expression 4.

P _i represents the sampling feature amount for each correct data, and N _p represents the number of correct data. Further, fp _i is a Haar feature amount at each sampling point, and R represents the number of sampling points.

  Similarly, the feature amount when the coordinates other than the correct answer point are set as the gazing point from the face image is sampled (S24) and stored (S25), and this is set as a failure learning data set. Assuming that the failure learning data set is N, N is expressed as follows.

Note that n _i represents the sampling feature amount for each failed data, and N _n represents the number of failed data. Also, fn _i is the Haar feature quantity at each sampling point, R represents the number of sampling points.

  It is determined whether or not the acquisition of the sampling feature amount has been completed for all the prepared learning face images (S26), and when the learning images still remain (S26-NO), the above processing is repeated. If the acquisition of the sampling feature amount is completed for all the learning images (S26-YES), the process proceeds to S27.

  The discriminator is trained using the correct learning data set P and failure learning data set N obtained in this way (S27). For example, when a linear discriminant function is used, the learning is performed as follows.

First, the average and variance of the learning data set are obtained. Each mean and dispersion of the correct training dataset, m _p, and S _p, the average and dispersion failure training dataset m _n, when the S _n, which are expressed as number 6.

  Using these statistical values, the discriminant function g (x) is determined as shown in Equation 7 below.

_{Incidentally,} k p, _{k n} is an arbitrary constant, for _example, may be set as _k p ₌ k n ₌ 1. Further, since the discriminant function obtained by multiplying w by a constant can be regarded as the same as the original discriminant function, for example, | w | = 1, g (m _p )> 0, g (m _n ) <0. It is good to choose what becomes.

  In addition to this, the learning process can be performed by an arbitrary linear / nonlinear discriminant function such as SVM.

<Detection process>
Next, processing for detecting the position of the eye from the input image (hereinafter referred to as input image) performed by the identification unit 5 will be described with reference to the flowchart of FIG. The identification unit 5 performs the following detection process using the identification function obtained by the learning process described above.

  First, an input image is acquired from the image input unit 2 (S30). Then, as preprocessing for detecting the eye position, the face detection unit 3 detects the face position from the input image (S31). Any existing technique such as pattern matching may be applied to face detection.

  Then, image normalization similar to the learning process is performed (S32). For example, the input image is converted so that the face width (pixel width) becomes a predetermined size, and the straight line connecting both eyes becomes horizontal, and normalization is performed so that the average luminance value is 0 and the variance is 1.

  A gaze point for acquiring a feature value from the normalized input image is determined (S33). Next, the feature quantity acquisition unit 4 performs the feature quantity acquisition process described above for the determined gaze point (S34). The identification unit 5 substitutes the acquired feature amount into the identification function g (x) (S35), and determines whether the output is positive (S36). If g (x)> 0 (S36-YES), the gaze point is output as the eye position (S37). If g (x) <0 (S36-NO), it is determined whether the entire image has been scanned (S38). If not all have been scanned (S38-NO), the next point is scanned and scanning is performed. If completed (S38-YES), no eyes are detected from the image.

  In the above description, the point where the discriminator output is positive for the first time among the scanning points is detected as the eye position, but the discriminator output is obtained for all points, and the position where the maximum value is obtained is the eye position. May be detected as

<Effect of embodiment>
As described above, in the present embodiment, the eye position is detected by acquiring the feature amount by combining the Haar feature amount and the retina sampling. Since the Haar feature value is used, the feature value can be acquired by high-speed processing, and not only the gaze point but also the surrounding information can be efficiently acquired by the retina sampling. In addition, since various feature quantities can be acquired by appropriately designing combinations of Haar feature quantities, accurate eye detection can be performed by using feature area patterns suitable for eye detection. .

<Modification>
The feature amount acquisition apparatus according to the present invention may be used to detect feature points (mouth, ears, or entire face) other than eyes from a face image. Further, it may be used not for feature point detection but for face matching and face recognition. In addition to the face, it may be used for detection / collation processing performed on the whole person (whole body), an X-ray image, or an organ in a CT image. Further, in addition to the detection / collation processing, the image processing may be used for any image processing such as processing for searching for an image similar to a specific image and processing for classifying an image according to a predetermined standard. That is, the feature quantity acquisition device according to the present invention can be used for any processing that requires a feature quantity of an image.

It is a figure which shows the functional block of the feature point detection apparatus concerning this embodiment. It is a figure which shows the example of the Haar feature-value used in this embodiment. It is a figure explaining an integral image. It is a figure which shows the example of a retina structure. It is a figure which shows the example of the combination of Haar feature-value in each sampling point used in this embodiment. It is a flowchart which shows the flow of a feature-value acquisition process. It is a flowchart which shows the flow of a discriminator learning process. It is a flowchart which shows the flow of a feature point detection process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Feature point detection apparatus 2 Image input part 3 Face detection part 4 Feature-value acquisition part 5 Identification part

Claims (6)

A feature amount acquisition device that acquires a feature amount at a gazing point from an input image,
The difference between the pixel values of the first region and the second region determined for each sampling point is obtained from each of the plurality of sampling points discretely arranged with reference to the gazing point, and the plurality of sampling points are obtained. A pixel value difference at a point is acquired as a feature amount at the gazing point;
The feature-value acquisition apparatus characterized by the above-mentioned.   The feature quantity acquisition apparatus according to claim 1, wherein patterns of the first area and the second area defined for the sampling point are all the same pattern.   The pattern of the first region and the second region defined for at least one sampling point of the plurality of sampling points is the first region defined for other sampling points and The feature amount acquisition apparatus according to claim 1, wherein the feature amount acquisition apparatus is different from the pattern of the second region.   4. The feature amount acquisition apparatus according to claim 1, wherein the first area and the second area are a rectangular area or a combination of a plurality of rectangular areas. A feature amount acquisition method for acquiring a feature amount at a gazing point from an input image,
Information processing device
From each of a plurality of sampling points discretely arranged with reference to the gazing point, a difference between the pixel values of the first region and the second region determined for each sampling point is acquired,
A difference between pixel values at the plurality of sampling points is acquired as a feature amount at the gazing point.
A feature amount acquisition method characterized by that. A feature amount acquisition program for acquiring a feature amount at a gazing point from an input image,
In the information processing device,
From each of a plurality of sampling points discretely arranged with reference to the gazing point, a difference between pixel values of the first region and the second region determined for each sampling point is acquired,
The difference between the pixel values at the plurality of sampling points is acquired as the feature amount at the gazing point.
A feature amount acquisition program characterized by that.

Images