JP4888217B2 - Person attribute estimation device - Google Patents

Description

  The present invention relates to a person attribute estimation device for estimating attributes such as a gender and an age group of a person from an image.

  Conventionally, attribute information such as a person's sex and age group has been utilized for marketing. For example, in a convenience store or the like, when a customer purchases a product, a store clerk guesses the attribute of the customer and stores it as product purchase information.

  Such attribute information is desired to be obtained automatically and with high accuracy.

  As a conventional technique in view of such a problem, there has been proposed a person attribute estimation device that automatically estimates attributes such as a sex and an age group of a target person from a face image of the target person taken from an arbitrary face direction. (See Patent Document 1).

  However, in such a device, since the age group is generally estimated based on the feature amount related to facial wrinkles, the shape of the facial organs, the illumination state of the face, etc. may adversely affect the estimation of the age group. is there. For example, when the target person is laughing, the shape of the facial organs such as the mouth and eyes changes, which may cause wrinkles in the mouth and corners of the eyes, and may be estimated to be older than the actual age. In addition, when exposed to direct sunlight, part of the face may be out of focus, and in such a state, the wrinkles that should have been originally cannot be obtained from the image, and the lower age group than actual May be estimated. Thus, there is a possibility that an incorrect age group is estimated depending on the face state at the time of photographing.

Therefore, when automatically estimating the age group, a measure for avoiding the adverse effects of the above-described states (face organ shape, face illumination state, etc.) is important.
JP 2003-242486 A JP 2006-038553 A

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for accurately estimating the attributes of a person from an image.

  In order to achieve the above object, the present invention adopts the following configuration.

A human attribute estimation apparatus according to the present invention is a human attribute estimation apparatus that estimates an attribute of a person from an image, and includes face state determination means for determining a face state in the image, and a plurality of predetermined age groups A score calculating means for calculating a score representing the probability that the face corresponds to the age group, a portion that adversely affects attribute estimation in the image based on the state of the face, and the attribute estimation The score correction means for correcting the score and the age group corresponding to the corrected score representing the highest probability among the corrected scores for each age group are reduced so that the influence of the part that adversely affects the person is reduced . Attribute estimation means regarded as an attribute.

  The score used for attribute estimation of the present invention is corrected based on the face state. As a result, the adverse effect of the face state on the attribute estimation can be reduced, so that it is possible to reduce erroneous estimation and improve accuracy.

  The score may be an integrated score obtained by integrating a plurality of partial scores respectively calculated from a plurality of parts in the image. Thereby, since a partial score can be obtained from each of a plurality of portions in the image, attribute estimation can be performed with higher accuracy than when attribute estimation is performed using only one type of score.

  The score correction means may correct the integrated score by identifying a portion that adversely affects attribute estimation based on the face state and excluding a partial score corresponding to a portion that adversely affects the attribute estimation. As a result, it is possible to eliminate the influence of the partial score of the part that adversely affects the attribute estimation, so that it is possible to reduce erroneous estimation and improve accuracy.

  The score correction means may correct the integrated score by identifying a part that adversely affects attribute estimation based on the face state and changing a value of a partial score corresponding to the part that adversely affects the attribute estimation. . Thereby, since the value of the partial score of the part that adversely affects the attribute estimation can be corrected to an appropriate value, it is possible to reduce erroneous estimation and improve accuracy.

  The score is an integrated score weighted for each partial score, and the score correction unit identifies a part that adversely affects attribute estimation based on the state of the face, and a part that adversely affects the attribute estimation The integrated score may be corrected by making the weight of the corresponding partial score relatively smaller than the weight of the other partial scores. As a result, a corrected score with an emphasis on the partial score of the portion suitable for attribute estimation can be obtained, so that it is possible to reduce erroneous estimation and improve accuracy.

  The face state may include one or more of a face orientation, a face organ shape, and a face illumination state. As a result, the adverse effects of the face direction, the shape of the facial organ, and the illumination state of the face on the attribute estimation can be reduced, so that the false estimation can be reduced and the accuracy can be improved.

  Prohibiting attribute estimation by the attribute estimation means when two or more of the face orientation, facial organ shape, and face illumination state in the image adversely affect attribute estimation Good. As a result, faces that are not suitable for the attribute estimation means can be excluded from the estimation target, or attribute estimation of the person can be attempted by other methods, so that erroneous estimation can be reduced and accuracy can be improved.

  Further, the present invention may be regarded as a person attribute estimation device having at least a part of the above means, a person attribute estimation method including at least a part of the above processing, or a person attribute estimation for realizing the method. It can also be understood as a program or a storage medium storing the program. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  According to the present invention, since the score used for attribute estimation is corrected based on the face state, the attribute can be estimated with high accuracy.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings.

<Device configuration>
FIG. 1 is a block diagram showing a functional configuration of a person attribute estimation device according to an embodiment of the present invention. This person attribute estimation device is a device that performs person attribute estimation processing for estimating a person's age group (age) from an image. For example, by using an image obtained by a surveillance camera such as a convenience store, the age group is determined. It can be stored as customer information, and the customer information can be used for marketing and the like. The present invention can be applied to existing facilities such as monitoring devices installed in buildings, vending machines, ATMs, and the like. Note that the present invention is not limited to a monitoring device, and can be applied to devices including a photographing device such as a mobile phone, a personal computer, and a PDA. In addition, the person attribute processing according to the present invention can be performed using images recorded on a hard disk, a DVD, or the like without using images obtained from the photographing apparatus in real time.

  The person attribute estimation device includes a plurality of functional elements shown in FIG. 1, that is, an imaging unit 11, an image storage unit 12, a face detection unit 13, a feature amount extraction unit 14, a face state determination unit 15, a reference feature amount storage unit 16, and a score. A calculation unit 17, a score correction unit 18, an attribute estimation unit 19, a result storage unit 20, and a display unit 21 are provided. In the present embodiment, these functional elements are realized by an arithmetic processing unit of a computer executing software (program) and controlling hardware resources such as a camera, a memory, and a display as necessary. However, these functional elements may be configured by a dedicated chip.

  As the imaging unit 11, a monitoring camera including an optical system and an imaging element (CCD, CMOS sensor, etc.) can be preferably applied.

  The image storage unit 12 is a storage device that temporarily stores an image to be processed. As the storage device, any specific technique such as a volatile memory or a nonvolatile memory may be applied.

  The face detection unit 13 has a function of detecting a face from an image by image processing. Any technology of existing face detection processing may be applied to the face detection processing by the face detection unit 13. For example, a method for detecting a face by template matching using a reference template corresponding to the outline of the entire face, a method for detecting a face by template matching based on a facial organ (eyes, nose, ears, etc.), chroma key processing Detecting vertices such as the head, detecting a face based on the vertices, detecting a region close to the skin color, detecting that region as a face, and learning with a teacher signal using a neural network And a method for detecting a face-like area as a face.

  The feature quantity extraction unit 14 first detects feature points from the face detected by the face detection unit 13. Characteristic points are points that become features such as both ends of the mouth, the corners of the eyes, the eyes, and the nose. In addition, the feature quantity extraction unit 14 can specify a facial organ (eyes, mouth, nose, etc.) from the detected feature points. Any technique of existing feature point detection processing may be applied to the feature point detection processing by the feature amount extraction unit 14. For example, a method that detects a pattern that indicates the position of a face feature point and detects the feature point by performing matching using the learned data, and detects a feature point by pattern matching inside the detected face Any existing method may be used, such as a method to perform.

  In addition, the feature quantity extraction unit 14 identifies the feature unit based on the position of the detected feature point and the positional relationship with other feature points. The feature portion is a region of interest for extracting a feature amount used for attribute (age group) estimation. The characteristic part is preferably a part where a change in appearance due to aging appears remarkably. In the present embodiment, four characteristic portions of “right eye”, “left eye”, “nose”, and “mouth” are considered. It should be noted that the position of the feature point, the positional relationship with other feature points, and the correspondence with the feature portion are stored in advance in the feature amount extraction unit 14.

  Then, the feature amount extraction unit 14 extracts a feature amount from each feature portion. As the feature amount, for example, the relative position of the feature point, the color density of the whole or part of the feature portion, the number of wrinkles, and the like are extracted.

The face state determination unit 15 has a function of determining the state of the face detected by the face detection unit 13. In the present embodiment, the orientation of the face is determined based on the positional relationship of the facial organs, the shape of the facial organ is determined based on the positional relationship of the contour points, and the face is determined based on the color density (luminance value) of the facial organs. The lighting state is determined. Here, the contour point is a point representing the contour of the facial organ.

  The reference feature amount storage unit 16 is a storage device that stores a reference feature amount. The reference feature amount is an average of feature amounts of a plurality of persons belonging to the same age group. In the present embodiment, reference feature values are stored for each of the five age groups of “10s and below”, “20s”, “30s”, “40s”, and “50s and above”. . Furthermore, the reference feature amount is divided according to the feature portion. As the storage device, any specific technique such as a nonvolatile memory or a hard disk may be applied.

  The score calculation unit 17 has a function of calculating a score for each age group using the feature amount extracted from the face detected by the face detection unit 13 and the reference feature amount. The score represents the probability that the face detected by the face detection unit 13 corresponds to the age group. A score is calculated for each feature and is called a partial score. And what integrated the partial score is called an integrated score. The partial score in the present embodiment is a similarity between the feature amount for each feature portion specified by the feature amount extraction unit 14 and the reference feature amount of the corresponding feature portion. In this embodiment, for each of the above age groups, the average value of the partial scores of that age group is used as the integrated score. In this way, by using the integrated values of partial scores respectively obtained from a plurality of portions in an image, it is possible to perform attribute estimation with higher accuracy than performing attribute estimation using only one type of score.

  The score correction unit 18 is a function that corrects the integrated score based on the face state determined by the face state determination unit 15. A specific correction method will be described later.

  The attribute estimation unit 19 is a function that regards the age group corresponding to the corrected score representing the highest probability among the corrected scores for each age group as the attribute of the person.

  The result storage unit 20 is a storage device that stores the result output from the attribute estimation unit 19. As the storage device, any specific technique such as a nonvolatile memory or a hard disk may be applied.

  As the display unit 21, any specific technique such as a liquid crystal display may be applied.

<Person attribute estimation function>
The function and processing flow of the person attribute estimation device will be described with reference to the flowcharts of FIGS. 2, 3, 4, and 5.

  The human attribute estimation device in the present embodiment executes the human attribute estimation function at regular time intervals. When the person attribute estimation function is activated, an image captured by the imaging unit 11 is captured for use in the person attribute estimation process (step S11). The input image is stored in the image storage unit 12 together with shooting time information. In addition, the user can confirm the image for a person attribute estimation process with the display part 21 as needed.

  Next, the face detection unit 13 detects a face in the image captured by the imaging unit 11 (step S12).

  Then, the feature amount extraction unit 14 selects one of the faces detected by the face detection unit 13 (step S13), and extracts the feature amount from the face (step S14).

  Next, the score calculation part 17 calculates the partial score of each characteristic part for every age group (step S15).

  Then, the face state determination unit 15 determines the face direction (step S16).

<Face orientation determination>
FIG. 3 shows a specific example of the process of step S16.

  First, the face state determination unit 15 determines the face orientation based on the positional relationship of the facial organs (step S21). Specifically, the face state determination unit 15 outputs any one of “up”, “down”, “right”, “left”, and “front” as the determination result.

  When the determination result is “front” (step S22; YES), the process proceeds to step S17 as it is (without correcting the score). On the other hand, when the determination result is not “front” (step S22; NO), the score correction unit 18 corrects the partial score based on the orientation of the face (step S23), and counts (+1) the number of corrections (step S24). ), Go to step S17. This is because a face organ that does not appear much in the image may be generated depending on the face orientation, and the partial score calculated from the feature portion including the face organ is considered to be low in reliability. For example, when the determination result is “upper”, the right eye and the left eye are identified as facial organs that adversely affect attribute estimation. And the score correction | amendment part 18 reduces the weight of the partial score calculated from the characteristic part containing the right eye or the left eye. When the determination result is “lower”, the mouth is identified as a facial organ that adversely affects attribute estimation, and the weight of the partial score calculated from the feature including the mouth is lowered. When the determination result is “right”, the weight of the partial score calculated from the feature part including the right eye is lowered. When the determination result is “left”, the part calculated from the feature part including the left eye Reduce the score weight. In the present embodiment, “up”, “down”, “right”, “left”, and “front” are considered as the determination results of the face orientation. (An amount indicating how much the face direction deviates from the front direction) may be further output. In such a case, the amount by which the weight is decreased may be increased in proportion to the amount of deviation, or the weight may be decreased by a certain amount depending on whether or not the face direction is the front.

  Next, the face state determination unit 15 determines the shape of the facial organ (step S17).

<Face organ shape determination>
FIG. 4 shows a specific example of the process of step S17.

  First, the face state determination unit 15 determines the shape of the facial organ based on the positional relationship between the contour points (step S31). Specifically, the face state determination unit 15 detects a plurality of contour points based on the positions of feature points for each facial organ. Then, the shape of the facial organ is determined by comparing the shape formed by the plurality of contour points (the shape of the facial organ) with the shape of the facial organ in a normal state stored in advance. As the determination results, the face state determination unit 15 determines that “the right eye is closed (or thin)”, “the left eye is closed (or thin)”, “both eyes are closed (or thin)”, “the mouth is closed Either “rising (or falling)” or “normal state” is output.

When the determination result is “normal state” (step S32; YES), the process proceeds to step S18 as it is (without correcting the score). On the other hand, when the determination result is not “normal state” (step S32; NO), the score correction unit 18 corrects the partial score based on the shape of the facial organ (step S33), and counts the number of corrections (+1) ( Step S34), the process proceeds to step S18. The facial organ may be wrinkled due to the shape of the facial organ, and since the wrinkle is age information, the partial score calculated from the feature including the facial organ is considered to be unreliable. is there. For example, when the determination result is “the mouth is raised”, the mouth is identified as a facial organ that has an adverse effect on attribute estimation. And the score correction | amendment part 18 reduces the weight of the partial score calculated from the characteristic part containing a mouth. When the determination result is “both eyes closed”, the right eye and the left eye are identified as facial organs that have an adverse effect on attribute estimation, and the weight of the partial score calculated from the feature part including the right eye or the left eye is lowered. Similarly, when the determination result is “the right eye is closed”, the weight of the score calculated from the feature including the right eye is reduced, and when the determination result is “the left eye is closed”, the left eye The weight of the score calculated from the included feature is lowered. In the present embodiment, the determination result of the shape of the facial organ is “right eye is closed”, “left eye is closed”, “both eyes are closed”, “mouth is raised”, “normal state”. However, as a determination result, the number of wrinkles generated in the facial organ that adversely affects attribute estimation, the degree of deformation of the facial organ, and the like may be further output. In such a case, the amount of weight reduction may be increased in proportion to the number of wrinkles and the degree of deformation of the facial organs, or a certain amount depending on whether the facial organs are in a normal state or not. The weight may be lowered.

  Next, the face state determination unit 15 determines the illumination state of the face (step S18).

<Lighting state judgment>
FIG. 5 shows a specific example of the process of step S18.

  First, the face state determination unit 15 determines the illumination state of the face based on the brightness value of the facial organ (step S41). Specifically, the face state determination unit 15 calculates the luminance value of each facial organ by integrating the luminance values of feature points for each facial organ. Then, the illumination state of the face is determined by comparing the luminance value of each facial organ with the luminance value (reference luminance value) at the time of the forward light stored in advance. Further, the face state determination unit 15 determines whether the entire face is bright, the entire face is dark, or is obliquely illuminated by comparing the brightness values of the face organs with the brightness values of other face organs. As the determination result, the face state determination unit 15 determines that “upper (meaning that the lighting state of the upper part of the face is bad)”, “lower”, “right”, “left”, “entire (the lighting state of the entire face is One of “good” or “good” is output.

  When the determination result is “good” (step S42; YES), the process proceeds to step S19. When the determination result is not “good” (step S42; NO), it is determined whether or not the determination result is “whole”. (Step S43). If the determination result is “whole” (step S43; YES), it is difficult to estimate the attribute from the face, so the process of step S18 ends, and the process proceeds to step S20. When the determination result is not “whole” (step S43; NO), the score correction unit 18 corrects the partial score based on the illumination state of the face (step S44), counts the number of corrections (+1) (step S45), Proceed to step S19. Specifically, when there is a facial organ whose lighting state is too bright, the surroundings of the facial organ may be blown out and a feature amount including wrinkle information may not be obtained. In addition, when there is a dark facial illumination, there is a possibility that a shadow caused by the facial organ is erroneously detected as a wrinkle. Since the partial score calculated from the feature part including such a facial organ is considered to be low in reliability, the weight of the partial score is reduced. For example, when the determination result is “upper”, the right eye and the left eye are identified as facial organs that adversely affect attribute estimation. And the score correction | amendment part 18 reduces the weight of the partial score calculated from the characteristic part containing the right eye or the left eye. When the determination result is “lower”, the mouth is identified as a facial organ that adversely affects attribute estimation, and the weight of the partial score calculated from the feature including the mouth is lowered. When the determination result is “right”, the weight of the partial score calculated from the feature part including the right eye is lowered. When the determination result is “left”, the part calculated from the feature part including the left eye Reduce the score weight. In this embodiment, “upper”, “lower”, “right”, “left”, “whole”, and “good” are considered as the determination results of the facial illumination state, but the determination result is calculated. A deviation amount between the luminance value and the reference luminance value may be further output. In such a case, the amount by which the weight is decreased may be increased in proportion to the amount of deviation, or the weight may be decreased by a certain amount depending on whether the lighting condition of the face is bad.

  In step S19, the score correction unit 18 determines whether or not the number of corrections is three (that is, whether or not correction has been made for all states). When the number of corrections is less than 3 (step S19; NO), the process proceeds to step S21. When the number of corrections is 3 (step S19; YES), the face attribute estimation is stopped and the face is detected. (Step S20), the process proceeds to step S22.

  In step S21, the attribute estimation unit 19 regards the age group corresponding to the corrected score representing the highest probability among the corrected scores (integrated scores) for each age group as the attribute of the person.

  Next, in step S22, it is determined whether there is a face that is not used for attribute estimation among the faces detected in step S12. If it is determined that there is another face (step S22; YES), steps S13 to S21 are repeated until there is no other face. When the person attribute estimation process is performed for all the faces detected in step S12 (step S22; NO), the process proceeds to step S23.

  In step S <b> 23, the estimation result of the attribute estimation unit 19 is stored in the result storage unit 20. Information on the estimation result stored in the result storage unit 20 includes the image used for the person attribute estimation process, the shooting time of the image, the number of detected faces, the position coordinates of the face, and the number of detections for each attribute. Information is preferred. The user can confirm the person attribute estimation processing result on the display unit 21 by inputting the photographing time and the display method of the face detection processing result according to the purpose. The display method may be divided by age group, such as the number of detections of teenagers or less / total number of detected faces or the number of detected detections / total number of detected faces in the twenties, or captured at the photographing time. An image corresponding to the face may be displayed at the face detection position.

  As described above, in this embodiment, the score correction unit 18 corrects the partial score based on the face state, thereby correcting the integrated score used for attribute estimation. As a result, the adverse effect of the face state on the attribute estimation can be reduced, so that it is possible to reduce erroneous estimation and improve accuracy.

  As an example, a human attribute estimation process when the face 31 (FIG. 6) is selected in step S13 will be described. The person of the face 31 has an age group of “20s”, the face direction is “front”, the shape of the facial organ is “the mouth is raised”, and the illumination state of the face is “good”.

  First, feature amounts are extracted from a plurality of feature portions (the right eye feature portion 32a, the left eye feature portion 32b, the nose feature portion 32c, and the mouth feature portion 32d) identified by the feature amount extraction unit 14 (step S14). A score for each age group is calculated using the amount and the reference feature amount (step S15). In the example of FIG. 7, “10 years or younger, right eye; 0.2, left eye; 0.2, nose; 0.25, mouth; 0.02”, “20s, right eye; 0.3, left eye; 3, nose; 0.3, mouth; 0.03 "," 30s, right eye; 0.2, left eye; 0.2, nose; 0.17, mouth; 0.55 "," 40s, right eye; 0.15, left eye; 0.15, nose; 0.15, mouth; 0.2, “50s and above, right eye; 0.15, left eye; 0.15, nose; 0.13, mouth; As in “2”, a standardized value is calculated so that the total of the partial scores is 1 for each age group. Therefore, the integrated score is “10 or less; 0.1675”, “20s; 0.2325”, “30s; 0.28”, “40s; 0.1625”, “50s or more; 14 ". If the attribute is estimated at this time, the integrated score of “30's” is the highest, so the attribute of the person of the face 31 who is “20's” is erroneously estimated as “30's”. Therefore, in the present embodiment, the score correction unit 18 corrects the score as follows based on the face state.

As described above, the face 31 has a face orientation of “front”, a facial organ shape of “mouth is raised”, and a face illumination state of “good”. to correct. Since the shape of the facial organ of the face 31 is “the mouth is raised”, the weight of the partial score calculated from the feature portion (mouth feature portion 32d) including the mouth is lowered. Specifically, the partial score calculated from the mouth feature part 32d is set to 1/5. Therefore, the corrected partial score is “10 years or younger, right eye; 0.2, left eye; 0.2, nose; 0.25, mouth; 0.004”, “20s, right eye; 0.3, left eye. 0.3, nose; 0.3, mouth; 0.006 "," 30s, right eye; 0.2, left eye; 0.2, nose; 0.17, mouth; 0.1 "," 40s " , Right eye; 0.15, left eye; 0.15, nose; 0.15, mouth; 0.04, "50s and above, right eye; 0.15, left eye; 0.15, nose; 0.13, mouth 0.04 ”, and the integrated score after correction is“ 10 or less; 0.1635 ”,“ 20s; 0.2265 ”,“ 30s; 0.1675 ”,“ 40s; 0.1225 ” , "50s and above; 0.1175" (FIG. 7). As a result, the score of “20's” is the highest (because the number of corrections is one, so the process proceeds to step S21 (step S19; NO)), and the attribute of the person of the face 31 is correctly set to “20's” in step S21. Determined.

  In this embodiment, the feature amount includes the position of the feature point, the whole or a part of the color of the feature portion, the number of wrinkles, the number of wrinkles, and the like. Any amount may be used as long as it can obtain aging information such as. In addition, the necessary number of feature points, facial organs, and feature parts can be detected and specified, and the greater the number, the higher the accuracy of attribute estimation can be expected.

  In the present embodiment, the orientation of the face is determined based on the positional relationship of the facial organs, the shape of the facial organ is determined based on the positional relationship of the contour points, and the illumination state of the face is determined based on the luminance value of the facial organs. However, any method may be used as long as it can know the orientation of the face, the shape of the facial organ, and the illumination state of the face. For example, for the face orientation, a template of the position of the facial organ for each face orientation may be prepared. Further, the illumination state of the face may be determined based on the luminance value of the feature point instead of the luminance value of the facial organ. Further, the face state determination unit 15 of the present embodiment determines all of the face orientation, the shape of the facial organ, and the lighting state of the face, but one or more of these may be determined, Other states may be determined as long as they adversely affect attribute estimation.

  In addition, although the age group of this embodiment is defined in increments of 10 years, the width of the age group can be changed as necessary. For example, “5 years old or younger”, “5 to 10 years old”, “10 to 15 years old”, etc. may be set in increments of 5 years, or the width of the age group may be changed exponentially. Good.

  Note that the reference feature amount in the present embodiment is an average of the feature amounts of a plurality of persons belonging to the same age group, but the reference feature amount may be an age group of a plurality of persons itself, Any feature amount representing an age group may be used. In addition, the partial score is a similarity between the feature amount for each feature portion specified by the feature amount extraction unit 14 and the reference feature amount of the corresponding feature portion (the sum of the partial scores is 1 for each age group). However, the partial score may not be standardized, or may be calculated by a different method depending on the definition of the reference feature amount. For example, when the reference feature amount is an age group of a plurality of persons, the number of similar persons in the age group is counted for each feature part, and the number may be used as a partial score. In the present embodiment, the average value of the partial scores is the integrated score, but the integrated score may be a total value or may be defined by a function.

  In the present embodiment, four “right eye”, “left eye”, “nose”, and “mouth” are considered as the characteristic portions, but not limited to these, various portions can be specified. For this purpose, the feature quantity extraction unit 14 may store the correspondence between the feature point and the feature part to be specified in advance.

In this embodiment, the integrated score is corrected by making the weight of the partial score corresponding to the characteristic part that adversely affects the attribute estimation smaller than the weight of the other partial scores. The weight of the partial score of the highly characteristic feature portion may be increased, or the partial score corresponding to the feature portion that adversely affects the attribute estimation may be excluded. Also, the score may be corrected by changing the value of the partial score corresponding to the feature that adversely affects the attribute estimation (for example, the partial score is changed to the value of the partial score corresponding to the age group one level above) Or changing only the partial score of a predetermined age group), or a combination of the above correction methods.

  In this embodiment, a facial organ that has an adverse effect on attribute estimation is identified based on the face state, and the partial score calculated from the feature including the facial organ is corrected. You may specify the feature point and feature part which give a bad influence. In addition, a region having an adverse effect may be specified based on the specified facial organs. For example, when the detected face is facing left, the partial score calculated from the feature including the whole or part of the left half of the face may be corrected instead of the feature including the left eye.

  In the present embodiment, when the illumination state of all the feature portions is bad, or when all of the face direction, the shape of the facial organ, and the illumination state of the face are corrected, the face is excluded from the estimation target. However, it may be a case where two or more states are corrected among the orientation of the face, the shape of the facial organ, the illumination state of the face, or other states. In this way, when states that adversely affect attribute estimation overlap, it is possible to prevent an estimation result with low reliability from being output by prohibiting attribute estimation using the face image. In such a case (when the attribute cannot be determined from the face), the attribute may be estimated from a portion other than the face. For example, the attribute may be estimated from information such as clothes, silhouette, height and stride.

  In this embodiment, it is determined whether or not the attribute can be estimated from the face by counting the number of corrections, but not the number of corrections, only the state of the face is determined before correction, It may be determined whether the attribute can be estimated by counting the number of times that correction is required.

FIG. 1 is a block diagram showing a functional configuration of the person attribute estimation device. FIG. 2 is a flowchart showing a flow of processing of the person attribute estimation device. FIG. 3 is a flowchart showing the flow of processing in step S16. FIG. 4 is a flowchart showing the flow of processing in step S17. FIG. 5 is a flowchart showing the flow of processing in step S18. FIG. 6 is a diagram illustrating an example of the face detected by the face detection unit 13 and its characteristic part. FIG. 7 is a diagram illustrating an example of a score before correction and a score after correction.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image pick-up part 12 Image memory | storage part 13 Face detection part 14 Feature-value extraction part 15 Face state determination part 16 Reference feature-value memory | storage part 17 Score calculation part 18 Score correction part 19 Attribute estimation part 20 Result storage part 21 Display part 31 Face 32a- 32d feature

Claims (9)

A human attribute estimation device for estimating a human attribute from an image,
Face state determining means for determining the state of the face in the image;
For each of a plurality of predetermined age groups, score calculating means for calculating a score representing the probability that the face corresponds to the age group;
A score correction unit that identifies a part that adversely affects attribute estimation in the image based on the state of the face, and that corrects the score so that the influence of the part that adversely affects the attribute estimation is reduced ;
Attribute estimation means that regards the age group corresponding to the corrected score representing the highest probability among the corrected scores for each age group as the attribute of the person,
A person attribute estimation device comprising: The person attribute estimation apparatus according to claim 1, wherein the score is an integrated score obtained by integrating a plurality of partial scores respectively calculated from a plurality of parts in the image. The score correction means identifies a portion that adversely affects attribute estimation based on the face state, and corrects the integrated score by excluding a partial score corresponding to a portion that adversely affects the attribute estimation. 2. The person attribute estimation device according to 2. The score correction means identifies a portion that adversely affects attribute estimation based on the face state, and corrects the integrated score by changing a value of a partial score corresponding to the portion that adversely affects the attribute estimation. Item 3. The person attribute estimation device according to Item 2. The score is an integrated score weighted for each partial score,
The score correction means identifies a portion that adversely affects attribute estimation based on the face state, and sets the weight of the partial score corresponding to the portion that adversely affects the attribute estimation relative to the weight of other partial scores The person attribute estimation device according to claim 2, wherein the integrated score is corrected by reducing the integrated score.   The person attribute estimation device according to claim 1, wherein the face state includes one or more of a face orientation, a face organ shape, and a face illumination state. Attribute estimation by the attribute estimation unit is prohibited when two or more of the face orientation, facial organ shape, and face illumination state in the image have an adverse effect on attribute estimation. The person attribute estimation device according to claim 6. A computer that estimates human attributes from images
Determine the state of the face in the image,
For each of a plurality of predetermined age groups, a score representing the probability that the face corresponds to that age group is calculated,
Identifying a portion that adversely affects attribute estimation in the image based on the state of the face, correcting the score so that the influence of the portion that adversely affects the attribute estimation is reduced ,
A person attribute estimation method that regards an age group corresponding to a corrected score representing the highest probability among corrected scores for each age group as an attribute of the person. A computer that estimates human attributes from images
Determining the state of the face in the image;
For each of a plurality of predetermined age groups, calculating a score representing the probability that the face corresponds to that age group;
Identifying a portion that adversely affects attribute estimation in the image based on the state of the face, and correcting the score so that the influence of the portion that adversely affects the attribute estimation is reduced ;
Of the corrected scores for each age group, the age group corresponding to the corrected score representing the highest probability is regarded as the attribute of the person,
A person attribute estimation program for executing.

Images