JP2011128916A - Object detection apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect objects such as faces without increasing the number of identifying apparatuses. <P>SOLUTION: A plurality of identifying apparatuses different in face direction to be identified are used to discriminate faces. The identifying apparatuses comprise a plurality of weak identifying apparatuses. The identifying apparatuses are classified into a prestage weak identifying apparatus group WC-F for determining a face direction, and a poststage weak identifying apparatus group WC-B for identifying a face facing the direction that can be identified by each of the identifying apparatuses. The pre- and post-stage weak identifying apparatus groups WC-F and WC-B produce first and second scores, respectively. A face is detected according to the sum of products of the first and second scores in each identifying apparatus. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an object detection apparatus and method for detecting an object such as a human face from a detection target image, and a program for causing a computer to execute the object detection method.

  Conventionally, the color distribution of a person's face area in a snapshot photographed by a digital camera is examined to correct the skin color, or a person in a digital image photographed by a digital video camera of a surveillance system is recognized. Has been done. In such a case, since it is necessary to detect a face region corresponding to a person's face in the digital image, various techniques for detecting a face in the digital image have been proposed so far. Among them, a classifier that combines multiple weak classifiers (weak classifiers) generated by machine learning learning using sample images as a face detection method that is considered to have excellent detection accuracy and robustness. A method using this is known.

  This method uses a face sample image group composed of a plurality of different face sample images and a non-face sample image group composed of a plurality of different non-face sample images that are known not to be faces. A classifier that can learn whether or not an image is a face image is generated and prepared, and a partial image is detected in an image that is a face detection target (hereinafter referred to as a detection target image). A method of detecting a face on a detection target image by sequentially cutting out, determining whether or not the partial image is a face using the above discriminator, and extracting a region of the partial image determined to be a face It is.

  By the way, the image input to the discriminator described above includes not only an image with the face facing forward, but also an image in which the face is rotated on the image plane (hereinafter referred to as “in-plane rotation”) or a face. An image rotating in the plane (hereinafter referred to as “out-of-plane rotation”) is input. Here, the rotation range of the face that can be discriminated by one discriminator is limited, and the rotation is about 30 degrees for an in-plane rotated image and about 30 to 60 degrees for an out-of-plane rotated image. If there is, it can be determined whether it is a face or a non-face. For this reason, in order to support a wider range of face orientations, a plurality of discriminators capable of discriminating images in the respective orientations are prepared, and whether or not each discriminator has a face in a specific direction is determined. A multi-class discrimination method has been proposed in which discrimination is performed and whether a face is determined from the final output of each discriminator is determined.

  In the multi-class discrimination method, it is determined whether or not a face is included in the previous stage of a plurality of weak classifiers (weak classifier groups) constituting each classifier, and the classifier having the largest score is obtained. However, a technique for reducing processing for discrimination by determining whether or not the face is a face using a group of weak classifiers in the subsequent stage has been proposed (see Patent Document 1 and Non-Patent Document 1). Furthermore, a method has been proposed in which discrimination is performed for all discriminators, and the outputs of all discriminators are added to determine whether or not a face is included in the detection target image.

JP 2007-66010 A Paul Viola and Michael Jones, Rapid object detection using a boosted cascade of features, IEEE CVPR, 2001

  In the multi-class discrimination method described above, a plurality of discriminators having different discriminating face orientations learn the face images in the discriminating directions, so that the discriminating face is discriminated. It can be performed with high accuracy. However, if a discriminator is prepared so that a face in any orientation can be detected, it is necessary to configure the discriminator so that a single discriminator can discriminate a face in a wide angle. Specifically, when the angle of the face facing the front is 0 degree, the faces facing 0 degree ± 15 degrees, 30 degrees ± 15 degrees, 60 degrees ± 15 degrees, and 90 degrees ± 15 degrees to the left and right respectively Is prepared as a sample image for learning, a discriminator capable of discriminating a front face from a sample image of 0 ° ± 15 °, and a face facing 30 ° from left / right by a sample image of 30 ° ± 15 ° A discriminator capable of discriminating a face facing 60 degrees left and right from a sample image of 60 degrees ± 15 degrees, and a face discriminating from 90 degrees left and right from a sample image of 90 degrees ± 15 degrees It is necessary to learn each classifier.

  However, if a face having a wide angle range can be discriminated in this way, the number of weak discriminators in one discriminator becomes very large, and as a result, the face cannot be detected at high speed. Further, when the discriminator is configured in this way, faces whose directions are different in units of 30 degrees can be detected with high accuracy, but the detection accuracy of faces facing angles between corresponding angles (for example, 15 degrees and 45 degrees). Will still decline.

  The present invention has been made in view of the above circumstances, and an object thereof is to detect a specific type of object such as a face with high accuracy without increasing the number of discriminators.

An object detection apparatus according to the present invention includes a plurality of discriminators each having different discriminating directions of the object, each of which includes a plurality of weak discriminators in which feature amounts extracted from an object to be discriminated are learned in advance. In an object detection apparatus including a determination unit that detects the object from the detection target image using a feature amount extracted from the image,
A plurality of weak classifiers of each classifier is divided into a weak classifier group in the previous stage and a weak classifier group in the subsequent stage, and the weak classifier group in the previous stage is learned to determine the orientation of the object, The latter weak classifier group is learned to detect an object corresponding to the direction of the object that can be discriminated by the classifier to which each subsequent weak classifier belongs,
The discriminating unit obtains a first score that is an output of the preceding weak discriminator group and a second score that is an output of the subsequent weak discriminator, and the first score in each of the plurality of discriminators. It is a means for detecting the object on the basis of the sum of the product of the score of 1 and the second score for all the discriminators.

  In the object detection device according to the present invention, it is preferable that the weak classifier group in the previous stage shares the feature amount in at least a part of the plurality of classifiers.

  The weak classifier group has a structure in which a plurality of weak classifiers are linearly combined, and the weak classifier calculates at least one feature amount in the detection target image and uses this feature amount to determine an object. To do. For this reason, each weak discriminator learns to discriminate an object using feature amounts in a plurality of sample images. “A weak classifier group in the previous stage shares a feature quantity in at least a part of the plurality of classifier groups” means that corresponding weak classifiers among a plurality of classifiers use the same feature quantity. Means learning. Thus, the weak discriminator trained using the same feature amount discriminates an object using the same feature amount in the detection target image. Note that all of the corresponding weak classifiers among a plurality of classifier groups need not be learned with a common feature amount, and at least some weak classifiers have been learned with a common feature amount. That's fine.

  The weak classifiers are divided into a weak classifier group at the front stage and a weak classifier group at the rear stage. Especially when the weak classifier group at the front stage is shared with each classifier group. It is preferable that the number of weak classifier groups at the front stage is larger than the number of weak classifier groups at the rear stage.

  In the object detection apparatus according to the present invention, the preceding weak classifier group and the subsequent weak classifier group may be connected in series.

  Further, in the object detection device according to the present invention, the discriminator rotates the reference sample image in which the object faces a predetermined direction, and the discrimination target of the reference sample image on the plane of the reference sample image, Learning is performed using at least one of a plurality of in-plane rotation sample images with different rotation angles and a plurality of out-of-plane rotation sample images with different rotation angles obtained by rotating the direction of the discrimination target in the reference sample image. Good.

An object detection method according to the present invention includes a plurality of discriminators each having different discriminating directions of the object, each including a plurality of weak discriminators in which feature amounts extracted from an object to be discriminated are previously learned. In the object detection method for detecting the object from the detection target image using the feature amount extracted from the image,
A plurality of weak classifiers of each classifier is divided into a weak classifier group in the previous stage and a weak classifier group in the subsequent stage, and the weak classifier group in the previous stage is learned to determine the orientation of the object, The latter weak classifier group is learned to detect an object corresponding to the direction of the object that can be discriminated by the classifier to which each subsequent weak classifier belongs,
Obtaining a first score that is the output of the preceding weak classifier group, and a second score that is the output of the subsequent weak classifier;
The object is detected on the basis of the sum of the product of the first score and the second score in each of the plurality of classifiers for all the classifiers.

  In addition, you may provide as a program for making a computer perform the object detection method by this invention.

  According to the present invention, the first score indicating the direction of the object is acquired by the weak classifier group in the previous stage, and the object corresponds to the object direction that can be determined by each classifier by the weak classifier group in the subsequent stage. A second score representing is obtained. Then, an object is detected based on the sum of the multiplied values of the first score and the second score in each of the plurality of classifiers for all the classifiers. For this reason, it is possible to integrate the discrimination results obtained by a plurality of discriminators having different discriminating object orientations, and to detect the object. As a result, as in the methods of Patent Literature 1 and Non-Patent Literature 1 As compared with the case of detecting an object having only the direction of the object, an object having a different direction can be detected flexibly. Therefore, the object detection accuracy can be improved without increasing the number of weak classifiers.

  In addition, by making the weak classifier group in the previous stage share the feature amount in at least a part of the plurality of classifiers, one classification process in at least a part of the plurality of classifiers can be performed in one weak classifier. It will be possible. Therefore, in the plurality of classifiers, the number of weak classifier groups in the previous stage can be reduced, and as a result, the number of weak classifiers in the group of weak classifiers in the previous stage can be reduced.

  Further, the preceding weak classifier group and the subsequent weak classifier group are continuously connected to each other so that the former weak classifier group and the subsequent weak classifier group are continuously connected. Since this can be done, the processing speed can be improved.

The block diagram which shows the structure of the face detection system in this embodiment Schematic showing how the window is scanned Schematic block diagram showing the configuration of the face detection unit Schematic block diagram showing the configuration of the candidate detection unit Block diagram showing the configuration of the discriminator Diagram for explaining feature sharing Flow chart showing processing performed by classifier The flowchart which shows the flow of a process in the face detection system by this embodiment. Detailed detection process flowchart Figure showing an example of a face sample image Flow chart showing the learning method of the classifier The figure which shows the method of deriving the histogram of the weak classifier The figure which shows the example of the sample image discriminate | determined as a face by the weak classifier group of the front | former stage

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a face detection system to which the object detection device of the present invention is applied. This face detection system detects a face included in a digital image. As shown in FIG. 1, the face detection system 1 multi-resolutions a detection target image S0 that is a target for detecting a face to generate a plurality of images having different resolutions (hereinafter referred to as resolution images) 10. And a face detection unit 20 that detects an image (hereinafter referred to as a face image) F0 representing a face included in the detection target image S0.

  The multi-resolution conversion unit 10 converts the resolution (image size) of the detection target image S0 to normalize the resolution into an image having a predetermined resolution, for example, a VGA size (640 × 480 pixels) rectangular size. Then, the multi-resolution conversion unit 10 generates a plurality of resolution images S1 to S3... Having different resolutions as shown in FIG. 2 by performing resolution conversion based on the standardized detection target image S0. Note that the standardized detection target image S0 is also included in the resolution image.

  In the present embodiment, as shown in FIG. 2, a window W having a set number of pixels (for example, 32 pixels × 32 pixels) is scanned in a resolution image Sk (k = 0 to m), and the window A partial image B having a set number of pixels is generated by cutting out an area surrounded by W. As a result, even if the face (discrimination target) does not fit in the window W in the high-resolution resolution image, it can be placed in the window W on the low-resolution image, and detection of faces of various sizes is possible. Can be performed reliably.

  The face detection unit 20 performs face detection processing on each of a plurality of resolution images Sk (hereinafter referred to as resolution image group Sg) generated by the multi-resolution conversion unit 10, and detects a face image F0 in each resolution image Sk. To do. FIG. 3 is a schematic block diagram showing the configuration of the face detection unit 20. As shown in FIG. 3, the face detection unit 20 controls each unit described later to mainly perform sequence control in the face detection process, and a resolution image used for the face detection process from the resolution image group Sg. A resolution image selection unit 22 that sequentially selects Sk in descending order of size, and a window that cuts out a partial image B that is a determination target of whether or not it is a face image in the resolution image Sk selected by the resolution image selection unit 22 A window setting unit 23 that sequentially sets W while shifting its position, a candidate determination unit 24 that determines whether or not the cut out partial image B is a face image, and a portion that is determined to be a face image The determination unit 25 further determines whether or not an image (hereinafter referred to as a candidate image CP) is a face image.

  The detection control unit 21 controls the resolution image selection unit 22 and the window setting unit 23 so as to perform face detection processing for detecting the face image F0 for each resolution image Sk in the resolution image group Sg. For example, the resolution image selection unit 22 is appropriately instructed to select the resolution image Sk, the window setting unit 23 is instructed about the setting condition of the window W, and the obtained detection result is output. Note that the window setting condition includes a range on the image where the window W is set, a movement interval (detection roughness) of the window W, and the like.

  Under the control of the detection control unit 21, the resolution image selection unit 22 sequentially selects resolution images Sk to be subjected to face detection processing from the resolution image group Sg in descending order of size (in order of fine resolution). Note that the face detection method in this embodiment determines whether or not the partial image B is a face image for the partial images B sequentially cut out on each resolution image, and the partial image that is determined to be a face image. Since this is a technique for detecting the face image in the detection target image S0 by extracting the area B, the resolution image selection unit 22 sets the size of the face to be detected in the detection target image S0 while changing each time. It can be said that it is equivalent to setting the face to be detected while changing the size of the face from small to large.

  The window setting unit 23 sequentially sets the window W while moving it on the resolution image Sk selected by the resolution image selection unit 22 based on the window setting condition set by the detection control unit 21.

  The candidate discriminating unit 24 has a function of performing binary discrimination as to whether or not the partial image B is a face image, and includes a candidate discriminator 30 having a plurality of weak discriminators WC as shown in FIG. Here, the candidate discriminator 30 has both an in-plane rotated image in which the discrimination target is rotated on the image plane and an out-of-plane rotated image in which the orientation of the discrimination target in the image is rotated as a face. It is to be determined.

  The candidate discriminator 30 has a cascade structure in which a plurality of weak discriminators WC are linearly coupled, and the weak discriminator WC obtains at least one feature amount related to the distribution of pixel values (luminance) of the partial image B. By calculating, a feature amount is extracted from the partial image B, and using this feature amount, it is determined whether or not the partial image B is a face image. The candidate discriminator 24 discriminates whether or not it is a face image using the discrimination result in the weak discriminator WC. In the present embodiment, a determination result CR as to whether or not the partial image B is a face image is output based on the total score for determination calculated by each weak classifier WC.

  The determination unit 25 determines whether or not the partial image B determined as a face image, that is, the candidate image CP is a face image when the candidate determination unit 24 determines that the partial image B is a face image. It is a discriminator that further discriminates. FIG. 5 is a diagram illustrating a configuration of the determination unit 25. As illustrated in FIG. 5, the determination unit 34 includes n classes of determination devices 25-1 to 25-n having different recognizable face directions, and a determination result output unit 25 -L. Here, the n class discriminator 25-i (i = 1 to n) includes an in-plane rotation discriminator capable of discriminating the face orientation on the image plane and an out-of-plane discriminator capable of discriminating the face orientation in the image. A rotation discriminator. The in-plane rotation discriminator is a 0 degree in-plane rotation discriminator capable of discriminating a face whose angle between the vertical direction of the image and the face center line is 0 degree, and a 30 degree in-plane rotation capable of discriminating a face of 30 degrees. For example, the discriminator includes a plurality of discriminators capable of discriminating faces having different in-plane rotation angles by 30 degrees within a range of 30 degrees to 330 degrees, for example. For example, the 0-degree in-plane rotation discriminator can discriminate a face whose rotation angle is in the range of −15 degrees (= 345 degrees) to +15 degrees centering on 0 degrees.

  Similarly, the out-of-plane rotation discriminator can discriminate a face whose orientation (angle) of the face in the image is 0 degrees, that is, a 0-degree out-of-plane rotation discriminator capable of discriminating a front face and a 30-degree face. It consists of a discriminator capable of discriminating faces with different out-of-plane rotation angles by 30 degrees in the range of −90 degrees to +90 degrees, for example. For example, the 0-degree out-of-plane rotation discriminator can discriminate a face whose rotation angle is in the range of -15 degrees to +15 degrees with 0 degrees as the center.

  Each discriminator 25-i has a cascade structure in which a plurality of weak discriminators WC are linearly coupled as shown in FIG. 5, and the weak discriminator WC has a pixel value (luminance) of the candidate image CP. ) Is calculated, and it is determined whether or not the candidate image CP is a face image using the feature amount.

  The plurality of weak classifiers WC included in each classifier 25-i are divided into a preceding weak classifier group WC-F and a subsequent weak classifier group WC-B. The former weak classifier group WC-F is for discriminating the in-plane and out-of-plane directions of the candidate image CP, and the latter weak classifier group WC-B belongs to the weak classifier group WC. The discriminator 25-i is for discriminating whether the face corresponds to the face orientation that can be discriminated. For this reason, in the present embodiment, the sample images used for learning are slightly different between the weak classifier group WC-F at the front stage and the weak classifier group WC-B at the rear stage. Note that the weak classifier learning will be described later. In this embodiment, the score finally obtained in the weak classifier group WC-F at the preceding stage of all the classifiers 25-i is output, and the weak classifiers at the subsequent stage of all the classifiers 25-i. The score finally obtained in the group WC-B is output. Note that the score output from the weak classifier group WC-F in the previous stage is referred to as a first score, and the score output from the weak classifier group WC-B in the subsequent stage is referred to as a second score.

  Note that at least some of the weak classifiers WC included in the preceding weak classifier group WC-F configuring each classifier 25-i share the feature quantity among the classifiers 25-i. That is, at least some weak classifiers WC have learned using the same feature amount, and determine the face orientation using the same feature amount in the candidate image CP. In FIG. 5, it is assumed that the sharing of the feature quantity is indicated by hatching the weak classifier WC.

  Here, in FIG. 5, for the sake of explanation, the weak discriminator WC sharing the feature quantity of each discriminator 25-i is shown by hatching, but in each discriminator 25-i, One weak classifier that shares the quantity may be created. FIG. 6 is a diagram for explaining feature amount sharing. In FIG. 6, for the sake of explanation, only four discriminators 25-1 to 25-4 are shown, and further, only four weak discriminators WC preceding each discriminator 25-1 to 25-4 are shown. ing. The first to fourth weak classifiers in the classifiers 25-1 to 25-4 are shown as weak classifiers 25-1-1 and 25-1-2, respectively.

  As shown in the upper side of FIG. 6, the first-stage weak discriminator shares the feature quantity among all the discriminators 25-1 to 25-4, and the second-stage weak discriminator is the weak discriminator 25-1-. 2, 25-2-2, 25-3-2 share the feature value, and the third-stage weak classifier shares the feature value between the weak classifiers 25-1-3 and 25-4-3, It is assumed that the weak discriminator at the fourth stage shares the feature quantity among the weak discriminators 25-2-4, 25-3-4, and 25-3-4. In this case, since only one classifier needs to be created as the weak classifier sharing the feature amount, the weak classifiers in the first to fourth stages in the classifiers 25-1 to 25-4 are shown in FIG. It will be combined as shown below. Therefore, the number of weak classifiers can be reduced from 16 to 8.

  Next, specific processing in the determination unit 25 will be described. FIG. 7 is a flowchart showing processing performed by each discriminator 25-i included in the discriminator 25. In the following description, the processing in each discriminator 25-i is performed in parallel. Of course, the processing in each discriminator 25-i may be performed sequentially.

  First, in the previous weak classifier group WC-F in each classifier 25-i, the first weak classifier WC determines the face direction in the candidate image CP with respect to the candidate image CP. A feature amount is calculated from the image CP (step ST1), a score is calculated with reference to a score table to be described later according to the feature amount (step ST2), and the score calculated by the weak classifier immediately before is calculated by itself. Is added to calculate a cumulative score (step ST3). In the first weak classifier, since there is no previous weak classifier, the self-calculated score is used as the cumulative score as it is. Next, it is determined whether or not the cumulative score for all of the weak classifier groups WC-F in the previous stage has been calculated (step ST4). If step ST4 is negative, the process proceeds to determination by the next weak classifier ( Step ST5) and return to step ST1. Thereby, the cumulative score for all weak classifiers constituting the weak classifier group WC-F in the previous stage is calculated. If step ST5 is affirmed, the accumulated score is output as the first score P1-i (i = 1 to n) (step ST6).

  Subsequently, the process proceeds to the processing by the weak classifier group WC-B in the subsequent stage. Subsequent to step ST6, the first weak classifier WC in the subsequent weak classifier group WC-B calculates a feature quantity from the candidate image CP (step ST7), and refers to the score table according to the feature quantity to obtain a score. Is calculated (step ST8), and the score calculated by itself is added to the score calculated by the previous weak discriminator to calculate the cumulative score (step ST9). In the first weak classifier in the subsequent weak classifier group WC-B, since there is no previous weak classifier, the self-calculated score is added to the first score P1-i to obtain a cumulative score. Next, it is determined whether or not the cumulative score for all of the subsequent weak classifier groups WC-B has been calculated (step ST10). If step ST10 is negative, the process proceeds to determination by the next weak classifier ( Step ST11) and return to step ST7. Thereby, the cumulative score for all weak classifiers constituting the weak classifier group WC-F in the previous stage is calculated. If step ST10 is affirmed, the cumulative score for the subsequent weak classifier group WC-B is output as the second score P2-i (i = 1 to n) (step ST12).

Next, the discrimination result output unit 25-L multiplies the first score P1-i and the second score P2-i, adds the multiplication results for all the discriminators 25-i, and performs the candidate image CP. The final score PL is calculated (step ST13). The final score PL is calculated by the following formula (1). Note that Σ represents adding the values of (P1-i) × (P2-i) from i = 1 to n. PL = Σ (P1-i) × (P2-i) (1)
Then, the determination result output unit 25-L determines whether or not the candidate image CP is a face image depending on whether or not the final score PL is greater than or equal to a predetermined threshold value, and outputs the determination result R (step ST14). .

  In the present embodiment, the detection control unit 21, the resolution image selection unit 22, the window setting unit 23, the candidate determination unit 24, and the determination unit 25 function as the determination unit of the present invention.

  Next, the flow of processing in the face detection system 1 will be described. FIG. 8 is a flowchart showing the flow of processing in the face detection system according to this embodiment. As shown in FIG. 8, when the detection target image S0 is input to the multi-resolution converting unit 10 (step ST21), the multi-resolution converting unit 10 multi-resolutions the detection target image S0 to obtain a resolution composed of a plurality of resolution images Sk. An image group Sg is generated (step ST22). In response to the instruction from the detection control unit 21, the face detection unit 20 uses the resolution image selection unit 22 to select resolution images Sk from the resolution image group Sg in descending order of image size (step ST23). Next, the detection control unit 21 instructs the window setting unit 23 to set the window W at the initial position, that is, the first target pixel on the selected resolution image (step ST24). The window setting unit 23 sets a window W on the selected resolution image, cuts out the partial image B using the set window W (step ST25), and inputs the partial image B to the candidate determination unit 24 (step ST26).

  The candidate determination unit 24 determines whether or not the partial image B is a face image with respect to the input partial image B, and the detection control unit 21 acquires the determination result CR (step ST27). It is determined whether or not the result CR is that the partial image B is a face image (step ST28). If the determination result CR is that the partial image B is not a face image (No in step ST28), the detection control unit 21 determines that the currently extracted partial image B is the partial image located at the last pixel of interest, that is, the last image It is determined whether or not it is a partial image (step ST29), and if it is determined that the partial image B is not the last partial image, the position where the window W is set is set to the position of the next pixel of interest (that is, the next position). ) (Step ST30), the process returns to step ST25, and the window setting unit 23 cuts out a new partial image B.

  When it is determined that the partial image B is the last partial image, the detection control unit 21 is the image in which the currently selected resolution image Sk is determined last, that is, the last resolution image Sm. Whether or not it is the last resolution image Sm, the detection process is terminated and the detection result is output (step ST32). On the other hand, if it is determined that the resolution image is not the last resolution image, the process returns to step ST23, and the resolution image selection unit 22 selects a resolution image that is one step smaller than the currently selected resolution image. Detection is performed.

  On the other hand, if the determination result CR is that the partial image B is a face image, the candidate determination unit 24 determines the partial image B as the candidate image CP and performs more detailed detection processing (step ST33). FIG. 9 is a flowchart of detailed detection processing. In the detailed detection process, since the candidate determination unit 24 determines that the partial image B is the candidate image CP, the detection control unit 21 inputs the candidate image CP to the determination unit 25 (step ST41). The determination unit 25 determines whether the input candidate image CP is a face image, the detection control unit 21 acquires the determination result R (step ST42), and the process proceeds to step ST29 of the flowchart shown in FIG. move on. By performing the above processing, an image including a face facing in various directions can be detected from the detection target image S0.

  The detection result is output when a face cannot be detected from the detection target image S0. When a face is detected in the detection target image S0, the face on the detection target image S0 is output. The coordinates of the position of the detected partial image are output.

  Next, a learning method (generation method) of the discriminator will be described. Note that learning is performed for each type of discriminator, that is, for each orientation of the face to be discriminated.

  The sample image group to be learned is standardized by the size of the window W, a plurality of sample images (face sample image group) known to be faces and a plurality of samples known to be non-faces It consists of an image (non-face sample image group).

  The face sample image has a face orientation corresponding to the number of classes of the discriminator 25-i. Specifically, as shown in FIG. 10A, an in-plane sample image consisting of 12 types of images in which the face arranged at the set position (for example, the center) is rotated by 30 °, and FIG. As shown in the figure, it consists of out-of-plane rotated sample images consisting of seven types of images in which the orientation of the face arranged at the set position (for example, the center) is rotated by ± 30 °. Each face sample image has a standardized face position and size.

  Then, using such face sample image group and non-face sample image group, learning of the discriminator 25-i is performed for each face direction, and 19 types of discriminators are generated. Hereinafter, a specific learning method will be described.

  FIG. 11 is a flowchart showing a learning method of the classifier. In the present embodiment, the weak classifier group WC-F in the previous stage shares the feature quantity among the classifiers 25-i, and which feature quantity is shared is determined appropriately by the user at the time of learning. You just have to choose. First, a weight or importance is assigned to each sample image. First, the initial value of the weight of all sample images is set equal to 1 (step ST51). Next, a feature amount is acquired from the sample image, and a weak semi-separator is created for the feature amount (step ST52). As the feature amount, a difference value between pixel values (luminance values) between two predetermined points in the sample image can be used. In the present embodiment, the histogram for the feature quantity is used as the basis of the score table of the weak classifier.

  The creation of a weak classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 12, the feature quantities for creating this weak classifier are in the point O1 at the center of the right eye on the sample image and the cheek portion on the right side in the face sample image. The difference value of the pixel value at the point O2 is used. Note that the coordinate position for obtaining a feature value for creating a weak classifier is the same in all sample images. Then, a feature amount is obtained for the face sample image, and a histogram thereof is created. Here, the value that can be taken by the feature amount depends on the number of luminance gradations of the image, but if it is a 16-bit gradation, there are 65536 different values for the difference value of one pixel value. A large number of samples, time and memory are required for detection. For this reason, in the present embodiment, the feature quantity is divided and quantized by an appropriate numerical value width and converted into an n-value (for example, n = 100). As a result, the number of combinations of feature amounts is n, so the number of data representing feature amounts can be reduced.

  Similarly, a histogram is also created for the non-face sample image. For the non-face sample image, a pixel value at a position corresponding to a pixel for acquiring a feature amount on the face sample image is used. A histogram obtained by taking logarithm values of the ratios of the frequency values indicated by these two histograms and representing the histogram is shown on the rightmost side of FIG. 12 and used as the basis of the score table of the weak classifier. The value of each vertical axis indicated by the histogram of the weak classifier is hereinafter referred to as a score. According to this weak classifier, an image showing the distribution of the combination of feature amounts corresponding to a positive score is highly likely to be a face, and it can be said that the possibility increases as the absolute value of the score increases. Conversely, an image showing the distribution of the combination of feature amounts corresponding to a negative score is highly likely not to be a face, and the possibility increases as the absolute value of the score increases. In step ST52, a plurality of weak classifiers in the above-described histogram format are created for combinations of feature quantities that can be used for discrimination.

  Subsequently, the weak classifier that is most effective for determining whether or not the image is a face facing a specific direction is selected from the plurality of weak half-classifiers created in step ST52. The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak discriminators are compared, and the weak discriminator showing the highest weighted correct answer rate is selected (step ST53). That is, in the first step ST53, since the weight of each sample image is equal to 1, the number of sample images for which the weak discriminator can simply determine whether the image is a face facing a specific direction is simply determined. The most numerous are selected as the most effective weak classifiers. On the other hand, in the second step ST53 after the weight of each sample image is updated in step ST55, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step ST53 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the combination of the weak classifiers selected so far, that is, using the weak classifiers selected so far (in the learning stage, the weak classifiers do not necessarily have to be linearly combined) ) Whether the result of determining whether or not each sample image is a face image facing in a specific direction has exceeded a predetermined threshold, the rate at which it matches the answer of whether or not it is actually a face image Is confirmed (step ST54). Here, the current weighted sample image group or the sample image group with equal weight may be used for evaluating the correct answer rate of the combination of weak classifiers. When the predetermined threshold value is exceeded, learning is completed because it is possible to determine whether the image is a face with a sufficiently high score using the weak classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step ST56 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far. In step ST56, the weak discriminator selected in the latest step ST53 is excluded so as not to be selected again.

  Next, the weak discriminator selected in the most recent step ST53 cannot correctly determine whether the face is in a specific direction or not, the weight of the sample image is increased, and whether or not the image is a face is correct. The weight of the sample image that can be discriminated is reduced (step ST55). The reason for increasing or decreasing the weight in this way is that in the selection of the next weak classifier, importance is placed on images that could not be correctly determined by the already selected weak classifier, and whether or not those images are faces is correct. This is because a weak discriminator that can be discriminated is selected to enhance the effect of the combination of the weak discriminators. Subsequently, the process returns to step ST53, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  Steps ST53 to S56 are repeated, and when a weak discriminator corresponding to a combination of feature amounts is selected as a weak discriminator suitable for discriminating whether or not the face is in a specific direction, the step If the correct answer rate confirmed in ST54 exceeds the threshold, the type of weak discriminator used for discriminating whether or not the face is in a specific direction and the discriminating condition are determined (step ST57), thereby learning. finish. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to constitute one classifier. For each weak classifier, a score table for calculating a score according to the combination of feature amounts is generated based on the obtained histogram. Note that the histogram itself can also be used as a score table. In this case, the discrimination point of the histogram is directly used as a score. In this manner, the 19 kinds of discriminators described above are generated by performing learning for each face sample image group.

  Here, in the present embodiment, the weak classifier group WC-F in the previous stage is for determining the face orientation of the candidate image CP, and the weak classifier group WC-B in the subsequent stage is the weak classifier. This is for discriminating whether or not the discriminator 25-i to which the vessel group WC belongs is a face corresponding to the face orientation that can be discriminated. For this reason, in this embodiment, when a predetermined number of weak classifiers are selected, the weak classifier group WC-F in the previous stage is configured from the weak classifiers selected so far. Then, using the weak classifier group WC-F in the previous stage, face discrimination is performed for the face sample image group and the non-face sample image group. Here, since only the weak classifier group WC-F in the previous stage is in the middle of learning by the classifier 25-i, a face in a specific direction cannot be accurately identified, so that it may be mistaken for a face in a specific direction. In some cases, a non-face sample image is identified as a face in a specific direction.

  FIG. 13 is a diagram illustrating an example of a sample image that is determined to be a face by the weak classifier group WC-F in the previous stage in the classifier capable of discriminating a face facing the front. Among the four sample images shown in FIG. 13, it is clear that the sample images SP1 to SP3 are faces facing the front, but the sample image SP4 is clearly not a face. For this reason, in the present embodiment, among the sample images determined to be faces in a specific direction by the weak classifier group WC-F in the previous stage, a sample image that is clearly not a face is used as a non-face sample image. It is not used for learning of the classifier group WC-B. In this way, after selecting a sample image to be used for learning, the weak classifiers constituting the subsequent weak classifier group WC-B are learned following the weak classifier group WC-F in the previous stage. Note that, at the time of learning the subsequent weak classifier group WC-B, the threshold value used in step ST54 is set higher than the threshold value used for learning the previous weak classifier group WC-F, so that The discriminator group WC-B can perform discrimination with high accuracy.

  In the case of adopting the above learning method, if the weak discriminator provides a reference for discriminating between a face image and a non-face image using a combination of feature amounts, The data is not limited to any data, and may be any data, for example, binary data, a threshold value, a function, or the like. Further, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 12 may be used. Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  Thus, according to the present embodiment, the first score P1-i representing the face direction is acquired by the weak classifier group WC-F in the previous stage, and each classifier is acquired by the weak classifier group WC-B in the subsequent stage. A second score P2-i indicating that the face corresponds to the face orientation that can be identified by 25-i is acquired. Then, the face is detected based on the sum of all the discriminators 25-i of the multiplication values of the first score P1-i and the second score P2-i in each of the discriminators 25-i. It is a thing. For this reason, it is possible to detect the face by integrating the discrimination results by the plurality of discriminators 25-i having different face directions that can be discriminated. Therefore, as in the methods of Patent Literature 1 and Non-Patent Literature 1. Compared with detecting a face only in a specific direction, a face with a different orientation can be detected flexibly. Therefore, face detection accuracy can be improved without increasing the number of weak classifiers.

  In addition, the weak classifier group WC-F in the previous stage shares a feature amount in at least a part of the plurality of classifiers 25-i, thereby performing a plurality of discrimination processes in at least a part of the plurality of classifiers 25-i. This can be performed in one weak classifier WC. Therefore, the number of weak classifier groups WC-F in the previous stage can be reduced, and as a result, the number of weak classifiers WC in the weak classifier group WC-F in the previous stage can be reduced.

In the above embodiment, the detection target is a human face, but other objects such as a human hand may be detected. In this case, the discriminator may perform learning using a sample image group including an object and a sample image group including no object. As described above, the face detection system according to the embodiment of the present invention has been described. A program for causing a computer to execute each process in a portion corresponding to the object detection device of the present invention is also an embodiment of the present invention. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

DESCRIPTION OF SYMBOLS 1 Face detection system 10 Multi-resolution part 20 Face detection part 21 Detection control part 22 Resolution image selection part 23 Window setting part 24 Candidate discrimination | determination part 25 Discrimination part

Claims (6)

A plurality of weak classifiers that have previously learned feature quantities extracted from objects to be discriminated, each having a plurality of discriminators having different orientations of the distinguishable objects, and using feature quantities extracted from detection target images In the object detection apparatus provided with a determination unit for detecting the object from the detection target image,
A plurality of weak classifiers of each classifier is divided into a weak classifier group in the previous stage and a weak classifier group in the subsequent stage, and the weak classifier group in the previous stage is learned to determine the orientation of the object, The latter weak classifier group is learned to detect an object corresponding to the orientation of the object that can be discriminated by the classifier to which each subsequent weak classifier belongs,
The discriminating unit obtains a first score that is an output of the preceding weak discriminator group and a second score that is an output of the subsequent weak discriminator, and the first score in each of the plurality of discriminators. An object detection apparatus, comprising: means for detecting the object based on a sum of multiplication values of the score of 1 and the second score for all the discriminators.   The object detection apparatus according to claim 1, wherein the weak classifier group in the previous stage shares the feature quantity in at least a part of the plurality of classifiers.   3. The object detection apparatus according to claim 1, wherein the preceding weak classifier group and the subsequent weak classifier group are connected in series.   A reference sample image in which the object is directed in a predetermined direction, and a plurality of in-plane rotation sample images having different rotation angles obtained by rotating the determination target of the reference sample image in the plane of the reference sample image And learning using at least one of a plurality of out-of-plane rotated sample images with different rotation angles obtained by rotating the direction of the discrimination target in the reference sample image. 4. The object detection device according to any one of items 3. A plurality of weak classifiers that have previously learned feature quantities extracted from objects to be discriminated, each having a plurality of discriminators having different orientations of the distinguishable objects, and using feature quantities extracted from detection target images In the object detection method for detecting the object from the detection target image,
A plurality of weak classifiers of each classifier is divided into a weak classifier group in the previous stage and a weak classifier group in the subsequent stage, and the weak classifier group in the previous stage is learned to determine the orientation of the object, The latter weak classifier group is learned to detect an object corresponding to the direction of the object that can be discriminated by the classifier to which each subsequent weak classifier belongs,
Obtaining a first score that is the output of the preceding weak classifier group and a second score that is the output of the subsequent weak classifier;
An object detection method, comprising: detecting the object based on a sum of multiplication values of the first score and the second score in each of the plurality of classifiers for all the classifiers. A plurality of weak classifiers that have previously learned feature quantities extracted from objects to be discriminated, each having a plurality of discriminators having different orientations of the distinguishable objects, and using feature quantities extracted from detection target images In a program for causing a computer to execute an object detection method for detecting the object from the detection target image,
A plurality of weak classifiers of each classifier is divided into a weak classifier group in the previous stage and a weak classifier group in the subsequent stage, and the weak classifier group in the previous stage is learned to determine the orientation of the object, The latter weak classifier group is learned to detect an object corresponding to the direction of the object that can be discriminated by the classifier to which each subsequent weak classifier belongs,
The program acquires a first score that is an output of the preceding weak classifier group, and a second score that is an output of the subsequent weak classifier;
Causing the computer to execute a procedure for detecting the object based on a sum of the product of the first score and the second score of each of the plurality of classifiers for all the classifiers. A featured program.

Images