JP2007312249A - Unit and method for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a technique for improving an object detection rate, while reducing incorrect detection. <P>SOLUTION: An image processing unit for detecting an object from a photographed image includes an acquisition means for acquiring photography information at the time of photography; a modification means for modifying the detection condition of the above object by a detection means corresponding to the above photography information; and the detection means for detecting the object from the above feature information by extracting feature information of the object from the photographed image using the above photography condition. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a subject detection technique in an image.

  Conventionally, in the field of image recognition and speech recognition, a recognition target is detected by realizing a recognition processing algorithm specialized for a specific recognition target by computer software or hardware using a dedicated parallel image processor. Things are known.

  In particular, there are some known methods for detecting a face as a specific recognition target from an image including the face.

  For example, a technique for detecting a subject from a matching result between shape data of each component of a face and an input image is disclosed (see Patent Document 1). In this technique, shape data such as eyes, mouth, and nose are prepared, and first two eyes are obtained. Subsequently, when obtaining the mouth, nose, etc., the search area for the facial parts such as the mouth, nose, etc. is limited based on the position of the eyes. In other words, this algorithm does not detect face parts that make up the face such as eyes, mouth, and nose in parallel, but first finds the eyes and uses the results to detect face parts such as mouth and nose in order. ing.

In addition, a camera having a function of detecting an area corresponding to a human face from images obtained for a monitor by image processing has been proposed. According to this camera, regardless of the position of the person's face in the captured image, it is possible to focus on the person's face and take a picture with an optimum exposure for the person's face.
Japanese Patent Laying-Open No. 2005-56387 (page 32, FIG. 9)

  However, in the conventional method described above, a face may not be detected from an image depending on the situation at the time of shooting. For example, in an image when a flash is emitted during photographing, a person's eyes may be thinner than normal due to a flashing light, and may not be recognized as eyes.

  In addition, there are scenes where the face is not facing the camera when shooting movies or continuous shooting, in which case the positional relationship between the eyes and mouth changes compared to when the face is facing the camera. So it may not be recognized as a face.

  In addition, in a camera that performs focus and exposure control based on the position information of the detected face information, if not only the main subject but also surrounding people are detected, the focus and exposure may be adjusted for the wrong person instead of the main subject. is there.

  In order to solve the above-described problems, the present invention provides an image processing apparatus that detects a subject from a captured image, an acquisition unit that acquires shooting information at the time of shooting, and an object that is detected by the detection unit according to the shooting information. Changing means for changing the detection condition; and detecting means for extracting feature information of the subject from the photographed image based on the shooting condition and detecting the subject from the feature information.

  The present invention is also an image processing method for detecting a subject from a captured image, the step of acquiring shooting information at the time of shooting, the step of changing the detection condition of the subject according to the shooting information, and the detection condition To detect feature information of the subject from the photographed image and detect the subject from the feature information.

  According to the present invention, it is possible to improve the detection rate and reduce false detection by selecting a detection condition according to the state of the subject predicted from the photographing information and detecting the subject.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

[First Embodiment]
FIG. 1 is a block diagram showing main components of an imaging apparatus according to a first embodiment in which an image processing apparatus according to the present invention is mounted.

  In FIG. 1, a microcomputer (hereinafter referred to as CPU) 100 includes a signal input circuit 104 for detecting various operation switch groups 114 of the camera, an image sensor 106, a photometric sensor 107, a focus detection sensor 108, a strobe. 109, a display unit 110, an external memory 102, a video signal processing circuit 103, and an image processing circuit 101 are connected.

  Further, the CPU 100 exchanges a lens control signal 115 with a photographing lens (not shown) via the lens communication circuit 105 to control the focal position and the diaphragm.

  The CPU 100 includes a ROM 111 storing a program for controlling the operation of the entire apparatus, a RAM 112 for temporarily storing variables, and an EEPROM 113 for storing various parameters.

  The setting of various camera modes is determined by the setting of the switch group 114. The imaging apparatus according to the present embodiment continuously shoots a moving subject, an auto mode in which general photographing is performed on various subjects, a portrait mode programmed to be suitable for photographing a person, and a moving subject. It has shooting modes such as a continuous shooting mode to be performed, an exposure correction shooting mode in which the exposure can be taken more or less than usual, and a moving image shooting mode.

  The CPU 100 controls the photometry sensor 107 and the focus detection sensor 108 to detect the focus state and brightness of the subject, and determine the focus lens position, aperture value, and exposure time of a photographing lens (not shown). It is also determined whether or not the flash 109 needs to emit light.

  The aperture value is controlled via the lens communication circuit 105, and the photographing operation is performed by controlling the strobe 109 and the image sensor 106.

  The image signal captured by the image sensor 106 is subjected to signal processing in the video signal processing circuit 103 and stored in the external memory 102. Here, the signal processing includes noise removal processing, white balance processing, compression encoding processing, decoding processing, and the like.

  The display unit 110 displays an image subjected to signal processing by the video signal processing circuit 103, various menus, and warnings and instructions for the user.

  The image processing circuit 101 includes a subject detection block 116, a detection parameter selection block 117, a detection parameter group 118, a determination block 119, a determination level determination block 120, and a shooting information input block 121. The image processing circuit 101 detects position information corresponding to a person's face, facial feature information, and the like from the image signal output from the video signal processing circuit 103.

  The shooting information input block 121 outputs shooting information such as various camera modes set by the setting of the switch group 114 and the presence or absence of strobe light emission to the detection parameter selection block 117 and the determination level determination block 120.

  The detection parameter selection block 117 selects at least one or more detection parameters from a plurality of detection parameter groups 118 prepared in advance according to the shooting information input from the shooting information input block 121. The detection parameter group 118 is information for determining a person such as the shape of the eyes, the positional relationship between the eyes and the mouth, and the skin color.

  The determination level determination block 120 determines a determination level that is a determination criterion for a person's face according to the shooting information input from the shooting information input block 121.

  In the subject detection block 116, local features (here, eyes and mouth) are extracted from the image, and the image is evaluated with respect to the detection parameter selected in the detection parameter selection block 117. Feature information and the like are output to the CPU 100.

  The determination block 119 determines whether or not a person's face exists by comparing the evaluation result obtained by evaluating the image in the subject detection block 116 with the determination level determined by the determination level determination block 120.

  Here, the detection parameter group 118 and the determination level will be described in more detail with reference to FIGS.

  FIG. 8 is a diagram showing detection parameter items and determination levels corresponding to parameters in general imaging. In the present embodiment, six parameter parameters are exemplified.

  First, the “eye aspect ratio” of the detection parameters in FIG. 8 will be described with reference to FIG.

  FIG. 4 is an enlarged view of a human eye detected by the subject detection block 116, where the distance from the top of the eye to the corner of the eye is the width W, the distance perpendicular to the W direction is the height H, and the ratio of W to H is shown in FIG. Let W / H be the aspect ratio of the eyes.

  Next, the “positional relationship between eyes and mouth” of the detection parameters in FIG. 8 will be described with reference to FIG.

  FIG. 5 is an enlarged view of a person's eyes and mouth detected by the subject detection block 116. The distance from one eye to the other eye is D1, and the distance from both eyes to the mouth is D2 and D3, respectively. , D2, D3 ratios D1: D2, D1: D3 are “positional relationship between eyes and mouth”.

  Next, the detection parameter “eye level on the screen” in FIG. 8 will be described with reference to FIG.

  FIG. 6 is an enlarged view of a person's eyes and mouth detected by the subject detection block 116, and the slope Θ of the line connecting one eye to the other eye is expressed as “the level of the eye” with respect to the lower part of the screen. "

  Next, the detection parameter “face size on the screen” in FIG. 8 will be described with reference to FIG.

  FIG. 7 is an enlarged view of the size of the face on the screen detected by the subject detection block 116, where the area ΔS surrounded by the eyes and mouth, the area of the entire screen is S, and ΔS / S is “the face of the face on the screen”. “Size”.

  Note that “eye color information” is red, green, and blue color information of the pupil, and face color information is red, green, and blue skin color information of a portion surrounded by the eyes and mouth.

  The determination level in FIG. 8 is a determination level in general shooting, and the value varies depending on shooting conditions.

  The detection parameters and determination levels here are only examples and are not limited thereto.

<Description of operation>
Next, the operation of the imaging apparatus according to the first embodiment will be described in detail with reference to the flowchart of FIG.

  In FIG. 2, in S <b> 201, the CPU 100 controls the photometry sensor 107 and the focus detection sensor 108 to detect the degree of focus and the luminance of the subject. Further, the CPU 100 determines a focus lens position, an aperture value, and an exposure time of a photographing lens (not shown) according to the photographing mode set by the state of the switch group 114. Then, the aperture value is controlled via the lens communication circuit 105, and the strobe 109 and the image sensor 106 are controlled to perform a photographing operation. The captured image signal is subjected to predetermined signal processing by the video signal processing circuit 103 and then displayed on the display unit 110, stored in the external memory 102, and input to the subject detection block 116.

  In S202, the shooting information acquisition block 121 acquires shooting information when an image is shot. The imaging information is information such as various camera modes set by the switch group 114 and the presence or absence of flash emission. These pieces of information are output to the detection parameter selection block 117 and the determination level determination block 120.

  In S203, the detection parameter selection block 117 selects at least one detection parameter from the detection parameter group 118 according to the shooting information acquired by the shooting information input block 121.

  In S <b> 204, the determination level determination block 120 determines the determination level for each detection parameter according to the shooting information acquired by the shooting information input block 121.

  Here, the detection parameter selection operation in S203 and the determination level determination operation in S204 will be described in more detail.

  FIG. 9 is a diagram showing the shooting information, detection parameters, and evaluation level determination levels.

  In FIG. 9, in the portrait shooting mode, the subject and the imaging device are facing each other and the main subject is often photographed on the screen. In this case, it is necessary to reliably detect the main subject, not the person existing around the main subject. Therefore, the detection parameter selection block 117 selects all of the detection parameter groups 118. In addition, the determination level determination block 120 detects an object having a higher determination level of “positional relationship between eyes and mouth” representing the degree of confrontation and a determination level of “face size on the screen” larger. Set.

  When the flash is fired, the subject's eyes are thinner than usual, and the red light is often reflected by the pupil part, resulting in red eyes. Therefore, the detection parameter selection block 117 selects other than “eye color information” from the detection parameter group 118. In addition, the determination level determination block 120 sets the determination level of the “aspect ratio of the eyes” so that the subject can be detected even with a small detail. When portrait shooting mode and flash firing are set at the same time, priority is given to the judgment level at the time of flash firing for “eye color information” and “eye aspect ratio”, and other detection parameters are portrait shooting mode. The determination level is given priority. As described above, it is determined in advance which shooting mode is given priority for each detection parameter.

  During exposure compensation shooting, the user may intentionally lower or increase the subject brightness, and in many cases, the subject cannot be detected based on color information. Therefore, the detection parameter selection block 117 selects other than “eye color information” and “face color information” from the detection parameter group 118.

  In continuous shooting or moving image shooting mode, the subject is moving, so the subject and the imaging device often do not face each other. Therefore, the determination level determination block 120 further relaxes the determination levels of “the positional relationship between the eyes and the mouth” representing the degree of confrontation and “the level of the eyes on the screen”.

  For other imaging information, the detection parameter selection block 117 selects all of the detection parameter groups 118. The determination level determination block 120 sets a standard determination level as shown in FIG.

  In S205, the subject detection block 116 uses the input image to extract a shape portion corresponding to the eyes and mouth, which are local features of the face in the image. From the image including the background and the person as shown in FIG. 3A, only the image of the person's eyes and mouth is extracted as shown in FIG. Then, calculation is performed for each detection parameter selected by the detection parameter selection block 117 in S203.

  The local feature extraction method here is described in, for example, JP-A-2005-056387.

  In S206, the determination block 119 determines the evaluation result calculated for the detection parameter calculated by the subject detection block 116 in S205. Here, if all the calculated evaluation results are within the determination level range determined by the determination level determination block, it is determined that the face of the subject has been detected, and the process proceeds to S207. On the other hand, if even one of the calculated evaluation results is out of the determination level range, it is determined that the subject's face does not exist, and the series of operations ends.

  In S207, subject information such as the face position and facial expression of the subject from the eye and mouth positions detected in S205 is stored in the RAM 112, and is used for photometry and focus position control centering on the subject position at the next shooting. . Further, by storing the subject information in the external memory 102 in association with the image, it can be used as a condition for searching for the image.

  In the above description, one subject is described, but a plurality of subjects can also be detected.

  Further, the detection parameters and determination levels in FIG. 9 are examples, and are not limited to these.

  As described above, by selecting the optimum detection parameter and determination level according to the shooting information, it is possible to increase the detection rate of the subject and reduce the false detection.

  Next, an operation different from the flowchart of FIG. 2 will be described in detail with reference to the flowchart of FIG.

  In FIG. 10, in S1001, the CPU 100 controls the photometry sensor 107 and the focus detection sensor 108 to detect the in-focus state and luminance of the subject. Further, the CPU 100 determines a focus lens position, an aperture value, and an exposure time of a photographing lens (not shown) according to the photographing mode set by the state of the switch group 114. Then, the aperture value is controlled via the lens communication circuit 105, and the strobe 109 and the image sensor 106 are controlled to perform a photographing operation. The captured image signal is subjected to predetermined signal processing by the video signal processing circuit 103, and then displayed on the display unit 110, stored in the external memory 102, and output to the subject detection block 116.

  In S1002, shooting information when the shooting information input block 121 has shot an image is acquired. The shooting information is information such as various camera modes set by the switch group 114, the presence / absence of strobe light emission, and subject information about the last shot image stored in the RAM 112. These pieces of shooting information are output to the detection parameter selection block 117 and the determination level determination block 120.

  In S1003, the determination block 119 determines whether or not the shooting information acquired by the shooting information input block 121 in S1002 is the continuous shooting mode or the moving image shooting mode. If it is the continuous shooting mode or the moving image shooting mode, the process proceeds to S1004, and if it is other shooting information, the process proceeds to S1005.

  In step S <b> 1004, the determination block 119 determines whether or not a subject has been detected for the previously captured image. If the subject has been detected, the process proceeds to S1007, and if it has not been detected, the process proceeds to S1005.

  In step S <b> 1005, the detection parameter selection block 117 selects the first parameter from the detection parameter group 118 according to the shooting information acquired by the shooting information input block 121. The first parameter to be selected is the same as the detection parameter in FIG.

  In step S <b> 1006, the determination level determination block 120 determines a first determination level for each detection parameter according to the shooting information input from the shooting information input block 121. The first determination level is the same as the determination level in FIG.

  In S1007, the detection parameter selection block 117 selects the second parameter from the detection parameter group 118.

  In S1008, the determination level determination block 120 determines a second determination level for each detection parameter.

  Here, the detection parameter selection process in S1007 and the determination level determination process in S1008 will be described in detail with reference to FIG.

  FIG. 11 is a diagram illustrating detection parameter and evaluation value determination levels when a subject can be detected from a previously captured image in continuous shooting or moving image shooting mode.

  In continuous shooting or moving image shooting mode, if a subject can be detected from an image, the same subject often exists in the next image. Therefore, by performing subject detection with fewer detection parameters than the number of detection parameters in the continuous shooting and moving image shooting modes shown in FIG. 9, it is easy to detect the subject even if the subject moves and the face orientation changes greatly. Become. Also, by reducing the number of detection parameters, it is possible to shorten the time related to subject detection.

  Returning to the description of FIG. 10, in step S1009, the subject detection block 116 uses the input image to extract a shape portion corresponding to the eyes and mouth which are local features of the face. Then, in the operation of S1005 or S1007, calculation is performed for each detection parameter selected by the detection parameter selection block 117.

  In S1010, the determination block 119 determines the evaluation result calculated for each detection parameter calculated by the subject detection block 116 in S1009. Here, if all the calculated evaluation results are within the determination level range determined by the determination level determination block, it is determined that the face of the subject has been detected, and the process proceeds to S1011. On the other hand, if even one of the calculated evaluation results is out of the determination level range, it is determined that the subject's face does not exist, and the series of operations ends.

  In S1011, subject information such as the position of the subject's face and facial expression from the eye and mouth positions detected in S1009 is stored in the RAM 112 for later photometry and focus position control, etc., centering on the subject position at the time of shooting. Use. The subject information is stored in the external memory 102 in association with the image.

  As described above, by selecting the optimum detection parameter and evaluation value determination level according to the photographing information, it is possible to increase the detection rate and reduce erroneous detection. Furthermore, in continuous shooting and movie shooting modes, if the subject can be detected once from the image, it can follow the subject even if the subject moves and the orientation of the face changes significantly by changing the detection parameters and judgment level. Can be detected.

In addition, by reducing the number of detection parameters, it is possible to shorten the time for subject detection. For continuous shooting or movie shooting, even if the time interval between shooting is short, every image (one frame) A subject detection operation can be performed.
[Second Embodiment]
Next, the face detection process of the second embodiment will be described.

  FIG. 12 is a block diagram illustrating main components of the imaging apparatus according to the second embodiment in which the image processing apparatus according to the present invention is mounted.

  12, the image processing circuit 122 includes a subject detection block 123, a detection parameter group 124, a reliability determination block 125, a weighting block 126, and a photographing information input block 127. The image processing circuit 122 detects position information corresponding to the face of the subject, face feature information, and the like from the image signal input from the video signal processing circuit 103.

  The shooting information input block 127 outputs shooting information such as various camera modes set by the setting of the switch group 114 and the presence or absence of flash emission to the weighting block 120.

  The weighting block 126 determines a weighting coefficient according to the shooting information output from the shooting information input block 121 and outputs the coefficient to the determination block 125. The weighting coefficient is a coefficient indicating the importance of the reliability level described later.

  In the subject detection block 123, local features (here, eyes and mouth) are extracted from the image, and the image is evaluated with respect to the detection parameters of the detection parameter group 124. It outputs to CPU100.

  The determination block 125 determines the presence / absence of a subject from the reliability level that is a criterion for determining whether or not the face of the subject is based on the evaluation result obtained by evaluating the image in the subject detection block 123 and the weighting coefficient input from the weighting block 126. I do.

  In addition, about the structure same as FIG. 1, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  Here, the detection parameter group 124 and the reliability level will be described in detail with reference to FIG.

  FIG. 13 is a diagram showing detection parameters, evaluation values, and reliability levels.

  The detection parameter items are the same as those in FIG.

  The result calculated for each parameter is used as an evaluation result, and a reliability level corresponding to the evaluation result is determined in advance. Here, the reliability levels are represented by three types of 2, 1, 0, and “2” is the highest reliability and “0” is the lowest reliability.

  The detection parameters, evaluation values, and reliability levels described here are merely examples, and the present invention is not limited thereto.

<Description of operation>
Next, the operation of the imaging apparatus according to the second embodiment will be described in detail with reference to the flowchart of FIG.

  In FIG. 14, in S1401, the CPU 100 controls the photometry sensor 107 and the focus detection sensor 108 to detect the in-focus state and the brightness of the subject. Further, the CPU 100 determines a focus lens position, an aperture value, and an exposure time of a photographing lens (not shown) according to the photographing mode set by the state of the switch group 114. The aperture value is controlled via the lens communication circuit 105, and the photographing operation is performed by controlling the strobe 109 and the image sensor 106. The captured image signal is subjected to predetermined signal processing by the video signal processing circuit 103, displayed on the display unit 110, stored in the external memory 102, and output to the subject detection block 123.

  In S1402, the shooting information input block 127 acquires shooting information when the image is shot. The shooting information is information such as various camera modes set by the switch group 114 and the presence or absence of flash emission. These pieces of information are output to the weighting block 126.

  In S1403, the weighting block 126 determines a weighting coefficient according to the shooting information acquired by the shooting information input block 123. The weighting coefficient represents the importance of the reliability level for each detection parameter determined in the determination block 125.

  Here, the determination operation of the weighting coefficient in S1403 will be described in more detail.

  FIG. 15 is a diagram showing weighting coefficients for the photographing information and the reliability level.

  In the portrait shooting mode, the subject and the image pickup device face each other, and the main subject is often photographed mainly. In this case, it is necessary to reliably detect the main subject, not the person existing around the main subject. Therefore, the weighting block 126 determines the coefficient so as to increase the importance of the reliability level of the “positional relationship between eyes and mouth” and the “face size on the screen” representing the degree of confrontation.

  When the flash is fired, the subject's eyes are thinner than usual, and the red light is often reflected by the pupil and becomes red-eye. Therefore, the detection parameter selection block 117 reduces the importance of the reliability level of the “eye color information” and “eye aspect ratio” in the detection parameter group 118, and the coefficient so that the subject can be detected even with red eyes and fine eyes. To decide.

  During exposure compensation shooting, the user may intentionally lower or increase the subject brightness, and in many cases, the subject cannot be detected based on color information. Accordingly, the coefficient is determined with a low importance level of the reliability level of “eye color information” and “face color information”.

  In continuous shooting or moving image shooting mode, the subject is moving, so the subject and the imaging device often do not face each other. Accordingly, the coefficient is determined by lowering the importance of the reliability level of “the positional relationship between the eyes and the mouth” and “the level of the eyes on the screen”.

  In other imaging modes, the coefficients are determined so that the reliability levels of all detection parameters have the same importance.

  Here, adding the weighting coefficient of each detection parameter results in the same numerical value in each shooting mode, predicts the situation of the subject according to the shooting mode, and changes the importance level of the reliability level according to each detection parameter. Yes.

  In step S <b> 1405, the subject detection block 123 extracts a shape portion corresponding to the eyes and mouth that are local features of the face from the input image, and calculates an evaluation value for the detection parameter group 124.

  In S1406, reliability determination is performed by the determination block 125 from the evaluation result calculated for each detection parameter calculated in S1405 and the weighting coefficient determined in the operation of S1403. Here, for each detection parameter, the reliability level corresponding to the evaluation result is multiplied by the weighting coefficient, and all the multiplication results are added to obtain an overall reliability level. If the total reliability level is equal to or higher than the predetermined level, it is determined that the subject's face has been detected, and the process proceeds to S1407. On the other hand, when the total reliability level is lower than the predetermined level, it is determined that the face of the subject does not exist, and the series of operations is terminated.

  In step S1407, subject information such as the position of the subject's face and facial expression detected from the eye and mouth positions detected in step S1405 is stored in the RAM 112, and photometry and focus position control centered on the subject position are performed during the next shooting. The subject information is stored in the external memory 102 in association with the image.

  Also, the imaging information and the reliability level weighting coefficients in FIG. 15 are examples, and the present invention is not limited to these.

  As described above, it is possible to increase the detection rate of the subject and reduce false detection by determining the weighting coefficient of the optimum reliability level according to the photographing information.

[Other Embodiments]
The present invention includes a case where the computer program for realizing the functions of the above-described embodiments is achieved by supplying the system or apparatus directly or remotely. In that case, a computer such as a system reads and executes the computer program.

  Therefore, in order to realize the functional processing of the present invention with a computer, the computer program itself installed in the computer also implements the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium (storage medium) for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. In addition, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser can be used to connect to a homepage on the Internet, and the computer program itself of the present invention can be downloaded from the homepage. It can also be supplied by downloading a compressed file including an automatic installation function to a recording medium such as a hard disk. It can also be realized by dividing the computer program constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and a user who satisfies predetermined conditions downloads key information for decryption from a homepage via the Internet. You can also. In this case, the downloaded key information is used to execute the encrypted program and install it on the computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS or the like operating on the computer based on the instruction of the program may be a part of the actual processing. Alternatively, it can be realized by performing all of them.

  Furthermore, after the program read from the recording medium is written in the memory of a function expansion board inserted into the computer or a function expansion unit connected to the computer, the CPU of the board or the like performs a part of the actual processing. Alternatively, it can be realized by performing all of them.

1 is a block diagram showing a configuration of a first embodiment according to the present invention. It is a flowchart which shows the to-be-photographed object detection process by 1st Embodiment based on this invention. It is a figure explaining the local (eyes, mouth) extraction of the face in an image. It is a figure explaining "aspect ratio of eyes". It is a figure explaining "the positional relationship of an eye and a mouth." It is a figure explaining "the level of the eye in a screen". It is a figure explaining "the size of the face in a screen". It is a figure which shows a detection parameter and a determination level. It is a figure which shows imaging | photography information, a detection parameter, and a determination level. 7 is a flowchart showing another subject detection process according to the first embodiment of the present invention. It is a figure which shows the detection parameter and determination level when a to-be-photographed object can be detected with the front image. It is a block diagram which shows the structure of 2nd Embodiment which concerns on this invention. It is a figure which shows a detection parameter, an evaluation value, and a reliability level. It is a flowchart which shows the to-be-photographed object detection process by 2nd Embodiment which concerns on this invention. It is a figure which shows the weighting coefficient of imaging | photography information and a reliability level.

Explanation of symbols

100 CPU
DESCRIPTION OF SYMBOLS 101 Image processing circuit 102 External memory 103 Video signal processing circuit 104 Signal input circuit 105 Lens communication circuit 106 Image sensor 107 Photometric sensor 108 Focus detection sensor 109 Strobe 110 Display unit 111 ROM
112 RAM
113 EEPROM
114 Switch group 115 Lens control signal 116 Subject detection block 117 Detection parameter selection block 118 Detection parameter group 119 Determination block 120 Determination level determination block 121 Imaging information input block

Claims (22)

An image processing device for detecting a subject from a captured image,
Acquisition means for acquiring shooting information at the time of shooting;
Change means for changing the detection condition of the subject by the detection means according to the shooting information;
An image processing apparatus comprising: detection means for extracting feature information of a subject from a photographed image according to the photographing condition and detecting the subject from the feature information.   The image processing apparatus according to claim 1, wherein the detection unit extracts feature information of the subject using a plurality of detection conditions, and identifies the subject based on an evaluation result of the feature information. A first selection unit that selects at least one first condition from the plurality of detection conditions according to the photographing information;
The image processing apparatus according to claim 2, wherein the detection unit extracts feature information of the subject from the first condition and specifies the subject based on an evaluation result of the feature information. A first determination means for determining the presence or absence of a subject by comparing the evaluation result with a first determination level;
The image processing apparatus according to claim 1, wherein the first determination level is determined according to the shooting information. A second selection unit that selects at least one second condition from the plurality of detection conditions when the subject is detected from the next photographed image after the subject is detected by the detection unit;
5. The image processing apparatus according to claim 3, wherein the detection unit extracts feature information of the subject from the second condition and specifies the subject based on an evaluation result of the feature information.   The image processing apparatus according to claim 5, wherein the number of the second conditions is smaller than the number of the first conditions. A second determination means for determining the presence or absence of a subject by comparing the evaluation result under the second condition and a second determination level;
The image processing apparatus according to claim 5, wherein the second determination level is determined according to the shooting information.   The image processing apparatus according to claim 5, wherein the second determination level is different from the first determination level. A determining unit that determines a weighting coefficient for the detection condition according to the imaging condition;
The detection unit extracts feature information of the subject using a plurality of detection conditions, and specifies the subject from the evaluation result of the feature information and the reliability level of the subject obtained from the detection condition and the weighting coefficient. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein the shooting information includes shooting information in strobe shooting, continuous shooting, person shooting, moving image shooting, and exposure correction mode. An image processing method for detecting a subject from a captured image,
Acquiring shooting information at the time of shooting;
Changing a subject detection condition in accordance with the shooting information;
An image processing method comprising: detecting feature information of a subject from a captured image according to the detection condition, and detecting the subject from the feature information.   The image processing method according to claim 11, wherein in the detection step, feature information of the subject is extracted using a plurality of detection conditions, and the subject is specified based on an evaluation result of the feature information. A first selection step of selecting at least one first condition from the plurality of detection conditions according to the photographing information;
13. The image processing method according to claim 12, wherein in the detection step, feature information of the subject is extracted from the first condition, and the subject is specified based on an evaluation result of the feature information. A first determination step of determining the presence or absence of a subject by comparing the evaluation result with a first determination level;
The image processing method according to claim 11, wherein the first determination level is determined according to the shooting information. A second selection step of selecting at least one second condition from the plurality of detection conditions when the subject is detected from the next captured image after the detection of the subject by the detection step;
15. The image processing method according to claim 13, wherein in the detection step, feature information of the subject is extracted from the second condition, and the subject is specified based on an evaluation result of the feature information.   The image processing method according to claim 15, wherein the number of the second conditions is smaller than the number of the first conditions. A second determination step of determining the presence or absence of a subject by comparing the evaluation result under the second condition and a second determination level;
The image processing method according to claim 15 or 16, wherein the second determination level is determined according to the photographing information.   The image processing method according to claim 15, wherein the second determination level is different from the first determination level. A determination step of determining a weighting coefficient for the detection condition according to the imaging condition;
In the detection step, the feature information of the subject is extracted using a plurality of detection conditions, and the subject is identified from the evaluation result of the feature information and the reliability level of the subject obtained from the detection condition and the weighting coefficient. The image processing method according to claim 11.   20. The image processing method according to claim 11, wherein the shooting information includes shooting information in strobe shooting, continuous shooting, person shooting, moving image shooting, and exposure correction mode.   A program for causing a computer to execute the image processing method according to any one of claims 11 to 20.   A computer-readable storage medium storing the program according to claim 21.

Images