JP2008005438A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress a main face from being extremely frequently switched while having respect for a change in the state of an object existent within an imaging range when selecting the main face from the imaging range. <P>SOLUTION: A digital camera 200 selects the face of a human figure to be a main object from image data obtained from an image signal captured by a CCD 201. In selection processing, in addition to a weight calculated based on the states of a plurality of faces in a certain timing, a weight (fourth weight) calculated based on the distance between each of the plurality of faces and the main face in the previous timing is also utilized. The fourth face is calculated so as to become larger for the face in the shorter distance to the main face in the previous timing. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method for selecting a person's face as a main subject.

  2. Description of the Related Art Image pickup apparatuses (for example, digital cameras) having an AF (autofocus) function for automatically focusing on a subject and an AE (auto exposure) function for automatically determining exposure are known. In addition, when a person is a subject, a function of detecting a person or a person's face from an imaging range is provided in order to more accurately perform AF, AE, etc. (hereinafter referred to as “AF etc.”) according to the person. Imaging devices are also known. Furthermore, an imaging apparatus having a function of selecting a person to be a main subject or a person's face from a plurality of persons is also known in order to perform more accurate AF or the like even when there are a plurality of persons in the imaging range ( (See Patent Documents 1 to 4).

The imaging devices disclosed in Patent Documents 1 to 4 select a person or a person's face as a main subject based on the state of the subject existing in the imaging range. The state of the subject means, for example, various parameters representing the distance from the imaging device to the subject, the area of the subject in the imaging range, the position of the subject in the imaging range, the distance between subjects, and the like.
Japanese Patent Laid-Open No. 11-146405 JP 2001-215403 A JP 2004-317699 A JP 2002-512255 A

  According to the conventional technology for selecting a person or a person's face as a main subject, the selection process is performed based only on the state of the subject at a specific time, and the following problems may occur.

  FIG. 1 is a diagram illustrating a person's face detected by the imaging apparatus at time (timing) t = 1 to 4 and a person's face as a main subject (hereinafter referred to as “main face”). The part indicated by the solid frame (for example, left of FIG. 1A) is the main face, and the part indicated by the dotted frame (for example, right of FIG. 1A) is the other face. As shown in FIGS. 1A to 1D, when the states of the two faces are substantially the same, the main face is frequently switched due to a slight change in the state.

  It is troublesome for the user that the display of the EVF (electronic viewfinder) on the display device 213 changes frequently as shown in FIGS. In addition, when AF or the like is performed in accordance with the main face, if the main face is frequently changed, the AF lens or the like is also frequently driven in accordance therewith, and the stability and eventually the accuracy may be impaired.

  The present invention has been made in view of such a situation, and when selecting a main face from an imaging range, the main face is extremely frequently observed while respecting changes in the state of a subject existing in the imaging range. It aims at providing the technique which suppresses changing.

  In order to solve the above problems, an imaging apparatus according to the present invention includes an imaging unit that acquires image data, a face detection unit that detects a human face from the image data acquired by the imaging unit, and the face detection unit. A selection unit that selects a face to be a main subject from the detected face of the person, and when the selection unit has information on a face to be a main subject selected by the selection unit in the past, A first mode for selecting the face of the main subject in the image data from the information about the face as the main subject in the past and the information about the face of the person detected by the face detection means; and the selection means in the past If there is no information about the selected face as the main subject, the second mode for selecting the face as the main subject in the image data from the information about the face of the person detected by the face detection means. Characterized in that it has and.

  The imaging method of the present invention includes an imaging process for acquiring image data, a face detection process for detecting a human face from the image data acquired in the imaging process, and a human face detected in the face detection process. A selection step of selecting a face to be a main subject, and when the selection step includes information on the face to be a main subject selected in the past in the selection step, The first mode for selecting the face of the main subject in the image data from the information about the face and the information about the face of the person detected in the face detection step, and the main subject selected in the past in the selection step A second mode for selecting a face to be a main subject in the image data from information on the face of the person detected in the face detection step when no information about the face exists. To.

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the best mode for carrying out the invention.

  With the above configuration, according to the present invention, when selecting a main face from the imaging range, it is possible to prevent the main face from being changed extremely frequently while respecting changes in the state of the subject existing in the imaging range. It becomes possible.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The individual embodiments described below will help to understand various concepts, such as the superordinate concept, intermediate concept and subordinate concept of the present invention.

  The technical scope of the present invention is determined by the scope of the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
<Configuration and Operation of Imaging Device>
FIG. 2 is a block diagram illustrating a configuration example of a digital camera 200 that is an example of the imaging apparatus according to the embodiment. First, the operation of the digital camera 200 when the shutter button (not shown) is not pressed after the power is turned on will be described.

  The digital camera 200 drives the CCD 201 in various modes (described later), and acquires an image signal by photoelectric conversion every 1/30 seconds, for example. The acquired image signal is output to the OB unit 202, and signal processing is sequentially performed in subsequent blocks. The CCD 201 in this embodiment can operate in the normal still readout (all pixel readout) mode shown in FIG. Further, it can operate in a mode (addition / decimation mode) such as movie reading (vertical direction 2/8 reading) shown in FIG. 3B for EVF (electronic viewfinder) or moving image. For EVF display, it is necessary to read out charges at high speed. Generally, the CCD 201 is driven in the addition / thinning mode shown in FIG. There are various types of addition / decimation other than those shown in FIG. 3B, but the description is omitted because it is not the gist of the present invention.

  The image signal acquired by the CCD 201 driven for EVF is subjected to blacking by an OB (optical black) unit 202. A white balance (WB) unit 203 adjusts the white balance. Thereafter, the color conversion MTX 206 converts it into a CrCb color difference signal. The band is limited by an LPF (low-pass filter) 207. The CSUP circuit 208 suppresses the color difference signal in the saturation portion. On the other hand, the image signal after white balance adjustment is sent to the luminance signal generation circuit 214 to create a luminance signal. Thereafter, the edge enhancement circuit 215 performs contour correction. The color difference signal and the edge-enhanced luminance signal after the suppression processing in the CSUP circuit 208 are converted into RGB signals by the RGB conversion circuit 209. The gamma correction circuit 210 performs gamma correction. After that, it is converted into a YUV signal by the color luminance conversion circuit 211, resized to an EVF display image size, for example, an image size of 320 pixels × 240 pixels, and transferred to the display device 213.

  On the other hand, the resized YUV signal is also transferred to the face detection module (VBPF unit 217, eye detection unit 218, non-face filter pattern recognition unit 219). The face detection module periodically performs face detection processing using the resized YUV signal. Then, the detected face information (face coordinate value, eye coordinate value, reliability information, etc., details will be described later) is written in the face register 224. Here, the detection processing cycle by the face detection module and the EVF update cycle by the display device 213 do not necessarily match, and can be set independently. Since the face detection method is not the main point of the present invention, detailed description thereof is omitted.

  The face detection module can detect a person's face from image data by predetermined image processing. For example, it is determined whether there are two eyes by using the fact that the pupil color is a certain color such as black or brown and has a certain size. Then, it is determined whether or not the detected interval between the two eyes is close to the interval between the two eyes in a general face. Further, it may be determined whether or not a skin color area having an area close to a general face area exists in the image data by using the fact that the face color is a constant color such as skin color or black.

  Next, the operation of the digital camera 200 after a shutter button (not shown) is half pressed will be described.

  When the user presses a shutter button (not shown) halfway, the system controller 220 reads the latest face information from the face register 224 at that time. Here, when there are two or more detected faces, main face selection processing for selecting a face (main face) to be a main subject from the detected faces is performed. Details of the main face selection process will be described later. The main face selection process may be started not when the shutter button (not shown) is pressed halfway but when the digital camera 200 is turned on.

  The system controller 220 uses the main face information determined in the main face selection process to set the AF distance measurement frame and the photometric weight used in the AE in the AE / EF control unit 221 and the AF control unit 222. Then, the AF operation and the AE operation are performed, and when completed, the user is notified through the display device 213 and the like. Since the specific processing contents of the AF and AE operations are not the main point of the present invention, description thereof will be omitted.

  Here, before and after the shutter button is pressed halfway, the CCD 201 continuously acquires image signals at a predetermined timing and displays them on the display device 213 or uses them for main face selection processing. Note that the image data is acquired.

<Main face selection process>
FIG. 4 is a flowchart showing a flow of main face selection processing by the digital camera 200 according to the first embodiment. The processing of each step of this flowchart can be realized by the system controller 220 executing a program stored in a ROM (not shown), for example. Hereinafter, it is assumed that the system controller 220 performs each step unless otherwise specified.

(Step S401)
A variable counter that counts the number of times the main face selection process has been performed is initialized to zero.

(Step S402)
A face detection result at a predetermined timing t is acquired. The face detection result is information on a face existing in image data obtained by the image signal acquired by the CCD 201 being resized by the resizing unit 223 thereafter.

  FIG. 5 is a diagram illustrating an example of face information. As face information, there are a center coordinate (x, y) of a detected face, a face size size indicating the length of one side when the face is a square, and a reliability value indicating the probability that the detected face is a face. To be acquired.

  The face size can be set to such a size that the ratio of the skin color area included in the square of a predetermined size becomes a predetermined value by utilizing the fact that most of the face is a skin color.

  For example, the reliability is based on the fact that the pupil color is black, whether there are two eyes, how far the two eyes are, and whether there is a nose between the two eyes. Determined based on the above. In this embodiment, the reliability is expressed in 10 levels of 1 to 10, with 1 indicating the highest probability of the face.

(Step S403)
If there is only one detected face, the process proceeds to step S417, where Counter = 0 is set. And it returns to step S402 and repeats the same process. On the other hand, if there are a plurality of detected faces, the process proceeds to step S404. In step S417, the counter value may be set to 1 or more, and the face information of the main face may be stored in the face register 224 with one detected face as the main face.

(Step S404)
When Counter = 0, the process proceeds to step S405. Otherwise, the process proceeds to step S409.

(Step S405)
The calculation processing in steps S405 to S408 is performed on all face information read in step S102 (that is, information on all detected faces).

  First, the first weight is calculated using the reliability of the detected face and the reliability-weight characteristic graph shown in FIG. FIG. 6 is a graph in which the input (x-axis) is the detected face reliability and the output (y-axis) is the first weight. In the present embodiment, the first weight is set to 1 until the reliability is 3, and the reliability 3 to 5 are connected with a straight line so that the first weight becomes 0 at the reliability 5, and the reliability is 5 or more. In this case, the first weight is 0.

(Step S406)
The second weight is calculated using the detected face size and the face size-weight characteristic graph shown in FIG. FIG. 7 is a graph in which the input (x-axis) is the detected face size and the output (y-axis) is the second weight. In the present embodiment, the face sizes 0 to 20 are connected by a straight line so that the weight is 0 when the face size is 0 (the unit is, for example, a pixel, and so on), and the weight is 0.2 when the face size is 20. . Further, the face sizes 20 to 30 are connected by a straight line so that the weight becomes 1 when the face size is 30, and the weight is set to 1.0 when the face size is 30 or more.

(Step S407)
For the detected face, distance information dist between the center coordinate value of the detected face and, for example, the center coordinate value of the screen, as shown in FIG. 5, is calculated, and the distance information dist and the distance-weight characteristic shown in FIG. A third weight is calculated using the graph. In the present embodiment, the face detection target image data is the image data for EVF display of the display device 213, and therefore the size is 320 pixels × 240 pixels. Therefore, the distance from the center coordinates (160, 120) is calculated using the three-square theorem. Then, the third weight is obtained using the graph of FIG. In the present embodiment, the distances 10 to 80 are connected by a straight line so that the weight is 1.0 when the distance is 10 and the weight is 0 when the distance is 80, and the weight is 0 when the distance is 80 or more.

(Step S408)
The final weight is calculated by multiplying the first to third weights calculated in steps S405 to S407, and the process proceeds to step S415. The calculation of the final weight is not limited to multiplication. For example, the final weight may be obtained by the sum of the first to third weights.

(Step S409)
The process proceeds from step S404 to S409 when the value of Counter is not 0, that is, when the main face is selected in the previous main face selection process and the main face information is stored in the face register 224. is there. In step S409, information on the main face selected last time is acquired from the face register 224, and a distance from each of the face information acquired in step S402 is calculated. That is, as shown in FIG. 9, the distance from the coordinate values of the main face selected at the previous timing t-1 (hereinafter referred to as “previous main face”) for all of the plurality of faces detected at the timing t. dist_preMainFace is calculated using Equation 1 below.

(Step S410)
A fourth weight is calculated using the distance from the previous main face calculated in step S409. For example, the face having the shortest distance from the previous main face is set as a main face candidate at timing t, and a value kHoseiWeightMainFace (for example, 1.5) preset as the main face candidate correction weight is assigned as the fourth weight. Furthermore, kHoseiWeightOtherFace (= 1.0) is assigned to the detected face other than the main face candidate as the fourth weight. When there are a plurality of faces having the shortest distance from the previous main face, all the faces having the shortest distance are set as main face candidates, and kHoseiWeightMainFace is assigned as the fourth weight.

  Alternatively, the fourth weight may be calculated using a previous distance-weight characteristic graph that determines the fourth weight from the distance dist_preMainFace from the previous main face calculated by Formula 1 as shown in FIG.

(Steps S411 to S413)
Here, the same processing as the processing in steps S405 to S407 is performed.

(Step S414)
According to Equation 2, the final weight of each detected face is calculated by multiplying the fourth weight calculated in Step S410 by the first to third weights calculated in Steps S411 to S413. The calculation of the final weight is not limited to multiplication. For example, the final weight may be obtained by the sum of the first to fourth weights.

(Step S415)
According to the final weights for each of the detected faces calculated in step S408 or S414, the face with the largest weight is determined (selected) as the main face at timing t. Then, the face information of the selected main face is stored in the face register 224.

(Step S416)
The value of Counter is incremented by 1 (Counter = Counter + 1), and the process returns to step S102 to repeat the same processing.

  Here, although not explicitly shown in FIG. 4, the digital camera 200 may be configured to reset the value of Counter to 0 when a predetermined time has elapsed since the last time the main face was selected. That is, in principle, the digital camera 200 selects the main face using the previous main face. If the previous main face is too old in time, the digital camera 200 follows steps S405 to S408 and S415 regardless of the previous main face. A main face may be selected.

<Specific example of main face selection processing>
The flow of the main face selection process has been described above with reference to FIG. Here, a specific example of main face selection processing when two faces (referred to as “face 1” and “face 2”) are detected in a continuous scene (timing t = 1 to 4) as shown in FIG. This will be described with reference to FIG. 12 showing face information at each timing and the above-described flowchart (FIG. 4). FIG. 13 is a diagram showing first to fourth weights, final weights, distances from the previous main face, and main face selection results for each face at each timing.

(Main face selection at t = 1)
The operation proceeds from step S101 to step S402 in FIG. 4. Since there are two faces detected in step S403, the process proceeds to step S404, and because Counter = 0, the process proceeds to step S405.

  In step S405, a first weight (Weight1) based on the reliability of each detected face is calculated. In this embodiment, the reliability is 1 for both faces 1 and 2, so that Weight 1 is 1.

  In step S406, the second weight (Weight2) is calculated from the face size, as shown in FIG.

  In step S407, a third weight (Weight 3) is calculated based on the distance from the center, as shown in FIG.

  In step S408, the final weight (Total_Weight) for each face is calculated by multiplying the values of weights 1 to 3.

  In step S415, since the final weight of face 1 is larger than the final weight of face 2, face 1 is selected as the main face. In FIG. 13, it is shown that a main face is one having a value of 1 in the column “main face”.

  In step S416, the value of Counter is increased by one.

(Main face selection at t = 2)
In step S402, face information is read, and the process proceeds to step S409 through steps S403 and S404.

  In step S409, the distance between the center coordinates of the face detected at t = 2 and the center coordinates of the main face (previous main face, here face 1) at t = 1 is calculated. Then, the face with the shortest distance is set as a main face candidate at t = 2. From FIG. 13, the distance between face 1 and the previous main face is 10, the distance between face 2 and the previous main face is 105, and face 1 is the main face candidate.

  In step S410, based on the result of step S409, the fourth weight of the face closest to the previous main face (face 1) is set to 1.5, for example, and the others are set to 1.0.

  In steps S411 to S413, first to third weights for each face are calculated.

  In step S14, the final weight at t = 2 is calculated using the first to fourth weights. From FIG. 13, the final weight of face 1 is 0.42, and the final weight of face 2 is 0.35.

  Accordingly, in step S415, face 1 is selected as the main face.

(Main face selection at t = 3)
The processing path is the same as the main face selection process at t = 2, and face 1 is selected as the main face.

(Main face selection at t = 4)
The processing path is the same as the main face selection process at t = 3, but the third weight based on the distance from the center of face 1 is 0, so the final weight is also 0, and face 2 is selected as the main face. The

(When not using the fourth weight)
Here, for comparison, a case where the main face is selected without using the fourth weight at t = 1 to 4 will be described with reference to FIGS. 1 and 14. Even if two faces are present at exactly the same positions as in the case of FIGS. 11 and 13 described above, it can be seen that the main face frequently changes as shown in FIGS.

<Summary of First Embodiment>
As described above, according to the present embodiment, the digital camera 200 selects the face of the person who is the main subject from the image data obtained from the image signal acquired by the CCD 201. In the selection process, in addition to the weight calculated based on the state of each of a plurality of faces at a certain timing, a weight calculated based on the distance between each of the plurality of faces and the main face at the previous timing (first (Weight of 4) is also used. The fourth weight is calculated so that the face closer to the main face at the previous timing becomes larger.

  Thereby, when selecting the main face from the imaging range, it is possible to prevent the main face from being changed extremely frequently while respecting the change in the state of the subject existing in the imaging range.

[Second Embodiment]
In the first embodiment, the distance between the detected face and the previous main face is used in the main face selection process. In the second embodiment, the distance difference and the size difference between the face detected in the current image data and the main face selected in the past (hereinafter referred to as “reference face”) used as a reference are also included. Use to select the main face. This is a configuration focusing on the high probability that the main face exists in the vicinity of the center of the imaging range in general shooting. That is, consider a case where the main face selected from the current image data has moved to some extent from the position of the reference face. In this case, based on the verification data that the main face selection (steps S405 to S408 and S415 in FIG. 4) without considering the previous main face position has a higher probability of meeting the user's intention. ing.

  In the present embodiment, the configuration of the digital camera 200 (see FIG. 2) is the same as that of the first embodiment, and thus the description thereof is omitted.

<Main face selection process>
FIG. 15 is a flowchart showing a flow of main face selection processing by the digital camera 200 according to the second embodiment. The processing of each step of this flowchart can be realized by the system controller 220 executing a program stored in a ROM (not shown), for example. Hereinafter, it is assumed that the system controller 220 performs each step unless otherwise specified. In addition, steps that perform the same processing as in the first embodiment (FIG. 4) are denoted by the same reference numerals, and description thereof is omitted.

(Steps S1508 to S1510)
If Counter = 0 in step S404, the main face is selected in step S1508 by the same processing as in steps S405 to S408 and S415 in FIG. In step S1509, the face information of the selected main face is stored in the face register 224 as a reference face. In step S1510, the value of Counter becomes 1, and the process returns to step S402. The face information of the selected main face is also stored in the face register 224 as the previous main face.

  Note that, based on the same idea as in the first embodiment, in step S417, one detected face may be set as the main face, and face information of this main face may be stored in the face register 224 as a reference face. In this case, Counter = 1 is set in step S417. The face information of the selected main face is also stored in the face register 224 as the previous main face.

(Step S1501)
For each face acquired in step S402, the distance from the previous main face is calculated (see FIG. 9), and sorted in the order closer to the previous main face. The face closest to the previous main face is set as a main face candidate.

(Step S1502)
A distance between the center coordinates of the reference face and the main face candidate is calculated and set as a main face moving distance (Δdis_MainFace). The calculation method is the same as that shown in FIG. 9 except that the previous main face becomes the reference face.

(Step S1503)
If the main face moving distance calculated in step S1502 is less than or equal to the preset distance Th_dis, the process proceeds to step S1504. Otherwise, it is determined that the main face candidate is not actually the main face, and the process proceeds to step S1507. Here, the distance Th_dis can be set by multiplying the size of the reference face by a preset coefficient, for example, as shown in Equation 3.

Th_dis = RefFaceSize × kThHoseiVal1 (Formula 3)
RefFaceSize: Reference face size (length of one side)
KthHoseiVal1: constant (step S1504)
A difference value ΔS between the size of the reference face and the size of the main face candidate is calculated.

(Step S1505)
If the difference value ΔS is less than or equal to the preset difference value Th_size, the process proceeds to step S1506. Otherwise, it is determined that the main face candidate is not actually the main face, and the process proceeds to step S1507. Here, Th_size can be set by multiplying the size of the reference face by a preset coefficient, for example, as shown in Equation 4.

Th_size = RefFaceSize × kThHoseiVal2 (Formula 4)
RefFaceSize: Reference face size (length of one side)
KthHoseiVal2: constant (step S1506)
The main face candidate is determined as the main face, and the face information is stored in the face register 224 as the previous main face.

(Step S1507)
In the order sorted in step S1501, the next closest face to the previous main face is set as a main face candidate, the process returns to step S1502, and the same processing is repeated. When processing has been performed for all detected faces, the main face position has moved greatly, so it is determined that it is not appropriate to use the previous main face position, and the process advances to step S1508.

<Example of main face selection processing>
The flow of the main face selection process has been described above with reference to FIG. Here, as in the first embodiment, an example of main face selection processing when two faces are detected in a continuous scene (timing t = 1 to 4) as shown in FIG. This will be described with reference to FIG. 12 showing information and the above-described flowchart (FIG. 15). FIG. 16 is a diagram illustrating first to third weights, final weights, reference face coordinates, distances from a reference image, Th_dis, and main face selection results for each face at each timing. For the sake of simplicity, description regarding the face size is omitted.

(Main face selection at t = 1)
The operation proceeds from step S401 to step S402 in FIG. 15. Since there are two faces detected in step S403, the process proceeds to step S404, and because Counter = 0, the process proceeds to step S1508.

  In step S1508, a first weight (Weight1) based on the reliability of each detected face, a second weight (Weight2) based on the face size, and a third weight (Weight3) based on the center position of the face are calculated. Then, the final weight is calculated by multiplying them, the face with the largest final weight is set as the main face at t = 1, and the face information is stored in the face register 224 as the reference face. Also, Counter is set to 1.

  In this embodiment, the above calculation is performed for face 1 and face 2, and as shown in FIG. 16, the final weight of face 1 is 0.42, the final weight of face 2 is 0.35, and face 1 is the main face. Set to

(Main face selection at t = 2)
In step S402, face information is read, and the process proceeds to step S1501 through steps S403 and S404.

  In step S1501, the distance between the center coordinates and the center coordinates of the reference face (face 1 at t = 1 in this embodiment) is calculated for all of the plurality of faces detected at t = 2. Then, the plurality of detected faces are sorted in order of increasing distance from the reference face. In this embodiment, the distance between the face 1 and the reference face is 10, and the distance between the face 2 and the reference face is 105. As a result of the sorting, the order is the face 1 and the face 2. Therefore, first, the process after step S1502 is performed with face 1 having a short distance as the main face candidate.

  In step S1502, the distance between the main face candidate and the reference face is calculated. Here, it is 10.

  In step S1503, the distance is compared with a preset distance Th_dis (= 30) and is equal to or less than Th_dis, and thus the process proceeds to step S1504.

  Although the details of the face size are omitted, the difference value is equal to or smaller than Th_size, and the process proceeds from step S1505 to S1506.

  In step S1506, face 1 is set as the main face.

(Main face selection at t = 3)
Face 1 is selected as the main face by the same process as the main face selection process at t = 2.

(Main face selection at t = 4)
Up to step S1501, processing similar to the main face selection processing at t = 3 is performed.

  The distance between the face 1 and the reference face is 70, and the distance between the face 2 and the reference face is 105, both of which are larger than Th_dis. Therefore, the process proceeds from step S1503 to step 1507, and further proceeds to step S1508.

  In step S1507, as in the main face selection process at t = 1, the main face is selected based on the first to third weights. Here, since the final weight of face 1 is 0 and the final weight of face 2 is 0.35, face 2 is selected as the main face.

<Summary of Second Embodiment>
As described above, according to the present embodiment, the digital camera 200 uses the distance and the size difference between the detected face and the reference face in addition to the distance between the detected face and the previous main face. To select the main face.

  Thereby, in the digital camera, it is possible to prevent the main face from being changed extremely frequently while considering that the main face is more likely to exist in the center of the imaging range.

[Other Embodiments]
The processing of each embodiment described above may provide a system or apparatus with a storage medium storing software program codes embodying each function. The functions of the above-described embodiments can be realized by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, or the like can be used. Alternatively, a CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, or the like can be used.

  The functions of the above-described embodiments are not only realized by executing the program code read by the computer. In some cases, an OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. include.

  Furthermore, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Is also included.

It is a figure which shows the face of a person and the person who becomes a main subject which the imaging device detected in time t = 1-4 according to a prior art. It is a block diagram which shows the structure and operation | movement of the digital camera 200 which is an example of the imaging device which concerns on embodiment. 6 is a diagram illustrating an example of a charge reading method of the CCD 201. FIG. 6 is a flowchart showing a flow of main face selection processing by the digital camera 200 according to the first embodiment. It is a figure which shows an example of face information. It is a figure which shows the reliability-weight characteristic graph for determining the 1st weight used for a main face selection process based on reliability. It is a figure which shows the face weight-weight characteristic graph for determining the 2nd weight used for a main face selection process based on the size of a face. It is a figure which shows the distance-weight characteristic graph for determining the 3rd weight used for a main face selection process based on distance information dist. It is a figure which shows distance dist_preMainFace of the face detected in the timing t, and the main face (previous main face) selected in the timing t-1. It is a figure which shows the 4th weight used for a main face selection process based on the distance with the last main face based on the last distance-weight characteristic graph. It is a figure which shows the person's face and the person's face used as the main subject which the imaging device detected in time t = 1-4 according to embodiment. It is a figure which shows the face information of each of the two faces which the imaging device detected at the time t = 1-4. It is a figure which shows the 1st-4th weight with respect to each face at the time t = 1-4, a final weight, the distance with the last main face, and a main face selection result. It is a figure which shows the 1st-4th weight with respect to each face, the last weight, the distance with the last main face, and the main face selection result in the time t = 1-4 when not using a 4th weight. It is a flowchart which shows the flow of the main face selection process by the digital camera 200 based on 2nd Embodiment. It is a figure which shows the 1st-3rd weight with respect to each face in each timing, a final weight, a reference face coordinate, the distance with a reference image, Th_dis, and a main face selection result.

Explanation of symbols

201 CCD
213 Display device 217 VBPF unit 218 Eye detection unit 219 Non-face filter pattern recognition unit 220 System controller 223 Resize circuit 224 Face register

Claims (9)

Imaging means for acquiring image data;
Face detection means for detecting a human face from the image data acquired by the imaging means;
Selecting means for selecting a face to be a main subject from the face of the person detected by the face detecting means;
The selecting means is
If there is information about the face that will be the main subject selected by the selection means in the past, information about the face that will be the main subject that has been selected in the past and information about the face of the person detected by the face detection means A first mode for selecting the face of the main subject in the image data;
When there is no information about the face that is the main subject selected in the past by the selection means, the face that becomes the main subject in the image data is selected from the information about the face of the person detected by the face detection means. 2. An imaging device having two modes. The information about the face that is the main subject selected by the selection means in the past and the information about the face of the person detected by the face detection means include information about the position of the face,
For the selection in the first mode, the selection unit uses a distance between a face as a main subject selected by the selection unit in the past and a human face detected by the face detection unit. The imaging device according to claim 1.   3. The imaging apparatus according to claim 1, wherein the information about the face of the person detected by the face detecting unit includes a reliability indicating the likelihood of the face, a size of the face, and a position of the face. .   In the first mode, in the first mode, when the face to be the main subject previously selected by the selection unit is selected before a predetermined time in the first mode, the main mode is selected in the second mode. The imaging apparatus according to any one of claims 1 to 3, wherein a face to be a subject is selected. A storage unit that stores information about the face that is the main subject selected by the selection unit in the second mode;
In the first mode, when there is no information satisfying a predetermined relationship with the information stored in the storage unit among the information about the face of the person detected by the face detection unit in the first mode The imaging apparatus according to claim 1, wherein a face to be a main subject is selected in the second mode. The information stored in the storage means and the information about the face of the person detected by the face detection means include information about at least one of the face position and the face size,
The predetermined relationship means that the face position of the person detected by the face detection means is within a predetermined range from the face position stored in the storage means, and the face detection 6. The method according to claim 5, further comprising at least one of a difference between a face size detected by the means and a face size stored in the storage means being within a predetermined range. The imaging device described. An imaging process for acquiring image data;
A face detection step of detecting a human face from the image data acquired in the imaging step;
A selection step of selecting a face to be a main subject from the face of the person detected in the face detection step,
The selection step comprises:
If there is information about the face that will be the main subject selected in the past in the selection step, information about the face that will be the main subject in the past and information about the face of the person detected in the face detection step A first mode for selecting the face of the main subject in the image data;
When there is no information about the face that is the main subject selected in the past in the selection step, the face that is the main subject in the image data is selected from the information about the face of the person detected in the face detection step. 2. An imaging method comprising two modes.   A program for causing a computer to execute the imaging method according to claim 7.   A computer-readable storage medium in which the program according to claim 8 is recorded.

Images