JP5023757B2 - Imaging apparatus and program - Google Patents

Description

  The present invention relates to an imaging apparatus and a program that perform imaging by recognizing a face of a subject.

2. Description of the Related Art Conventionally, there has been known an imaging device that stores face images of a plurality of subjects in advance and preferentially performs autofocus (AF) on a face when a face image that matches any of them is detected. (For example, see Patent Documents 1 and 2). Specifically, the registered face image is compared and collated with the image output from the image sensor, and when both the patterns match, the image capturing process is automatically performed.
Japanese Patent Laid-Open No. 2003-224761

  By the way, the AF utilization technique applying the face recognition technique of the above-mentioned Patent Document 1 is simply to perform AF on the face when a registered face is detected, and the application technique is being pursued. There wasn't.

  Accordingly, an object of the present invention is to provide an imaging apparatus and a program that can effectively perform processing related to imaging of a plurality of subjects by using face recognition technology.

The imaging apparatus of the present invention includes a face image information registration unit that registers face image information used for identifying faces of a plurality of persons belonging to a predetermined group, an imaging unit that captures an image of a subject and generates image information, and the imaging Face detection means for detecting a plurality of face portions from the image information generated by the means, and face image information registered in the face image information registration means for each of the plurality of face portions detected by the face detection means; Face collation means for performing a comparison and collation to determine whether the person belongs to the predetermined group, the number of persons determined to be a person belonging to the predetermined group by the face collation means, and all the groups determination means for determining the difference between the number of the difference in number of the automatic photographing operation or warning operation in the shooting group photo intended for the predetermined group is determined by the determining means It is characterized by and a photographing control means for controlling.

The program according to the present invention registers a face image information for registering face image information used for identifying faces of a plurality of persons belonging to a predetermined group using a computer of an image pickup apparatus including an image pickup unit that picks up an image of a subject and generates image information A face detecting unit for detecting a plurality of face parts from the image information generated by the imaging unit, and a plurality of face parts detected by the face detecting unit are registered in the face image information registering unit. Face collation means for performing comparison and collation with the face image information to determine whether the person belongs to the predetermined group, and the number of persons determined to be a person belonging to the predetermined group by the face collation means When a determination means for determining the difference between the number of the entire group, the automatic photographing operation or warning operation in the shooting group photo intended for the predetermined group, wherein Is characterized and imaging control means for controlling in response to the difference between the number of people is determined, that is to function by cross section.

According to the present invention, it is possible to effectively take a group photo of a plurality of persons belonging to a predetermined group using face recognition technology, and it is possible to provide a more convenient imaging device.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to the first embodiment to which the present invention is applied.
The imaging apparatus (image processing apparatus) 100 according to the first embodiment detects (recognizes) a person's face from image data, and uses the detection (recognition) result to capture, browse, and process image data. Various processes for image data such as these are automated.
As shown in FIG. 1, the imaging apparatus 100 includes an imaging unit 1, an image recording unit 2, an image acquisition control unit 3, a personal face database (DB) 4, a group database (DB) 5, and an operation input unit. 6, a face detection unit 7, a detection information storage unit 8, a specific condition determination unit 9, a display unit 10, a process execution unit 11, and the like.

  The imaging unit 1 captures a still image or a moving image of a subject such as a person. Specifically, the imaging unit 1 includes an imaging lens 1a, an imaging element 1b, a signal processing unit 1c, an image memory 1d, and the like.

The imaging lens 1a has a focus function and a zoom function, and is composed of a plurality of imaging lenses.
The image sensor 1b is composed of a CCD (Charge Coupled Device), a CMOS (Complementary Metal-Oxide Semiconductor), or the like that converts a subject image that has passed through the imaging lens 1a into a two-dimensional image signal.
The signal processing unit 1c performs predetermined image processing on the image signal output from the image sensor 1b.
Here, the imaging lens 1a, the imaging element 1b, and the signal processing unit 1c constitute imaging means for imaging a subject and generating image information.
The image memory 1d temporarily stores an image signal after image processing.

Although not shown, the imaging unit 1 includes a focus driving unit and a zoom driving unit that are driven when the imaging unit 1 captures an object.
The focus drive unit drives the focus mechanism unit connected to the imaging lens 1a.
The zoom drive unit drives the zoom mechanism unit connected to the imaging lens 1a.

The image recording unit 2 records still images and moving images captured by the imaging unit 1 as image files. The image recording unit 2 is configured by, for example, a card-type nonvolatile memory (flash memory), a hard disk, or the like.
Here, the image recording unit 2 is a still image information storage unit that stores the image information generated by the imaging unit (imaging unit) 1 as a still image, and the image information continuously generated by the imaging unit 1 is a moving image. The moving image information storing means for storing as is configured.

  The image acquisition control unit 3 reads out image data from the image memory 1 d or the image recording unit 2 of the imaging unit 1 and outputs the image data to the processing execution unit 11 and the face detection unit 7. Specifically, the image acquisition control unit 3 is a plurality of targets for various shooting / image processing in the process execution unit 11 in response to a user instruction via the operation input unit 6 or a request from the process execution unit 11. Are sequentially acquired from the image memory 1d or sequentially read out from the image recording unit 2 and output to the processing execution unit 11 and the face detection unit 7.

The personal face DB 4 registers face image information related to a face image of each subject as a face image information registration unit.
Specifically, as shown in FIG. 2, the personal face DB 4 stores the subject name, face number, face image data, face feature data, priority, and the like in association with each other. For example, exemplifying “Mr. A” as a subject, the face number “A-1” includes image data obtained by imaging Mr. A from the front and features of the face viewed from the front (for example, eye position, nose). , The size of the mouth, and the like) and the orientation of the face, and the priority “2” relating to the preferential processing in the plurality of Mr. A's images is associated. Similarly, the face number “A-2” has the image data of Mr. A taken from 30 ° on the right, the feature of the face seen from the right 30 °, and the priority “5” among the images of Mr. A. (Worst) ”is associated, and the face number“ A-3 ”is the image data obtained by imaging Mr. A from the left 15 °, the facial features viewed from the left 15 °, and the images of the plurality of A's images. Is associated with the priority “1 (best)”. That is, the personal face DB 4 stores information for specifying the individual and the face direction based on the face image.

The group DB 5 registers a subject for each group to which the subject belongs as face image information registration means.
Specifically, as shown in FIG. 3, the group DB 5 stores a group name, the number of extra prints, members, and the like in association with each other. For example, exemplifying “Mr. A's family” as a group, the group name “Ms. A's family” has the number of extra prints “collectively 1” and the member names “Mr. A” and “Mr. C”. , “Mr. D”, “Mr. E”, and the like are associated with each other. Similarly, in the group “B's friend”, the number of extra prints “separate” and the member names “B”, “C”, “F” and the like are associated.

The operation input unit 6 is for performing a predetermined operation of the imaging apparatus 100. Specifically, although not shown, the operation input unit 6 includes a shutter button, a mode switching button, an input button, and the like.
The shutter button instructs to record (save) the image information of the subject imaged by the imaging unit 1.
The mode switch is for switching the imaging mode to a still-in movie processing mode, a motion print processing mode, an image deletion processing mode, an automatic shutter processing mode, a reprint number setting processing mode, or the like.
The input button is used to input registration contents of the personal face DB 4 and group DB 5, specific image determination conditions (described later), and the like. Specifically, the input button inputs a priority associated with the face direction (face number) for each subject (person).
Here, the input button (operation input unit 6) constitutes a face direction designating unit for designating the face direction of the subject instructing execution of the image information processing by the processing execution unit 11.

  Operation signals input from the buttons of the operation input unit 6 are output to the process execution unit 11.

The display unit 10 displays an image captured by the imaging unit 1. Specifically, the display unit 10 includes a display control unit, an image display unit, and the like, although illustration is omitted.
The display control unit includes a video memory (not shown) that temporarily stores display data output from the processing execution unit 11 as appropriate.
The image display unit includes a liquid crystal monitor that displays a predetermined image based on an output signal from the display control unit.

When the face detection function and the face recognition function are in the ON state, the face detection unit 7 individually detects a face image in this image data according to the image data output from the image acquisition control unit 3, The detected face image is recognized by referring to the personal face DB 4, and face individual detection information 8a described later is sequentially output.
Specifically, the face detection unit 7 first detects all face parts from the image data (image frame) output by the image acquisition control unit 3. For example, the face detection unit 7 detects all the skin color areas, determines whether or not there is a characteristic part (face part) corresponding to eyes, nose, or rho in each detected area. If it is determined that there is an image, the skin color area is regarded as a face image. Next, the face detection unit 7 stores the size determined from the position of each detected face image portion, the area of the skin color region, and the like in the detection information storage unit 8 as face individual detection information 8a. Then, the face detection unit 7 collates each face image portion with the personal face DB 4 and stores the name of the person corresponding to the record with the highest matching degree and the face number in the detection information storage unit 8 as face individual detection information 8a. Remember. In other words, by identifying the detected face image with the face image data in the personal face DB, or by comparing the feature data extracted from the detected face image with the face feature data in the personal face DB, the identification of the individual and the face Specify (estimate) the orientation of It should be noted that the degree of coincidence is unknown when a predetermined number of persons or more are not registered.

Note that if the target image frame is obtained during continuous imaging, the face detection unit 7 refers to the individual face detection information 8a corresponding to the immediately preceding image frame, and applies to the immediately preceding captured frame. A tracking process is performed on a detected face image portion based on a detection position shift or the like. Here, the face detection unit 7 limits the detection range and detection position in the image frame and the collation range and collation order with the personal face DB 4 in order to follow the detected face image portion in the tracking process. The process of doing is performed. The movement of each face part is specified by this tracking process, and the specified movement is stored in the detection information storage unit 8 as face individual detection information 8a.
Here, the face detection unit 7 constitutes a face detection unit that detects a face image from image data. Further, the face detection unit 7 determines whether or not the direction of the face of the subject input via the operation input unit 6 is equal to the direction of the face of the subject detected by the face detection unit 7. An angle determination unit is configured.
Furthermore, the face detection unit 7 constitutes a face collating means for comparing and collating the detected face image with the face image information registered in the personal face DB 4.
The face detection unit 7 constitutes a face related information acquisition unit that acquires various types of face related information related to the face image.

The face individual detection information 8a output by the face detection unit 7 will be described below with reference to FIG.
As shown in FIG. 4, the face individual detection information 8a is stored in association with the detection number, position, size, movement, person name, face number in the personal face DB 4, personal matching degree, etc. for each face image portion. Yes. For example, the detection number 1 will be described as an example. The position (coordinates) of the face image portion “213, 378”, the size of the face image portion “120”, the operation state “30 pixels / frame to the right”, the person name “Unknown” or the like is associated and stored. Similarly, the detection number 2 includes the position (coordinates) “563, 423” of the face image portion, the size “375” of the face image portion, the operation state “still”, the person name “Mr. B”, and the face number “ B-2 ”is associated with the degree of personal coincidence“ 90% ”, and the detection number 3 is the position (coordinates)“ 783,324 ”of the face image portion, the size“ 265 ”of the face image portion, and the operation state “Still”, person name “Mr. F”, face number “F-3”, personal matching degree “85%” and the like are stored in association with each other.

In addition, the face detection unit 7 refers to the group DB 5 to determine the detection status of all faces in the image data based on the individual face detection information 8a, and outputs the entire face detection information 8b.
Specifically, the face detection unit 7 refers to the group DB 5 to calculate the degree of matching of the group between the plurality of members in each group and the detected face images of the plurality of subjects (persons), and the best match is obtained. A group having a high degree is identified, and the identified group name, the degree of coincidence, and the number of extra prints corresponding to the group are stored in the detection information storage unit 8 as whole face detection information 8b. Here, the degree of coincidence is determined based on the number of matching persons. If the number of persons is the same, the degree of matching is determined based on the matching degree of the face image.
Next, the face detection unit 7 refers to the individual face detection information 8a, counts and stores the total number of detected face images, and then counts the total number of face images that can be specified by each individual to calculate the entire face. The detection information is stored in the detection information storage unit 8 as detection information 8b.

Furthermore, the face detection unit 7 specifies and stores the face image of the main person in the target image frame. As a method for specifying the main person, when the target image frame is obtained during continuous imaging, the face image portion with the least movement relative to the shooting frame is specified as the face image of the main person, If the target image frame is not obtained during continuous imaging, conditions such as whether it is located at the center in the imaging frame, is the largest in size, or exists in the AF area, are set in advance. The face image of the main person is specified by evaluating with the designated priority.
In addition, the movement direction of each of the face images being tracked is acquired as quantitative data by a face detection sensor, and the movement direction of the image pickup apparatus is quantitatively determined by an acceleration sensor mounted on the image pickup apparatus. Get with data. Then, each face image is analyzed to determine whether the face moving direction being tracked is the same direction or a different direction with respect to the moving direction of the imaging device. It is determined that the closest face image is the main subject to be photographed for the photographer. Then, it is also possible to automatically set the optimum setting for imaging the main subject so as to be ready for shooting, including AF processing.
As a result, if a plurality of face images are recognized during face recognition, it is difficult for the imaging device to automatically determine which person should be the main subject. By determining the movement of the subject, the imaging device can automatically determine the main subject.
Then, the priority corresponding to the face number having the highest degree of coincidence with the face image of the main person is specified with reference to the personal face DB 4 and stored in the detection information storage unit 8 as the entire face detection information 8b.
Further, the movement of the face image having the largest absolute movement in each face image is stored as the movement of the entire face image. Here, the absolute movement of the face image refers to the movement of the face image with respect to the ground, and based on the information of an acceleration sensor (not shown) built in the camera, the angle of view information indicated by the zoom magnification, etc. Is obtained by a method of converting the motion of each face image to the absolute motion with respect to the ground.

Hereinafter, the entire face detection information 8b output by the face detection unit 7 will be described with reference to FIG.
As shown in FIG. 5, the whole face detection information 8 b is associated with the total number of face detections, the total number of individual recognitions, the group, the group matching degree, the overall movement, the main person, the priority, the number of extra prints, and the like. For example, the total face detection information 8b includes a total face detection number “7”, a personal recognition total number “3”, a group “Mr. B family”, a group matching degree “80%”, an overall movement “with moving face”, a main person Those associated with “Mr. B” are stored.

The detection information storage unit 8 temporarily stores the face individual detection information 8a and the entire face detection information 8b, and is updated by the face detection unit 7 every time new image data is output from the image acquisition control unit. In addition, the detection information storage unit 8 displays the individual face detection information 8a and the entire face detection information 8b as an image when a file storage instruction is given to use the individual face detection information 8a and the entire face detection information 8b later. It is stored (saved) as a file with the same file name and a different extension.
Here, the detection information storage unit 8 records the individual face detection information 8a and the entire face detection information 8b (face related information) acquired by the face detection unit 7 and the image file recorded in the image recording unit 2 in association with each other. It constitutes face related information recording means.

The specific condition determination unit 9 determines execution conditions for various processes performed by the process execution unit 11 such as a still-in movie process. That is, the specific condition determination unit 9 sets the face individual detection information 8a and the whole face detection information 8b in response to the output of the new face individual detection information 8a and the whole face detection information 8b from the face detection unit 7. It is determined whether or not the specified condition determination information 9a matches, and if they match, the matching image data and instruction information indicating the matching timing are sequentially output.
Here, the specific condition determination unit 9 constitutes a processing target determination unit that determines image information to be processed by the processing execution unit 11 based on the determination result of the face angle of the subject by the face detection unit 7. .

Here, an example of the specific condition determination information 9a that can be arbitrarily set by the user will be described with reference to FIG.
As shown in FIG. 6, the specific condition determination information 9a includes, for example, the total number of face detections, the total number of individual recognitions, the group, the group matching degree, the overall movement, the main person, the face state (priority) of the main person, The size and the like are associated with each other. The specific condition determination information 9a is arbitrarily set or automatically set by the user according to the content and purpose of the process executed by the process execution unit 11, and for example, the main person “Mr. B”, What is associated with the face state “priority 1” of the main person is stored.
When face individual detection information 8a and whole face detection information 8b corresponding to the image data output from the image acquisition control unit 3 are already recorded as face information files (described later), the face detected by the face detection unit 7 The specific condition determination unit 9 does not update the individual detection information 8a and the entire face detection information 8b, and makes a determination using the individual face detection information 8a and the entire face detection information 8b already recorded in the face information file.

The process execution unit 11 performs image data shooting processing (still-in movie processing, still image shooting processing, etc.), image data browsing processing (image search processing, image display processing, etc.), Various processes for image data such as image data processing (motion print processing, image deletion processing, extra number setting processing, etc.) are performed. In these processes, a process for a plurality of image data is performed, but specific image data or specific timing is processed differently from other image data or other timing. Normally, the specific image data and specific timing are instructed by the user's manual operation. However, when the face detection function and the face recognition function by the face detection unit 7 are set to the ON state, This specific image data and specific timing are automatically instructed by the specific condition determination unit 9.
The process execution unit 11 determines a plurality of image data to be processed manually according to a user instruction or automatically according to execution contents and purpose, and a plurality of image data to be processed with respect to the image acquisition control unit 3. Request to get image data. Then, in response to this request, a process for a plurality of image data output by the image acquisition control unit 3 is executed, and the image data or timing related to the instruction information instructed by the specific condition determination unit 9 is set to the specific image data or the timing. Assuming that it is a specific timing, processing is performed differently from other image data and other timing.

  As described above, according to the imaging apparatus 100 of the first embodiment, various processes for image data are performed using information on the face of a subject (person) detected (recognized) in the image data. It is possible to automate, and it is possible to flexibly and easily set the conditions for automation using information relating to the face.

[Embodiment 2]
In the second embodiment, the imaging apparatus 100 according to the first embodiment is applied to the automation of still-in movie processing. In particular, a still image is shot during moving image shooting according to the face angle designated for each subject. The timing can be automatically determined.
In the second embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

When the still execution movie process is executed by the process execution unit 11, the specific condition determination unit 9 serves as a stored image determination unit based on the determination result of the face angle of the subject by the face detection unit 7. Among a plurality of continuous image frames (moving image data) stored in a predetermined image frame, a predetermined image frame stored as a still image in the image recording unit 2 is determined. That is, the specific condition determination unit 9 obtains an image frame with a face orientation of priority 1 corresponding to the face state of the main person in the specific condition determination information 9a during the capturing of a moving image by the image capturing unit 1. Instruction information related to an instruction to determine the image frame as a storage target still image is output to the process execution unit 11.
Here, the still-in movie processing is a function for taking a still image during movie shooting. In the case of normal still-in movie processing that is not automated, the movie is shot when the photographer presses the shutter during movie shooting. The shooting is temporarily interrupted to record and record a still image, and after the still image shooting, the moving image shooting is resumed.

Next, the still-in movie process automated will be described with reference to FIG.
As shown in FIG. 7, first, the still-in movie processing mode is set based on a predetermined operation of the mode setting button of the operation input unit 6. At this time, conditions for capturing a still image during moving image shooting are also set in the specific condition determination information 9a. Here, it is assumed that a still image is shot and recorded when the orientation of Mr. B's face becomes the favorite orientation during video recording, and the main person in the specific condition determination information 9a is set to “Mr. B”. Is set to “priority 1”. When the still-in movie processing mode is set, “priority 1” is set by default, and when changing this, the user designates an arbitrary priority or a range of priority (step) S1).
Next, the process execution unit 11 determines whether or not the face detection function and the face recognition function by the face detection unit 7 are in an ON state, and when they are in an ON state, the following automatic still-in movie process is performed. Execute. Note that in the OFF state, normal still-in movie processing is executed.

  The process execution unit 11 is obtained by performing continuous imaging with the imaging unit 1 within a specified period based on a predetermined operation (instruction for starting and ending moving image shooting) of a predetermined input button of the operation input unit 6. The image acquisition control unit 3 is instructed to sequentially output image data (step S2).

Subsequently, the face detection unit 7 refers to the personal face DB 4, a plurality of face image information of the personal face DB 4, and image data (image frame) sequentially output from the image acquisition control unit 3 in response to the instruction. ) Are compared (step S3), and face individual detection information 8a is output (step S4).
The specific condition determination unit 9 receives the face individual detection information 8a output from the face detection unit 7, and based on the face individual detection information 8a, the priority (here, the specific condition determination information 9a) is specified. It is determined whether or not it matches the image information of the face orientation with the priority 1) corresponding to the most favorite face orientation (step S5). Here, if it is determined that the image information matches the image information of the face orientation of priority 1 (step S5; YES), the specific condition determination unit 9 designates the matched image data as a target image for still image capturing. Is output. Alternatively, the instruction information specifying the coincidence timing as the still image capturing timing is output (step S6).

When the instruction information output from the specific condition determination information 9a is input, the process execution unit 11 automatically acquires an image frame related to the instruction information as a still image. Alternatively, a still image is automatically photographed and recorded at the photographing timing related to the instruction information (step S7).
The above process is repeated until the output of the image data from the image acquisition control unit 3 is completed (step S8; YES).

Next, the processing execution unit 11 processes the plurality of image frames output from the image acquisition control unit 3 into a moving image file and records the processed image data in the image recording unit 2, and the captured image data as individual still images. Recording is performed in the recording unit 2 (step S9).
As a result, the still-in movie process is terminated.

As described above, according to the imaging apparatus 100 of the second embodiment, whether the face direction of the subject specified by the operation input unit 6 is equal to the face direction of the subject detected by the face detection unit 7 or not. Therefore, based on the determination result, it is possible to determine the image data to be processed by the process execution unit 11, so that the image data is determined according to the face angle specified for each subject. The various processes can be preferentially performed.
Further, in the second embodiment, the still-in movie process can be automated, and the priority of the face angle of the favorite (favorite) of the subject is increased and set in the specific condition determination information 9a. When the image data of the angle is generated by the imaging unit 1, it can be automatically acquired and recorded in the image recording unit 2, and a more convenient imaging device can be provided.

[Embodiment 3]
In the third embodiment, the imaging apparatus 100 according to the first embodiment is applied to the automation of image deletion processing, and in particular, an image to be deleted can be automatically determined according to the face angle designated for each subject. It is a thing.
In the third embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

  When the process execution unit 11 executes the image deletion process, the specific condition determination unit 9 serves as a deletion image determination unit based on the determination result of the face angle of the subject by the face detection unit 7. Determine the image information to be deleted. That is, when the specific condition determination unit 9 acquires image information of the face orientation with the lowest priority corresponding to the deletion target image (not shown) of the specific condition determination information 9a, the specific condition determination unit 9 sets the image information as the deletion target image. Instruction information related to the instruction to be determined is output to the process execution unit 11.

  The process execution unit 11 deletes the image information stored in the image recording unit 2 as an image deletion unit. Specifically, the process execution unit 11 actually deletes image information to be deleted or lists image information to be deleted.

Next, an automated image deletion process will be described with reference to FIG.
In the following image deletion process, a person to be deleted is selected in advance, and registered as specific condition determination information 9a together with the face orientation of the lowest priority (for example, priority 5) of the person. It is assumed that When the image deletion processing mode is set, “priority 5” is set by default, and when changing this, the user designates an arbitrary priority or a range of priorities.

As shown in FIG. 8, first, the image deletion processing mode is set based on a predetermined operation of the mode setting button of the operation input unit 6 (step S11).
Next, the process execution unit 11 determines whether or not the face detection function and the face recognition function by the face detection unit 7 are in an ON state, and when they are in an ON state, the following automatic image deletion process is performed. Execute (if the image is in the OFF state, execute a normal image deletion process).
Based on a predetermined operation of a predetermined input button of the operation input unit 6, the process execution unit 11 sequentially selects image files in a specified image set (group) among the image files recorded in the image recording unit 2. The image acquisition control unit 3 is instructed to output (step S12).

Subsequently, the face detection unit 7 refers to the personal face DB 4 and compares the plurality of face image information of the personal face DB 4 with the image file sequentially output from the image acquisition control unit 3 in response to the instruction. After collation (step S13), the face individual detection information 8a is output (step S14).
The specific condition determination unit 9 receives the entire face detection information 8b output from the face detection unit 7, and based on the entire face detection information 8b, the priority (here, the specific condition determination information 9a) is specified. It is determined whether or not it matches the image information of the face orientation of priority 5) corresponding to the face orientation with the worst reflection (step S15). If it is determined that the image information matches the image information of the face orientation of priority 5 (step S15; YES), the specific condition determination unit 9 specifies instruction information for designating the matched image data as an image deletion target image. Output (step S16).

When the instruction information output from the specific condition determination information 9a is input, the process execution unit 11 acquires the image file related to the instruction information as a deletion target image and adds it to the deletion list (step S17).
The above process is repeated until the output of the image file from the image acquisition control unit 3 is completed (step S18; YES).

And the process execution part 11 determines whether the user confirmation mode before deletion is set (step S49), and if the user confirmation mode is not set (step S19; NO), it will be in a deletion list. The listed image file is immediately (automatically) deleted (step S20).
On the other hand, if the user confirmation mode is set (step S19; YES), the deletion list is displayed as a list and deleted according to the user's instruction (step S21).
Thus, the image deletion process is terminated.

As described above, according to the imaging apparatus 100 of the third embodiment, whether the face orientation of the subject specified by the operation input unit 6 is equal to the face orientation of the subject detected by the face detection unit 7 or not. Therefore, based on the determination result, it is possible to determine the image data to be processed by the process execution unit 11, so that the image data is determined according to the face angle specified for each subject. The various processes can be preferentially performed.
Further, in the third embodiment, the image deletion process can be automated, and the priority of the angle of the face that the subject does not like so much is lowered and set in the specific condition determination information 9a, whereby the angle of the face is determined. Can be obtained from the image recording unit 2 and automatically listed as deletion candidates, and an image of only the user's favorite face can be left in the image recording unit 2, which is more convenient. An imaging device with high performance can be provided.

[Embodiment 4]
In the fourth embodiment, the imaging apparatus 100 according to the first embodiment is applied to the automation of the motion print process. In particular, the main image and the surrounding image in the motion print process are displayed according to the face angle specified for each subject. It can be automatically determined.
In the fourth embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

  Here, the motion print processing is to generate (print) one still image by arranging and synthesizing a plurality of frame images extracted from moving image data in a predetermined layout, and is not an automated motion print In the case of processing, the user selects a frame that is mainly arranged (for example, arranged in the center) in the predetermined layout.

  In the fourth embodiment, as illustrated in FIG. 9, specific condition determination information 209 a configured from a plurality of determination conditions 901 to 90 n is stored, and the process execution unit 11 performs a motion print process according to each determination condition.

As shown in FIG. 10A, the specific condition determination information 209a includes, for example, each condition number (determination condition), face detection total number, personal recognition total number, group, group matching degree, overall movement, main person, main The face state (priority) of the person, the size of the main person, and the like are associated with each other. In addition, as shown in FIG. 10B, each condition number associates each process being activated with the handling of a specific image (specific timing).
For example, as the specific condition determination information 209a, the determination condition of condition number 1 includes the main person “Mr. B”, the face state “priority 1” of the main person, the process “motion print” being activated, and the specific image Is handled as “main image M”. Similarly, the judgment condition of condition number 2 is that the main person “Mr. B”, the face state “priority 2 to 4” of the main person, the process “motion print” being activated, and the specific image “ Handling as “sub) image” is associated.

  When the motion printing process is executed by the process execution unit 11, the specific condition determination unit 9 is used as a main image determination unit by the process execution unit 11 based on the determination result of the face angle of the subject by the face detection unit 7. Image information related to the main image of the composite image to be generated is determined. That is, the specific condition determination unit 9 acquires the image information of the priority 1 face orientation corresponding to the face state of the main person in the specific condition determination information 209a during the imaging of a plurality of consecutive images by the imaging unit 1. In this case, the instruction information related to the instruction to determine the image information as the main image M is output to the process execution unit 11.

That is, the process execution unit 11 generates one composite image from a plurality of continuous image frames (moving image data) as a composite image generation unit. Specifically, the process execution unit 11 selects one image frame to be the main image M from among the plurality of image frames, and the image frame to be the sub image S arranged around the main image M. A predetermined number is selected (see FIG. 11).
Here, the process execution unit 11 selectively executes an operation corresponding to the condition determined to be matched by the specific condition determination unit 9.

Hereinafter, the motion print process will be described with reference to FIGS.
It is assumed that the motion print process is executed when the face detection function and the face recognition function by the face detection unit 7 are ON.

As shown in FIG. 12, first, the motion print processing mode is set based on a predetermined operation of the mode setting button of the operation input unit 6 (step S31).
Next, the process execution unit 11 sequentially outputs image data obtained by continuously capturing images with the imaging unit 1 within a specified period based on a predetermined operation of a predetermined input button of the operation input unit 6. In this way, the image acquisition control unit 3 is instructed (step S32).

Subsequently, the face detection unit 7 refers to the personal face DB 4 and compares and collates the plurality of face image information of the personal face DB 4 with the image data (image frame) output from the image acquisition control unit 3. (Step S33), the entire face detection information 8b is output (Step S34).
Then, the specific condition determination unit 9 has a priority 1 corresponding to the face state of the main person related to the condition number 1 of the specific condition determination information 209a based on the entire face detection information 8b output and input from the face detection unit 7. It is determined whether the image information matches the image information of the face direction (step S35). Here, when it is determined that the image information matches the image information of the face orientation of priority 1 (step S35; YES), the specific condition determination unit 9 outputs instruction information specifying the matched image data as the main image M. (Step S36).
If it is determined in step S35 that the image information does not match the image information of the priority 1 face orientation (step S35; NO), the specific condition determination unit 9 relates to condition number 1 of the specific condition determination information 209a. It is determined whether or not it matches the image information of the face orientations of priorities 2 to 4 corresponding to the face state of the main person (step S37). Here, if it is determined that the image information matches the image information of the face orientations with priorities 2 to 4 (step S37; YES), the specific condition determination unit 9 arranges the matched image data around the main image M. Instruction information to be designated as the sub image S is output (step S38).

When the instruction information output from the specific condition determination unit 9 is input, the process execution unit 11 adds a flag indicating that it matches each condition number of the specific condition determination information 209a to the corresponding image frame (step S39). ).
The above process is repeated until the output of the image data from the image acquisition control unit 3 is completed (step S40; YES).

  Next, the process execution unit 11 processes the plurality of image frames output from the image acquisition control unit 3 into a moving image file together with flag information, and causes the image recording unit 2 to record it (step S41).

If it is not instructed to execute printing immediately, the processing shown in FIG. 13 is executed at an arbitrary timing at a later date in accordance with an instruction from the user.
As shown in FIG. 13, when the execution processing unit arranges and synthesizes a plurality of image frames included in the moving image file according to predetermined layout information, the execution processing unit selects the image frame to which the flag corresponding to the condition number 1 is added. The image M is arranged with the largest size at the center position (step S42).
Next, the processing execution unit 11 arranges the image frame to which the flag corresponding to the condition number 2 is added as a sub image S in a small size around the main image M according to the layout information (step S43).

The process execution unit 11 causes the image recording unit 2 to record the single combined image that has been arranged and synthesized in accordance with an instruction from the user, or prints it using a printing device connected to the imaging apparatus (step S44). .
Thus, the motion print process is completed.

As described above, according to the imaging apparatus 200 of the fourth embodiment, whether or not the face direction of the subject specified by the operation input unit 6 and the face direction of the subject detected by the face detection unit 7 are equal. Therefore, based on the determination result, it is possible to determine the image data to be processed by the process execution unit 11, so that the image data is determined according to the face angle specified for each subject. The various processes can be preferentially performed.
That is, in the motion print process, the priority of the favorite (favorite) face angle of the subject is set high in the specific condition determination information 209a so that the image data of the face angle is captured by the imaging unit 1. Once generated, it can be automatically acquired as the main image M, and a more convenient imaging device can be provided. In addition, by making the determination conditions of the specific condition determination information 209a plural, and making the priority of the sub-image S arranged around the main image M lower than the priority of the main image M, the sub-image S is automatically set. Can be acquired.
Therefore, it is possible to automatically create a motion print in which the main image M and the sub image S are properly arranged at predetermined positions.

  In step S32, the image acquisition control unit 3 sequentially outputs the image data acquired by the imaging unit 1. However, the image acquisition control unit 3 is not limited to this, and is recorded in the image recording unit 2, and is recorded by the user. The image files in the instructed image set (group) may be output in order.

  Further, in the above-described motion print processing, based on a predetermined operation of a predetermined input button of the operation input unit 6, a person name (for example, “A child”) of a subject to be subjected to the motion print processing is set. Then, the face detection unit 7 may preferentially select the face image information related to the person from among the plurality of face image information in the personal face DB 4 and obtain the data as data for comparison and collation. Thereby, the face image information related to the person can be preferentially processed.

  Furthermore, the sub image S is selected and acquired according to the condition number 2 of the specific condition determination information 209a. However, the sub image S is not limited to this, and for example, the sub image S is arranged with a certain interval with respect to the main image M. Alternatively, an image frame or an image frame having a large amount of change between before and after may be selected.

  In the above embodiment, the processing according to the plurality of determination conditions 901 to 90n is applied to the motion print processing. However, the present invention is not limited to this. For example, still-in movie processing, automatic shutter You may make it apply to various processes, such as a process and an image deletion process.

[Embodiment 5]
In the fifth embodiment, the imaging apparatus 100 according to the first embodiment is applied to the automation of related processing related to group photo shooting and group photo shooting. In particular, the group detected (recognized) in a shooting frame. When a member of a photo is in a predetermined state, a still image is automatically taken or related processing related to still image shooting is performed.
In the fifth embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

  As illustrated in FIG. 14, the imaging apparatus 300 according to the fifth embodiment stores specific condition determination information 309a including a plurality of determination conditions 901 to 90n, and the process execution unit 11 performs automatic shutter processing and the like according to each determination condition. A related process related to the process is performed.

That is, as shown in FIGS. 15A and 15B, the specific condition determination information 309a includes, for example, the determination condition of condition number 1 is the group “Mr. B's family” and the group matching degree “70˜ “80%” is associated with the process “automatic photographing” being activated and “warning that there are not enough members in the group” at a specific timing. Similarly, the determination condition of condition number 2 is the group “Mr. B's family”, the group matching degree “80% or more”, the overall movement “with movement”, and the process “automatic shooting” being activated. It is associated with “warning that all members of the group are not stationary” at the timing. Similarly, the determination condition of condition number 3 is specified as the group “family of Mr. B”, the group matching degree “80% or more”, the entire movement “no movement”, and the process “automatic shooting” being activated. Performing “shooting and recording” at the timing is associated.
Here, the group matching degree “70 to 80%” matches, for example, when all members except one photographer who operates the camera are gathered when Mr. B's family is four in total. The group matching degree “80% or more” is a numerical value set so as to match when all four people are together. The group matching degree is changed according to the number of people in the entire group, and the number of people may be designated instead of the percentage.

  When the automatic shutter process is executed by the process execution unit 11, the specific condition determination unit 9 serves as a stored image determination unit in the image recording unit 2 based on the determination result of the face angle of the subject by the face detection unit 7. Determine the image information to be saved. That is, the specific condition determination unit 9 has a group matching degree of the group “Mr. B's family” in the specific condition determination information 309a of 80% or more during the capturing of the through image by the imaging unit 1, and the entire movement. The instruction information related to the instruction to determine the image information in a state where there is no more as a still image is output to the process execution unit 11.

  The processing execution unit 11 executes imaging-related processing related to imaging of the subject by the imaging unit 1 according to the number of face images collated by the face detection unit 7 as imaging-related processing execution means. In other words, the process execution unit 11 corresponds to the face image detected by the face detection unit 7 and the collation result for each person and each group, that is, the face image information registered in the individual face DB 4 and the members of the group DB 5. In accordance with the collation result, for example, when the face detection unit 7 collates face images of members of a predetermined group, the group matching degree is equal to or higher than a predetermined ratio, and the entire movement is lost, the imaging unit 1 An automatic shutter process for automatically capturing an image of the subject is executed.

  The process execution unit 11 sets the number of extra prints of the image data captured by the imaging unit 1 as the extra number setting unit. Specifically, the process execution unit 11 corresponds to the face image detected by the face detection unit 7 and the collation result for each individual and each group, that is, the face image information registered in the personal face DB 4 and the group DB 5. The number of extra prints is set according to the collation result with the member.

Next, automatic shutter processing will be described with reference to FIG.
It is assumed that the automatic shutter process is executed when the face detection function and the face recognition function by the face detection unit 7 are ON.

As shown in FIG. 16, first, the automatic shutter processing mode is set based on a predetermined operation of the mode setting button of the operation input unit 6 (step S51).
Next, the process execution unit 11 performs monitor display image data obtained by continuously capturing images within the specified period based on a predetermined operation of a predetermined input button of the operation input unit 6. Are sequentially output to the image acquisition control unit 3 (step S52).

  Subsequently, the face detection unit 7 refers to the personal face DB 4 and compares and collates the plurality of face image information of the personal face DB 4 with the image data (image frame) output from the image acquisition control unit 3. The face individual detection information 8a is output (step S53). Further, the face detection unit 7 refers to the group DB 5 to collate a plurality of members in each group with a plurality of detected persons, and outputs the entire face detection information 8b (step S54).

  Then, the specific condition determination unit 9 matches each determination condition defined in condition number 1 or condition number 2 of the specific condition determination information 309a based on the entire detection information 8b output and input from the face detection unit 7. It is determined whether or not (step S55). Here, if it is determined that the condition number 1 matches each determination condition (step S55; YES), the specific condition determination unit 9 issues an instruction to issue a “warning for lack of members” at the timing when the image data matches. Output information. If it is determined that the condition number 2 matches each determination condition, instruction information for issuing a “warning that all members of the group are not stationary” is output (step S56). Further, when it is determined in step S55 that the determination conditions of condition number 1 and condition number 2 do not match (step S55; NO), the specific condition determination unit 9 sets the condition number 2 of the specific condition determination information 309a. It is determined whether or not each of the specified determination conditions is met (step S57). Here, if it is determined that each of the determination conditions of condition number 2 matches (step S57; YES), the specific condition determination unit 9 outputs instruction information for performing “shooting and recording” at the timing when the image data matches. (Step S58).

When the instruction information related to “warning” output from the specific condition determination information 309a is input, the process execution unit 11 controls the display unit 10 to display a predetermined warning screen or from a speaker (not shown). A predetermined warning sound is generated (step S59).
Further, when the instruction information related to “shooting and recording” output from the specific condition determination information 309a is input, the process execution unit 11 acquires the image frame related to the instruction information as a still image (step S60).
The above process is repeated until the output of the image data from the image acquisition control unit 3 is completed (step S61; YES).

Next, the process execution unit 11 causes the image recording unit 2 to record the captured image data as individual still images (step S62).
Thereby, the automatic shutter process is terminated.

Next, the extra number setting process will be described with reference to FIG.
In the following extra number setting process, it is assumed that a predetermined group (for example, “A's family”) is registered in advance in the specific condition determination information 309a.

As shown in FIG. 13, first, the extra number setting processing mode is set based on a predetermined operation of the mode setting button of the operation input unit 6 (step S71).
Next, the process execution unit 11 selects an image in a specified image set (group) among the image files recorded in the image recording unit 2 based on a predetermined operation of a predetermined input button of the operation input unit 6. The image acquisition control unit 3 is instructed to output the files in order (step S72).

Subsequently, the face detection unit 7 refers to the personal face DB 4 and compares and collates the plurality of face image information of the personal face DB 4 with the image data (image frame) output from the image acquisition control unit 3. The face individual detection information 8a is output (step S73). Further, the face detection unit 7 refers to the group DB 5 and collates a plurality of members in each group with a plurality of detected persons, and outputs the entire face detection information 8b (step S74).
Then, whether the specific condition determination unit 9 matches the predetermined group defined in the specific condition determination information 309a based on the individual face detection information 8a and the whole face detection information 8b input from the face detection unit 7 It is determined whether or not (step S75). Here, if it is determined that it matches the predetermined group (step S75; YES), the specific condition determination unit 9 subtracts “number of group members−1” from the total number of personal recognitions of the entire face detection information 8b. The number of extra prints is set (step S76).
On the other hand, if it is determined that it does not match the predetermined group (step S75; NO), the specific condition determination unit 9 sets a value corresponding to the total number of personal recognitions of the entire face detection information 8b as the number of extra prints (step S77). .
Thus, the extra number setting process is completed.

  As described above, according to the imaging apparatus 300 of the fifth embodiment, the imaging related to the imaging of the subject by the imaging unit 1 according to the number of face images collated by the face detection unit 7 and the amount of motion of the face image. Since the processing can be executed, it is possible to effectively capture a plurality of subjects by using the face recognition technology. That is, the comparison between the face image detected by the face detection unit 7 and the face image information registered in the personal face DB 4 and the group DB 5 can be performed for each individual and for each group. Can be taken automatically by taking a shutter release in a state in which the movement of the face image is eliminated. As a result, it is possible to appropriately capture the group photo without failing. For example, in group photo shooting, if all members except one gather in front of the camera, a warning will be made in advance knowing that if one of the other members gathers it will automatically be taken, etc. Even if all members are ready, a warning will be given if all members are not stationary, so it can stop moving at the necessary time, and all members Since the automatic shooting is performed when all the members are stationary and all the members are stationary, the camera operator does not take a picture in a still moving state just entering the shooting frame.

  Further, by registering a predetermined group in the group DB 5 in the extra number setting automatic setting process, it is possible to recognize the group and set the extra number required for the entire group in advance. In the family living together, since the number of photos for the number of people is not required, it is possible to automatically set the troublesome number of extra prints such as setting as one sheet.

In the above-described automatic shutter processing, the image is automatically picked up when the group matching degree is 80% or more and other conditions are met. However, the present invention is not limited to this, and face detection is performed. Imaging may be performed when a plurality of pieces of face image information registered in the personal face DB 4 are collated by the unit 7.
As a result, when automatic shutter processing is performed in combination with self-timer shooting, commemorative shooting without failure can be performed, and the convenience of the imaging apparatus can be further improved.

Furthermore, in the above-described automatic shutter processing, in order to prevent a person other than the registered face from being out of focus, it may be switched to the pan focus mode and photographed. That is, when a plurality of people are photographed, if they are lined up in the depth direction, there is a high possibility that the image will be out of focus. Therefore, a face is detected and an automatic determination is performed with the recognized face size.
Specifically, when the face detection unit 7 collates one or more face images already registered in the imaging apparatus, all the detected face sizes are acquired. When two or more face images are recognized in the collation with the personal face DB 4, the registered face image and the smallest face size recognized in the image frame, that is, A comparison is made between the person who is farthest and the size of other face images that are not registered in the personal face DB 4 but are detected. Here, if there is a difference of a certain size or more in the face image which is not registered, the mode is switched to the pan focus mode. Further, when one face image can be recognized in the collation with the personal face DB 4, comparison is made with all the detected face sizes, and a face image that is at least one size larger than the recognized face image. If exists, switch to pan focus mode.
Accordingly, it is possible to avoid an imaging in which a person who is not registered in a group photo or the like is out of focus by using pan focus, and it is possible to provide a more convenient imaging apparatus.

In the above embodiment, the face image detected in the image frame during continuous shooting is a face image registered in the personal face DB 4, and there are n face images (n If it is recognized that it is greater than or equal to 1), tracking processing is started on the face image that has been registered, and imaging is performed with an automatic shutter while n face images are being tracked. Then, immediately after the end of imaging, it is confirmed whether or not tracking is possible, and imaging is continued until n faces cannot be tracked.
Thereby, while the face image can be tracked, imaging can be continuously performed, and various facial expressions of the subject of the registered face image can be easily captured.

  In the above-described embodiment, the processing according to the plurality of determination conditions 901 to 90n has been described as being applied to the automatic shutter processing and the extra number setting processing. However, the present invention is not limited to this. You may make it apply to various processes, such as a process, a still-in movie process, and an image deletion process.

[Embodiment 6]
In the sixth embodiment, the imaging apparatus 100 according to the first embodiment is applied to automation of various shooting mode setting processes and various shooting condition setting processes, and in particular, for a plurality of persons (faces) detected in a shooting frame. The photographing mode and the photographing conditions to be set are automatically determined according to the personal identification (recognition) and the group specific situation.
In the sixth embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

As illustrated in FIG. 18, the imaging apparatus 400 according to the sixth embodiment includes a face information storage control unit 12 that generates and records a face information file from the individual face detection information 8 a and the entire face detection information 8 b.
In other words, the face information storage control unit 12 records and stores the face individual detection information 8a and the entire face detection information 8b as a face information file 12a that can be reused by another imaging apparatus. Specifically, the face information storage control unit 12 includes a memory card that is detachable from the main body of the imaging apparatus 400, and the face information file 12a (the face individual detection information 8a and the whole face detection information 8b) is stored in the memory card. ) And the image file associated with the face information file 12a.
Here, the face information storage control unit 12 constitutes another file storage means for storing the face information file 12a and the image file as separate files.

When the face individual detection information 8a and the whole face detection information 8b are integrated as the face information file 12a, the specific condition determination information 409a includes, for example, each condition number (determination condition) as shown in FIG. ), The total number of face detections, the total number of individual recognitions, the group, the group matching degree, the overall movement, the main person, the face state (priority) of the main person, the size of the main person, and the like. In addition, as shown in FIG. 19B, each condition number is associated with each process to be activated, the activation process condition, and the like.
For example, as the specific condition determination information 409a, the determination condition of condition number 1 is associated with the face detection total number “0”, the personal recognition total number “0”, the process “landscape shooting mode” to be started, and the like. Similarly, the determination condition of condition number 2 includes the face detection total number “1”, the personal recognition total number “0”, the size of the main person “20% or more”, the process to be started “single person photographing mode”, the process The condition “focus on the recognized person” is associated. Similarly, the determination condition of condition number 3 is associated with the face detection total number “2 or more”, the personal recognition total number “0”, the overall movement “fast”, the activation process “sports shooting mode”, and the like. Yes. Similarly, the determination condition of condition number 4 includes the face detection total number “1 or more”, the personal recognition total number “1”, the process “snap shooting mode” to be started, the processing condition “focus on the main person”, and the like. It is associated. Similarly, the determination condition of condition number 5 associates the total number of personal recognition “2 or more” with the process “group photo shooting mode” to be started. Similarly, the determination condition of condition number 6 is that the total number of personal recognition is “2 or more”, the entire movement is “present”, the process to be started “automatic continuous shooting possible mode”, and the processing condition “repeated photographing every fixed time”. Are associated with each other. Similarly, the determination condition of condition number 7 is that the total number of individual recognition is “1 or more”, the main person “unknown”, the process “AF mode” to be activated, and the process condition “focus between the recognized individual and the main person”. "Set" is associated. Similarly, the determination condition of condition number 8 includes the group matching degree “80% or more”, the main person “unknown”, the process “pan focus mode” to be activated, and the process condition “all the groups and the main person”. “Set focus” or the like is associated.

  The specific condition determination unit 9 always compares the entire face detection information 8b output from the face detection unit 7 with the specific condition determination information 409a, and outputs a matching condition number if there is a matching condition. .

When the condition number is output from the specific condition determination unit 9, the process execution unit 11 switches to the shooting mode or shooting condition corresponding to the condition number.
When the specific condition determination information 409a as shown in FIG. 19 is set, when no face is detected in the shooting frame, a landscape shooting mode such as setting the focus to a distant view is automatically set. When one face of a certain size is detected, the camera automatically switches to single-person shooting mode, such as setting the focus to the foreground, and when two or more fast-moving faces are detected Automatically switches to a sport shooting mode such as increasing the shutter speed, and if a registered person is recognized, it is automatically switched to a snap shooting mode suitable for personal shooting and recording. When a plurality of persons are recognized, the mode is automatically switched to a group photo shooting mode suitable for group photo recording.

As described above, according to the imaging apparatus 400 of the sixth embodiment, the number of face detections and the number of individual recognitions are specified separately, and the shooting mode or shooting conditions are switched according to the combination of the specified numbers. It is possible to automatically set the optimal shooting mode or shooting conditions more flexibly.
Further, in another setting example of the specific condition determination information 409a shown in FIG. 19, when a plurality of registered persons are recognized and the whole is moving, by automatically switching to the automatic continuous shooting mode. Thus, it becomes possible to easily obtain a photographed image at the moment when all persons face the front. In addition, the focus mode can be flexibly and appropriately set according to the registered personal recognition status and the main human recognition status.

[Embodiment 7]
In the seventh embodiment, the imaging apparatus 100 according to the first embodiment is applied to automation of image search / display processing.
In the seventh embodiment, the processing contents and setting contents of the specific condition determination unit 9 and the processing execution unit 11 in the first embodiment are specialized for the purpose of the above processing.

  The operation input unit 6 can input and specify processing conditions related to image search processing and image display processing by the processing execution unit 11 according to the conditions related to the individual face detection information 8a and the entire face detection information 8b (face related information). It constitutes a processing condition designating means.

The process execution unit 11 performs face information storage control based on the processing conditions specified via the operation input unit 6 and the individual face detection information 8a and the entire face detection information 8b (face related information) as image information selection means. A processing target such as an image search process and an image display process is selected from a plurality of image files associated with the face information file 12a recorded in the unit 12.
That is, the process execution unit 11 searches for a desired image file corresponding to the processing condition designated via the operation input unit 6 among the plurality of image files recorded in the image recording unit 2 as an image search unit. To do.
Further, the process execution unit 11 serves as an image display control unit to select a desired image file corresponding to the processing conditions designated via the operation input unit 6 from among the plurality of image files recorded in the image recording unit 2. It is displayed on the display unit 10. In other words, the process execution unit 11 causes the display unit 10 to display the face portion of the subject magnified by the image processing control unit (process execution unit 11).

Next, an automated image search display process will be described with reference to FIG.
In order to automate the search and display of recorded and stored image data, the face information as indicated by the individual face detection information 8a and the whole face detection information 8b described above is associated with the captured image data. Must be recorded.

  At the time of shooting, first, shooting of a subject by the imaging unit 1 is started (step S101), and when a face image (person) exists in the image frame output from the imaging unit 1, the face detection unit 7 The detection information 8a and the entire face detection information 8b are output (step S102).

  The face information storage control unit 12 stores various face information based on the output individual face detection information 8a and the entire face detection information 8b as a “face information file 12a” as a file with a different file extension and the same file name as the image data. (Step S103).

Thereafter, an image search condition (for example, a large face) and an image display condition are set as specific condition determination information 9a based on a predetermined operation of the operation input unit 6 at an arbitrary timing (step S104), and a plurality of images to be searched When data is specified, the image acquisition control unit 3 sequentially outputs a plurality of image data specified as search targets in accordance with instructions from the processing execution unit 11. When the face information file corresponding to the image data has not yet been recorded, the face detection unit 7 and the face information storage control unit 12 newly output the individual face detection information 8a and the entire face detection information 8b, The information file 12a "is saved.
The specific condition determination unit 9 compares the search condition set in the specific condition determination information 9a with the contents of the face information file corresponding to the image data. If the conditions match, the specific condition determination unit 9 displays the image data as a display target. Instruction information designated as (search image) is output (step S105).
Then, the process execution unit 11 acquires only the image data designated as the display target (search image) from among the plurality of image data in the image recording unit 2 (step S106). Based on the contents of the face information file corresponding to the image data designated as the target (search image) and the image display conditions set in the specific condition determination information 9a, the image data is output to the display unit 10 for display. Specifically, when enlargement display of the main person is designated as the image display condition, the face information file is referred to specify the position and size of the face of the main person, and the portion is enlarged and displayed. (Step S107).

As described above, according to the imaging apparatus 400 of the seventh embodiment, when a photographed image data includes a person's face, image search is performed using various information related to the person's face. And display can be controlled. Further, the face information file 12a can be recorded in association with the image data, and the image data associated with the face image file is processed by the process execution unit 11 based on the processing conditions designated by the operation input unit 6. Therefore, the image-related processing can be efficiently performed to speed up the processing.
That is, it is not necessary to perform face detection or face recognition by the face detection unit 7 after image data is expanded on the RAM each time in image search processing or image display processing, and by referring to the face image file Thus, playback display based on the face and image search using the face as a key can be performed efficiently.

  In addition, by saving the face image file as a separate file together with the image data, the data can be easily transferred to an external device, and the search based on the face can be properly performed by a viewer that does not have a camera function. Can do.

  Further, in the above embodiment, the face information file 12a is stored as a separate file, but may be stored in a management area such as a header in the image file.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, as shown in FIG. 21, the personal face DB 4 may be integrated for each person to form the personal face integrated DB 13.
That is, if there are not enough records to be integrated, the face information file 12a generated by the imaging apparatus 400 according to the fourth embodiment is read from the external memory and added to the personal face DB 4. Then, based on a plurality of records of a person with the same name, feature data of an individual's face not related to the face orientation is extracted, and the feature data is stored in the personal face integration DB 13. After that, the face orientation of each of the plurality of face images stored corresponding to each priority of the person is determined by a process as described later, and the range of the face orientation corresponding to each priority is specified. And remember.
Specifically, as shown in FIG. 21, the personal face integration DB 13 includes each person's name, feature data not related to the face orientation, and priority 1 (best) to priority 5 (worst). Data and the like are stored in association with each other. For example, “Mr. A” will be described as an example of the subject. As the information indicating Mr. A's feature data and the range of the face orientation of priority 1, the right 15 ° to 30 ° right indicates the priority 2 face. Information indicating the range of orientation is associated with 15 ° from the right to 15 ° to the left, information indicating the range of the orientation of the face with priority 5 is associated with 30 ° from the left to 90 ° to the left. Similarly, “Mr. B” is information indicating the range of the face orientation of priority 2 from the right 10 ° to the left 10 ° as information indicating the feature data of Mr. B and the face orientation range of priority 1 The left 20 ° to the left 40 ° are associated with each other, and the right 70 ° to the right 90 ° and the like are associated with each other as information indicating the range of the face orientation with the priority 5.

  When the individual face detection information 508a is generated using the personal face integration DB 13 described above, each face image portion detected by the face detection unit 7 is collated with the personal face integration DB 13, and the degree of coincidence is highest. The name of the person corresponding to the record is stored. Here, it is determined as unknown if no person with a matching degree equal to or higher than a predetermined level is registered. In addition, the face detection unit 7 determines and stores the face direction based on the amount of deviation of the face parts such as eyes, nose, and rho included in the face image from the face center ( (See FIG. 23). In FIG. 22, only the portion different from the face individual detection information 8 a in FIG. 4 is represented in the face individual detection information 508 a. Further, the specific condition determination unit 9 determines whether the face direction determined by the face detection unit 7 is included in the range of the face direction stored in the personal face integration DB 13 according to the specified face direction. It is determined whether the detected face orientations are equal.

It is a block diagram which shows schematic structure of the imaging device of Embodiment 1 to which this invention is applied. It is a figure which shows typically personal face DB of the imaging device of FIG. It is a figure which shows typically group DB of the imaging device of FIG. It is a figure which shows typically the face individual detection information produced | generated by the face detection part of the imaging device of FIG. It is a figure which shows typically the whole face detection information produced | generated by the face detection part of the imaging device of FIG. It is a figure which shows typically the specific condition determination information which concerns on the determination process by the specific condition determination part of the imaging device of FIG. It is a flowchart which shows an example of the operation | movement which concerns on the still-in movie process by the process execution part of the imaging device of Embodiment 2 to which this invention is applied. It is a flowchart which shows an example of the operation | movement which concerns on the image deletion process by the process execution part of the imaging device of Embodiment 3 to which this invention is applied. It is a block diagram which shows schematic structure of the imaging device of Embodiment 4 to which this invention is applied. It is a figure which shows typically the specific condition determination information which concerns on the determination process by the specific condition determination part of the imaging device of FIG. It is a figure which shows typically the synthesized image produced | generated by the motion print process by the process execution part of the imaging device of FIG. It is a flowchart which shows an example of the operation | movement which concerns on the motion print process of FIG. FIG. 13 is a diagram showing a continuation of the motion print process in FIG. 12. It is a block diagram which shows schematic structure of the imaging device of Embodiment 5 to which this invention is applied. It is a figure which shows typically the specific condition determination information which concerns on the determination process by the specific condition determination part of the imaging device of FIG. 15 is a flowchart illustrating an example of an operation related to an automatic shutter process performed by a process execution unit of the imaging apparatus in FIG. 15 is a flowchart illustrating an example of an operation related to an extra number setting process by a process execution unit of the imaging apparatus of FIG. It is a block diagram which shows schematic structure of the imaging device of Embodiment 6 to which this invention is applied. It is a figure which shows typically the specific condition determination information which concerns on the determination process by the specific condition determination part of the imaging device of FIG. It is a flowchart which shows an example of the operation | movement which concerns on the image search display process by the imaging device of Embodiment 7 to which this invention is applied. It is a figure which shows typically personal face integration DB of the imaging device of a modification. It is a figure which shows typically the face separate detection information produced | generated by the face detection part of the imaging device of FIG.

Explanation of symbols

100, 200, 300, 400 Imaging device (image processing device)
1 Imaging unit (imaging means)
2 Image recording unit 4 Personal face DB (face image information registration means)
5 Group DB
6 Operation input part (Processing condition designation means)
7 Face detection unit (face detection means, face collation means, face related information acquisition means)
8 Detection information storage unit (face-related information recording means)
9 Specific condition determination unit (processing object determination means, main image determination means, stored image determination means, deletion image determination means)
10 Display section (image display means)
11 Process execution unit (imaging related process execution means, image information selection means, image processing means, composite image generation means, image deletion means, extra number setting means, image search means, image processing control means, image display control means)

Claims (6)

Face image information registration means for registering face image information used for identifying faces of a plurality of persons belonging to a predetermined group;
Imaging means for imaging a subject to generate image information;
Face detection means for detecting a plurality of face portions from the image information generated by the imaging means;
Each of the plurality of face portions detected by the face detection means is compared with face image information registered in the face image information registration means to determine whether or not the person belongs to the predetermined group. Face matching means to
Determining means for determining a difference between the number of persons determined to be a person belonging to the predetermined group by the face matching means and the number of persons of all the groups;
Shooting control means for controlling an automatic shooting operation or warning operation in shooting a group photo for the predetermined group according to a difference in the number of persons determined by the determination means;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the imaging control unit performs automatic imaging when the number of persons determined to be a person belonging to the predetermined group becomes equal to the number of persons in the group. .   3. The imaging apparatus according to claim 1, wherein the imaging control unit issues a warning when the number of persons determined to be a person belonging to the predetermined group is smaller than the number of persons in the group. .   The imaging apparatus according to claim 3, wherein the imaging control unit issues a warning when the number of persons determined to be a person belonging to the predetermined group is smaller by one than the number of persons of all the groups. .   The photographing control means performs automatic photographing when the number of persons determined to be a person belonging to the predetermined group is equal to the number of persons of all the groups and all of the groups are stationary. The imaging device according to claim 2. A computer of an imaging apparatus including an imaging unit that images a subject and generates image information,
Face image information registration means for registering face image information used for identifying faces of a plurality of persons belonging to a predetermined group;
Face detection means for detecting a plurality of face portions from the image information generated by the imaging means;
Each of the plurality of face portions detected by the face detection means is compared with face image information registered in the face image information registration means to determine whether or not the person belongs to the predetermined group. Face matching means to
Determining means for determining a difference between the number of persons determined to be a person belonging to the predetermined group by the face matching means and the number of persons of all the groups;
Shooting control means for controlling an automatic shooting operation or warning operation in shooting a group photo for the predetermined group according to a difference in the number of persons determined by the determination means;
A program characterized by making it function.

Images