JP5142825B2 - Image display device and image display method - Google Patents

Description

  The present invention relates to an image display device and an image display method.

  To check whether each person in the shot image has a suitable expression for shooting, such as whether his eyes are closed, the face of the person is displayed when there are many people in the shooting angle of view, such as a group photo. Because it is small, it becomes difficult.

Therefore, Patent Document 1 proposes an imaging apparatus that cuts out and displays a plurality of images so that a smaller number of people than the reference number is displayed when the number of detected faces is larger than the reference number when the captured image is reproduced. ing. Further, Patent Document 2 detects a face in a photographed image, detects a region where eyes exist from the face region, determines whether a person in the photographed image has eyes closed, and a person who has eyes closed An image pickup apparatus that instructs a photographer to perform re-shooting in the case where there is an image is proposed.
JP 2006-309661 A JP 2006-254229 A

  However, in Patent Document 1, it is difficult to know which part of the captured image is the enlarged image of the cut-out image, and all persons appearing in the captured image are confirmed unless a plurality of cut-out images are confirmed. I can't do it. On the other hand, since Patent Document 2 determines whether or not a person in a captured image has closed eyes, it cannot be determined whether or not the face is suitable for shooting even if the eyes are open.

  It is an object of the present invention to provide an image display device and an image display method that allow a photographer to easily check whether an identification target that appears in a captured image before and after shooting is in a state suitable for shooting. .

An image display device according to one aspect of the present invention includes a detection unit that detects an identification region that satisfies a condition of a specific subject as an identification target from a first image, and the detection unit detects a plurality of identification regions from the first image. A center area specifying means for specifying any of the plurality of identification areas as a central area when detected, and a non-central area that is an identification area different from the central area among the plurality of identification areas. A second image is generated by enlarging the image of the identification area after moving each non-central area so as to approach the central area along a vector composed of the center and the center of the central area. It has an image enlarging means and a display means for displaying the first image and the second image .
According to another aspect of the present invention, there is provided an image display device that detects a detection area that satisfies a condition of a specific subject as an identification target from a first image and a plurality of identification areas detected from the first image. And an image enlarging means for generating a second image by moving and enlarging the image of each of the identification areas so that the respective positions of the plurality of identification areas approach in the first image, Display means for displaying one image and the second image, and determination means for determining whether or not the number of identification areas detected by the detection means is equal to or greater than a threshold, wherein the determination means is equal to or greater than the threshold The second image is displayed on the display means when it is determined, and the first image is displayed when it is determined that the determination means is less than a threshold value.

Another image display method as aspects of the present invention is an image display method of displaying an image on a display unit, detecting a satisfying identification area of the specific subject as an identification target from the first image And, when a plurality of identification areas are detected from the first image, a step of specifying any identification area as a central area from the plurality of identification areas, and an identification different from the central area among the plurality of identification areas The plurality of identification areas are expanded after each non-central area is moved closer to the central area along a vector composed of the center of each non-central area and the center of the central area. And generating a second image, and displaying the second image on a display means .
An image display method according to another aspect of the present invention is an image display method for displaying an image on a display unit, and a detection step of detecting an identification region satisfying a condition of a specific subject as an identification target from a first image; , When detecting a plurality of identification regions from the first image, moving and enlarging the images of the identification regions so that the respective positions of the plurality of identification regions approach in the first image. And generating a second image, a determination step for determining whether the number of identification regions detected in the detection step is greater than or equal to a threshold value, and And displaying the second image on a display unit, and displaying the first image on the display unit when it is determined that the second image is less than a threshold value.

  ADVANTAGE OF THE INVENTION According to this invention, the image display apparatus and image display method which can make a photographer confirm easily whether the identification target reflected on a picked-up image before and after imaging | photography is in the state suitable for imaging | photography can be provided. .

  FIG. 1 is a block diagram of an image display apparatus 10 of this embodiment (hereinafter referred to as a digital camera 10 in this embodiment). Note that the image display device of the present invention may be an imaging device (for example, a digital camera or a digital video camera) that records and reproduces data. Further, the present invention can be applied to an electronic apparatus having an imaging function and a reproduction display device that reproduces and displays image data. Here, the electronic device having an imaging function is, for example, a PDA (Personal Digital Assistant), a mobile phone, or the like.

  The digital camera 10 includes an imaging unit that images a subject, a central processing unit (CPU) 20, an operation unit 35, a VRAM 40, a video encoder 42, a memory 44, and a display unit 50.

  The imaging unit includes an imaging lens 12, an imaging element 14, an imaging circuit 16, and an image processing circuit 18. The light that has passed through the imaging lens 12 enters the imaging device 14, is sequentially read out as an image signal, and is added to the imaging circuit 16. The imaging circuit 16 includes a gain adjustment circuit, and the obtained image data is sent to the image processing circuit 18. The image processing circuit 18 performs image processing such as white balance correction, gamma correction, and contrast correction in accordance with commands from the CPU 20. The image data input to the image processing circuit 18 is converted into a luminance signal (Y signal) and a color difference signal (CrC1 signal) and subjected to predetermined processing such as gamma correction and then stored in the VRAM 40. .

  FIG. 2 is a block diagram of the CPU 20. The CPU 20 comprehensively controls each circuit in the digital camera 10 based on an input from the operation unit 35. When outputting the captured image to the display unit 50, the YC signal is read from the VRAM 40 and sent to the video encoder 42. The video encoder 42 converts the input YC signal into a predetermined display format (NTSC or the like) and outputs it to the display unit 50. In addition, the YC signal of each frame processed at a predetermined frame rate is alternately written in the A area and the B area of the VRAM 40, and the YC signal is written in the A area and the B area of the VRAM 40. YC signals written from different areas are read out. In this way, the YC signal in the VRAM 40 is periodically rewritten, and the video signal generated from the YC signal is supplied to the display unit 50, whereby the image being captured is displayed on the display unit 50 in real time. . Then, the photographer can confirm the shooting angle of view from the through image displayed on the display unit 50.

  The CPU 20 includes a face detection unit 21, a number determination unit 22, a face region trimming unit 23, a trimming region average area calculation unit 24, a trimming region enlargement / reduction unit 25, a center region specification unit 26, and a center coordinate calculation unit 27. The CPU 20 further includes a center coordinate distance calculation unit 28, a movement order determination unit 29, a face area movement unit 30, and an image enlargement unit 31.

  The face detection unit 21 performs face detection on the image within the shooting angle of view (first image) displayed by the display unit 50. The number determination unit 22 determines whether or not the number of faces detected by the face detection unit 21 is equal to or greater than the first threshold or equal to or less than the second threshold (4 to 21 in this embodiment). The face area trimming unit 23 forms an area (a “face area” described later) surrounding the face detected by the face detection unit 21. The coordinates of the rectangular area surrounding the face area become an identification area indicating the position of the subject that satisfies the condition as the identification target. The trimming area average area calculation unit 24 calculates the average area of the trimmed face area. The trimming area enlargement / reduction unit 25 enlarges or reduces the trimmed face areas so as to have an average area.

  When the face detection unit 21 detects a plurality of faces in the first image, the center region specifying unit 26 specifies a center region that is a central face region among the plurality of faces. The center coordinate calculation unit 27 calculates the center coordinates of the face area. The center-coordinate distance calculation unit 28 calculates the distance between the center of gravity of the first image and the center coordinates of each face area, or the distance between the center area and a non-center area that is an area different from the center area. The movement order determination unit 29 determines the movement order in the non-center area. The face area moving unit 30 generates a second image by moving each non-center area closer to the center area along a vector composed of the center of the non-center area and the center of the center area among the plurality of face areas. To do. The image enlargement unit 31 enlarges the second image within a range that can be displayed by the display unit 50 and generates a third image.

  FIG. 3 is a block diagram of the operation unit 35. The operation unit 35 includes a release switch 36 that performs focus adjustment and photographing, and a mode dial (mode setting unit) 37 that sets an operation mode. The mode dial 37 can set photographing modes such as a portrait mode, a landscape mode, a night view mode, a sports mode, and a group photo mode. Among these, when a specific mode (for example, group photo mode) is selected, at least one of finder image display (through image display) 37a, review display 37b, and reproduction display 37c is executed. The viewfinder image display (through image display) 37a displays an image within the shooting angle of view before shooting. The review display 37b displays on the display unit an image that has been captured for a certain period of time after shooting by the imaging unit (or immediately after shooting). The reproduction display 37c reproduces and displays the image stored in the memory 44 for storing the image.

  The memory 44 stores captured images. The operation unit 35 can select an image to be stored in the memory 44 and an image to be discarded. The display unit 50 is a liquid crystal display unit that is provided on the back surface of the digital camera 10 and displays an image. The digital camera 10 includes a finder (view window), and the display unit 50 can display not only a photographed image but also a finder image (through image) before photographing.

  Hereinafter, the operation (image display method) of the finder image display 37a when the mode dial 37 is set to a specific mode will be described with reference to FIG. FIG. 4 is a flowchart for explaining the operation of the CPU 20 in the finder image display. In FIG. 4, “S” is an abbreviation for a step. The image display method shown in FIG. 4 is used for display by the display unit 50 and can be executed or embodied as a program.

  First, an instruction to display a through image on the display unit 50 is given, and the face detection unit 21 performs face detection processing in the through image (S101). The method of face detection is not limited. For example, the face detection unit 21 determines whether or not the hue of each pixel of the captured image is included in the skin color range and divides the skin color region into a non-skin color region, and detects an edge in the image to detect the image. Each part inside is classified into an edge part or a non-edge part. The face detection unit 21 extracts, as a face candidate area, an area that is formed of pixels classified in the skin color area and is classified as a non-edge part, and is surrounded by pixels that are determined to be edge parts. It is determined whether the region corresponds to the face, and the face region is extracted based on the determination result. When the face detection unit 21 determines that no face exists, the display unit 50 displays a through image (S103).

  When the face detection unit 21 detects a face (S101), it is determined whether there are a plurality of faces (S102). When the face detection unit 21 determines that there are a plurality of faces (S102), the number determination unit 22 counts the number of faces (S103). When the face detection unit 21 determines that a plurality of faces do not exist (S102), the display unit 50 displays a through image (S103).

  The number-of-persons determination unit 22 determines whether the number of faces (number of persons) N is within a threshold range (S105). As described above, in this embodiment, the number determination unit 22 determines whether N is equal to or greater than the first threshold (4 or more) and equal to or less than the second threshold (21 or less) (3 <N <22). to decide. When the number determination unit 22 determines that N does not satisfy this condition (S105), the display unit 50 displays a through image (S103). This is because when the number of people within the shooting angle of view is small (less than the first threshold), there is often no need to enlarge the face of the person because the face of the person is already sufficiently large. is there. On the other hand, if the number of people is too large, enlarging and displaying the faces of all the people will make the image difficult to see. Note that the photographer can freely set the threshold range.

  Next, the face area trimming unit 23 trims the minimum rectangular area surrounding each face based on the face position information detected by the face detecting unit 21 (S106). The trimmed area is hereinafter referred to as “face area”. FIG. 5 shows the first image F1 including the face areas P1 to P5 generated by the face area trimming unit 23.

  Next, the trimming region average area calculation unit 24 calculates the average value of the face region areas created in S106, and the trimming region enlargement / reduction unit 25 centers the rectangular center of gravity so that each face region area is equal to the average value. The point is enlarged or reduced (S107).

  Next, the center area specifying unit 26 specifies the center area Kc among the face areas K1 to K5 obtained by enlarging or reducing the face areas P1 to P5 that have undergone S107 (S108). At this time, as shown in FIG. 6, the center coordinate calculation unit 27 obtains the center G of the figure formed by connecting the centers of gravity g1 to g5 of the face regions K1 to K5 with a straight line so as not to self-intersect. Then, the center area specifying unit 26 calculates distances L1 to L5 between the center of gravity G and the centers of gravity g1 to g5, and specifies the face area having the shortest distance as the center area Kc. In FIG. 6, since L3 is the minimum, the central area Kc is the face area K3.

  Next, as shown in FIG. 7, the center coordinate distance calculation unit 28 centroid point g3 of the face region K3 which is the center region Kc and centroid points g1, g2, g4 of other face regions (non-center regions), The distances l1, l2, l4, and l5 between g5 are calculated. Then, the movement order determination unit 29 performs ordering from the smallest one (S109). In this embodiment, the order is l4, l2, l1, and l5.

  Next, the face area moving unit 30 follows a vector heading to the straight line K3 connecting the centroid g3 of the central area KK3 and the centroids g4, g2, g1, and g5 of the non-central areas according to the order determined by the movement order determining unit 29. The face areas K1, K2, K4, and K5 are moved. The face area moving unit 30 sequentially moves the face area until it contacts any of the face areas K1 to K5 to generate an image (second image) (S110). The second image is shown in FIG.

  Next, the image enlargement unit 31 calculates a minimum rectangle surrounding all the face regions K1 to K5 together with the aspect ratio of the display unit 50 (S111). Then, an image (third image) F3 enlarged so that the calculated rectangular image is maximized on the display unit 50 is generated and output to the display unit 50 as shown in FIG. 9 (S112).

  By the method described above, the faces of the persons within the shooting angle of view are enlarged and displayed with an equal size, and further displayed in a positional relationship corresponding to the shooting angle of view. For this reason, the photographer can intuitively confirm the facial expression of the person within the shooting angle of view at once. At this time, an area different from the face area may leave the background as it is, or may be a monochrome display such as black or blue. Further, in order to convey the overall atmosphere, as shown in FIG. 10, a reduced version of the entire captured image may be displayed on the display unit 50 together with the third image.

  In this embodiment, the operation (image display method) of the finder image display 37a when the mode dial 37 is set to a specific mode has been described. However, the operation of the review display 37b when the mode dial 37 is set to a specific mode and the operation of the playback display 37c are also displayed in a similar manner, so that the photographer can intuitively display the face of the person within the shooting angle of view. It becomes possible to confirm facial expressions at once.

  In this embodiment, the detected face area in the imaging screen is used as the identification area, but a specific subject may be detected by image detection. For example, it may be possible to cut out and detect a subject image from the background. In addition, the identification area in the imaging screen may be determined by inputting a position in the imaging screen from an external input unit or detecting the line of sight of the photographer who is looking at the viewfinder.

1 is a block diagram of a digital camera according to an embodiment. It is a block diagram of CPU shown in FIG. It is a block diagram of the operation part shown in FIG. 3 is a flowchart for explaining the operation of the CPU shown in FIG. 1 in a finder image display mode. It is a figure which shows the example which displays the result of S106 shown in FIG. 4 on a display part. It is a figure which shows the example which displays the result of S108 shown in FIG. 4 on a display part. It is a figure which shows the example which displays the result of S109 shown in FIG. 4 on a display part. It is a figure which shows the example which displays the result of S110 shown in FIG. 4 on a display part. It is a figure which shows the example which displays the result of S112 shown in FIG. 4 on a display part. It is a figure which shows another example at the time of displaying the result of S112 shown in FIG. 4 on a display part.

Explanation of symbols

10 Digital camera 20 CPU
DESCRIPTION OF SYMBOLS 21 Face detection part 22 Number determination part 24 Trimming area | region average area calculation part 25 Trimming area expansion / contraction part 30 Face area movement part 31 Image expansion part 35 Operation part 37 Mode dial (mode setting part)
50 display section

Claims (8)

Detecting means for detecting an identification region satisfying a condition of a specific subject as an identification target from the first image;
When the detection unit detects a plurality of identification regions from the first image, a center region specifying unit that specifies any one of the plurality of detected identification regions as a central region;
Along a vector consisting of respective centers and the center of the central region of the central area non-centered regions is different identification region and out of the previous SL plurality of identification areas, approaching each non central region to the central region after moving to the image enlarging means for generating a second image by enlarging the image of the identification area,
Images display you further comprising a display means for displaying the first image and the second image. Detecting means for detecting an identification region satisfying a condition of a specific subject as an identification target from the first image;
When a plurality of identification areas are detected from the first image, the images of the identification areas are moved and enlarged so that the positions of the plurality of identification areas approach in the first image. Image enlarging means for generating a second image;
Display means for displaying the first image and the second image;
Determining means for determining whether or not the number of identification areas detected by the detecting means is equal to or greater than a threshold ;
When the determination unit determines that the threshold value is greater than or equal to a threshold value, the display unit displays the second image;
An image display device, wherein the first image is displayed when the determination means determines that it is less than a threshold value. The image display device further includes a mode setting unit for setting an operation mode,
It said mode setting section, in the case of setting the specific mode, the image display apparatus according to claim 1 or 2, characterized in that for displaying the second image. 4. The image according to claim 3 , wherein in the specific mode, the second image is displayed at least one of before shooting, a fixed time after shooting, and when playing back an image stored in a memory. Display device. Wherein the display means, the image display apparatus according an image obtained by reducing the first image in any one of claims 1 to 4, characterized in that to display together with the second image. Average area calculating means for calculating an average area of the plurality of identification regions;
Scaling means for enlarging or reducing the plurality of identification regions so that the plurality of identification regions are equal to the average area calculated by the average area calculation unit;
The image enlarging means moves the image of each identification area in the first image so that the positions of the plurality of identification areas enlarged or reduced by the enlargement / reduction means approach each other. the image display apparatus according to any one of claims 1 to 5. An image display method for displaying an image on a display means,
A detection step of detecting an identification region satisfying a condition of a specific subject as an identification target from the first image;
If it detects a plurality of identification areas from the first image, identifying a central region of one of the identification area from the plurality of identification areas,
Each non-central region approaches the central region along a vector composed of the center of the non-central region, which is an identification region different from the central region among the plurality of identification regions, and the center of the central region. after moving to the steps of generating a second image by enlarging the plurality of identification areas,
Images displayed how to characterized by the step of displaying on the display unit the second image. An image display method for displaying an image on a display means,
A detection step of detecting an identification region satisfying a condition of a specific subject as an identification target from the first image;
When a plurality of identification areas are detected from the first image, the images of the identification areas are moved and enlarged so that the positions of the plurality of identification areas approach in the first image. Generating a second image;
A determination step of determining whether or not the number of identification regions detected in the detection step is greater than or equal to a threshold;
The step of displaying the second image on the display unit when it is determined in the determination step that the threshold value is equal to or greater than the threshold value, and the step of displaying the first image on the display unit when it is determined that the value is less than the threshold value. An image display method characterized by the above.