JP4991685B2 - Stereoscopic image processing apparatus and stereoscopic image processing method - Google Patents

Description

  The present invention relates to a stereoscopic image processing apparatus and a stereoscopic image processing method that can easily perform alignment of a plurality of images respectively captured from a plurality of viewpoints.

  There is known a stereoscopic image processing apparatus that outputs a stereoscopic image captured from a plurality of viewpoints to a display capable of stereoscopic display.

  Three-dimensional display methods include, for example, a lenticular method that uses a display with a kamaboko-shaped lenticular lens in front, and a light direction control method that controls the direction of the backlight that illuminates the back of the LCD (liquid crystal display device) in a time-sharing manner Also referred to as “time-division light direction control backlight method”).

  By the way, in order to stereoscopically view a stereoscopic image with high quality, it is necessary to align the left eye image and the right eye image.

  Patent Document 1 describes a configuration for correcting the tilt and positional deviation of left and right binocular images.

Japanese Patent Application Laid-Open No. 2004-228667 describes a video camera finder with a guideline, and a user who takes a picture adjusts the subject so that the subject is at the center of the guideline and shoots left and right images.
JP 2004-12918 A Japanese Patent Laid-Open No. 10-322725

  However, it takes time when the user performs an image alignment operation. Further, if corresponding point detection processing such as template matching is performed in order to automatically perform alignment processing, the amount of calculation increases.

  In the configuration described in Patent Document 1, since feature points are extracted from each of the left eye image and the right eye image, it takes time and cannot be said to be simple.

  The configuration described in Patent Document 2 is inconvenient because it is necessary for the user to position the imaging target in accordance with the guidelines when shooting.

  The present invention has been made in view of such circumstances, and a stereoscopic image processing apparatus and a stereoscopic image processing method that do not require a user alignment operation and can perform image alignment with a small amount of calculation. The purpose is to provide.

  In order to achieve the above object, the present invention provides an image acquisition means for acquiring a first captured image and a second captured image captured from a plurality of viewpoints, and the first captured image and the second captured image. Detecting a specific target in each of the images, and detecting mark information having representative coordinates of the detection mark of the specific target in the first captured image and representative coordinates of the detection mark of the specific target in the second captured image Specific object detection means to be generated; difference vector calculation means for calculating a difference vector between representative coordinates of the detection mark in the first captured image and representative coordinates of the detection mark in the second captured image; and the difference By adjusting the difference between the display position of the specific target of the first captured image and the display position of the specific target of the second captured image based on the vector, A parallax amount adjusting unit that adjusts the parallax amount of the specific target between one captured image and the second captured image; and the first captured image and the second captured image in which the parallax amount is adjusted There is provided a stereoscopic image processing apparatus comprising display control means for displaying on a displayable display means.

  According to the present invention, since the parallax amount is adjusted using the coordinates of the detection mark in the stereoscopic image processing apparatus having the detection function of the specific target, a user's alignment operation is not required and the amount of calculation is small. Can be used for image alignment.

  In one aspect of the present invention, the difference vector calculation means calculates a center coordinate of the detection mark in each of the first captured image and the second captured image, and the difference in the first captured image is calculated. A difference between the center coordinate of the detection mark and the center coordinate of the detection mark in the second captured image is calculated as the difference vector.

  In one aspect of the present invention, the parallax amount adjusting means matches the display position of the specific target.

  In one aspect of the present invention, the parallax amount adjusting means increases the parallax amount of the specific target.

  In one aspect of the present invention, the parallax amount adjusting means changes the display position of the specific target by image processing.

  In one embodiment of the present invention, a first imaging unit having a first imaging lens, a first imaging element on which a subject image is formed via the first imaging lens, a second imaging lens, Camera shake is corrected by moving at least one of the second imaging unit having a second imaging element on which a subject image is formed via the second imaging lens, and the imaging lens and the imaging unit. A parallax amount adjusting unit that changes the display position of the specific target using the camera shake correcting unit.

  In one embodiment of the present invention, a first imaging unit having a first imaging lens, a first imaging element on which a subject image is formed via the first imaging lens, a second imaging lens, A second imaging unit having a second imaging element on which a subject image is formed via the second imaging lens; and a convergence for changing a convergence angle of the first imaging lens and the second imaging lens. Angle change means, and the parallax amount adjustment means changes the display position of the specific target using the convergence angle change means.

In one aspect of the present invention, provided with a shooting instruction means for inputting a shooting instruction,
Movement of the display position of the specific target obtained from the difference vector when the imaging instruction is input by the imaging instruction unit in a state where the captured image with the parallax amount adjusted is displayed on the display unit The image processing apparatus includes recording control means for recording at least one of the vector and the difference vector as parallax amount adjustment information on the recording medium in association with the first captured image and the second captured image.

  In one aspect of the present invention, the image acquisition unit acquires the captured image from a recording medium or by communication, and the parallax amount adjustment unit displays the acquired captured image on the display device. The parallax amount is adjusted.

  According to the present invention, a specific target is detected in each of the steps of obtaining a first captured image and a second captured image captured from a plurality of viewpoints, and each of the first captured image and the second captured image. Obtaining detection mark information having representative coordinates of the detection mark of the specific target, and representative coordinates of the detection mark in the first captured image and the detection mark in the second captured image. Calculating a difference vector with respect to the representative coordinates of the first image, and based on the difference vector, a difference between the display position of the specific target of the first captured image and the display position of the specific target of the second captured image Adjusting the parallax amount of the specific target, and causing the display means to display the first captured image and the second captured image with the parallax amount adjusted. To provide a stereoscopic image processing method characterized by comprising: a flop, a.

  In addition, the present invention provides a step of acquiring a first captured image and a second captured image captured from a plurality of viewpoints, and a specific target part in each of the first captured image and the second captured image. Detecting part information having representative coordinates of the part to be identified, representative coordinates of the part in the first captured image, and representative of the part in the second captured image A step of calculating a difference vector with respect to coordinates, and based on the difference vector, a display position of the part to be specified in the first captured image and a display position of the part to be specified in the second captured image Adjusting the difference between the specific target, the step of adjusting the parallax amount of the specific target, and the first captured image and the second captured image in which the parallax amount is adjusted are displayed on the display means. To provide a stereoscopic image processing method characterized by comprising the steps that, the.

  According to the present invention, it is possible to perform image alignment without requiring a user alignment operation and with a small amount of calculation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a front perspective view showing an external configuration of a digital camera to which the present invention is applied. FIG. 2 is a rear perspective view showing an external configuration of a digital camera to which the present invention is applied.

  The digital camera 10 of the present embodiment is a digital camera (corresponding to the compound-eye digital camera of the present invention) provided with a plurality of (two are illustrated in FIG. 1) imaging means (also referred to as an imaging system). Images can be taken from a plurality of viewpoints (two left and right viewpoints are illustrated in FIG. 1).

  In the present embodiment, two imaging units are illustrated for convenience of explanation, but the present invention can be similarly applied to three or more imaging units. The arrangement of the imaging means (mainly the photographing lenses 14R and 14L) may not be one horizontal row along the horizontal direction, but may be two-dimensionally arranged. Stereo shooting, multi-viewpoint, or shooting in all directions may be used.

  The camera body 12 of the digital camera 10 is formed in a rectangular box shape, and a pair of photographing lenses 14R and 14L, a strobe 16 and the like are provided on the front surface thereof as shown in FIG. On the upper surface of the camera body 12, a shutter button 18, a power / mode switch 20, a mode dial 22, and the like are provided.

  On the other hand, on the back of the camera body 12, as shown in FIG. 2, a monitor 24, a zoom button 26, a cross button 28, a MENU / OK button 30, a DISP button 32, a BACK button 34, and a portrait / landscape switch button 36 Etc. are provided.

  Although not shown, the bottom of the camera body 12 is provided with a tripod screw hole, an openable / closable battery cover, and the like. A battery storage chamber for storing a battery, a memory is provided inside the battery cover. A memory card slot or the like for installing a card is provided.

  Each of the pair of left and right photographing lenses 14R and 14L is constituted by a retractable zoom lens, and has a macro photographing function (proximity photographing function). The photographing lenses 14R and 14L are extended from the camera body 12 when the digital camera 10 is turned on. In addition, since the zoom mechanism, the retracting mechanism, and the macro photographing mechanism in the photographing lens are well-known techniques, description of the specific configuration is omitted here.

  The strobe 16 is composed of a xenon tube, and emits light as necessary when shooting a dark subject or when backlit.

  The shutter button 18 is composed of a two-stroke switch that includes a so-called “half-press” and “full-press”. When the digital camera 10 shoots a still image (for example, when the still image shooting mode is selected with the mode dial 22 or when the still image shooting mode is selected from the menu), when the shutter button 18 is pressed halfway, a shooting preparation process, that is, AE ( Automatic exposure (AF), AF (auto focus), and AWB (automatic white balance) processing are performed, and when fully pressed, image capture / recording processing is performed. Also, during movie shooting (for example, when the movie shooting mode is selected with the mode dial 22 or when the movie shooting mode is selected from the menu), when the shutter button 18 is fully pressed, shooting of the movie starts and when the shutter button 18 is fully pressed again, shooting is performed. Exit. Depending on the setting, the moving image can be shot while the shutter button 18 is fully pressed, and the shooting can be terminated when the full press is released. A shutter button dedicated to still image shooting and a shutter button dedicated to moving image shooting may be provided.

  The power / mode switch 20 functions as a power switch of the digital camera 10 and also functions as a switching unit that switches between the playback mode and the shooting mode of the digital camera 10, and includes “OFF position”, “playback position”, and “shooting position”. It is slidably provided between the two. The digital camera 10 is set to the playback mode when the power / mode switch 20 is positioned at the “playback position”, and is set to the shooting mode when it is positioned at the “shooting position”. Further, when it is positioned at the “OFF position”, the power is turned off.

  The mode dial 22 is used for setting the shooting mode. The mode dial 22 is rotatably provided on the upper surface of the camera body 12, and “2D still image position”, “2D moving image position”, “3D still image position”, “3D moving image position” are not shown by a click mechanism (not shown). It is provided so that it can be set. The digital camera 10 is set to a 2D still image shooting mode for shooting a 2D still image by setting the mode dial 22 to “2D still image position”, and the 2D / 3D mode switching flag is set to the 2D mode. A flag representing that is set. Also, by setting the “2D moving image position”, a 2D moving image shooting mode for shooting a 2D moving image is set, and a flag indicating the 2D mode is set in the 2D / 3D mode switching flag.

  In addition, by setting to “3D still image position”, a 3D still image shooting mode for shooting a 3D still image is set, and a flag indicating the 3D mode is set in the 2D / 3D mode switching flag. . Further, by setting the “3D moving image position”, a 3D moving image shooting mode for shooting a 3D moving image is set, and a flag indicating the 3D mode is set in the 2D / 3D mode switching flag. The CPU 110, which will be described later, refers to the 2D / 3D mode switching flag and grasps whether it is the 2D mode or the 3D mode.

  The monitor 24 is a display device such as a color liquid crystal panel. The monitor 24 is used as an image display unit for displaying captured images, and is used as a GUI when various settings are made. In addition, the monitor 24 sequentially displays images (through images) continuously captured by the imaging elements 134R / L during shooting, and is used as an electronic viewfinder.

  The zoom button 26 is used for a zoom operation of the photographing lenses 14R and 14L, and includes a zoom tele button for instructing zooming to the telephoto side and a zoom wide button for instructing zooming to the wide angle side.

  The cross button 28 is provided so that it can be pressed in four directions, up, down, left, and right, and a function corresponding to the setting state of the camera is assigned to the button in each direction. For example, at the time of shooting, a function for switching on / off of the macro function is assigned to the left button, and a function for switching the strobe mode is assigned to the right button. In addition, a function for changing the brightness of the monitor 24 is assigned to the upper button, and a function for switching ON / OFF of the self-timer is assigned to the lower button. Further, during playback, a frame advance function is assigned to the left button, and a frame return function is assigned to the right button. Also, a function for changing the brightness of the monitor 24 is assigned to the upper button, and a function for deleting the image being reproduced is assigned to the lower button. In various settings, a function for moving the cursor displayed on the monitor 24 in the direction of each button is assigned.

  The MENU / OK button 30 is used to call a menu screen (MENU function), and is used to confirm selection contents, execute a process, etc. (OK function), and is assigned according to the setting state of the digital camera 10. Is switched.

  On the menu screen, for example, all adjustment items that the digital camera 10 has such as image quality adjustment such as exposure value, hue, ISO sensitivity, number of recorded pixels, self-timer setting, photometry method switching, and whether or not to use digital zoom. Settings are made. The digital camera 10 operates according to the conditions set on this menu screen.

  The DISP button 32 is used to input an instruction to switch the display contents of the monitor 24, and the BACK button 34 is used to input an instruction to cancel the input operation.

  The vertical / horizontal shooting switching button 36 is a button for instructing whether to perform shooting in vertical shooting or horizontal shooting.

  The input / output connector 38 is used for wired communication with the outside. It is possible to input a captured image to the digital camera 10 via the input / output connector 38.

  FIG. 3 is an explanatory diagram for explaining a structural example of the monitor 24 capable of three-dimensional display (also referred to as “stereoscopic display”). This example is a lenticular method, and uses a monitor 24 in which a lenticular lens having a semi-cylindrical lens group is arranged on the front surface.

  A lenticular lens 24a is arranged on the front surface of the monitor 24 (in the z-axis direction where the observer's viewpoint (left eye EL, right eye ER) exists). The lenticular lens 24a is configured by connecting a plurality of cylindrical convex lenses in the x-axis direction in FIG.

  The display area of the three-dimensional image (also referred to as “stereoscopic image”) displayed on the monitor 24 includes a left-eye strip image display area 24L and a right-eye strip image display area 24R. The strip image display area 24L for the left eye and the strip image display area 24R for the right eye each have an elongated strip shape in the y-axis direction in FIG. 3 of the screen, and are alternately arranged in the x-axis direction in FIG. The

  Each convex lens constituting the lenticular lens 24a has a strip image display area including a pair of left eye strip image display areas 24L and a right eye strip image display area 24R based on a given observation point of the observer. It is formed at a position corresponding to the region 24c.

  In FIG. 3, the left eye strip image displayed in the left eye strip image display area 24 </ b> L of the monitor 24 is incident on the left eye EL of the observer due to the photorefractive action of the lenticular lens 24 a. The right eye strip image displayed in the right eye strip image display area 24R of the monitor 24 is incident on the right eye ER of the observer due to the light refraction action of the lenticular lens 24a. Therefore, the left eye of the observer sees only the strip image for the left eye, and the right eye of the observer sees only the strip image for the right eye. Stereoscopic viewing is possible by right-and-left parallax from the right-eye image that is a set of ophthalmic strip images.

  Although the case where the lenticular method is used as an example of the structure of the monitor 24 for 3D display has been described with reference to FIG. 3, the present invention is not particularly limited to the lenticular method.

  For example, the left-eye image and the right-eye image are each formed into a strip shape that is cut out in the vertical direction of the image, and displayed side by side alternately, and the image is shown to the observer through a slit similarly cut in the vertical direction, You may use the parallax barrier (parallax barrier) system which delivers a left eye image to an observer's left eye, and a right eye image to a right eye, respectively. Other space division methods may be used.

  In addition, a light direction control method (time-division light direction control backlight) that controls the direction of the backlight illuminating the back of the LCD (liquid crystal display device) constituting the monitor 24 in the right eye direction and the left eye direction of the observer in a time division manner. May also be used. The light direction control method is Kentaro Toyooka, Tetsuya Miyashita, Tatsuo Uchida, “Three-dimensional display using time-division light direction control backlight”, 2000 Proceedings of the Japanese Liquid Crystal Society Annual Meeting, pp.137-138 (2000) Or in Japanese Patent Application Laid-Open No. 2004-20684. A so-called scan backlight method described in Japanese Patent No. 3930021 may be used.

  The left and right images may be displayed alternately, and the image may be stereoscopically viewed by using image separation glasses.

  As the monitor 24, for example, a liquid crystal display device or an organic EL device is used. Self-emission or a method of controlling the amount of light with a separate light source may be used. In addition, a system such as a polarized light system, an anaglyph, and a naked eye system are not limited. Moreover, the system which laminated | stacked the liquid crystal and organic EL in the multilayer may be sufficient.

  FIG. 4A schematically illustrates a situation in which the face 90 is imaged as a specific target (also referred to as “main subject”) by the digital camera 10. FIG. 4B shows a state seen from above. FIG. 5A shows a left eye image 91L imaged by the imaging element 134L via the left-eye imaging lens 14L and a right eye imaged by the imaging element 134R via the right-eye imaging lens 14R. An example of the image 91R is shown. If no adjustment processing of the display position is performed on the left eye image 91L and the right eye image 91R, as shown in FIG. It is rare that the position at which the optical axes of the photographing lenses 14L and 14R intersect with the position of the face 90 that is the specific target coincides.

  FIG. 4C schematically shows a state in which the face 90 of another person is watched by both eyes 94L and 94R of the person. In the case of a person, by adjusting the optical axis of both eyes 94L and 94R to intersect each other at the face 90 that is watched, the left eye image and the right eye image are visually recognized as matching. . On the other hand, when attention is paid to an object farther than the face 90, the face 90 closer to the object being watched is as shown in FIG. A parallax amount d exists and is visually recognized.

  Hereinafter, various embodiments for adjusting the amount of parallax will be described in the case where the specific target of interest is a face.

(First embodiment)
In the first embodiment, a plurality of captured images (left-eye image and right-eye image) respectively captured at a plurality of viewpoints, and detection mark information generated by detecting a specific target in each of the plurality of captured images, And the difference (parallax amount) in the display position of the specific target (display coordinates on the monitor 24) is adjusted between the plurality of captured images.

  In this example, in order to facilitate understanding of the invention, a case where a face is detected as a specific target as is often done with a general digital camera will be described as an example. It is not particularly limited to the face. For example, the present invention can also be applied when detecting a car, a building, or the like as a specific target.

  FIG. 6 is a block diagram showing an internal configuration of the digital camera 10 shown in FIGS. 1 and 2 in the first embodiment. The elements shown in FIGS. 1 and 2 are denoted by the same reference numerals, and the description of the contents already described is omitted.

  As shown in FIG. 6, the digital camera 10 of the present embodiment is configured to be able to acquire image signals from each of the two imaging systems, and includes a CPU 110, a difference vector calculation unit 70, a movement vector calculation unit 72, and a parallax amount. Adjustment unit 74, operation unit 112, ROM 116, flash ROM 118, SDRAM 120, VRAM 122 (image display memory), photographing lens 14 (14R, 14L), zoom lens control unit 124 (124R, 124L), focus lens control unit 126 (126R, 126L), aperture control unit 128 (128R, 128L), image sensor 134 (134R, 134L), image sensor control unit 136 (136R, 136L), analog signal processing unit 138 (138R, 138L), and A / D converter 140. (140R, 140L), image input controller 141 (141R, 141L), digital signal processing unit 142 (142R, 142L), AF detection unit 144, AE / AWB detection unit 146, compression / decompression processing unit 152, media control unit 154, memory card 156, display control unit 158 , A power control unit 160, a battery 162, and a strobe control unit 164.

  The imaging means 11L for the left eye mainly includes a photographing lens 14L, a zoom lens control unit 124L, a focus lens control unit 126L, an aperture control unit 128L, an image sensor 134L, an image sensor control unit 136L, an analog signal processing unit 138L, and an A / A. It includes a D converter 140L, an image input controller 141L, a digital signal processing unit 142L, and the like.

  The imaging means 11R for the right eye mainly includes a photographing lens 14R, a zoom lens control unit 124R, a focus lens control unit 126R, an aperture control unit 128R, an image sensor 134R, an image sensor control unit 136R, an analog signal processing unit 138R, and an A / A. It comprises a D converter 140R, an image input controller 141R, a digital signal processing unit 142R, and the like.

  Hereinafter, digital image data obtained by imaging the subject with the imaging units 11L and 11R is referred to as “captured image”. The captured image obtained by the left-eye imaging unit 11L is referred to as a “left-eye image”, and the captured image obtained by the right-eye imaging unit 11R is referred to as a “right-eye image”.

  The CPU 110 functions as a control unit that controls the overall operation of the camera, such as shooting, display, and recording, and controls each unit according to a predetermined control program based on an input from the operation unit 112.

  The operation unit 112 includes the shutter button 18, the power / mode switch 20, the mode dial 22, the zoom button 26, the cross button 28, the MENU / OK button 30, the DISP button 32, the BACK button 34, shown in FIGS. A vertical shooting / horizontal shooting switching button 36 and the like are included.

  The ROM 116 connected via the bus 114 stores a control program executed by the CPU 110 and various data necessary for control (AE / AF control cycle and the like described later). Various setting information relating to the operation of the digital camera 10 such as setting information is stored.

  The SDRAM 120 is used as a calculation work area for the CPU 110 and is also used as a temporary storage area for image data, and the VRAM 122 is used as a temporary storage area dedicated to image data for display.

  A pair of left and right photographing lenses 14R and 14L (also collectively referred to as photographing lens 14) are zoom lenses 130ZR and 130ZL (also collectively denoted as zoom lens 130Z), focus lenses 130FR and 130FL (collectively focus lenses). 130F), which includes the apertures 132R and 132L, and is arranged on the camera body 12 with a predetermined interval.

  The zoom lenses 130ZR and 130LR are driven by a zoom actuator (not shown) to move back and forth along the optical axis. The CPU 110 controls the position of the zoom lens by controlling the driving of the zoom actuator via the zoom lens control units 124R and 124L, and zooms the photographing lenses 14R and 14L.

  The focus lenses 130FR and 130FL are driven by a focus actuator (not shown) to move back and forth along the optical axis. The CPU 110 controls the position of the focus lens by controlling the drive of the focus actuator via the focus lens control units 126R and 126L, and performs focusing of the photographing lenses 14R and 14L.

  The diaphragms 132R and 132L are constituted by, for example, iris diaphragms, and are operated by being driven by a diaphragm actuator (not shown). The CPU 110 controls the aperture amount (aperture value) of the diaphragms 132R and 132L by controlling the driving of the diaphragm actuator via the diaphragm controllers 128R and 128L, and controls the amount of light incident on the image sensors 134R and 134L.

  The CPU 110 drives the left and right photographing lenses 14R and 14L in synchronism when driving the zoom lenses 130ZR and 130ZL, the focus lenses 130FR and 130FL, and the apertures 132R and 132L constituting the photographing lenses 14R and 14L. That is, the left and right photographing lenses 14R and 14L are always set to the same focal length (zoom magnification), and focus adjustment is performed so that the same subject is always in focus. In addition, the aperture is adjusted so that the same incident light amount (aperture value) is always obtained.

  The image sensors 134R and 134L are composed of color CCDs having a predetermined color filter array. In the CCD, a large number of photodiodes are two-dimensionally arranged on the light receiving surface. The optical image of the subject formed on the light receiving surface of the CCD by the photographing lenses 14R and 14L is converted into signal charges corresponding to the amount of incident light by the photodiode. The signal charges accumulated in each photodiode are sequentially read out from the image sensors 134R and 134L as voltage signals (image signals) corresponding to the signal charges based on the drive pulses given from the image sensor controllers 136R and 136L in accordance with instructions from the CPU 110. It is.

  The imaging elements 134R and 134L have an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.

  In the present embodiment, a CCD is used as the image sensor, but an image sensor having another configuration such as a CMOS sensor can also be used.

  Analog signal processing units 138R and 138L are correlated double sampling circuits (CDS) for removing reset noise (low frequency) included in the image signals output from the image sensors 134R and 134L, amplify the image signals, and are constant An AGS circuit for controlling the magnitude of the level is included, and the image signals output from the image sensors 134R and 134L are subjected to correlated double sampling processing and amplified. The A / D converters 140R and 140L convert the analog image signals output from the analog signal processing units 138R and 138L into digital image signals. The image input controllers 141R and 141L take in the image signals output from the A / D converters 140R and 140L and store them in the SDRAM 120. The digital signal processing units 142R and 142L take in the image signal stored in the SDRAM 120 in accordance with a command from the CPU 110, perform predetermined signal processing, and generate a YUV signal including the luminance signal Y and the color difference signals Cr and Cb.

  The digital signal processing units 142R and 142L perform various processes such as offset processing, white balance adjustment processing, gamma correction processing, RGB interpolation processing, RGB / YC conversion processing, noise reduction processing, contour correction processing, color correction, and light source type determination processing. Perform digital correction. The digital signal processing unit 142 (142R, 142L) may be configured by a hardware circuit, or the same function may be configured by software.

  The AF detection unit 144 captures R, G, and B color image signals captured from one image input controller 141R, and calculates a focus evaluation value necessary for AF control. The AF detection unit 144 includes a high-pass filter that allows only a high-frequency component of the G signal to pass, an absolute value processing unit, a focus area extraction unit that extracts a signal within a predetermined focus area set on the screen, and a focus area An integration unit for integrating the absolute value data is included, and the absolute value data in the focus area integrated by the integration unit is output to the CPU 110 as a focus evaluation value.

  During the AF control, the CPU 110 searches for a position where the focus evaluation value output from the AF detection unit 144 is maximized, and moves the focus lenses 130FR and 130FL to the position to perform focusing on the main subject. . That is, during the AF control, the CPU 110 first moves the focus lenses 130FR and 130FL from the close range to the infinity, sequentially acquires the focus evaluation value from the AF detection unit 144 in the moving process, and the focus evaluation value becomes maximum. Detect position. Then, the position where the detected focus evaluation value is maximum is determined as the in-focus position, and the focus lenses 130FR and 130FL are moved to that position. Thereby, the subject (main subject) located in the focus area is focused.

  The AE / AWB detection unit 146 takes in image signals of R, G, and B colors taken from one image input controller 141R, and calculates an integrated value necessary for AE control and AWB control. That is, the AE / AWB detection unit 146 divides one screen into a plurality of areas (for example, 8 × 8 = 64 areas), and calculates an integrated value of R, G, and B signals for each divided area.

  At the time of AE control, the CPU 110 acquires an integrated value of R, G, B signals for each area calculated by the AE / AWB detection unit 146, obtains the brightness (photometric value) of the subject, and obtains an appropriate exposure amount. Set the exposure to obtain That is, sensitivity, aperture value, shutter speed, and necessity of strobe light emission are set.

  In addition, the CPU 110 adds the integrated values of R, G, and B signals for each area calculated by the AE / AWB detection unit 146 to the digital signal processing unit 142 during AWB control. The digital signal processing unit 142 calculates a gain value for white balance adjustment based on the integrated value calculated by the AE / AWB detection unit 146. Further, the digital signal processing unit 142 detects the light source type based on the integrated value calculated by the AE / AWB detection unit 146.

  The compression / decompression processing unit 152 performs compression processing in a predetermined format on the input image data in accordance with a command from the CPU 110 to generate compressed image data. Further, in accordance with a command from the CPU 110, the input compressed image data is subjected to a decompression process in a predetermined format to generate uncompressed image data. In the digital camera 10 according to the present embodiment, a still image is subjected to compression processing conforming to the JPEG standard, and a moving image is subjected to compression processing conforming to the MPEG2 standard.

  The media control unit 154 controls reading / writing of data with respect to the memory card 156 in accordance with a command from the CPU 110.

  The display control unit 158 controls display on the monitor 24 in accordance with a command from the CPU 110. That is, in accordance with a command from the CPU 110, the input image signal is converted into a video signal (for example, NTSC signal, PAL signal, SCAM signal) to be displayed on the monitor 24 and output to the monitor 24. The graphic information is output to the monitor 24.

  The power control unit 160 controls power supply from the battery 162 to each unit in accordance with a command from the CPU 110.

  The strobe control unit 164 controls the light emission of the strobe 16 in accordance with a command from the CPU 110.

  The face detection unit 148 detects a face in each of the left eye image and the right eye image, and includes face detection frame information having at least representative coordinates of the face detection frame in the left eye image and representative coordinates of the face detection frame in the right eye image. Is generated. That is, a specific target is detected in each captured image, and detection mark information having at least representative coordinates of the detection mark of the specific target in each captured image is generated.

  Here, the face is detected based on an evaluation value indicating the likelihood of a face. The face detection frame indicates the position or range of the face in the captured image. In the digital camera 10 shown in FIG. 6, even when general AF (automatic focus control) or AE (automatic exposure control) is difficult, it is possible to detect a face and perform appropriate focusing and AF by performing AF or AE on the face area. An exposed subject image can be obtained. Further, in a shooting scene where the face is important, the face is detected, and a specific correction (for example, correction selected by the user) is performed on the face area. Further, by displaying the face detection frame on the monitor 24, the user can determine whether or not the digital camera 10 has detected a face.

  The difference vector calculation unit 70 acquires the face detection frame information generated by the face detection unit 148 and calculates a difference vector between the representative coordinates of the face detection frame in the left eye image and the representative coordinates of the face detection frame in the right eye image. calculate. For example, a difference vector between the center coordinates of the face detection frame in the left eye image and the center coordinates of the face detection frame in the right eye image is calculated. When the face detection frame is a quadrangle and the inclination and size of the face detection frame are the same for the left eye image and the right eye image, a difference vector may be obtained using the coordinates of one of the four vertices as representative coordinates. .

  The movement vector calculation unit 72 adjusts the parallax amount of the face between the left eye image and the right eye image by a parallax amount adjustment unit 74 (to be described later), so that the left side is based on the difference vector calculated by the difference vector calculation unit 70. At least one of a movement vector of the eye image and a movement vector of the right eye image is calculated.

  The parallax amount adjustment unit 74 moves at least one of the left-eye image and the right-eye image according to the movement vector calculated by the movement vector calculation unit 72, so that the display position of the face of the left-eye image and the right-eye image Adjust the difference with the face display position. Specifically, the difference in the face display position on the display screen of the monitor 24 is adjusted. That is, the amount of parallax of the face is adjusted.

  As an example of adjusting the amount of parallax, first, there is an adjustment to match the display position of the face. Second, there is an adjustment that increases the amount of parallax of the face. Third, there is an adjustment for setting the parallax amount of the face to the target amount. Which of the first to third adjustments is to be performed is instructed by the operation unit 112. For example, in the normal portrait mode, the face display positions are matched and the parallax amount is set to “0”.

  There are various specific configurations of the parallax adjustment unit 74.

  First, by moving at least one of the left eye image and the right eye image on the display screen of the monitor 24 by image processing, the display position of the face of the left eye image and the display position of the face of the right eye image There is a configuration for adjusting the difference (that is, the amount of parallax). Actually, the display position is adjusted when the left eye image and the right eye image are written in the VRAM 122. A pixel value of black (or white, etc.) is set for the display pixel of the monitor 24 which becomes blank by moving the image. For example, both the left eye image and the right eye image are moved by ½ of the difference vector to match the face display position.

  Second, there is a configuration in which the display position of the face is moved using a camera shake correction mechanism that corrects camera shake by moving the image sensor 134 or the photographic lens 14. For example, in the case of a CCD (imaging device) shift type, both the left eye imaging device 134L and the right eye imaging device 134R are moved by a movement amount corresponding to ½ of the difference vector. Match the display position of. In this example, the amount of movement of the image sensor 134 corresponding to one pixel (1 pixel) is obtained in advance. That is, the correspondence between the movement amount of the image sensor 134 and the movement amount of the display position on the monitor 24 is obtained in advance. Alternatively, it has a feedback function and moves the image sensor 134 until the face display positions match.

  Third, there is a configuration in which the display position of the face is moved using a convergence angle changing mechanism that changes the convergence angle between the left-eye imaging lens 14L and the right-eye imaging lens 14R. The convergence angle is an angle (θc in FIG. 4B) formed by the optical axis of the left-eye imaging lens 14L and the optical axis of the right-eye imaging lens 14R. In this example, the rotation amount of the photographing lens 14 corresponding to one pixel (1 pixel) is obtained in advance. Alternatively, the photographing lens 14 is rotated until it has a feedback function and the face display positions match.

  FIG. 7 is a flowchart showing a flow of an example of photographing processing in the first embodiment to which the present invention is applied. This process is executed according to a program by the overall control of the CPU 110 in FIG. 6 when the shutter button 18 in FIG. 1 is half-pressed. This processing may be started when the photographing mode is selected by the power / mode switch 20.

  In step S1, a left-eye image captured by the left-eye image sensor 134L and a right-eye image captured by the right-eye image sensor 134R are acquired. In the digital camera 10 of FIG. 6, the signal charge is read from the photodiode (imaging pixel) of the image sensor 134 under the control of the image sensor control unit 136, a digital image signal composed of the signal charge is generated, and the image signal Is captured by the image input controller 141 and stored in the SDRAM 120.

  In step S <b> 2, the difference vector calculation unit 70 acquires face detection frame information from the SDRAM 120. Face detection processing for detecting a face (face image) in the left eye image and right eye image is performed by the face detection unit 148, and face detection frame information generated as a result of the face detection processing is stored in the SDRAM 120. Yes. The face detection frame information includes the representative coordinates of the face detection frame in the left eye image and the representative coordinates of the face detection frame in the right eye image.

  FIG. 8A shows a left eye image 91L and its face detection frame 92L as an example, and FIG. 8B shows a right eye image 91R and its face detection frame 92R as an example. In this example, a square mark (face detection mark) is used as a face detection frame, and the coordinates of the four points constituting the mark are used as representative coordinates to adjust the parallax amount of the face described later.

  In this example, the coordinates of the four points of the face detection frame indicating the position of the face in the left eye image are (Lx1, Ly1), (Lx2, Ly2), (Lx3, Ly3), (Lx4, Ly4). . Also, the coordinates of the four points of the face detection frame indicating the position of the face in the right eye image are (Rx1, Ry1), (Rx2, Ry2), (Rx3, Ry3), (Rx4, Ry4).

  Note that AF (automatic focus control) and AE (automatic exposure control) of each of the imaging units 11L and 11R are performed based on the face detection frame information generated by the face detection unit 148. For example, AE and AF of the imaging means 11L for the left eye are performed based on the face detection frame information for the left eye, and AE and AF of the imaging means 11R for the right eye are performed based on the face detection frame information for the right eye. Is called.

  In step S3, based on the face detection frame information, the difference vector calculation unit 70 determines the coordinates of the four points on the face detection frame in the left-eye captured image and the four points on the face detection frame in the right-eye captured image. The difference from the coordinates is calculated.

  In this example, as shown in the following equation, coordinate differences t1 to t4 are calculated for each of the four corners of the face detection frame.

[Equation 1]
^{t1 = ((Lx1-Rx1)} 2 + (Ly1-Ry1) 2) 0.5
^{t2 = ((Lx2-Rx2)} 2 + (Ly2-Ry2) 2) 0.5
^{t3 = ((Lx3-Rx3)} 2 + (Ly3-Ry3) 2) 0.5
t4 = ((Lx4-Rx4) ² + (Ly4-Ry4) ² ) ^0.5
In step S4, it is determined whether the coordinate differences t1 to t4 are the same. If t1 to t4 are not the same, steps S5 to S8 are performed, and if t1 to t4 are the same, steps S9 to S11 are performed.

In step S5, the difference vector calculating portion 70, FIG. 9 (A) the center point _{C L} coordinates of the face detection frame in the left eye image 91L shown _{(C Lx,} C _Ly) to calculate the, and FIG. 9 (B) center point _{C R} of the coordinates of the face detection frame of the right eye image 91R shown _{(C Rx,} C _Ry) is calculated. C _Lx , C _Ly , C _Rx , and C _Ry are shown in the following equations.
[Equation 2]
C _Lx = (Lx1 + Lx2 + Lx3 + Lx4) / 4
C _Ly = (Ly1 + Ly2 + Ly3 + Ly4) / 4
C _Rx = (Rx1 + Rx2 + Rx3 + Rx4) / 4
C _Ry = (Ry1 + Ry2 + Ry3 + Ry4) / 4
In step S6, the difference vector calculating portion 70, as shown in FIG. 10 (A), the center point of the center point C _L coordinate and the right-eye image 91R of the face detection frame of the face detection frame of the left-eye image 91L C A difference vector tc _LR with respect to the coordinates of _R is calculated. The difference vector tc _LR is shown in the following equation.

[Equation 3]
tc _LR = [C _Rx , C _Ry ] − [C _Lx , C _Ly ]
In step S7, the movement vector calculation unit 72 calculates the movement vector (tc _LR / 2) of the left eye image and the right eye image, and in step S8, the parallax amount adjustment unit 74 performs the operation as shown in FIG. By moving the left eye image 91L and the right eye image 91R by half of the difference vector tc _LR (0.5 × tc _LR / 2), the center coordinates of the face detection frame of the left eye image 91L and the right eye image 91R The center coordinates of the face detection frame are substantially matched. That is, the display position of the face of the left eye image and the display position of the face of the right eye image are substantially matched. Thereby, the parallax amount of the face is set to “0” between the left eye image and the right eye image.

If the difference vectors t1 to t4 of the points in the face detection frame are the same in step S4, the difference vector calculation unit 70 causes the face detection frame for the left eye image and the face detection frame for the right eye image in step S9. Then, a difference vector ti _LR of coordinates between corresponding points Ri (Rxi, Ryi) and Li (Lxi, Lyi) corresponding to each other is calculated (shown in FIG. 10B). Here, i is any one of 1-4.

In step S10, the movement vector calculation unit 72 calculates the movement vector (0.5 × ti _LR ) of the left eye image 91L and the right eye image 91R, and the parallax amount adjustment unit 74 calculates the left eye in step S11. The center point coordinates of the face detection frame of the left eye image and the center point coordinates of the face detection frame of the right eye image are substantially matched by moving the image 91L and the right eye image 91R by the respective movement vectors. That is, the display position of the face of the left eye image and the display position of the face of the right eye image are substantially matched. Accordingly, the parallax amount of the face is set to “0” between the left eye image and the right eye image.

  In step S12, the display control unit 158 causes the monitor 24 to display the right eye image and the left eye image with the parallax amount adjusted as a through image.

  In step S13, it is determined whether or not the shutter button 18 has been fully pressed. When the shutter button 18 is fully pressed, the process proceeds to step S14. When the shutter button 18 is not fully pressed, the process returns to step S1.

  When the shutter button is fully pressed, in step S14, the media control unit 154 causes the media control unit 154 to display display positions of the captured image (the right eye image and the left eye image) and the face of the left eye image and the face of the right eye image. The information used for the alignment (position alignment information) is associated and recorded in the memory card 156.

  Here, as the alignment information, there are a difference vector (a parallax amount before adjustment), a movement vector, a parallax amount after adjustment (“0” in this example), and the like. At least one of the difference vector and the movement vector is recorded in association with the captured image.

  12A to 12C show examples of association between the alignment information and the captured image. In the example shown in FIG. 12A, a header including alignment information, a left eye image, and a right eye image are recorded as one stereoscopic image file 191. In the example shown in FIG. 12B, the header and the left eye image including the alignment information of the left eye image are set as the left eye image file 192, and the header and the right eye image including the alignment information of the right eye image are converted to the right. Each is recorded as an eye image file 193. In the example shown in FIG. 12C, an alignment information file 194 including alignment information, a left eye image file 195 including a left eye image, and a right eye image file 196 including a right eye image are recorded. In the case shown in FIGS. 12B and 12C, a plurality of files may be stored in the same folder, or a part of the file name may be stored in the same folder.

(Second Embodiment)
In the second embodiment, a plurality of captured images (left-eye image and right-eye image) respectively captured at a plurality of viewpoints are acquired, and specific target parts are detected and generated from the plurality of captured images. The amount of parallax of the specific target is adjusted by adjusting the difference in the display position (display coordinates on the monitor 24) of the part of the specific target among the plurality of captured images using the part information. In other words, unlike the first embodiment in that the part information is used, it is the same as the first embodiment in that the parallax amount of the specific target is adjusted between a plurality of captured images.

  In this example, in order to facilitate the understanding of the invention, as is often done with a general digital camera, an example of detecting a part (hereinafter referred to as “face part”) with a face as a specific target is taken as an example. However, the specific object is not particularly limited to the face in the present invention. For example, the present invention can be applied to a case where a part is detected with a car, a building, or the like as a specific target.

  FIG. 13 is a block diagram showing an internal configuration of the digital camera 10 shown in FIGS. 1 and 2 in the second embodiment. In addition, the same code | symbol is attached | subjected to the same component as the component shown in FIG. 1, FIG. 2, and FIG. 6 of 1st Embodiment, The description is abbreviate | omitted about the content already demonstrated.

  In FIG. 13, the face part detection unit 149 detects face parts (for example, eyes, mouth, etc.) in each of the left eye image and the right eye image, and the representative coordinates and right eye image of each face part in the left eye image. Part information having at least representative coordinates of each face part is generated.

  Here, the face part is detected based on an evaluation value indicating the likelihood of the face part. In the digital camera 10 of FIG. 13, even when general AF (automatic focus control) or AE (automatic exposure control) is difficult, a face part is detected and AF or AE is performed on the face area including the face part and the periphery. Thus, it is possible to obtain a subject image that is appropriately focused and exposed. Further, in a shooting scene where the face is important, a face part is detected, and a specific correction (for example, correction selected by the user) is performed on the face part and the face area including the periphery. Further, by displaying a face part detection frame indicating the position or range of the face part on the monitor 24, the user can determine whether or not the digital camera 10 has detected the face part.

  The difference vector calculation unit 170 acquires the part information generated by the face part detection unit 149, and calculates a difference vector between the representative coordinates of the face part in the left eye image and the representative coordinates of the face part in the right eye image. For example, in the left eye image and the right eye image, a difference vector of coordinates between points corresponding to each other of the corresponding face parts (for example, the left eye for the left eye and the right eye for the right eye) is calculated. You may calculate the difference vector of the center coordinate of the detection frame of the face parts corresponding to each other.

  The movement vector calculation unit 172 and the parallax amount adjustment unit 174 are the same as the movement vector calculation unit 72 and the parallax amount adjustment unit 74 of FIG. 6 in the first embodiment. The movement vector calculation unit 172 of the present embodiment calculates a movement vector of the left eye image and a movement vector of the right eye image based on the difference vector calculated by the difference vector calculation unit 170 of the present embodiment. The parallax amount adjustment unit 174 according to the present embodiment displays the face part of the left eye image by moving at least one of the left eye image and the right eye image according to the movement vector calculated by the movement vector calculation unit 172. The difference between the position and the display position of the face part of the right eye image is adjusted. Thereby, the parallax amount of the face is adjusted between the left eye image and the right eye image.

  The image rotation processing unit 176 detects the tilt of the face in the left eye image and the right eye image based on the representative coordinates of the plurality of face parts indicated by the part information, and applies to at least one of the left eye image and the right eye image. By performing rotation processing on the left eye image and the right eye image, the inclination of the face is matched. For example, the rotation angle of the image is calculated based on the coordinates of both eyes.

  The image enlargement / reduction processing unit 178 detects the size ratio of the face between the left eye image and the right eye image based on the representative coordinates of the plurality of face parts indicated by the part information, and among the left eye image and the right eye image By performing enlargement / reduction processing on at least one of the images, the size of the face is matched between the left eye image and the right eye image. For example, the ratio of the face sizes is detected based on the distance between both eyes, and the enlargement / reduction ratio of the image is calculated.

  FIG. 14 is a flowchart showing an example of a shooting process to which the image processing method of the present invention is applied. This process is executed according to a program by the overall control of the CPU 110 in FIG. 13 when the shutter button 18 in FIG. 1 is half-pressed. This processing may be started when the photographing mode is selected by the power / mode switch 20.

  Step S21 is the same as step S1 of the first embodiment shown in FIG. 7 and has already been described, so the description thereof is omitted here.

  In step S <b> 22, the difference vector calculation unit 170 acquires part information from the SDRAM 120. Face part detection processing for detecting face parts (face part images) in the left eye image and the right eye image is performed by the face part detection unit 149, and part information generated as a result of the face part detection processing is stored in the SDRAM 120. Stored. The part information includes representative coordinates of each face part in the left eye image and representative coordinates of each face part in the right eye image.

  FIG. 15A shows a left eye image as an example, and FIG. 15B shows a right eye image as an example. In the following, for the sake of simplicity, it is assumed that only eyes are detected as face parts and only the coordinates of the face part detection frame indicating the position of the eyes are used. A square mark is used as the face part detection frame, and the coordinates of the four points constituting the mark are used as the representative coordinates of the eyes to adjust the parallax amount of the face, which will be described later.

In this example, the left eye coordinates of the four points of the face part detecting frame indicating the position of the left eye in the ^{_{image, (L P 1x1, L P}} 1y1), (L P 1x2, L P 1y2), (L P 1x3 , ^L P _1y3 ), ( ^L P _1x4 , ^L P _1y4 ). Left eye right eye coordinates of the four points of the face part detecting frame indicating the position of the ^{_{image, (L P 2x1, L P}} 2y1), (L P 2x2, L P 2y2), (L P 2x3, L P 2y3) , ( ^L P _2x4 , ^L P _2y4 ).

Right coordinates of the four points of the face part detecting frame indicating the position of the left eye in the eye ^{_{image, (R P 1x1, R P}} 1y1), (R P 1x2, R P 1y2), (R P 1x3, R P 1y3) , ( ^R P _1x4 , ^R P _1y4 ).

Right eye right eye coordinates of four points of the face part detecting frame indicating the position of the ^{_{image, (R P 2x1, R P}} 2y1), (R P 2x2, R P 2y2), (R P 2x3, R P 2y3) , ( ^R P _2x4 , ^R P _2y4 ).

  Note that AF (automatic focus control) and AE (automatic exposure control) of each of the imaging units 11L and 11R are performed based on the part information generated by the face part detection unit 149. For example, AE and AF of the imaging means 11L for the left eye are performed based on the face part detection information for the left eye, and AE and AF of the imaging means 11R for the right eye are performed based on the face part detection information for the right eye. Is called.

  In step S23, the difference vector calculation unit 170 calculates the center coordinates of each face part in each image (left eye image and right eye image). In this example, the center coordinates of the eye detection frame (face part detection frame) are calculated as the eye center coordinates.

Center coordinates of the left-eye detection frame of the left eye image ^{_{(L P cx1, L P cy1}} ) is calculated by the following equation.

[Equation 4]
^{_{L P cx1 = (L P 1x1}} + L P 1x2 + L P 1x3 + L P 1x4) / 4
^L P _cy1 = ( ^L P _1y1 + ^L P _1y2 + ^L P _1y3 + ^L P _1y4 ) / 4
Center coordinates of the right eye detection frame of the left eye image ^{_{(L P cx2, L P cy2}} ) is calculated by the following equation.

[Equation 5]
^{_{L P cx2 = (L P 2x1}} + L P 2x2 + L P 2x3 + L P 2x4) / 4
^L P _cy2 = ( ^L P _2y1 + ^L P _2y2 + ^L P _2y3 + ^L P _2y4 ) / 4
Center coordinates of the left-eye detection frame of the right eye image ^{_{(R P cx1, R P cy1}} ) is calculated by the following equation.

[Equation 6]
^{_{R P cx1 = (R P 1x1}} + R P 1x2 + R P 1x3 + R P 1x4) / 4
^{_{R P cy1 = (R P 1y1}} + R P 1y2 + R P 1y3 + R P 1y4) / 4
Center coordinates of the right eye detection frame of the right eye image ^{_{(R P cx2, R P cy2}} ) is calculated by the following equation.

[Equation 7]
^{_{R P cx2 = (R P 2x1}} + R P 2x2 + R P 2x3 + R P 2x4) / 4
^{_{R P cy2 = (R P 2y1}} + R P 2y2 + R P 2y3 + R P 2y4) / 4
In step S24, the image rotation processing unit 176 calculates the slopes of the straight lines a _L and a _R passing through both eyes in each image (left eye image and right eye image). In this example, the slopes a _L and a _R of straight lines passing through the center coordinates of the two eye detection frames are calculated.
^{_{a L = (L P cy2 -}} L P cy1) / (L P cx2 - L P cx1)
^{_{a R = (R P cy2 -}} R P cy1) / (R P cx2 - R P cx1)
In step S25, the image rotation processing unit 176 compares the inclinations a _L and a _R , performs rotation processing on the image having a large inclination, and matches the images with the smaller inclination. Thereby, the inclination of the face is matched between the left eye image and the right eye image.

When a _L > a _R , the left eye image is rotated so as to have the same inclination a _R.

When a _L <a _R , the left eye image is rotated so as to have the same inclination a _L.

In step S26, the image enlargement / reduction processing unit 178 calculates the distances d _L and d _R between the eyes in each image (left eye image and right eye image) as shown in FIGS. 16 (A) and 16 (B). To do.
^{_{d L = ((L P cX2}} - L P cX1) 2 + (L P cY2 - L P cY1) 2) 0.5
^{_{d R = ((R P cX2}} - R P cy1) 2 + (R P cY2 - R P cY1) 2) 0.5
In step S27, the image enlargement / reduction processing unit 178 obtains the ratio b (= d _L / d _R ) between both eyes as the face size ratio, and doubles the right eye image by b. As a result, the left eye image 91L and the right eye image 91R have the same face size.

In step S28, the movement vector calculation unit 172 calculates the movement vector (tc _LR / 2) of the left eye image and the right eye image, and in step S29, the left eye image 91L and the right eye image 91R are respectively converted into the difference vector tc. _By moving the _LR by half (tc _LR / 2), the center coordinates of the corresponding face parts in the left eye image 91L and the right eye image 91R are substantially matched. That is, the parallax amount of the face is set to “0” between the left eye image and the right eye image.

  Steps S30, S31, and S32 are the same as steps S12, S13, and S14 of the first embodiment shown in FIG. 7 and have already been described.

  The case where the specific target for adjusting the parallax amount is a face has been described above as an example, but the specific target is not limited in the present invention. Further, although the case where the detection mark is a square face detection frame has been described as an example, the shape of the detection mark information is not limited in the present invention. For example, a circular detection mark or a symbol other than a frame (for example, an arrow) may be used. The detection mark information may be information indicating at least the position or range of the face on the captured image. Further, although the case where the part information is the coordinate information of the eyes (or the eye detection frame) has been described as an example, the parts information may be information indicating at least the position or range of the part on the captured image.

  Moreover, although the case where the captured image is acquired from the image sensor 134 at the time of imaging has been described as an example, the present invention is not limited to such a case. The present invention may be applied even when acquiring from a recording medium such as the memory card 156, or when acquiring via communication via the input / output connector 38. For the communication, a known communication technology such as USB (Universal Serial Bus) communication may be used.

  Further, the present invention is not limited to the digital camera 10 and can be applied to various image processing apparatuses capable of stereoscopic display. Needless to say, the monitor 24 may be provided outside the image processing apparatus.

  The present invention is not limited to the examples described in the present specification and the examples illustrated in the drawings, and various design changes and improvements may be made without departing from the scope of the present invention. is there.

The front perspective view which shows the external appearance structure of the digital camera to which this invention is applied The rear perspective view which shows the external appearance structure of the digital camera to which this invention is applied Explanatory drawing for demonstrating the structural example of the monitor which can be displayed stereoscopically (A) and (B) are schematic diagrams schematically showing a situation in which a face is imaged by a digital camera, and (C) is a schematic diagram schematically showing a state in which the face is noticed by both eyes of a person. (A) is explanatory drawing which shows an example of a left eye image and a right eye image, (B) is explanatory drawing which shows typically how to look when a parallax amount is large. 1 is a block diagram illustrating an example of an internal configuration of a digital camera according to a first embodiment. The flowchart which shows the flow of an example of the imaging | photography process in 1st Embodiment to which this invention is applied. (A) is explanatory drawing which shows a left eye image and its face detection frame, (B) is explanatory drawing which shows a right eye image and its face detection frame (A) is explanatory drawing which shows the center coordinate of the face detection frame in a left eye image, (B) is explanatory drawing which shows the center coordinate of the face detection frame in a right eye image (A) is explanatory drawing which shows the difference vector of the center coordinate of a face detection frame, (B) is explanatory drawing which shows the difference vector of the coordinate of the corresponding points of a face detection frame. Explanatory drawing which shows the amount adjustment of the parallax of the face by image movement Explanatory drawing used for explaining registration information and image recording The block diagram which shows an example of an internal structure of the digital camera in 2nd Embodiment. The flowchart which shows the flow of an example of the imaging | photography process in 2nd Embodiment to which this invention is applied. (A) is explanatory drawing which shows a left eye image and its eye detection frame, (B) is explanatory drawing which shows a right eye image and its eye detection frame (A) is explanatory drawing which shows the space | interval of both eyes in a left eye image, (B) is explanatory drawing which shows the space | interval of both eyes in a right eye image Explanatory drawing showing the difference vector of the eye center coordinates

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 11 (11R, 11L) ... Imaging means, 14 (14R, 14L) ... Shooting lens, 18 ... Shutter button, 20 ... Power supply / mode switch, 24 ... Monitor, 70, 170 ... Difference vector calculation part, 72, 172 ... movement vector calculation unit, 74, 174 ... parallax amount adjustment unit, 110 ... CPU, 122 ... VRAM (image display memory), 134 (134R, 134L) ... image sensor, 148 ... face detection unit, 149 ... face Parts detection unit, 154... Media control unit, 176... Image rotation processing unit, 178.

Claims (10)

Image acquisition means for acquiring a first captured image and a second captured image captured from a plurality of viewpoints;
A specific target is detected in each of the first captured image and the second captured image, and representative coordinates of the detection mark of the specific target in the first captured image and the specific target in the second captured image are detected. Specific object detection means for generating detection mark information having representative coordinates of the detection mark;
Difference vector calculation means for calculating a difference vector between a representative coordinate of the detection mark in the first captured image and a representative coordinate of the detection mark in the second captured image;
The parallax amount of the specific target is adjusted by adjusting a difference between the display position of the specific target of the first captured image and the display position of the specific target of the second captured image based on the difference vector. A parallax amount adjusting means to adjust;
Display control means for displaying the first captured image and the second captured image with the parallax amount adjusted on a display means capable of stereoscopic display;
A stereoscopic image processing apparatus comprising:   The difference vector calculation means calculates center coordinates of the detection mark in each of the first captured image and the second captured image, and the center coordinate of the detection mark in the first captured image The stereoscopic image processing apparatus according to claim 1, wherein a difference between the detection mark and the center coordinate of the second captured image is calculated as the difference vector. It said parallax amount adjustment means, three-dimensional image processing apparatus according to claim 1 or 2, characterized in that to match the display position of the specific subject. It said parallax amount adjustment means, three-dimensional image processing apparatus according to claim 1 or 2, characterized in that to increase the amount of parallax of the particular subject. It said parallax amount adjustment means, three-dimensional image processing apparatus according to any one of claims 1 to 4, characterized in that to change the image processing the display position of the specific object. A first imaging means having a first imaging lens and a first imaging element on which a subject image is formed via the first imaging lens;
A second imaging means having a second imaging lens and a second imaging element on which a subject image is formed via the second imaging lens;
A camera shake correction unit that corrects camera shake by moving at least one of the photographing lens and the imaging unit;
With
It said parallax amount adjustment means, three-dimensional image processing apparatus according to any one of claims 1 to 4, wherein the changing the display position of the specific subject using the camera shake correction means. A first imaging means having a first imaging lens and a first imaging element on which a subject image is formed via the first imaging lens;
A second imaging means having a second imaging lens and a second imaging element on which a subject image is formed via the second imaging lens;
A convergence angle changing means for changing a convergence angle of the first photographing lens and the second photographing lens;
With
It said parallax amount adjustment means, three-dimensional image processing apparatus according to any one of claims 1 to 4, wherein the changing the display position of the specific object by using the convergence angle changing means. A shooting instruction means for inputting a shooting instruction;
Movement of the display position of the specific target obtained from the difference vector when the imaging instruction is input by the imaging instruction unit in a state where the captured image with the parallax amount adjusted is displayed on the display unit And a recording control unit configured to record at least one of the vector and the difference vector as parallax amount adjustment information on a recording medium in association with the first captured image and the second captured image. Item 5. The stereoscopic image processing apparatus according to any one of Items 1 to 4 . The image acquisition means acquires the captured image from a recording medium or by communication,
Said parallax amount adjustment means, when displaying the captured image acquired in the display device, according to any one of claims 1 to 5 and 8, characterized in that the adjustment of the parallax amount 3D image processing apparatus. Obtaining a first captured image and a second captured image captured from a plurality of viewpoints;
Obtaining detection mark information having representative coordinates of the detection mark of the specific target, generated by detecting the specific target in each of the first captured image and the second captured image;
Calculating a difference vector between a representative coordinate of the detection mark in the first captured image and a representative coordinate of the detection mark in the second captured image;
The parallax amount of the specific target is adjusted by adjusting a difference between the display position of the specific target of the first captured image and the display position of the specific target of the second captured image based on the difference vector. Adjusting steps,
Displaying the first captured image and the second captured image in which the amount of parallax is adjusted on display means capable of stereoscopic display;
A stereoscopic image processing method comprising:

Images