JP2015012304A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily guide a user to a desired photographing state.SOLUTION: When an image acquired by an image acquisition unit is deviated greatly from a desired state, an image generation unit generates a greatly deviated image where the deviation from the desired state is increased. This technique can be applied to an apparatus with a photographing function equipped with a photographing function of photographing an image such as a digital camera, a smartphone, a mobile phone, a tablet, a PC, and any image utilization application which uses an image photographed by the photographing apparatus such as a digital camera regardless of whether or not having the photographing function.

Description

  The present technology relates to an image processing device, an image processing method, and a program, and in particular, for example, an image processing device, an image processing method, and an image processing device that can easily guide a user to a desired shooting state. Regarding the program.

  In recent years, for example, a photographing function for photographing an image is often implemented in various devices such as a smartphone, a mobile phone, a tablet, and a PC (Personal Computer). Accordingly, images are frequently taken not only by digital cameras as dedicated devices for taking images but also by devices other than cameras equipped with a shooting function.

  Furthermore, as a shooting function is implemented in various devices, various applications that use images shot by the shooting function have been developed.

  As an image utilization application that is an application that uses an image photographed by the photographing function, for example, an image of a user's eye is photographed, and authentication of whether or not the user is a predetermined user is performed using the eye image. There is an iris authentication device that performs iris authentication (see, for example, Patent Document 1).

JP 2009-199392

  By the way, a captured image that is an image captured by a user using a device (hereinafter also referred to as a device with a photographing function) including a photographing function including a digital camera is, for example, a desired request requested by an image use application or the like. It may be out of the shooting state.

  However, the user may not be able to recognize that the captured image has deviated from the desired shooting state, and in this case, the user who cannot recognize that the captured image has deviated from the desired shooting state. It is difficult to adjust a device with a photographing function so that a photographed image is in a desired photographing state.

  The present technology has been made in view of such a situation, and makes it possible to easily guide a user to a desired shooting state.

  The image processing apparatus or the program according to an embodiment of the present technology is configured to increase the deviation from the desired shooting state when the acquired image acquired by the image acquisition unit that acquires the image is shifted from the desired shooting state. An image processing apparatus including an image generation unit that generates a shifted image, or a program for causing a computer to function as such an image processing apparatus.

  The image processing method according to an embodiment of the present technology generates a large shift image in which a shift from the desired shooting state is large when the acquired image acquired by the image acquisition unit that acquires the image is shifted from the desired shooting state. An image processing method including the step of:

  In the present technology, when an acquired image acquired by an image acquisition unit that acquires an image is deviated from a desired shooting state, a large shift image in which the shift from the desired shooting state is increased is generated.

  Note that the image processing apparatus may be an independent apparatus or an internal block constituting one apparatus.

  The program can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.

  According to the present technology, it is possible to easily guide a user to a desired shooting state.

  In addition, the effect described in this specification is an illustration to the last, and the effect of this technique is not limited to the effect described in this specification, and there may be an additional effect.

It is a block diagram which shows the structural example of one Embodiment of the image processing apparatus to which this technique is applied. It is a flowchart explaining the process of an image processing apparatus. It is a figure explaining the case where a user photographs the user himself. It is a figure explaining the 1st specific example of a deformed image. It is a figure explaining the 2nd specific example of a deformed image. It is a figure explaining the 3rd specific example of a deformed image. It is a figure explaining the 4th specific example of a deformed image. It is a figure explaining the 5th specific example of a deformed image. It is a figure explaining the 6th specific example of a deformed image. It is a figure explaining the 7th specific example of a deformed image. It is a figure explaining the example of the method of calculation of the deformation amount by the deformation amount calculation part 31. FIG. And FIG. 18 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied.

  <One embodiment of an image processing apparatus to which the present technology is applied>

  FIG. 1 is a block diagram illustrating a configuration example of an embodiment of an image processing apparatus to which the present technology is applied.

  The image processing apparatus in FIG. 1 can be applied to various devices such as a smartphone, a mobile phone, a tablet, and a PC. Here, it is assumed that the image processing apparatus in FIG. 1 is applied to, for example, a smartphone.

  In FIG. 1, an image processing apparatus as a smartphone includes an image acquisition unit 10, an image determination unit 20, a deformation processing unit 30, a display unit 40, an image processing unit 50, a storage unit 60, an operation unit 70, and a control unit 80. Have.

  The image acquisition unit 10 acquires an image in accordance with, for example, the operation of the operation unit 70 by the user, and sends the acquired image, which is the acquired image, to the image determination unit 20, the deformation processing unit 32, and the image processing unit 50. Supply.

  That is, the image acquisition unit 10 is, for example, an imaging unit that captures an image. The image acquisition unit 10, the deformation processing unit 32, and the image are captured as an acquired image by capturing the image. Supply to the processing unit 50.

  In addition, the image acquisition unit 10 functions as, for example, an interface with a network such as the Internet or an interface with an image providing apparatus that can provide an image such as a digital camera. An image or the like provided from the apparatus can be acquired by wired communication or wireless communication.

  In the following description, the image acquisition unit 10 is, for example, an imaging unit that captures an image, and the captured image captured by the imaging unit is transferred from the image acquisition unit 10 that is the imaging unit to the image determination unit 20. It is supplied to the deformation processing unit 32 and the image processing unit 50.

  The image determination unit 20 determines whether or not the captured image (acquired image) from the image acquisition unit 10 is deviated from a desired shooting state, and displays the displacement determination information indicating the determination result as the deformation processing unit 30 and The image is supplied to the image processing unit 50.

  That is, the image determination unit 20 includes an actual displacement amount calculation unit 21, a desired shooting state setting unit 22, and a determination unit 23.

  The actual displacement amount calculation unit 21 is supplied with a captured image from the image acquisition unit 10 and with desired shooting state information from the desired shooting state setting unit 22.

  The actual displacement amount calculation unit 21 calculates an actual displacement amount that is a shift amount indicating a shift between the state of the captured image from the image acquisition unit 10 and the desired shooting state represented by the desired shooting state information from the desired shooting state setting unit 22. Calculate and supply to the determination unit 23.

  For example, in response to a request from the image processing unit 50, the desired shooting state setting unit 22 sets a desired shooting state that is an optimal shooting state of the shot image for the image processing unit 50 (image processing thereof). Desired shooting state information representing the desired shooting state is supplied to the actual displacement amount calculation unit 21.

  The determination unit 23 determines whether the captured image from the image acquisition unit 10 is deviated from the desired photographing state based on the actual displacement amount from the actual displacement amount calculation unit 21, and deviation determination information representing the determination result. Are supplied to the deformation processing unit 30 and the image processing unit 50.

  The deviation determination information includes the actual displacement amount from the actual displacement amount calculation unit 21.

  The deformation processing unit 30 generates a deformed image as a large shift image with a large shift from the desired shooting state based on the shift determination information supplied from the determination unit 23 of the image determination unit 20 and supplies the deformed image to the display unit 40. To do.

  That is, the deformation processing unit 30 includes a deformation amount calculation unit 31 and a deformation image generation unit 32.

  The deformation amount calculation unit 31 is supplied with deviation determination information from the determination unit 23 of the image determination unit 20.

  For example, when the displacement determination information from the determination unit 23 indicates that the captured image is shifted from the desired shooting state, the deformation amount calculation unit 31 includes, for example, the actual displacement amount included in the shift determination information from the determination unit 23. Accordingly, a deformation amount of the deformed image as a shift amount (large shift amount) from the desired shooting state of the large shift image is calculated and supplied to the deformed image generation unit 32.

  When the deviation determination information indicates that the photographed image is not deviated from the desired photographing state, the deformation amount calculation unit 31 supplies, for example, 0 to the deformation image generation unit 32 as the deformation amount.

  The deformed image generation unit 32 uses a captured image or a predetermined graphics image supplied from the image acquisition unit 10 to deviate from the desired shooting state by a deformed amount supplied from the deformed amount calculation unit 31 (with a captured image and A large deviation image with a large deviation from the desired photographing state, that is, a deformed image in which a deviation from the desired photographing state is deformed, for example, is generated and supplied to the display unit 40.

  When the deformation amount supplied from the deformation amount calculation unit 31 is 0, the deformed image generation unit 32 displays the captured image or the predetermined graphics image supplied from the image acquisition unit 10 as it is. To supply.

  The display unit 40 includes a liquid crystal panel, an organic EL (Electro-Luminescence) panel, and the like, and is supplied with an image (a captured image, a predetermined graphics image, or a captured image) supplied from the deformed image generating unit 32 of the deformed processing unit 30. Or a deformed image obtained by deforming a predetermined graphics image).

  The image processing unit 50 is, for example, an image use application that uses a captured image from the image acquisition unit 10. The image processing unit 50 encodes the captured image from the image acquisition unit 10, stores (stores) it in the storage unit 60, and the like. Predetermined image processing using the photographed image is performed.

  That is, the image processing unit 50 requests the desired shooting state setting unit 22 for a shooting image in the shooting state that is optimal for the image processing unit 50 (image processing thereof).

  When the image processing unit 50 is supplied with deviation determination information indicating that the photographed image is not deviated from the desired photographing state (that the photographed image is in the desired photographing state) from the determining unit 23, Then, predetermined image processing is performed using the captured image supplied from the image acquisition unit 10.

  Here, as the image use application, for example, there is an authentication device that performs iris authentication and face authentication to unlock the image processing apparatus as a smartphone. In addition, as an image utilization application, for example, when photographing a face image mainly of a face such as an ID photo, macro shooting with an extremely close-up of the subject, or any other shooting, the shooting is assisted. There are photographing assistance devices.

  The storage unit 60 is a semiconductor memory, a hard disk, or the like, and stores (saves) a captured image supplied from the image processing unit 50 in accordance with control from the image processing unit 50.

  The operation unit 70 is a physical button, a virtual button displayed on a touch screen (configured integrally with the display unit 40), or the like, and is operated by a user. The operation unit 70 supplies an operation signal corresponding to a user operation to the control unit 80.

  The control unit 80 controls each block constituting the image processing apparatus as a smartphone according to an operation signal from the operation unit 70 or the like.

  FIG. 2 is a flowchart for explaining processing of the image processing apparatus as the smartphone of FIG.

  The desired shooting state setting unit 22 has already set a desired shooting state that is the optimum shooting state of the shot image for the image processing unit 50 (image processing thereof) in response to a request from the image processing unit 50 or the like. Then, it is assumed that the desired shooting state information representing the desired shooting state is supplied to the actual displacement amount calculation unit 21.

  In step S11, the image acquisition unit 10 acquires, for example, by capturing an image, and supplies the acquired image to the actual displacement calculation unit 21, the deformed image generation unit 32, and the image processing unit 50, and the processing is performed in step S12. Proceed to

  In step S <b> 12, the actual displacement amount calculation unit 21 is the actual displacement that is a deviation amount of the captured image (state) from the image acquisition unit 10 with respect to the desired shooting state represented by the desired shooting state information from the desired shooting state setting unit 22. The amount is calculated and supplied to the determination unit 23, and the process proceeds to step S13.

  In step S <b> 13, the determination unit 23 determines whether the captured image from the image acquisition unit 10 is deviated from the desired shooting state based on the actual displacement amount from the actual displacement amount calculation unit 21.

  When it is determined in step S <b> 13 that the photographed image has deviated from the desired photographing state, the determination unit 23 adds to the displacement determination information that the photographed image has deviated from the desired photographing state from the actual displacement amount calculating unit 21. The actual displacement amount is supplied to the deformation amount calculation unit 31, and the process proceeds to step S14.

  In step S <b> 14, the deformation amount calculation unit 31 follows the shift determination information from the determination unit 23 to the effect that the captured image has shifted from the desired shooting state, and changes the deformation image according to the actual displacement amount included in the shift determination information. The deformation amount is calculated and supplied to the deformed image generation unit 32, and the process proceeds to step S15.

  In step S15, the deformed image generation unit 32 uses the captured image supplied from the image acquisition unit 10, for example, to generate a deformed image in which the shift of the captured image is deformed by the deformed amount from the deformed amount calculating unit 31. , The process proceeds to step S16.

  In step S <b> 16, the deformed image from the deformed image generating unit 32 is displayed on the display unit 40.

  Then, the process returns from step S16 to step S11. Hereinafter, the processes of steps S11 to S16 are repeated until it is determined in step S13 that the photographed image is not deviated from the desired photographing state.

  Therefore, while the photographed image is deviated from the desired photographing state (determined as being determined), the deformed image is displayed on the display unit 40.

  On the other hand, when it is determined in step S13 that the photographed image is not deviated from the desired photographing state, that is, when the photographed image is in the desired photographing state, the determination unit 23 deviates from the desired photographing state. The deviation determination information indicating that there is no data is supplied to the deformation amount calculation unit 31 and the image processing unit 50, and the process proceeds to step S17.

  Here, when the deviation determination information indicates that the photographed image is not deviated from the desired photographing state, the deformation amount calculation unit 31 supplies 0 to the deformation image generation unit 32 as the deformation amount. Then, when the deformation amount supplied from the deformation amount calculation unit 31 is 0, for example, the deformed image generation unit 32 supplies the captured image supplied from the image acquisition unit 10 to the display unit 40 as it is.

  As a result, the display unit 40 displays the captured image itself acquired by the image acquisition unit 10 in step S17, that is, the captured image in the desired shooting state, and the process proceeds to step S18.

  As described above, when the photographed image is in the desired photographing state, the display unit 40 displays the photographed image in the desired photographing state instead of the deformed image.

  In step S18, the control unit 80 determines whether or not the operation unit 70 has been operated, that is, for example, whether or not a shutter button (release button) (not shown) included in the operation unit 70 has been operated.

  If it is determined in step S18 that the operation unit 70 has not been operated, the process returns to step S11.

  If it is determined in step S18 that the operation unit 70 has been operated, the process proceeds to step S19, and the image processing unit 50 enters a captured image supplied from the image acquisition unit 10, that is, a desired shooting state. Predetermined image processing such as storing the captured image in the storage unit 60 is performed, and the processing ends.

  In FIG. 2, when the operation unit 70 is operated after the photographed image is in the desired photographing state, the image processing unit 50 performs image processing. When the desired shooting state is reached, image processing can be performed without waiting for the operation of the operation unit 70.

  Alternatively, when the photographed image is in a desired photographing state while the operation unit 70 is being operated, the image processing unit 50 can perform image processing.

  <Specific examples of deformed images>

  Hereinafter, a specific example of a deformed image will be described by taking as an example the case where the image processing unit 50 as an image use application is, for example, an iris authentication device.

  FIG. 3 is a diagram illustrating a case where the user photographs the user himself / herself for iris authentication in the image processing apparatus as the smartphone in FIG. 1.

  In the image processing apparatus as the smartphone in FIG. 3, the photographing unit that captures an image as the image acquisition unit 10 is provided on the same surface as the display screen of the display unit 40.

  Now, as an operation mode of an image processing apparatus as a smartphone, a normal mode in which a captured image captured by the image acquisition unit 10 is displayed as a so-called through image and a deformed image are displayed as a through image on the display unit 40. Suppose there is a deformation mode. Note that switching of the operation mode to the normal mode or the deformation mode can be performed, for example, according to the operation of the operation unit 70 by the user.

  In the normal mode, the captured image captured by the image acquisition unit 10 is displayed on the display unit 40 as a through image. While viewing the captured image as the through image displayed on the display unit 40, the user can change the direction of the smartphone (the imaging direction) or the smartphone so that the user himself / herself as the subject reflected in the captured image is in an appropriate state. The position of the user, the user's own posture, and the like can be adjusted.

  Then, when the captured image is in a state that the user thinks appropriate, that is, for example, when the user reflected in the captured image is in a size that the user thinks appropriate and placed in a position that the user thinks appropriate, etc. The user operates a shutter button (not shown) included in the operation unit 70.

  When the user operates the shutter button, the image processing unit 50 performs predetermined image processing such as storing the captured image captured by the image acquisition unit 10 in the storage unit 60, for example.

  Here, FIG. 3 shows an example of display of a through image on the display unit 40 in the normal mode.

  In FIG. 3, the captured image captured by the image acquisition unit 10 is displayed on the display unit 40 as a through image as it is.

  In FIG. 3 (the same applies to the drawings described later), a visual image is shown. The visual image is an image of a user who is visually recognized when viewing a user facing away from a third party.

  That is, in FIG. 3 (the same applies to the drawings described later), a user who faces the display screen side of the display unit 40 of the image processing apparatus as a smartphone is illustrated, and a user who faces such a display screen side Is difficult to illustrate so that the state of the user's facial expression or the like can be confirmed.

  Therefore, in FIG. 3 (the same applies to the drawings to be described later), a third party is on the other side so that the facial expression of the user facing the other side (the display screen side of the display unit 40) can be confirmed. If it is, the visual image which is an image of the user who the third party can visually recognize is shown.

  In the normal mode, as described above, when the captured image is in a state considered appropriate by the user, image processing using the captured image, such as storing the captured image in the storage unit 60, is performed.

  Therefore, as image processing using a captured image, for example, when iris authentication is performed, a captured image in a capturing state appropriate for iris authentication is not always captured in the normal mode.

  That is, for a user unfamiliar with iris authentication, it is possible to recognize that the captured image deviates from the appropriate imaging state for iris authentication because the captured image appropriate for the iris authentication is unknown. First, it may be difficult to capture a captured image that is appropriate for iris authentication.

  Therefore, in the deformation mode, it is determined whether or not the captured image is deviated from a desired photographing state that is an appropriate photographing state for iris authentication. If the photographed image deviates from the desired photographing state, the photographed image is captured as desired. The deformed image that is deformed in the direction opposite to the adjustment direction of the image processing device as a smartphone, that is, the deformed image that emphasizes the deviation from the desired shooting state is displayed. It is displayed on the unit 40 as a through image.

  By viewing the deformed image as a through image, the user easily recognizes that the photographed image is deviated from the desired photographing state, and using the deformed image as a teacher on the other hand, the photographed image is brought into the desired photographing state. In addition, the orientation of the image processing apparatus as a smartphone can be adjusted.

  Therefore, according to the deformation mode, it is possible to easily guide the user to the desired shooting state.

  Note that the elements of a captured image that can be in a desired shooting state include, for example, the position, size (field angle, area), angle (orientation), focus, facial expression, etc. of the subject in the captured image.

  As the desired shooting state, for example, the position of the subject shown in the shot image is a position on a vertical line that bisects the shot image left and right, and the size (area) of the subject shown in the shot image is It occupies more than a predetermined percentage of the area of the shot image, the direction of the subject shown in the shot image is facing directly, the focus is on the subject and the subject is not blurred, The facial expression of the person as the subject in the photographed image is a facial expression with the eyelids open as usual, and the entire iris is reflected in the photographed image.

  In addition, as the desired shooting state, for example, it is possible to adopt that the facial expression of the person as the subject reflected in the shot image is a smiling expression, a closed expression, etc. it can.

  FIG. 4 is a diagram for explaining a first specific example of a deformed image.

  In FIG. 4, the position of the user as the subject shown in the captured image is shifted to the right from the position (predetermined position) on the center line that is a vertical line that bisects the captured image to the left and right. Yes.

  Now, assuming that the desired shooting state is a state where the position of the user shown in the shot image is a position on the center line, the actual displacement amount calculation unit 21 of the image determination unit 20 determines the position of the user shown in the shot image. The amount of displacement in the right direction from the position on the center line is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the photographed image is not in the desired photographing state based on the rightward displacement amount as the actual displacement amount, and calculates the rightward displacement amount. The deviation determination information that is included is supplied to the deformation processing unit 30.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the size of the displacement amount is increased according to the displacement amount in the right direction as the actual displacement amount included in the displacement determination information from the image determination unit 20. The amount of rightward misalignment is calculated as the deformation amount.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed image obtained by deforming the captured image supplied from the image acquiring unit 10 by a deformed amount is generated and displayed on the display unit 40.

  That is, in the deformation processing unit 30, the amount of rightward misalignment, which is the amount of rightward misalignment as the actual displacement, is displayed as a deformed amount in the captured image based on the amount of deformation. An image in which the position of the user is significantly shifted from the position on the center line in the right direction relative to the actual displacement amount is generated as a deformed image.

  Then, on the display unit 40, the above-described deformed image, that is, an image in which the position of the user shown in the captured image is significantly shifted from the position on the center line to the right than the actual displacement amount is displayed. The user recognizes that the position of the user shown in the captured image is shifted to the right from the position on the center line, and the captured image is not an appropriate image (for the image processing unit 50 as an image utilization application). be able to.

  In this case, while viewing the deformed image, the user corrects (or eliminates) a large amount of displacement in the right direction as the deformation amount deformed in the deformed image with respect to the position or orientation of the image processing apparatus as the smartphone or the user himself / herself. ), And finally, an appropriate captured image, that is, a captured image in which the position of the user reflected in the captured image is on the center line can be obtained.

  For example, in the actual displacement amount calculation unit 21 of the image determination unit 20, whether or not the captured image is in a desired imaging state in which the position of the user reflected in the captured image is a position on the center line. Recognizing a user's face as a subject in a captured image using face recognition technology as a target, and calculating the distance between the center of the face and the center line of the captured image as an actual displacement amount Thus, the determination unit 23 can determine based on the actual displacement amount.

  Note that the method for determining whether or not the captured image is in a desired capturing state in which the position of the user reflected in the captured image is a position on the center line is limited to the method using the face recognition technique as described above. Is not to be done.

  FIG. 5 is a diagram for explaining a second specific example of a deformed image.

  In FIG. 5, the user as the subject reflected in the captured image is slightly slightly caused by the distance between the user and the image processing apparatus (the image acquisition unit 10) as a smartphone being too close or too far. I am blurred.

  Assuming that the desired shooting state is a state in which the user shown in the shot image is in focus, the actual displacement amount calculation unit 21 of the image determination unit 20 determines that the user shown in the shot image is not in focus. The amount of blur to be expressed is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the captured image is not in the desired shooting state based on the blur amount as the actual displacement amount, and the displacement determination information including the blur amount is the deformation processing unit 30. To be supplied.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the amount of blur obtained by increasing the amount of blur according to the amount of blur as the actual displacement amount included in the deviation determination information from the image determination unit 20 is changed to the deformation amount. Calculated as a quantity.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed image obtained by deforming the captured image supplied from the image acquiring unit 10 by a deformed amount is generated and displayed on the display unit 40.

  That is, in the deformation processing unit 30, an image obtained by further blurring the user reflected in the photographed image based on the deformation amount, with the amount of blurring obtained by increasing the amount of blurring as the actual displacement amount as the deformation amount, Is generated as

  The display unit 40 displays the above-described deformed image, that is, an image in which the user reflected in the captured image is further blurred, so that the user is not focused on the user reflected in the captured image, and the captured image is captured. It can be recognized that the image is not an appropriate image (for the image processing unit 50 as an image utilization application).

  In this case, while viewing the deformed image, the user properly adjusts (displaces) the distance between the image processing apparatus as the smartphone and the user himself to correct the amount of blur as the deformed amount deformed in the deformed image. The distance can be adjusted to a proper distance, and finally, an appropriate captured image, that is, a captured image focused on the user reflected in the captured image can be obtained.

  The determination as to whether or not the photographed image is in a desired photographing state in which the user reflected in the photographed image is in focus can be performed as follows, for example.

  That is, the actual displacement amount calculation unit 21 of the image determination unit 20 calculates the contrast (brightness / darkness difference) of the attention area in time series with the area including the user shown in the captured image as the attention area for the latest predetermined period. The maximum contrast, which is the maximum value in the series contrast, is detected.

  Then, the actual displacement amount calculation unit 21 obtains a subtraction value obtained by subtracting the contrast of the latest photographed image (its attention area) from the maximum contrast as a blur amount, and uses the blur amount as the actual displacement amount to determine the determination unit 23. The determination unit 23 can determine based on the actual displacement amount whether or not the captured image is in a desired capturing state in which the user who is reflected in the captured image is in focus.

  In addition, the deformed image generating unit 32 of the deforming processing unit 30 filters the captured image (an area including the user shown in the image) with a low-pass filter that narrows the pass bandwidth according to the amount of blur as the amount of deformation. Thus, an image obtained by further blurring the user shown in the captured image can be generated as a deformed image.

  Note that a method for determining whether or not a photographed image is in a desired photographing state in which the user reflected in the photographed image is in focus, and a method for generating a deformed deformed image of the user reflected in the photographed image are as follows. The method is not limited to the above-described method.

  FIG. 6 is a diagram for explaining a third specific example of the deformed image.

  In FIG. 6, the user's facial expression as a subject shown in the captured image is a facial expression with the eyelid slightly closed.

  Now, assuming that the desired shooting state is a state in which the user's facial expression reflected in the captured image is a facial expression with the eyelids normally opened so that the entire iris can be seen, the actual displacement amount calculation of the image determination unit 20 is calculated. In the unit 21, a facial expression shift amount, which is a shift amount in which the facial expression of the user reflected in the photographed image is shifted from a facial expression in which the eyelids are normally opened (predetermined facial expression), is calculated as an actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the photographed image is not in the desired photographing state based on the expression displacement amount as the actual displacement amount, and the displacement determination information including the expression displacement amount is subjected to deformation processing. Supplied to the unit 30.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the facial expression deviation amount is increased in accordance with the facial expression deviation amount as the actual displacement amount included in the deviation determination information from the image determination unit 20. Is calculated as the deformation amount.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed image obtained by deforming the captured image supplied from the image acquiring unit 10 by a deformed amount is generated and displayed on the display unit 40.

  That is, in the deformation processing unit 30, the facial expression deviation amount obtained by increasing the facial expression deviation amount as the actual displacement amount is set as the deformation amount, and the facial expression of the user reflected in the photographed image based on the deformation amount is, for example, the eyelid An image having an expression that is almost closed is generated as a deformed image.

  Then, the deformed image as described above, that is, an image in which the facial expression of the user reflected in the captured image is a facial expression in which the eyelid is almost closed is displayed on the display unit 40. It can be recognized that the user's facial expression is not a facial expression in which the eyelids are normally opened, and the captured image is not an appropriate image (for the image processing unit 50 as an image utilization application).

  In this case, the user looks at the deformed image and adjusts the user's own facial expression by opening the eyelid, so that the appropriate photographed image, that is, the user's facial expression reflected in the photographed image becomes a facial expression that normally opens the eyelid. Therefore, it is possible to obtain a photographed image in which the entire iris can be seen (reflected).

  Whether the photographed image is in a desired photographing state in which the user's facial expression reflected in the photographed image is a facial expression with the eyelids normally opened so that the entire iris can be seen is, for example, the image determination unit 20. In the actual displacement amount calculation unit 21, the user's eyelid as a subject reflected in the photographed image is overlapped with the iris using the eyelid detection technique or the like for the photographed image (hereinafter also referred to as eyelid overlap condition). ) As the actual displacement amount, the determination unit 23 can determine based on the actual displacement amount.

  In addition, the deformed image generating unit 32 of the deforming processing unit 30 holds, for example, a series of captured images obtained by capturing a blinking user as a blinking image, and a blinking image of an eyelid overlap according to the amount of deformation. By replacing the eye part of the latest photographed image with the eye part, an image showing a user whose eyelids are more closed than the user shown in the photographed image can be generated as a deformed image.

  A method for determining whether a photographed image is in a desired photographing state in which a user's facial expression reflected in the photographed image is a facial expression with the eyelids normally opened so that the entire iris can be seen, and The method for generating a deformed image in which a user whose eyelids are more closed than the user shown in the captured image is not limited to the above-described method.

  FIG. 7 is a diagram illustrating a fourth specific example of a deformed image.

  In FIG. 7, the user (posture) as the subject shown in the captured image is tilted without facing.

  Assuming that the desired shooting state is a state where the user shown in the shot image is facing the user, the actual displacement amount calculation unit 21 of the image determination unit 20 tilts the user shown in the shot image with respect to the facing direction. The tilt angle is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the photographed image is not in the desired photographing state based on the inclination angle as the actual displacement amount, and the displacement determination information including the inclination angle is the deformation processing unit 30. To be supplied.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the inclination angle obtained by increasing the inclination angle according to the inclination angle as the actual displacement amount included in the deviation determination information from the image determination unit 20 is the deformation amount. Calculated as a quantity.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed image obtained by deforming the captured image supplied from the image acquiring unit 10 by a deformed amount is generated and displayed on the display unit 40.

  That is, in the deformation processing unit 30, the inclination angle obtained by increasing the inclination angle as the actual displacement amount is set as the deformation amount, and based on the deformation amount, the user shown in the photographed image is more directly directed to the facing direction. An inclined image is generated as a deformed image.

  The display unit 40 displays the deformed image as described above, that is, an image in which the user reflected in the captured image is more inclined, so that the user does not face the user reflected in the captured image, and the captured image is displayed. However, it can be recognized that the image is not appropriate (for the image processing unit 50 as an image utilization application).

  In this case, the user looks at the deformed image and adjusts the posture of the image processing apparatus as a smartphone by tilting in the yaw direction, the roll direction, or the pitch direction, for example, and an appropriate captured image, that is, the captured image. It is possible to obtain a captured image in which the user shown in the screen is facing (to the image processing apparatus (image acquisition unit 10 thereof) as a smartphone).

  Whether or not the photographed image is in a desired photographing state in which the user reflected in the photographed image is directly facing is determined by, for example, the face recognition technique for the photographed image in the actual displacement amount calculation unit 21 of the image determination unit 20. In the determination unit 23, the orientation of the face of the user as a subject reflected in the captured image is detected, and the inclination angle of the face orientation with respect to the facing direction is calculated as an actual displacement amount. The determination can be made based on the actual displacement amount.

  In addition, for example, when the image processing apparatus as a smartphone has a built-in gyro sensor or acceleration sensor, for example, the posture of the smartphone when the user reflected in the photographed image is facing (hereinafter referred to as the facing posture). ) Is acquired from the gyro sensor or acceleration sensor as one of the calibration operations of the smartphone, and the user performs the operation in advance, so that the current orientation of the smartphone is acquired. By calculating the difference between the posture and the facing posture as the actual displacement amount, based on the actual displacement amount, whether or not the captured image is in the desired shooting state that the user reflected in the captured image is facing Can be determined.

  Note that the method for determining whether or not the captured image is in the desired capturing state in which the user shown in the captured image is facing is not limited to the method described above.

  FIG. 8 is a diagram for explaining a fifth specific example of a deformed image.

  In FIG. 8, as in the case of FIG. 6, the user's facial expression as a subject shown in the captured image is a facial expression with the eyelid slightly closed.

  Now, assuming that the desired shooting state is a state in which the user's facial expression reflected in the captured image is a facial expression with the eyelids normally opened so that the entire iris can be seen, the actual displacement amount calculation of the image determination unit 20 is calculated. In the unit 21, a facial expression deviation amount, which is a deviation amount in which the facial expression of the user reflected in the photographed image deviates from the facial expression in which the eyelids are normally opened, is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the photographed image is not in the desired photographing state based on the expression displacement amount as the actual displacement amount, and the displacement determination information including the expression displacement amount is subjected to deformation processing. Supplied to the unit 30.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the facial expression deviation amount is increased in accordance with the facial expression deviation amount as the actual displacement amount included in the deviation determination information from the image determination unit 20. Is calculated as the deformation amount.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed deformed image is generated according to the deformed amount and displayed on the display unit 40.

  In FIG. 6, the deformation processing unit 30 generates a deformed image using the captured image. However, the deformed image is not a captured image, but a character such as a 3D (Dimension) avatar or a GUI (Graphical User Interface). It can be generated using other graphics images.

  In FIG. 8, a deformed image is generated using a bear character (graphics image thereof).

  That is, in the deformation processing unit 30, a facial expression expression of the bear character is reflected in the captured image based on the deformation amount, with the expression displacement amount obtained by increasing the expression displacement amount as the actual displacement amount as the deformation amount. An image having a facial expression with the eyelid closed is generated as a deformed image.

  Then, on the display unit 40, the deformed image as described above, that is, an image in which the facial expression of the bear character is a facial expression with a closed eyelid rather than the user reflected in the captured image, the user can Recognizing that the facial expression of the user in the captured image is not a facial expression with the eyelids normally opened, and that the captured image is not an appropriate image (for the image processing unit 50 as an image utilization application). Can do.

  In this case, the user looks at the deformed image and adjusts the user's own facial expression by opening the eyelid, so that the appropriate photographed image, that is, the user's facial expression reflected in the photographed image becomes a facial expression that normally opens the eyelid. Therefore, it is possible to obtain a photographed image in which the entire iris can be seen (reflected).

  Note that the deformed image generated using a graphics image reminiscent of a person (user) such as the bear character in FIG. 8 is not limited to the case in FIG. 6, but in the case of FIG. 4, FIG. 5, FIG. It can also be applied to.

  Further, as the graphics image used for generating the deformed image, an image other than the graphics image reminiscent of a person such as the bear character in FIG. 8 can be adopted.

  That is, FIG. 9 is a diagram for explaining a sixth specific example of a deformed image.

  In FIG. 9, as in the case of FIG. 5, the user as the subject shown in the captured image is slightly blurred.

  Assuming that the desired shooting state is a state in which the user shown in the shot image is in focus, the actual displacement amount calculation unit 21 of the image determination unit 20 determines that the user shown in the shot image is not in focus. The amount of blur to be expressed is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the captured image is not in the desired shooting state based on the blur amount as the actual displacement amount, and the displacement determination information including the blur amount is the deformation processing unit 30. To be supplied.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the amount of blur obtained by increasing the amount of blur according to the amount of blur as the actual displacement amount included in the deviation determination information from the image determination unit 20 is changed to the deformation amount. Calculated as a quantity.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed deformed image is generated according to the deformed amount and displayed on the display unit 40.

  Here, in FIG. 5, the deformation processing unit 30 generates a deformed image using a captured image. However, as described above, the deformed image can be generated using a graphics image instead of a captured image. .

  In FIG. 9, for example, when the image processing unit 50 as an image use application that uses a captured image is an authentication device that performs iris authentication or face authentication, this is displayed when the authentication device is performing authentication. A deformed image is generated using a GUI (hereinafter, also referred to as “authenticating GUI”) on which characters “authenticating” are drawn.

  That is, the deformation processing unit 30 uses the blur amount obtained by increasing the amount of blur as the actual displacement amount as the deform amount, and based on the deform amount, the authenticating GUI is more blurred than the user reflected in the captured image. An image is generated as a deformed image.

  The display unit 40 displays the deformed image as described above, that is, the authenticating GUI that is more blurred than the user reflected in the captured image, so that the user is not focused on the user reflected in the captured image. It can be recognized that the captured image is not an appropriate image (for the image processing unit 50 as an image utilization application).

  In this case, while viewing the deformed image, the user sets the distance between the image processing apparatus as the smartphone and the user himself to an appropriate distance so as to correct the blur amount as the deformed amount deformed in the deformed image. After adjustment, an appropriate captured image, that is, a captured image focused on the user reflected in the captured image can be obtained.

  FIG. 10 is a diagram for explaining a seventh specific example of a deformed image.

  In FIG. 10, as in the case of FIG. 7, the user (or posture) as a subject reflected in the captured image is tilted without facing.

  Assuming that the desired shooting state is a state where the user shown in the shot image is facing the user, the actual displacement amount calculation unit 21 of the image determination unit 20 tilts the user shown in the shot image with respect to the facing direction. The tilt angle is calculated as the actual displacement amount.

  Then, the determination unit 23 of the image determination unit 20 determines that the photographed image is not in the desired photographing state based on the inclination angle as the actual displacement amount, and the displacement determination information including the inclination angle is the deformation processing unit 30. To be supplied.

  In the deformation amount calculation unit 31 of the deformation processing unit 30, the inclination angle obtained by increasing the inclination angle according to the inclination angle as the actual displacement amount included in the deviation determination information from the image determination unit 20 is the deformation amount. Calculated as a quantity.

  Further, in the deformed image generating unit 32 of the deforming processing unit 30, a deformed deformed image is generated according to the deformed amount and displayed on the display unit 40.

  Here, in FIG. 7, the deformation processing unit 30 generates a deformed image using the captured image. However, in FIG. 10, for example, as in the case of FIG. 9, authentication as a graphics image instead of a captured image is performed. A deformed image is generated using the middle GUI.

  That is, in the deformation processing unit 30, an inclination angle obtained by increasing the inclination angle as the actual displacement amount is used as a deformation amount, and an image in which the authenticating GUI is inclined is generated as a deformation image based on the deformation amount. The

  The display unit 40 displays the deformed image as described above, that is, the authenticating GUI tilted with respect to the user reflected in the captured image, so that the user does not face the user reflected in the captured image. It can be recognized that the captured image is not an appropriate image (for the image processing unit 50 as an image utilization application).

  In this case, the user looks at the deformed image, adjusts the attitude of the image processing apparatus as a smartphone by tilting it in the yaw direction, the roll direction, the pitch direction, etc., and appears in an appropriate captured image, that is, the captured image. It is possible to obtain a captured image that the user is facing (to the image acquisition device 10 of the image processing apparatus as a smartphone).

  Note that a deformed image generated using a graphics image that does not recall a person such as the authenticating GUI in FIGS. 9 and 10 is difficult to obtain a deformed image of a human (user) expression. The present invention can be applied to the cases of FIGS. 4, 5, and 7 that deform other than the above expression.

  <Calculation of deformation amount>

  FIG. 11 is a diagram for explaining an example of a method of calculating the deformation amount by the deformation amount calculation unit 31 of FIG.

  The deformation amount calculation unit 31 can calculate the deformation amount with respect to the actual displacement amount included in the deviation determination information from the determination unit 23, for example, according to a predetermined interpolation function.

  FIG. 11A shows an example of a first interpolation function for calculating the deformation amount.

  The first interpolation function in FIG. 11A is a linear interpolation function that obtains a linear output value with respect to an input value. According to the first interpolation function, the actual displacement amount is multiplied by a predetermined number (> 1). The deformation amount of the value is calculated.

  FIG. 11B shows an example of a second interpolation function for calculating the deformation amount.

  The second interpolation function in FIG. 11B is a spline interpolation function that obtains a non-linear output value with respect to the input value. According to the second interpolation function, the actual displacement amount with a small value is a linear function. A deformation amount having a value larger than that in the case of a certain first interpolation function is calculated, and a deformation amount having a value that converges to a constant value is calculated for a large displacement value.

  Therefore, according to the deformation amount calculated by the second interpolation function, for example, for a captured image in which the user's position is shifted from the position on the center line (to the right) as shown in FIG. When the positional deviation amount from the position on the center line of the user position is small, a deformed image in which the positional deviation amount is greatly deformed (emphasized), that is, a deformed image in which the user position is significantly displaced is generated.

  On the other hand, when the amount of positional deviation is large, a deformed image in which the amount of positional deviation is somewhat deformed (not much deformed), that is, for example, a deformed image in which the position of the user is substantially deviated from the amount of positional deviation. An image is generated.

  According to the deformation amount calculated by the second interpolation function as described above, when the displacement amount is large, it is possible to prevent the user from protruding from the image frame in the deformed image, or the user can move out of the image frame. The extent of protrusion can be reduced.

  FIG. 11C shows an example of a third interpolation function for calculating the deformation amount.

  The third interpolation function in FIG. 11C is a constant interpolation function that can obtain a non-linear output value with respect to the input value. According to the third interpolation function, the third interpolation function changes stepwise with respect to the actual displacement amount. A deformation amount is calculated.

  The third interpolation function is effective when the actual displacement amount does not change even if the actual displacement amount changes to some extent. For example, it is effective when the change in the actual displacement amount need not be sequentially reflected as the deformation image. .

  For example, as shown in FIG. 8, a facial expression deviation amount in which a user's facial expression reflected in a photographed image deviates from a facial expression in which the eyelids are normally opened is calculated as an actual displacement amount, and a deformation amount obtained from the actual displacement amount is calculated. On the basis of this, when an image in which the facial expression of the bear character is a facial expression in which the eyelid is closed more than the user shown in the captured image is generated as a deformed image, the third interpolation function is used to calculate the deformed amount. be able to.

  That is, for example, as a (graphics) image of a bear character, there are three types of images: an image with the eyelids fully open, an image with the eyelids half closed (open), and an image with the eyelids fully closed An image is prepared, and an image to be generated as a deformed image can be selected from the above-described three types of images according to the deformed amount that changes stepwise with respect to the actual displacement amount.

  In this case, it is possible to reduce the memory area for holding the character image and the processing cost for drawing the character image on the display unit 40.

  FIG. 11D shows an example of a fourth interpolation function for calculating the deformation amount.

  The fourth interpolation function in FIG. 11D is a linear interpolation function similar to the first interpolation function in FIG. 11A, but the actual displacement amount is a function that is calculated as a deformed amount as it is.

  The fourth interpolation function in FIG. 11D is used when the displacement amount is used as a deformation amount as it is, and an image that has not been deformed is displayed on the display unit 40 as a through image (when it is not desired to perform any deformation). be able to.

  In the deformed mode, when the deformed amount is calculated by the fourth interpolation function and the deformed image is generated using the captured image, the display unit 40 acquires the image as in the normal mode. The captured image captured by the unit 10 is displayed as a through image as it is.

  In calculating the deformation amount using the actual displacement amount, it is possible to use a function other than the interpolation function of FIG.

  As described above, in the image processing apparatus of FIG. 1 as a smartphone, when a captured image (acquired image) acquired by capturing an image is deviated from the desired shooting state, the desired shooting state Since a deformed image in which the deviation from is deformed is generated, the user can be easily guided in a desired shooting state.

  That is, the user naturally sees the deformed image without the user knowing the desired shooting state (for example, the user shown in the shot image is directly facing), and the natural deviation of the shot image from the desired shooting state. Thus, an operation (behavior) is performed to correct the image, and a captured image in a desired shooting state can be easily obtained.

  Therefore, when the image processing unit 50 as an image use application that uses a photographed image is, for example, an iris authentication device that performs iris authentication, a photographed image suitable for iris authentication can be easily obtained.

  Here, in the iris authentication device, for example, as described in, for example, Japanese Patent No. 3307936, an eye image that is an image of the user's eye is photographed with a camera, and an iris in the eye image is captured. After extracting the part and extracting the iris characteristics (iris data), the user is identified (authenticated) if the user matches and matches the iris data registered in the iris authentication device in advance. The

  In such an iris authentication device, in order to maintain the authentication accuracy, it is necessary to capture a high-definition and appropriate eye image of the eye, so that the user's eye image is displayed at a high resolution on the display screen, The eye image necessary for iris authentication is taken by aligning the eye position with a rectangular area in the display screen.

  However, it is difficult for a user who is unfamiliar with the iris authentication operation to align the position of the eye with the rectangular area in the high-resolution display screen, and it may take a long time to align the eye.

  In addition, even if the eyes can be aligned, some or all of the iris of the eye is covered by out-of-focus, blurred eye images, or closed eyelids. In other words, it may be difficult to obtain an eye image suitable for iris authentication of the iris authentication device due to the fact that the image is taken obliquely without facing the camera.

  Further, in Patent Document 1 described above, in the iris authentication apparatus, first, with respect to a low-resolution eye image, it is determined whether or not the eye shown in the eye image is in a rectangular area in the display screen, and is shown in the eye image. If the eye is in a rectangular area, the eye image is switched from a low-resolution eye image to a high-resolution eye image and displayed on the display screen, and the user is positioned on the rectangular area in the display screen. It is described to repeat matching.

  However, in such an iris authentication device, every time the resolution of the eye image is switched, the user must align the eye position with the rectangular area in the display screen, and even when the eye position can be aligned. Of course, as described above, the eye image may not be in focus.

  On the other hand, as described above, the image processing apparatus of FIG. 1 as a smartphone can easily guide the user to the desired shooting state, so that a shot image suitable for iris authentication can be easily obtained. .

  Further, in the image processing apparatus of FIG. 1 as a smartphone, a deformed image can be generated using a graphics image such as a predetermined character or a GUI instead of a captured image showing a user. For example, when the image processing unit 50 is an iris authentication device, it is possible to prevent the user from feeling disgusted due to the sneak shot of biometric information or a large display of an eye image. Can do.

  That is, for example, when a photographed image showing a user is displayed instead of a graphics image, another person can take a voyeur of biological information such as the user's iris (image) reflected in the photographed image, The user may feel disgusted with the image of the eye that appears largely in the image.

  When generating a deformed image using a graphic image such as a predetermined character or GUI instead of a photographed image shown by the user, it prevents the above-described voyeurism of biometric information and the user's aversion can do.

  Here, examples of the image utilization application that uses the photographed image include an authentication device that performs iris authentication and the like, a photographing assistance device that assists macro photographing, and the like, as described above.

  Regarding the authentication device, the user's position shown in the captured image is on the center line, that the user reflected in the captured image is in focus, and the user's facial expression reflected in the captured image shows the entire iris. As described above, it is possible to make all of the expression that the eyelid is open and that the user reflected in the photographed image is directly facing the desired photographing state.

  Also, in the case of assisting macro shooting in the shooting assisting device, the (large) subject appearing in the shot image is in focus, and the angle of view of the subject appearing in the shot image is within a predetermined range (predetermined) In the range from the minimum value to the predetermined maximum value) can be set as a desired photographing state.

  Note that the present technology is not limited to a digital camera, a smartphone, a mobile phone, a tablet, a PC, or the like, which has a shooting function, such as a digital camera. The present invention can be applied to any image utilization application that uses an image photographed by a photographing apparatus such as the above.

  That is, for example, the present technology shoots a user in a game in which a user is photographed, and a user reflected in an image obtained by the photographing is displayed on a TV (television receiver) or the like and progresses according to the user's action. By displaying a deformed image obtained by deforming a user's appropriate size and an appropriate position in the image, the user can be guided so that the user appears in an appropriate position with an appropriate size for the game.

  In addition, the present technology displays, for example, a deformed image in which a deviation from an appropriate image to be photographed by a camera is displayed in a so-called wearable glass having a small camera, so that the user can wear the wearable glasses. It can be applied to an application that guides the user to properly wear (apply).

  Furthermore, in the present technology, for example, in photographing with a digital camera, when the photographed image is inclined due to the digital camera being inclined in the yaw direction, the roll direction, or the pitch direction, the inclination is reduced. By displaying the deformed deformed image, it can be applied to an application that notifies the user that the digital camera is tilted and prompts the user to correct the tilt of the digital camera.

  Here, for example, a smartphone may be provided with a camera on both the display surface that performs main display and the surface opposite to the display surface, but the present technology is provided on the display surface. The present invention can be applied to both an image photographed by a provided camera (inward camera) and an image photographed by a camera (outward camera) provided on a surface opposite to the display surface.

  The present technology can be applied to both still images and moving images.

  <Description of computer to which this technology is applied>

  Next, the series of processes described above can be performed by hardware or software. When a series of processing is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

  Therefore, FIG. 12 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

  The program can be recorded in advance on a hard disk 105 or a ROM 103 as a recording medium built in the computer.

  Alternatively, the program can be stored (recorded) in the removable recording medium 111. Such a removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

  In addition to installing the program from the removable recording medium 111 as described above, the program can be downloaded to the computer via a communication network or a broadcast network, and can be installed in the built-in hard disk 105. That is, for example, the program is wirelessly transferred from a download site to a computer via a digital satellite broadcasting artificial satellite, or wired to a computer via a network such as a LAN (Local Area Network) or the Internet. be able to.

  The computer incorporates a CPU (Central Processing Unit) 102, and an input / output interface 110 is connected to the CPU 102 via a bus 101.

  The CPU 102 executes a program stored in a ROM (Read Only Memory) 103 according to a command input by the user by operating the input unit 107 or the like via the input / output interface 110. . Alternatively, the CPU 102 loads a program stored in the hard disk 105 to a RAM (Random Access Memory) 104 and executes it.

  Thus, the CPU 102 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result as necessary, for example, via the input / output interface 110, from the output unit 106, transmitted from the communication unit 108, and further recorded in the hard disk 105.

  The input unit 107 includes a keyboard, a mouse, a microphone, and the like. The output unit 106 includes an LCD (Liquid Crystal Display), a speaker, and the like.

  Here, in the present specification, the processing performed by the computer according to the program does not necessarily have to be performed in time series in the order described as the flowchart. That is, the processing performed by the computer according to the program includes processing executed in parallel or individually (for example, parallel processing or object processing).

  The program may be processed by one computer (processor), or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  Furthermore, in this specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .

  The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

  For example, the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and is jointly processed.

  In addition, each step described in the above flowchart can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Furthermore, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  In addition, this technique can take the following structures.

<1>
When the acquired image acquired by the image acquiring unit that acquires an image is deviated from a desired shooting state, the image processing unit includes an image generation unit that generates a large shift image with a large shift from the desired shooting state. apparatus.
<2>
The image processing device according to <1>, wherein the image generation unit generates the large shift image in which the position shift of the subject is increased when the position of the subject shown in the acquired image is shifted from a predetermined position.
<3>
The image processing device according to <1>, wherein the image generation unit generates the large deviation image in which the blur of the subject is increased when the subject reflected in the acquired image is blurred.
<4>
The image processing device according to <1>, wherein the image generation unit generates the large displacement image in which the subject's facial expression shift is increased when the facial expression of the subject reflected in the acquired image deviates from a predetermined facial expression.
<5>
The image processing device according to <1>, wherein the image generation unit generates the large deviation image in which the inclination of the subject is increased when the subject shown in the acquired image is tilted without facing.
<6>
The image processing device according to any one of <1> to <5>, wherein the image generation unit generates the large shift image using the acquired image.
<7>
The image processing device according to any one of <1> to <5>, wherein the image generation unit generates the large shift image using a predetermined graphics image.
<8>
An actual displacement amount calculation unit that calculates an actual displacement amount that is a deviation amount between the state of the acquired image acquired by the image acquisition unit and a desired shooting state;
A large deviation amount calculating unit that calculates a large deviation amount that is a deviation amount of the large deviation image according to the actual displacement amount;
The image generation unit generates a large shift image with a large shift from the desired shooting state according to the large shift amount,
The image processing device according to any one of <1> to <7>, wherein the large deviation amount changes linearly or nonlinearly with respect to the actual displacement amount.
<9>
The image acquisition unit;
The image processing apparatus according to any one of <1> to <8>, further comprising: a display unit that displays the large deviation image.
<10>
An image processing method including a step of generating a large shift image in which a shift from a desired shooting state is increased when an acquired image acquired by an image acquisition unit that acquires an image is shifted from a desired shooting state.
<11>
When the acquired image acquired by the image acquiring unit that acquires the image is deviated from the desired shooting state, the computer is used as an image generating unit that generates a large shift image with a large shift from the desired shooting state. A program to make it work.

  DESCRIPTION OF SYMBOLS 10 Image acquisition part, 20 Image determination part, 21 Actual displacement amount calculation part, 22 Desired imaging | photography state setting part, 23 Determination part, 30 Deformation process part, 31 Deformation amount calculation part, 32 Deformation image generation part, 40 Display part, 50 Image processing unit, 60 storage unit, 70 operation unit, 80 control unit, 101 bus, 102 CPU, 103 ROM, 104 RAM, 105 hard disk, 106 output unit, 107 input unit, 108 communication unit, 109 drive, 110 I / O interface 111 Removable recording media

Claims (11)

When the acquired image acquired by the image acquiring unit that acquires an image is deviated from a desired shooting state, the image processing unit includes an image generation unit that generates a large shift image with a large shift from the desired shooting state. apparatus. The image processing apparatus according to claim 1, wherein the image generation unit generates the large shift image in which the position shift of the subject is increased when the position of the subject shown in the acquired image is shifted from a predetermined position. The image processing apparatus according to claim 1, wherein the image generation unit generates the large shift image in which the blur of the subject is increased when the subject reflected in the acquired image is blurred. The image processing apparatus according to claim 1, wherein the image generation unit generates the large deviation image in which the expression deviation of the subject is increased when the expression of the subject reflected in the acquired image is deviated from a predetermined expression. The image processing apparatus according to claim 1, wherein the image generation unit generates the large deviation image in which the inclination of the subject is increased when the subject reflected in the acquired image is tilted without facing. The image processing apparatus according to claim 1, wherein the image generation unit generates the large displacement image using the acquired image. The image processing apparatus according to claim 1, wherein the image generation unit generates the large shift image using a predetermined graphics image. An actual displacement amount calculation unit that calculates an actual displacement amount that is a deviation amount between the state of the acquired image acquired by the image acquisition unit and a desired shooting state;
A large deviation amount calculating unit that calculates a large deviation amount that is a deviation amount of the large deviation image according to the actual displacement amount;
The image generation unit generates a large shift image with a large shift from the desired shooting state according to the large shift amount,
The image processing apparatus according to claim 1, wherein the large deviation amount changes linearly or nonlinearly with respect to the actual displacement amount. The image acquisition unit;
The image processing apparatus according to claim 1, further comprising: a display unit that displays the large deviation image. An image processing method including a step of generating a large shift image in which a shift from a desired shooting state is increased when an acquired image acquired by an image acquisition unit that acquires an image is shifted from a desired shooting state. When the acquired image acquired by the image acquiring unit that acquires the image is deviated from the desired shooting state, the computer is used as an image generating unit that generates a large shift image with a large shift from the desired shooting state. A program to make it work.

Images