JP2011022796A - Image processing method and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method and an image processor, allowing a user to easily decide content of an image when a generation image is displayed for the first time, in image processing of generating the image having an arbitrary focus distance, an aperture radius, and a viewpoint by use of a plurality of images captured from a plurality of viewpoints. <P>SOLUTION: The image processor extracts tag information including an acquisition condition of the image from a plurality of images of a photographic object captured from at least two different viewpoints, extracts an object from one of the plurality of images, calculates moving amount of the plurality of images based on the extracted tag information when the tag information is extracted, calculates moving amount of the plurality of images such that positions of the object overlap when the object is extracted, moves the plurality of images based on the calculated moving amount, and generates the output image from the plurality of images. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing method and an image processing apparatus that generate an image using a plurality of images having different viewpoint positions.

  Conventionally, there has been a method of generating an image with an arbitrary focus distance, depth of field, and viewpoint based on images obtained by photographing a subject from many viewpoints. For example, in Non-Patent Document 1, a plurality of images with different viewpoints are acquired by arranging a plurality of cameras in a grid on a two-dimensional plane. Then, a calculation process is performed based on the plurality of images, thereby generating an image focused on an arbitrary distance. Moreover, in patent document 1, it image | photographs using the plenoptic camera (Plenoptic Camera) which has arrange | positioned the micro lens array which arranged the micro lens in the grid | lattice form on the sensor. FIG. 14 is a diagram illustrating a structure example of a plenoptic camera. A plenoptic camera divides light incident on the camera by a microlens, thereby enabling images from different viewpoints to be acquired by one shooting. Therefore, an image with an arbitrary viewpoint, aperture diameter, and focus distance can be generated by performing calculation processing based on an image photographed by the plenoptic camera.

US 2007/0252074 A1

Bennet Wilburn, Neel Joshi, Vaibhav Vaish, Eino-Ville (Eddy) Talvala, Emilio Antunez, Adam Barth, Andrew Adams, Marc Levoy, Mark Horwitz, "High Perfomance Imaging Using Large Camera Arrays", USA, Proceedings of ACM SIGGRAPH 2005, 765-776.

  However, in the above-described Non-Patent Document 1 and Patent Document 1, when displaying an image obtained by calculation processing, it is necessary to give a viewpoint, a focus distance, and an aperture diameter as parameters. Therefore, an alternative process such as using a default value set in advance when the user does not give these parameters is necessary.

  For example, a case where a user displays a generated image focused on a specific focus distance based on an image captured by a camera array will be described. FIG. 15 (a) shows the relationship between the camera array and the object to be imaged. Assume that the subjects A, B, and C are arranged at distances D1, D2, and D3 from the camera array, respectively. Further, a default value of a parameter (here, a focal length) for performing the refocus processing is set to D0.

  First, a case where an image is displayed using default values will be described. FIG. 15B shows an image generated using default values based on an image photographed by the camera array. As shown in FIG. 15B, the subject is blurred when the default value is used. For this reason, there is a problem that it is difficult for the user to determine the content from the refocus image displayed first.

  Next, a case where the user gives a parameter (focal length) will be described. FIG. 15C shows a generated image obtained from an image captured by the camera array when the user gives a focal length. Here, a generated image focused on each subject is shown. In order to display these generated images, it is necessary to give a distance corresponding to the subject as a parameter. Here, in order to obtain images refocused on the subjects A, B, and C, distance values D1, D2, and D3 must be given, respectively. However, these distance values are not self-evident. Therefore, it is difficult to give these distance values when the image is first displayed. If distance values other than D1, D2, and D3 are given, the subject will be blurred in the same manner as when the default value is given. Therefore, the user determines the contents from the refocus image displayed first. It is hard to do.

  The present invention relates to image processing for generating an image of an arbitrary focus distance, aperture diameter, and viewpoint using a plurality of images photographed from a plurality of viewpoints. When a generated image is displayed for the first time, the user can easily display the contents of the image Provided are an image processing method and an image processing apparatus that enable determination.

  In order to achieve the above object, an image processing apparatus of the present invention is an image processing apparatus that processes a plurality of images of a subject captured from at least two different viewpoints, and an object is obtained from any one of the plurality of images. Extracting means for extracting; movement amount calculating means for calculating a movement amount of the plurality of images so that positions of the extracted objects overlap; and moving the plurality of images based on the calculated movement amount; And an image generation means for generating an output image from the plurality of images.

  The present invention relates to image processing for generating an image of an arbitrary focus distance, aperture diameter, and viewpoint using a plurality of images photographed from a plurality of viewpoints. It became easy to judge.

It is a block diagram which shows the structural example of the image processing apparatus of this embodiment. 6 is a diagram illustrating an example of imaging device information input from an input terminal 102. FIG. 1 is a diagram illustrating an example of an imaging device 106. FIG. It is a figure which shows the horizontal view angle and vertical view angle of a camera. 5 is a diagram illustrating an example of imaging data input from an input terminal 101 and an image included in the imaging data. FIG. It is a figure which shows the relationship between an image generation parameter and an imaging device. 6 is a flowchart illustrating an example of a processing procedure of an image generation parameter extraction unit 103. It is a flowchart which shows an example of the reconstruction parameter output procedure S805 of FIG. It is a figure which shows an example of the face area extraction process procedure S1001 of FIG. 8, and the distance value calculation procedure S1003 of a face area. It is a figure which shows the relationship between the space | interval of a camera, and the distance to a to-be-photographed object. 5 is a flowchart illustrating an example of a processing procedure of an image generation unit 104. It is a figure which shows an example of a face area extraction result and a generated image when only one face area is extracted and when a plurality of face areas are extracted. It is a figure which shows the example of a display part and a user interface part. It is the figure which showed the structure of a plenoptic camera, and the image image | photographed. It is a figure which shows the image produced | generated according to the relationship of a camera array and a to-be-photographed object, the image produced | generated with the default parameter, and the distance of a to-be-photographed object.

  In this embodiment, an arbitrary focus distance, aperture diameter, and viewpoint image are generated from a plurality of images obtained using a camera array in which a plurality of cameras are arranged in a grid on a two-dimensional plane and viewpoint information of each image. An image processing apparatus will be described. Here, it is assumed that the horizontal direction, the vertical direction, the resolution, and the viewing angle of each camera are all equal and the optical axes are all parallel. In addition, it is assumed that each camera can obtain an in-focus image at all pixels.

  <Configuration Example of Image Processing Apparatus According to this Embodiment> In FIG. 1, reference numeral 100 denotes an image processing apparatus. Reference numeral 101 denotes an input terminal for inputting later-described imaging data including a plurality of images with different viewpoints. Reference numeral 102 denotes an input terminal for inputting later-described imaging device information including viewpoint information of the camera used for shooting. 103 is an image generation parameter for extracting an image generation parameter, which will be described later, necessary for image generation processing based on the imaging data input from the input terminal 101 and the imaging device information input from the input terminal 102 It is an extraction unit. 104, based on the imaging data input from the input terminal 101, the imaging device information input from the input terminal 102, and the image generation parameters input from the image generation parameter extraction unit 103, It is an image generation part which performs an image generation process. Reference numeral 105 denotes an output terminal that outputs a generated image generated by the image generation unit 104. Reference numeral 106 denotes an imaging device used for shooting, which outputs imaging data. Reference numeral 107 denotes a memory unit in which the imaging data and imaging device information are recorded. Reference numeral 108 denotes a display unit that displays the generated image output from the image generation unit 105. Reference numeral 109 denotes a user interface unit that gives an operation instruction for the user to adjust image generation parameters. Note that the display unit 108 and the user interface unit 109 may be a common touch panel display unit.

  In the image processing apparatus 100 of the present embodiment, the image generation parameter extraction unit 103 extracts and generates image generation parameters based on the imaging data and imaging apparatus information input from the memory unit 106. Here, the image generation parameters include a focus distance, an aperture diameter, and a viewpoint position. Further, the image generation unit 104 generates an image based on the imaging data, imaging device information, and image generation parameters, and outputs the generated image to the display unit 108. Based on the image displayed on the display unit 108, the user can output image generation parameters from the user interface unit 109 to the image generation unit 104 to adjust the image generation parameters used in the image generation unit 104.

  In FIG. 1, the image generation parameter extraction unit 103 and the image generation unit 104 are the image processing device 100, but the image processing device may include all or part of the other components in FIG. In addition, among the constituent elements, the image generation parameter extraction unit 103 and the image generation unit 104 may be realized by a computer operating with software, and the image processing apparatus 100 may be realized by a general-purpose personal computer (PC). In that case, each function is achieved when the program stored in the storage medium is loaded into the RAM and executed by the CPU.

  (Configuration Example of Imaging Device Information) FIG. 2 is an example of imaging device information input from the input terminal 102 of FIG. Reference numeral 200 denotes imaging apparatus information of the present embodiment. The imaging device information 200 includes camera viewing angle information 201 including a horizontal viewing angle and a vertical viewing angle, and position information 202 of each camera.

  (Configuration Example of Imaging Device and Camera Position Information) FIG. 3 shows a configuration example of the imaging device used in this embodiment. In this embodiment, shooting is performed with 25 cameras arranged in a 5 × 5 grid. 300 is a u-axis indicating the horizontal direction of the camera, and 301 is a v-axis indicating the vertical direction of the camera. The horizontal direction and vertical direction of each camera are all parallel to the uv axis. In addition, the position of each camera is expressed in the uv coordinate system. In the example of FIG. 3, the origin of the uv coordinates is the central camera position, but it may be an arbitrary position. The 25 cameras shown in FIG. 3 are camera 1 to camera 25, and the positions on the uv coordinates corresponding to camera j are (uj, vj). The camera position information 202 in FIG. 2 stores the positions (u1, v1),..., (U25, v25) of each camera.

  (Horizontal Viewing Angle and Vertical Viewing Angle of Camera) FIG. 4A shows the relationship between the camera and the horizontal viewing angle. A broken line range is a horizontal photographing range, and an angle therebetween is defined as a horizontal viewing angle θ. FIG. 4B shows the relationship between the camera and the vertical viewing angle. The range indicated by the broken line is the shooting range in the vertical direction, and the angle therebetween is defined as the vertical viewing angle φ. In this embodiment, the horizontal viewing angle θ and the vertical viewing angle φ are the same for all cameras. Accordingly, the value of the horizontal viewing angle θ and the value of the vertical viewing angle φ are recorded in the viewing angle information 201 in FIG.

  (Configuration Example of Imaging Data) FIG. 5A is an example of imaging data input from the input terminal 101 in FIG. In FIG. 5A, the imaging data 600 of the present embodiment includes an image generation parameter that is tag information including imaging data acquisition conditions, and an image unit 604 that includes a plurality of images captured by the camera array. . Here, as the image generation parameters, a focus distance information tag 601 in which the focus distance is recorded, an aperture diameter information tag 602 in which the aperture diameter information is recorded, and a viewpoint position information tag 603 in which the viewpoint position is recorded. including. Details of these image generation parameters will be described later. In the present embodiment, the image unit 604 includes 25 images captured by 25 cameras. The tag information may be added to the imaging data as metadata at the time of imaging by the imaging device 106.

  FIG. 5B shows an example of an image included in the image unit 604. FIG. 5B shows three images A, B, and C taken by three different cameras A, B, and C. FIG. In the present embodiment, 25 images with different viewpoints captured by different cameras are included. The focus distance information tag 601, the aperture diameter information tag 602, and the viewpoint position information tag 603 are added to the imaging data 600 when shooting is performed with the imaging device 106. At this time, the tags 601 to 603 are given based on the camera information at the time of shooting. For example, you may give based on the focal distance and aperture value of the lens which are used for the camera. Alternatively, it is given based on information on the subject at the time of shooting. For example, you may give based on the distance information of the image | photographed subject and an imaging device. Further, it may be given as a fixed value at the time of photographing.

  Note that the imaging data 600 only needs to include the image portion 604, and the focus distance information tag 601, the aperture diameter information tag 602, and the viewpoint position information tag 603 are not necessarily included in the imaging data 600.

  (Example of Image Generation Parameter) The image generation parameters of the present embodiment output from the image generation parameter extraction unit 103 and input to the image generation unit 104 include a focus distance f, an aperture diameter r, and a viewpoint position (u0, v0). . Here, the focus distance f indicates a distance that is in focus when performing image generation processing. For example, when f = 1000 mm, an image in which a subject at a distance of 1000 mm from the camera array is in focus is generated. The viewpoint position (u0, v0) represents coordinates on the uv plane where the camera array is arranged. The opening diameter r indicates the opening diameter when performing image generation processing.

  FIG. 6 shows the relationship between the aperture diameter, the viewpoint position, and the camera array. In the image generation processing, processing is performed by regarding a camera group arranged within a radius r / 2 from the viewpoint position (u0, v0) as one lens. In FIG. 6, a range surrounded by a broken line is a camera group used in the image generation process. Here, the aperture diameter r determines the size of the blur of the subject that is out of focus when performing the image generation process. The blur increases as the value of the opening diameter r increases, and the blur decreases as the value of the opening diameter r decreases. The viewpoint position (u0, v0) represents the viewpoint position of the image obtained by the image generation process.

  <Operation Example of Image Generation Parameter Extraction Unit 103> The image generation parameter extraction unit 103 of the present embodiment extracts image generation parameters based on imaging data and imaging device information. Note that the image generation parameter extraction unit 103 may be configured by a hardware circuit, but in the present embodiment, the image generation parameter is extracted by the CPU operating in accordance with a program stored in the memory.

  FIG. 7 is a flowchart showing an example of the operation of the image generation parameter extraction unit 103. First, the imaging data 600 is input from the input terminal 101 of FIG. 1 to the image generation parameter extraction unit 103 (S801). Next, the imaging device information 200 is input to the image generation parameter extraction unit 103 from the input terminal 102 of FIG. 1 (S902). Next, the image generation parameter extraction unit 103 determines the presence / absence of tag information 601 to 603 in the imaging data 600 (S903). That is, tag information extraction is performed. When the image generation parameter extraction unit 103 determines that the tag information 601 to 603 is present, the image generation parameter extraction unit 103 outputs the image generation parameters to the image generation unit 104 based on the tag information 601 to 603 (S904). Then, the process ends. When the image generation parameter extraction unit 103 determines that there is no tag information 601 to 603, the image generation parameter extraction unit 103 outputs image generation parameters based on the object extracted from the image unit 604 in the imaging data 600 (S905). ). Then, the process ends. Details of the object extraction processing in step S905 will be described below.

  (Object Extraction Processing Example S905) A method for outputting image generation parameters based on object information in step S905 of FIG. 7 will be described. In this embodiment, an image generation parameter is output based on a face area as an object of interest in an image.

  FIG. 8 shows a flowchart of the object extraction process. First, an appropriate image is selected from a plurality of images included in the image unit 604 and face detection is performed (S1001). Next, the number of detected face areas is determined (S1002). If the number of face areas is one, the distance value of the extracted face areas is calculated based on a plurality of images (S1003). Then, based on the distance value of the face area, a parameter that generates an image that focuses on the face area is output (S1004). Then, the process ends and returns. If there are a plurality of face areas or no face area, an image generation parameter for generating a pan focus image is output (S1005). Then, the process ends and returns.

  Note that the image selected in step S1001 may be an image taken by any camera. Here, an image j photographed by the camera j is used. An existing algorithm may be used for the face detection method. Examples include algorithms that use template matching and algorithms that use the Hair-Like feature. In this embodiment, a face area is detected by template matching. The detailed procedure of the face area extraction example S1001 and the face area distance value calculation example S1003 in FIG. 8 will be described below.

  (Face Area Extraction Example S1001) FIG. 9A shows a flowchart of face area extraction processing. First, a skin color region is extracted by performing threshold processing on the input image (S1101). The extracted area is set as a face candidate area. Next, matching is performed on the face candidate areas using face image templates of various sizes (S1102). Finally, the face area is extracted by determining whether or not the face area is based on the likelihood calculated by the matching (S1103).

  FIG. 12 shows an example of the extracted face area. When only one face is included, one face area is extracted as shown in the left figure of FIG. When a plurality of faces are included, the same number of face regions are extracted as shown in the left diagram of FIG.

  (Face Area Distance Value Calculation Example S1003) When only one face area is extracted, in step S1003, the face area distance value is calculated using the face area extracted in step S1001 and a plurality of images. . As an algorithm for calculating the distance value, a stereo matching method, a multi-baseline stereo method, or the like can be applied. In the present embodiment, the distance value of the face area is calculated by the adjacent stereo matching method.

  The flow of processing for calculating the distance value of the face area is shown in the flowchart of FIG. First, matching is performed on two images using the extracted face area as a template, and the position of the face area in the two images is calculated (S1301). Then, triangulation is performed using the position of the face area in the two images, the position of the corresponding two cameras, and the angle of view of the camera, and the distance value of the face area is calculated (S1302).

  FIG. 10 shows an example in the case of calculating the distance value using two camera images. In the present embodiment, the distance value d to the subject is calculated using two adjacent cameras. The distance value d is expressed by the following formula based on α and β in FIG.

Here, α is calculated from the position of the camera A, the angle of view, and the position of the face area in the image taken by the camera A. β is calculated from the position of the camera B, the angle of view, and the position of the face area in the image taken by the camera B.

  (Reconstruction Parameter Generation Example S1004) In step S1004 in FIG. 8, a parameter for generating an image focused on the face area by using the distance value d of the face area calculated by the above formula as a focus distance. Is output. At this time, the viewpoint position and the opening radius may be arbitrary values. On the other hand, as image generation parameters for generating a pan-focus image in step S905, the viewpoint position is set to the position (uj, vj) of the camera j, and the opening radius is set to 0. At this time, the focus value may be an arbitrary value.

  The above-described face area distance value calculation and reconstruction parameter generation correspond to the movement amount calculation of a plurality of images such that the objects overlap.

  <Operation Example of Image Generation Processing Unit 104> The image generation processing unit 104 generates an output image based on the image generation parameters input from the image generation parameter extraction unit 103. Note that the image generation processing unit 104 may be configured by a hardware circuit, but in this embodiment, the image generation parameters are extracted by the CPU operating in accordance with a program stored in the memory.

  FIG. 11 is a flowchart illustrating an example of the operation of the image generation processing unit 104. First, imaging data 600 is input to the image generation processing unit 104 from the input terminal 101 of FIG. 1 (S1501). Next, the imaging device information 200 is input to the image generation processing unit 104 from the input terminal 102 in FIG. 1 (S1502). Next, image generation parameters are input from the image generation parameter extraction unit 103 (S1503). Here, the image generation parameters input from the image generation parameter extraction unit 103 are a focus distance f, an aperture diameter r, and a viewpoint position (u0, v0). Next, the image generation processing unit 104 calculates a diaphragm function based on the aperture diameter and the viewpoint position (S1504). Next, the image generation processing unit 104 calculates the shift amount of each image based on the focus distance and the imaging device information (S1505). Subsequently, the image generation processing unit 104 adds the images based on the aperture function and the shift amount, and generates an output image (S1506). Finally, the image generation processing unit 104 outputs the generated output image to the output terminal 105 in FIG. 1 (S1505).

  Hereinafter, each process will be described in more detail. First, in setting the aperture function in step S1504, the aperture function A (u, v) is defined as follows based on the aperture diameter r and the viewpoint position (u0, v0).

Here, if A (ui, vi) = 1 when the position of the camera i is (ui, vi), the image of the camera i is used for the image generation process. If A (ui, vi) = 0, it means that the image of the camera i is not used for the image generation process. That is, image generation processing is performed using only an image taken by a camera located within a radius r from the viewpoint position. Here, when the aperture diameter r is large, an image with a large blur is generated, and when the aperture diameter r is small, an image with a small blur is generated. When r = 0, a pan focus image is generated.

  In the calculation of the shift amount in step S1505, the shift amount of each image corresponding to the focus distance f is calculated. Based on the shift amount calculated here, each image is shifted and added to perform image generation processing. The horizontal shift amount Δx (j, f) and the vertical shift amount Δy (j, f) of the image j are expressed by the following equations.

here. W and H are the horizontal resolution and vertical resolution of the image, respectively, θ is the horizontal viewing angle of the camera, φ is the vertical viewing angle of the camera, and (ui, vi) is the position of the camera. This shift amount becomes larger as the camera is farther from the viewpoint position. Further, when the viewpoint position and the camera position are equal, the shift amount is zero.

  Further, in the generation of the reconstructed image in step S1506, image generation processing is performed by adding each image using the aperture function calculated in step S1504 and the shift amount for each image calculated in step S1505. When the image taken by the camera j is Ij and the image generation image is H, the image generation processing is expressed by the following equation.

N is the number of cameras.

(Example of Output Image Generation) FIG. 12A shows an example of a face area detection result and an image generated when only one face object is included in the image. As shown in FIG. 12A, when only one face area is detected, an image in which the face is in focus is generated. FIG. 12B shows an example of a face area detection result and an image to be generated when a plurality of face objects are included in the image. When two face areas are detected, a pan focus image is generated. <Example of Display Unit 108 and Interface Unit 109> FIG. 13 shows an example of the display unit 108 and the user interface unit 109. Reference numeral 1800 in FIG. 13A denotes a display screen of the display unit 108. Reference numeral 1801 denotes a focus distance adjustment knob included in the user interface unit 109. In FIG. 13A, a pan focus image, which is a generated image when two face areas are detected, is displayed on the display screen 1800 of the display unit 108.

  In the present embodiment, the user interface is implemented as software and is displayed on a display or a liquid crystal panel connected to a control device such as a personal computer. The user can adjust the focus distance included in the image generation parameter by moving the focus distance adjustment knob 1801 left and right. The focus distance can be reduced by moving the focus distance adjustment knob 1801 to the left, and the focus distance can be increased by moving the focus distance adjustment knob 1801 to the right.

When the focus distance adjustment knob 1801 is moved, the focus distance corresponding to the position of the focus distance adjustment knob 1801 is output to the image generation unit 104, and a new image is generated by the image generation unit 104. The generated image is displayed on the display screen 1800 of the display unit 108. FIG. 13B shows an example when the focus distance adjustment knob is moved. The display unit 108 displays not the pan-focus image displayed in FIG. 13A but the focused image at a specific position. In the present embodiment, tag information 601 in the imaging data 600 is displayed. The presence or absence of ~ 603 was determined. If there is, the image generation parameter is extracted using the tag information 601 to 603 in the imaging data 600, and if not, the information on any object extracted from the image in the imaging data 600 is used. Explained the method. However, processing is not limited to this form. For example, an object extraction process may be performed first, and it may be determined whether to use tag information based on the result. Further, the present invention can also be applied when both of these are used. For example, only focus distance information can be determined based on an object, and aperture information and viewpoint information can be determined using a tag.

  In this embodiment, the image is generated by calculating the shift amount of the image based on the depth information of the object, but the shift amount calculation method is not limited to the method of calculating based on the depth information. For example, a method of extracting objects from all input images and calculating the shift amount so that the objects overlap each other is also possible.

  <Effects of the Present Embodiment> As described above, according to the present embodiment, it is possible to generate an image by suitably selecting an image or a pan-focus image that is focused on a specific object.

  Other Embodiments The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (14)

An image processing apparatus that processes a plurality of images of a subject imaged from at least two different viewpoints,
Extraction means for extracting an object from any one of the plurality of images;
A movement amount calculating means for calculating a movement amount of the plurality of images so that the positions of the extracted objects overlap;
An image processing apparatus comprising: an image generation unit configured to move the plurality of images based on the calculated movement amount and generate an output image from the plurality of images.   The image processing apparatus according to claim 1, wherein the image generation unit generates a pan focus image when the extraction unit extracts a plurality of objects.   The image processing apparatus according to claim 1, wherein the object extracted by the extraction unit is a region of a subject of interest.   The image processing apparatus according to claim 3, wherein the region of the subject of interest is a face region. An image processing apparatus that processes a plurality of images of a subject imaged from at least two different viewpoints,
Extraction means for extracting tag information including image acquisition conditions from the plurality of images;
A movement amount calculating means for calculating a movement amount of the plurality of images based on the extracted tag information;
An image processing apparatus comprising: an image generation unit configured to move the plurality of images based on the calculated movement amount and generate an output image from the plurality of images.   The image processing apparatus according to claim 5, wherein the tag information includes a tag indicating a distance from the viewpoint to the subject.   The image processing apparatus according to claim 5, further comprising an adding unit that adds the tag information to the captured image.   The image processing apparatus according to claim 1, wherein the movement amount calculation unit calculates a movement amount based on a distance from the viewpoint to the subject. An input unit for inputting an instruction of a distance from the viewpoint to the subject by a user;
The image processing apparatus according to claim 8, wherein the movement amount calculation unit calculates movement amounts of the plurality of images based on an instruction from the user.   The image processing apparatus according to claim 1, further comprising a display unit that displays any one of the plurality of images or an output image generated by the image generation unit. . An image processing apparatus that processes a plurality of images of a subject imaged from at least two different viewpoints,
Tag information extraction means for extracting tag information including image acquisition conditions from the plurality of images;
Object extraction means for extracting an object from any one of the plurality of images;
When tag information is extracted by the tag information extraction unit, the movement amount of the plurality of images is calculated based on the extracted tag information, and when an object is extracted by the object extraction unit, the extraction is performed. A movement amount calculating means for calculating a movement amount of the plurality of images so that the positions of the objects that have been overlapped,
An image processing apparatus comprising: an image generation unit configured to move the plurality of images based on the calculated movement amount and generate an output image from the plurality of images. An image processing method for processing a plurality of images of a subject imaged from at least two different viewpoints,
A tag information extraction step of extracting tag information including image acquisition conditions from the plurality of images;
An object extraction step of extracting an object from any one of the plurality of images;
When tag information is extracted in the tag information extraction step, the movement amount of the plurality of images is calculated based on the extracted tag information, and when an object is extracted in the object extraction step, the extraction is performed. A movement amount calculation step of calculating movement amounts of the plurality of images so that the positions of the objects that have been overlapped,
An image processing method comprising: an image generation step of moving the plurality of images based on the calculated movement amount and generating an output image from the plurality of images.   A program for causing a computer to execute each step of the image processing method according to claim 12.   A computer-readable storage medium storing the program according to claim 13.

Images