JP4708619B2 - Stereoscopic image forming system and stereoscopic image forming method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention provides a three-dimensional image forming system in which a three-dimensional image is observed by generating a three-dimensional image from an image acquired by a camera or the like and superimposing an optical member such as a lenticular plate on the generated three-dimensional image. and 3D image formation To the law Related.
[0002]
[Prior art]
Conventionally, integral photography and three-dimensional images of lenticular plates are known as methods for forming stereoscopic images (see Takayoshi Ohkoshi: “Three-dimensional image engineering”, Sangyo Tosho, published in 1972).
[0003]
Such a conventional method for forming a stereoscopic image is based on a photographic method. For example, in the lenticular plate three-dimensional image, images obtained by photographing the subject from many viewpoints are acquired, and these images are printed on one photographic plate through the lenticular plate. The following (1) to (3) There was a problem of ▼.
[0004]
(1) Since images from multiple points of the subject are required, a photographing device such as a multi-lens camera has become large.
[0005]
{Circle around (2)} As with {circle around (1)}, a large-scale printing apparatus has been used for stereoscopic image formation.
[0006]
(3) Even with the above apparatus, adjustment and skill were required for shooting and printing.
[0007]
In order to overcome such problems, Japanese Patent Application Laid-Open No. 5-210181 discloses a method of generating more viewpoint images from a plurality of viewpoint images by interpolation, and using the electronic processing to generate the stereoscopic images. The method of forming is shown. That is, the number of viewpoints of an image that needs to be photographed is reduced by electronic interpolation of the image. In addition, recent digital photography has been used to simplify the formation of stereoscopic images.
[0008]
However, as long as a plurality of images are required, shooting is difficult, and there is a problem that moving subjects cannot be shot.
[0009]
In order to solve such a problem in photographing, the applicant of the present application first attaches a stereoscopic photograph adapter to the camera, shoots stereo images of two left and right viewpoints, and multi-viewpoint images from the stereo images by electronic interpolation. A method for acquiring and forming a stereoscopic image was proposed. However, in this case, a stereoscopic photograph adapter that is not required is necessary for normal photographing. Therefore, ideally, it is desirable to obtain a multi-viewpoint image from one image and form a stereoscopic image.
[0010]
On the other hand, in Japanese Patent Laid-Open No. 8-140116, distance measurement is performed for each of a plurality of blocks in an image at the time of shooting, a distance value is recorded together with the image, a multi-viewpoint image is acquired based on the distance value, A method of acquiring an image is shown.
[0011]
In addition, as a method of acquiring a multi-viewpoint image from one image, for example, a subject area is cut out from the background in the image, and the subject area cut out with respect to the background pops out to the front when viewed with the lenticular plate superimposed. As can be seen, a method is known in which a background image and a subject image are shifted by a predetermined amount of parallax when a multi-viewpoint image is generated.
[0012]
[Problems to be solved by the invention]
However, the conventional method described in Japanese Patent Laid-Open No. 8-140116 requires a special device for obtaining the distance values of a plurality of blocks at the time of shooting, and records the distance values together with the image data. Since it is necessary, a three-dimensional image cannot be formed from a normally taken image.
[0013]
In the conventional method for acquiring a multi-viewpoint image from a single image, when a three-dimensional image synthesized from the acquired multi-viewpoint image is observed through a lenticular plate, the subject cut out with respect to the background becomes a screen-like object. There is a problem that the three-dimensional effect becomes unnatural. In addition, cutting out the subject area requires manual work, and there has been a demand for simply specifying the subject area.
[0014]
Therefore, the present invention acquires a multi-viewpoint image from one image without performing special shooting, prints the multi-viewpoint image as a stereoscopic image, and superimposes an optical member such as a lenticular plate, A stereoscopic image forming system that allows you to observe a stereoscopic image of a subject with a natural stereoscopic effect with a simple operation. and 3D image formation The law The purpose is to provide.
[0015]
[Means for Solving the Problems]
In order to achieve the above object, a stereoscopic image forming system according to the present invention includes an image acquisition unit that acquires a subject image, and the subject image acquired by the image acquisition unit is arranged in an output image area. By doing so, an output confirmation image for confirming the position and size of the subject image on the output image area is obtained. Display control means for displaying on the display means, and displayed on the display means Included in output confirmation image Area acquisition means for acquiring a subject area from the subject image in response to an instruction selection of the subject area by the user for the subject image, and parallax map extraction means for extracting a parallax map representing the depth distribution of the subject from the subject area; A parallax map display control unit that displays the extracted parallax map on the display unit so as to overlap the output confirmation image; Based on the subject image and the parallax map, multi-viewpoint image generating means for generating a multi-viewpoint image for the subject from a plurality of viewpoints, and pixels of the same coordinates in each image of the multi-viewpoint image are arranged in the viewpoint array. Image synthesizing means for synthesizing a three-dimensional image by arranging so as to be adjacent pixels according to Corresponds to the output confirmation image displayed with the parallax map superimposed Output means for arranging and outputting the synthesized three-dimensional image in the output image region, and superimposing an optical member having a periodic structure on the output three-dimensional image, thereby providing a stereoscopic image of the subject. Is observed.
[0017]
The three-dimensional image forming method of the present invention includes a step of acquiring a subject image and arranging the acquired subject image in an output image region. By doing so, an output confirmation image for confirming the position and size of the subject image on the output image area is obtained. Display control process to be displayed on the display means, and displayed on the display means Included in output confirmation image Obtaining a subject area from the subject image in response to an instruction selection of the subject area by the user for the subject image, extracting a parallax map representing a depth distribution of the subject from the subject area, Displaying the extracted parallax map on the display for confirmation on the output confirmation image; A step of generating a multi-viewpoint image for the subject from a plurality of viewpoints based on the subject image and the parallax map, and pixels having the same coordinates in each image of the multi-viewpoint image are adjacent to each other according to the arrangement of the viewpoints A step of synthesizing a three-dimensional image by arranging the pixels, Corresponds to the output confirmation image displayed with the parallax map superimposed A step of arranging and outputting the synthesized three-dimensional image in the output image region, and superimposing an optical member having a periodic structure on the outputted three-dimensional image, thereby producing a stereoscopic image of the subject. Is observed.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Stereoscopic image forming system of the present invention and 3D image formation Legal Embodiments will be described with reference to the drawings.
[0020]
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration of a stereoscopic image forming system according to the first embodiment. In the figure, 1 is a subject. Reference numeral 2 denotes a digital camera. An instruction selection unit 3 such as a mouse and a keyboard has a user interface function. Reference numeral 4 denotes an image processing unit. Reference numeral 5 denotes a display unit such as a CRT display for displaying images and processing information.
[0021]
The instruction selection unit 3, the image processing unit 4, and the display unit 5 are configured by, for example, a general-purpose PC (Personal Computer). This general-purpose PC incorporates a hard disk as a recording medium in addition to a known CPU, ROM, RAM, and the like. A printing unit 6 is connected to the image processing unit 4 and prints image data and the like generated by the image processing unit 4. The camera 2, the image processing unit 4, and the printing unit 6 are connected via a USB (Universal Serial Bus).
[0022]
In the stereoscopic image forming system having the above configuration, first, an image photographed by the camera 2 is taken into the image processing unit 4. For example, image data shot and recorded by the camera 2 is activated on the hard disk in the PC via the USB interface by starting the driver software of the camera 2 in the PC that is the image processing unit 4 and performing a predetermined operation. Once recorded.
[0023]
For example, photographed image data is recorded on a CF (Compact Flash (registered trademark) card, so that if the image processing unit 4 has a PC card slot, the camera 2 and the image processing unit 4 are not connected once. The CF card is removed from the camera 2, the CF card is attached to a CF card adapter that can be attached to the PC card slot, and the CF card adapter is attached to the PC card slot, so that the image data recorded on the CF card is recorded on the PC hard disk. Can be handled in the same way as the processed image data.
[0024]
The image processing unit 4 generates a multi-viewpoint image of the subject with respect to the image captured in the image processing unit 4 in this way, combines the three-dimensional image, and outputs the synthesized image to the printing unit 6. This series of processing is executed as application software of a PC, for example.
[0025]
FIG. 2 is a flowchart showing a three-dimensional image forming process procedure in the image processing unit 4 of the stereoscopic image forming system. This processing program is stored in a ROM (not shown) in the image processing unit 4 corresponding to the PC main body, and is also executed by a CPU (not shown) in the image processing unit 4.
[0026]
First, in order to convert an image captured by the camera into data that can be handled by this processing program, the image is captured into the memory of the PC main body, which is the image processing unit 4 (step S1). At this time, an instruction such as the file name of the image data is given by the instruction selecting unit 3, and the designated file is read into the program. In addition, the image data is converted into three-channel two-dimensional array data or bit map composed of RGB. In particular, when the input image data is compressed image data such as the JPEG image format, it is difficult to process the compressed image data as it is. It is necessary to do.
[0027]
The image data captured in step S1 is displayed on the display unit 5 (step S2). At this time, the image is displayed in a plain rectangular area representing the paper so that the size and position when printing the image data on the predetermined paper can be confirmed. FIG. 3 is a diagram showing a display example of image data. In the figure, R is a plain rectangular area representing paper. I is the captured image. The size and position of the image in the paper are calculated in advance from parameters such as the size of the paper to be printed and the print size (resolution) per pixel of the image data. When the user designates the size, position, and direction (rotation within the paper surface) of the image by the instruction selection unit 3 and changes the parameters, the image displayed on the display unit 5 is updated accordingly.
[0028]
Then, a subject area is acquired (step S3). First, when the user views an image displayed on the display unit 5, the instruction selection unit 3 designates a subject area that is to be projected forward as a stereoscopic image. For example, as shown in FIG. 3, in a scene where a person stands in front of the background, the user designates a subject area along the outline of the person. FIG. 4 is a diagram showing an example in which a subject area is designated for the image of FIG. In the figure, the broken line portion is the contour of the subject specified by the user. The data of the subject area is stored in the memory as, for example, two-dimensional data having two values having the same size as the image data having the value 1 for the designated subject area and the value 0 for the background.
[0029]
Thereafter, a parallax map is generated from the subject area specified in step S3 (step S4). The parallax map represents the amount of deviation when the subject corresponding to each two-dimensional pixel of the image data is viewed from two viewpoints. A two-dimensional average value filter having a predetermined size is applied to binary subject area data to generate two-dimensional real number array data, which is used as a parallax map. By this average value filter processing, it is possible to expect an effect that the contour of the designated subject is blurred and the subject gradually jumps out of the background instead of the screen.
[0030]
Here, when the size of the two-dimensional average value filter is large, the stereoscopic effect becomes smoother, but the contour of the subject area designated by the user is blurred, and the intention of the user is not faithfully reflected in the formed stereoscopic image. If the size of the two-dimensional average value filter is small, the user's intention is reflected more faithfully, but the subject approaches a screen-like one. The size of the two-dimensional average value filter is preferably about 5 to 10% of the image size.
[0031]
In addition, if the size of the image data acquired in step S1 is large, the amount of calculation for the filtering process increases. Therefore, the subject area data is preliminarily reduced at a predetermined magnification to perform filtering, and then the parallax is restored to the original size. The map may be enlarged.
[0032]
In addition, as a method of obtaining an empirically desirable three-dimensional shape from the contour of the specified subject area, the contour shape is replaced with a polygon, the polygonal skeleton is estimated, and the height of each vertex constituting the skeleton is set to a polygonal shape. A method of obtaining a three-dimensional shape in a pseudo manner by determining the distance according to the distance from the boundary is known ("Teddy: A Sketching Interface for 3D Freeform Design", T. Igarashi, S. Matsuoka, H. Tanaka, See SIGGRAPH99 Conference Processings).
[0033]
According to this method, it is possible to obtain a three-dimensional shape assuming a thick cross section in a wide portion and a thin cross section in a narrow portion according to the width of the region. The above method may be applied as a method for obtaining a parallax map from a subject area. That is, the subject area acquired in step S3 is stored as line graphic data such as a chain code representing the contour shape, and the contour shape is first approximated to a polygon. A three-dimensional shape is acquired from the approximate polygon data by the above method, and the three-dimensional shape is subjected to perspective projection conversion from two predetermined viewpoints to obtain a parallax map of the image as a shift amount of a pixel position corresponding to each pixel. . For simplicity, the parallax amount may be determined so as to be proportional to the obtained height data of the three-dimensional shape.
[0034]
Thereafter, the parallax map is displayed superimposed on the image data (step S5). The product of the parallax map data and the pixel values of the RGB components of the image data is displayed as image data representing the parallax map.
[0035]
In the present embodiment, the parallax on the near side has a value of 0 and the back side has a value of 1, so that a darker image is displayed toward the front. FIG. 5 is a diagram showing a display example of a parallax map generated from the data of FIGS. 3 and 4. The user looks at the displayed parallax map and confirms the stereoscopic image to be created. That is, it is determined whether a desired parallax map is obtained from the display example in step S5 (step S6). If it is determined that the desired parallax map is obtained, the instruction selection unit 3 instructs the next process to be performed, and the process of step S7 is performed. On the other hand, if it is determined in step S6 that the parallax map needs to be corrected again, the instruction selecting unit 3 instructs the re-acquisition of the subject area, and the process returns to step S3 described above.
[0036]
Thereafter, a multi-viewpoint image is generated using a desired parallax map (step S7). Here, the generated image is a multi-viewpoint image forming a three-dimensional stripe image, and the size of each image depends on the number of images, the resolution to be printed, and the size. If the resolution of the printing unit 6 is RPdpi (dot per inch) and the print size is XP × YPinch, the size of the image to be printed is X (= RP × XP) × Y (= RP × YP) pixels. The number N of images is determined so that N = RP × RL in accordance with the pitch RLinch of the lenticular plate. Since N is an integer, an integer close to RP × RL is actually used as the number of images.
[0037]
For example, in the case of a printer resolution of 600 dpi and a lenticular plate pitch of 1 / 50.8 inch, an image of N = 12 is desirable. At this time, the size of the image at each viewpoint is H (= X / N) × V (= Y). Here, in practice, the print size is determined so that H and V are integers. For example, when H = 200 and V = 1800 pixels, X = 2400 and Y = 1800 pixels, and the print size is 4 × 3 inches.
[0038]
Actually, since the pitch of the lenticular plate needs to match the image period, the size changes slightly. This process is performed in step S8 described later. The image is generated by deforming the image acquired in step S1 using a parallax map.
[0039]
For example, when the image size is h (= 800) × v (= 600) pixels, an image of 200 × 600 pixels is generated according to the number of pixels H in the horizontal direction and the number of pixels v in the vertical direction. The number of pixels in the vertical direction is small because the number of pixels in the vertical direction of each image required for printing is significantly larger than the number of pixels in the horizontal direction. This is because of this.
[0040]
The viewpoint position of the image to be generated is determined so that a predetermined amount of parallax is generated. If the amount of parallax becomes too small, the stereoscopic effect when observing a stereoscopic image is impaired. Conversely, if the amount of parallax becomes too large, a stereoscopic image to be observed becomes unclear due to crosstalk with adjacent images. Further, the viewpoint positions are determined so that the viewpoints are arranged at equal intervals and symmetrically about the viewpoint position of the left image. This is because the viewpoint position of the image sequence is arranged at equal intervals so that the stereoscopic image can be observed stably. Also, the viewpoint position is symmetrically arranged around the viewpoint position of the left image, thereby deforming the image. This is because the amount is minimized, and a high-quality stereoscopic image can be stably obtained even if an error occurs in the parallax map depending on the subject and shooting conditions.
[0041]
Based on the parallax map obtained in step S4, the parallax corresponding to the subject position closest to the camera 2 and the parallax corresponding to the farthest subject position are obtained. In the present embodiment, since the parallax map is a real value from “0” to “1”, the parallax corresponding to the closest subject position is “0”, and the parallax corresponding to the farthest subject position is “1”. is there. Then, the viewpoint position is determined so that the closest subject is observed at a predetermined distance from the print surface at a predetermined observation position and the farthest subject is observed at a predetermined distance from the print surface.
[0042]
At this time, the parameters actually used for image generation are the ratio r of each image to the parallax of the pair of left and right stereo images, and the parallax adjustment amount sh in the perspective, corresponding to the viewpoint position. For example, when the ratio r = 0, it represents the left image itself, and when the ratio r = 1, it represents the image at the viewpoint position of the right image. If there is an error in the parallax map, errors may also occur in the parallax of the latest subject and the parallax of the farthest subject, so the parameters used for image generation do not affect the error from the statistical distribution of the entire parallax map You may make it obtain | require by such a method.
[0043]
Hereinafter, a method of generating each viewpoint image will be described. First, a new viewpoint image is generated by forward mapping using pixels of the left image. That is, first, the position (X in the new viewpoint image that maps each pixel of the left image. _N , Y _N ) From the parallax d at the pixel position (x, y) of the left image, the ratio r representing the viewpoint position, the size of the left image, and the size of the new viewpoint image.
[0044]
X _N = H / h × (x + r × (d−sh)), Y _N = Y (1)
The pixel at the pixel position (x, y) of the left image is changed to (X _N , Y _N ). This process is repeated for all pixels of the left image. Next, among the pixels of the new viewpoint image, the hole filling process is performed on the pixels to which no pixel is assigned from the left image. In the hole filling process, an effective pixel that is a predetermined distance away from the pixel to be obtained is searched, and a weighted average value using the distance of the pixel value as a parameter is assigned. If there is no valid pixel, the search range is expanded and the search is repeated. With this process, a new viewpoint image in which all pixels are valid is generated. This process is repeated for the number of viewpoints to obtain a multi-viewpoint image sequence.
[0045]
Then, a three-dimensional stripe image is synthesized from the multi-viewpoint image sequence (step S8). At this time, the three-dimensional image is synthesized so that pixels at the same coordinates of each image in the multi-viewpoint image sequence are arranged as adjacent pixels according to the viewpoint arrangement of the image. When the pixel value at the j-th viewpoint is Pjmn (where m and n are indices of the pixel array in the horizontal and vertical directions, respectively), the j-th image data is represented as the following two-dimensional array.
[0046]
P _j00 P _j10 P _j20 P _j30 ………
P _j01 P _j11 P _j21 P _j31 ………
P _j02 P _j12 P _j22 P _j32 ………
………
In the synthesis, the images of the respective viewpoints are decomposed into strips for each line in the vertical direction, and are synthesized by the number of viewpoints in the reverse order of the viewpoint positions. Therefore, the combined image is a stripe image as shown below.
[0047]
P _N00 ... P ₂₀₀ P ₁₀₀ P _N10 ... P ₂₁₀ P ₁₁₀ P _N20 ... P ₂₂₀ P ₁₂₀ ………
P _N01 ... P ₂₀₁ P ₁₀₁ P _N11 ... P ₂₁₁ P ₁₁₁ P _N21 ... P ₂₂₁ P ₁₂₁ ………
P _N02 ... P ₂₀₂ P ₁₀₂ P _N12 ... P ₂₁₂ P ₁₁₂ P _N22 ... P ₂₂₂ P ₁₂₂ ………
………
However, viewpoint 1 represents the left end image and N represents the right end image. Here, the reason for reversing the arrangement order of the viewpoint positions is that when observing with the lenticular plate, the image is observed in the left and right direction within one pitch of the lenticular. This three-dimensional stripe image has a size of X (= N × H) × v when the original multi-view image is an N viewpoint image having a size of H × v.
[0048]
Then, the pitch with the lenticular plate is matched with this three-dimensional stripe image. Since there are N RPdpi pixels for one pitch, one pitch is N / RPinch. Since the pitch of the lenticular plate is RLinch, the pitch is adjusted by multiplying the image by RL × RP / N in the horizontal direction. At this time, since the number of pixels in the vertical direction needs to be (RL × RP / N) × Y pixels, multiply the magnification by (RL × RP × Y) / (N × v) in the vertical direction. Match. Therefore, the scaling process is performed on the three-dimensional stripe image in the horizontal and vertical directions to obtain image data for printing. This scaling process is performed by, for example, bilinear interpolation. The synthesized three-dimensional stripe image can be stored in the hard disk in the PC.
[0049]
Finally, the image obtained in step S8 is printed (step S9), and this process ends.
[0050]
When the lenticular plate is overlaid on the printed image by the processing of steps S1 to S9 described above, a natural stereoscopic image reflecting the user's intention is observed. As described above, in this embodiment, it is possible to print a stereoscopic image from one image with a simple operation without performing special photographing.
[0051]
[Second Embodiment]
FIG. 6 is a diagram illustrating a configuration of a stereoscopic image forming system according to the second embodiment. The same components as those in the first embodiment are denoted by the same reference numerals and the description thereof is omitted. In the figure, reference numeral 7 denotes a shape model storage unit, which stores a plurality of standard three-dimensional shapes as data such as a wire frame model and a surface model.
[0052]
FIG. 7 is a flowchart showing a three-dimensional image forming process procedure in the image processing unit 4 of the stereoscopic image forming system. This processing program is stored in a ROM (not shown) in the image processing unit 4 corresponding to the PC main body, and is also executed by a CPU (not shown) in the image processing unit 4. The same step numbers as those in the first embodiment are denoted by the same step numbers and the description thereof is omitted.
[0053]
In the second embodiment, a process (step S3A) of selecting and arranging a shape model is performed instead of the process of acquiring the subject area in step S3 of the first embodiment.
[0054]
First, as in the first embodiment, the acquired image is displayed on the display unit 5 in accordance with the user's instruction by the processes in steps S1 and S2.
[0055]
Thereafter, the shape model is selected and arranged according to the user's instruction (step S3A). In this process, a function of drawing a three-dimensional shape model from an arbitrary direction and viewpoint and displaying it superimposed on an image and a function of deforming the three-dimensional shape model are executed.
[0056]
That is, a list of shapes stored in the shape model storage unit 7 is displayed on the display unit 5. The user uses the instruction selection unit 3 to select a shape model most likely to match the shape of the subject from the list. For example, the case where the user selects the shape model shown in FIG. 8 as the shape model of the subject with respect to the above-described image of FIG.
FIG. 8 is a diagram showing a shape model of a subject.
[0057]
The selected shape model is drawn in the standard direction and viewpoint position, and is displayed on the display unit 5 by being superimposed on the image. The user can freely move, rotate, and deform the shape model displayed in the image by the instruction selection unit 3, perform an operation of superimposing on the subject area in the image, and temporarily store the subject shape data. To do. FIG. 9 is a diagram showing how the selected shape model is matched with the image.
[0058]
A parallax map is generated from the subject shape data acquired in step S3A (step S4). When obtaining a parallax map from a three-dimensional shape, the method shown in step S4 of the first embodiment is used. That is, a perspective projection conversion is performed on a three-dimensional shape from two predetermined viewpoints to obtain a parallax map of the image. For simplicity, the parallax amount may be determined so as to be proportional to the obtained height data of the three-dimensional shape.
[0059]
Then, the obtained parallax map is displayed on the display unit 5 (step S5). The user determines whether or not the displayed parallax map is desired (step S6). If it is determined that the parallax map needs to be corrected, the process returns to step S3A and the same processing is repeated. At this time, the shape data displayed superimposed on the image is the shape data temporarily stored in step S3A. On the other hand, when a desired parallax map is obtained in step S6, the processing in steps S7 to S9 is performed as in the first embodiment, and a three-dimensional stripe image is printed.
[0060]
As described above, in the second embodiment, since a shape model is used instead of cutting out the subject area, a natural stereoscopic image can be formed from one image without cutting out the subject area. Furthermore, a more natural stereoscopic image can be formed by using a shape model more suitable for the subject.
[0061]
The above is the description of the embodiments of the present invention. However, the present invention is not limited to the configurations of these embodiments, and the functions shown in the claims or the functions of the configurations of the embodiments are included. Any configuration that can be achieved is applicable.
[0062]
For example, in the above-described embodiment, an image captured by the camera is temporarily recorded on the hard disk in the image processing unit. However, for example, another PC is prepared for image recording, and an image captured by the camera is stored in the PC. Once recorded, the image processing unit may read the image data via the network. In this case, a certain user A further takes a picture of the camera at a remote place, temporarily records the camera image on the portable PC, and then connects to the network to transfer the digital image data. Another user B connects a PC, which is an image processing unit to which a printing unit is connected at another point, to a network, directly imports digital image data into a processing program, prints a three-dimensional image, and displays the stereoscopic image. You may observe. Further, the user A may remotely acquire the stereoscopic image from a remote place by operating the image processing unit remotely.
[0063]
In the above-described embodiment, the generated three-dimensional stripe image is printed as it is, but an image template such as a photo frame, a Christmas card, or an animation character or a character template is synthesized with the three-dimensional stripe image, and the synthesized data is printed. You may make it do.
[0064]
Furthermore, in the above-described embodiment, the stereoscopic image forming system using the lenticular plate three-dimensional image method has been described. However, the present invention is not limited to this method, and the stereoscopic image forming system using the integral photography or the barrier method. It can also be applied to. In addition, although the case where the final output of the stereoscopic image is printing as a printer output has been shown, the present invention is not limited to this. It can also be applied to the display of stereoscopic images.
[0065]
Further, when the present invention is achieved by supplying software program code for realizing the functions of the above-described embodiments to the system, the program code itself realizes the new function of the present invention, and the program The storage medium storing itself and its program constitutes the present invention.
[0066]
In the above embodiment, the program code shown in the flowcharts of FIGS. 2 and 7 is stored in the storage medium. As a storage medium for supplying the program code, a ROM, a flexible disk, a nonvolatile memory card, or the like can be used.
[0067]
【The invention's effect】
According to the present invention, a multi-viewpoint image is acquired from one image without performing special shooting, the multi-viewpoint image is synthesized and printed as a stereoscopic image, and an optical member such as a lenticular plate is overlaid, A stereoscopic image of the subject can be observed. In addition, it is possible to obtain a stereoscopic image with a natural stereoscopic effect in which the subject does not form a screen with respect to the background. Furthermore, a stereoscopic image can be obtained by a simple operation.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of a stereoscopic image forming system according to a first embodiment.
FIG. 2 is a flowchart illustrating a three-dimensional image forming process procedure in an image processing unit 4 of the stereoscopic image forming system.
FIG. 3 is a diagram illustrating a display example of image data.
4 is a diagram illustrating an example in which a subject area is specified for the image in FIG. 3;
5 is a diagram showing a display example of a parallax map generated from the data of FIGS. 3 and 4. FIG.
FIG. 6 is a diagram illustrating a configuration of a stereoscopic image forming system according to a second embodiment.
FIG. 7 is a flowchart showing a 3D image forming process procedure in the image processing unit 4 of the stereoscopic image forming system.
FIG. 8 is a diagram illustrating a shape model of a subject.
FIG. 9 is a diagram showing how a selected shape model is matched with an image.
[Explanation of symbols]
1 Subject
2 Camera
3 Instruction selection part
4 Image processing section
5 display section
7 Shape model storage

Claims (12)

Image acquisition means for acquiring a subject image;
Display control for displaying on the display means an output confirmation image in which the position and size of the subject image on the output image area can be confirmed by arranging the subject image acquired by the image acquisition means in the output image area. Means,
Area acquisition means for acquiring a subject area from the subject image in response to an instruction selection of a subject area by a user for a subject image included in the output confirmation image displayed on the display means;
Parallax map extraction means for extracting a parallax map representing the depth distribution of the subject from the subject region;
A parallax map display control unit that displays the extracted parallax map on the display unit so as to overlap the output confirmation image;
Multi-viewpoint image generation means for generating a multi-viewpoint image for the subject from a plurality of viewpoints based on the subject image and the parallax map;
Image synthesizing means for synthesizing a three-dimensional image by arranging pixels having the same coordinates in each image of the multi-viewpoint image to be adjacent pixels according to the arrangement of the viewpoints;
Output means for arranging and outputting the synthesized three-dimensional image corresponding to the output confirmation image on which the parallax map is superimposed and displayed in the output image region;
By superimposing an optical member having a periodic structure on the output three-dimensional image,
A stereoscopic image forming system, wherein a stereoscopic image of the subject is observed. The subject area corresponds to each pixel of the subject image, and consists of a two-dimensional array of mask data representing whether the subject is a subject or the background,
The stereoscopic image forming system according to claim 1, wherein the parallax map extracting unit extracts the parallax map by blurring the mask data by a two-dimensional filter process.   The stereoscopic image forming system according to claim 1, wherein the parallax map extraction unit estimates a three-dimensional shape of a subject from the subject region and extracts the parallax map based on the estimated three-dimensional shape. .   The three-dimensional image forming system according to claim 1, wherein the output unit is a printing unit.   The three-dimensional image forming system according to claim 1, wherein the optical member is a lenticular plate. Image recording means for recording the subject image acquired by the image acquisition means as digital image data;
The stereoscopic image forming system according to claim 1, wherein the image recording unit transfers the digital image data via a network. Obtaining a subject image;
A display control step of displaying on the display means an output confirmation image in which the position and size of the subject image on the output image region can be confirmed by arranging the acquired subject image in the output image region;
Obtaining a subject area from the subject image in response to an instruction selection of a subject area by a user for a subject image included in the output confirmation image displayed on the display means;
Extracting a parallax map representing the depth distribution of the subject from the subject region;
Displaying the extracted parallax map on the display for confirmation on the output confirmation image;
Generating a multi-viewpoint image for the subject from a plurality of viewpoints based on the subject image and the parallax map;
Arranging the pixels of the same coordinates in each image of the multi-viewpoint image so as to be adjacent pixels according to the arrangement of the viewpoints, and synthesizing a three-dimensional image;
Arranging and outputting the synthesized three-dimensional image corresponding to the output confirmation image on which the parallax map is displayed superimposed on the output image region;
By superimposing an optical member having a periodic structure on the output three-dimensional image,
A stereoscopic image forming method, wherein a stereoscopic image of the subject is observed. The subject area corresponds to each pixel of the subject image, and consists of a two-dimensional array of mask data representing whether the subject is a subject or the background,
The stereoscopic image forming method according to claim 7 , wherein in the step of extracting the parallax map, the parallax map is extracted by blurring the mask data by a two-dimensional filter process. 8. The stereoscopic image according to claim 7 , wherein in the step of extracting the parallax map, a three-dimensional shape of a subject is estimated from the subject region, and the parallax map is extracted based on the estimated three-dimensional shape. Forming method. 8. The three-dimensional image forming method according to claim 7 , wherein printing is performed in the outputting step. The three-dimensional image forming method according to claim 7 , wherein the optical member is a lenticular plate. Recording the acquired subject image as digital image data;
The stereoscopic image forming method according to claim 7, further comprising a step of transferring the recorded digital image data via a network.

Images