JP4193292B2 - Multi-view data input device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a multi-view data input device for acquiring three-dimensional data.
[0002]
[Prior art]
Conventionally, a multi-view data input device is known as one of passive type three-dimensional data input devices. The multi-view data input device includes a plurality of input units for obtaining a two-dimensional image, a calculation unit for calculating three-dimensional data based on the obtained two-dimensional images, and the like.
[0003]
In order to generate three-dimensional data using such an apparatus, one of the obtained two-dimensional images is set as a reference image. For all coordinate points in the region indicating the subject in the reference image, coordinate points corresponding to them in other two-dimensional images are obtained. These corresponding points can be obtained by the gradient method or the correlation method using the shades of the pixels constituting the two-dimensional image. Based on the corresponding points, the positions of the points in the three-dimensional space are obtained by the principle of stereo vision. A set of position data of the obtained points is three-dimensional data about the subject.
[0004]
In the conventional apparatus, an imaging element that acquires a monochrome two-dimensional image is used in the left and right input units.
[0005]
[Problems to be solved by the invention]
However, when a monochrome image sensor is used, it is convenient to search for corresponding points, but since a color image cannot be obtained, color mapping for three-dimensional data is realized after generating three-dimensional data. Can not do it.
[0006]
For this reason, it is necessary to separately use a camera capable of shooting a color two-dimensional image and to perform shooting separately from shooting for three-dimensional data. For this reason, it takes time for photographing, and the positional relationship between photographing for three-dimensional data and photographing for a color two-dimensional image is not constant, so that it is not easy to perform alignment when performing color mapping. There was also a problem.
[0007]
The present invention has been made in view of the above-described problems. A two-dimensional image for three-dimensional data and a two-dimensional image for color mapping can be simultaneously acquired by one photographing, and the configuration is simple. An object of the present invention is to provide a multi-view data input device.
[0008]
  An apparatus according to the invention of claim 1CoveredA multi-view data input device having a plurality of image input units for inputting a two-dimensional image of a subject from a plurality of different viewpoint positions, wherein one image input unit of the plurality of image input unitsIn,An imaging device capable of acquiring a color two-dimensional image is provided,Other image input sectionIs provided with an image pickup device that acquires a monochrome two-dimensional image, and the color image pickup device and the monochrome image pickup device have the same resolution. For pixels of colors that are not output, the image data of each color is obtained for all pixels by performing interpolation using image data output from neighboring pixels, and the data value of the image data of each color for the same pixel Is added at a predetermined ratio to obtain image data indicating the luminance of each pixel, and for the monochrome image sensor, image data indicating the luminance is obtained from the image data output from each pixel for all pixels. likeIt is configured.
[0009]
  Claim 2And 6The device according to the inventionsoIsColor image sensor, Single-plate image sensor with color filterIt is.
[0010]
  Claim 3And 7In the apparatus according to the invention,The color image sensor is a multi-plate image sensor having a color filter.
[0011]
  Claim 4And 8The device according to the invention ofA calculation unit configured to calculate three-dimensional data of the subject based on the two-dimensional image;
[0012]
  A device according to the invention of claim 5Is,A multi-view data input device having a plurality of image input units for inputting a two-dimensional image of a subject from a plurality of different viewpoint positions, wherein one image input unit of the plurality of image input units Is provided with an image sensor that can acquire a color two-dimensional image, and the other image input unit is provided with an image sensor that acquires a monochrome two-dimensional image.SaidColorThe image sensorMonochromeThe resolution is higher than that of the image sensor.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
[First Embodiment]
FIG. 1 is a perspective view of a multi-lens input camera 5 showing a first embodiment of the present invention, FIG. 2 is a diagram showing an example of the configuration of a three-dimensional data generation apparatus 1 including the multi-lens input camera 5, and FIG. It is a figure which shows notionally the mode at the time of imaging | photography of the to-be-photographed object Q using the multi-lens input camera 5.
[0019]
As shown in FIG. 1, the multi-lens input camera 5 includes a camera body 11, image input units 12 and 13 having photographing lenses 12 a and 13 a, a texture projection unit 16, a shutter 155, and the like. Although not shown in FIG. 1, a processing circuit 15 is built in the multi-lens input camera 5.
[0020]
As shown in FIG. 2, the three-dimensional data generation device 1 includes an information processing device 4 and a multi-lens input camera 5. The information processing device 4 includes a processing device 21, a display device 22 having a display surface HG, an input device 23 including a keyboard and a mouse, and the like. The processing device 21 incorporates a CPU, RAM, ROM, other peripheral elements, an interface device, a hard disk device, a floppy disk device, a CD-ROM device, a modem, and other devices. As such an information processing apparatus 4, a personal computer in which an appropriate program is installed can be used.
[0021]
As shown in FIG. 2, the multi-lens input camera 5 is provided with display devices 12c and 13c made of a liquid crystal panel or the like for displaying the captured two-dimensional images QL and QR, respectively.
[0022]
Data transfer can be performed between the multi-lens input camera 5 and the information processing apparatus 4. It is also possible to display a two-dimensional image input from the multi-lens input camera 5 on the information processing apparatus 4. The information processing device 4 can generate three-dimensional data based on the two-dimensional image input from the multi-lens input camera 5 and display the generated three-dimensional data on the display surface HG.
[0023]
As shown in FIG. 3, the subject Q is captured with parallax by the two photographing lenses 12a and 13a together with the background QK, and is displayed as the two-dimensional images QL and QR on the respective display devices 12c and 13c. Yes. Based on these two-dimensional images QL and QR, corresponding points are searched as pre-processing for generating three-dimensional data. That is, one of the two two-dimensional images QL and QR is used as a reference image, and corresponding points corresponding to them in other two-dimensional images are obtained for all coordinate points in the region indicating the subject Q in the reference image. . Based on the corresponding points, three-dimensional data is obtained by calculation according to the principle of stereo vision. In the present specification, searching for a corresponding point or processing for that purpose may be referred to as “corresponding point search”.
[0024]
FIG. 4 is a diagram for explaining the concept of corresponding point search.
In FIG. 4, the image of the subject Q is formed on the imaging surfaces of the imaging elements 12 b and 13 b provided in the image input units 12 and 13. A two-dimensional image QL formed on the left image sensor 12b is set as a reference image. Note that the two-dimensional images QL and QR obtained by the imaging elements 12b and 13b are the same as the two-dimensional images QL and QR displayed on the display devices 12c and 13c. The corresponding point search is performed as follows.
[0025]
That is, in the corresponding point search, when an arbitrary point on the subject Q is set as the gazing point A, the point L1 indicating the gazing point A on the imaging surface of the imaging element 12b that captures the reference image is changed to the other imaging element 13c. This is an operation for determining what coordinate point corresponds to the image pickup surface. In the corresponding point search, the brightness of the two-dimensional images QL and QR can be used, and a conventionally known method such as a gradient method or a correlation method can be used. By using these methods, it can be seen in FIG. 4 that the point L1 on the imaging surface of the imaging device 12b corresponds to R1 on the imaging surface of the imaging device 13c. By performing such corresponding point search for all points on the subject Q, the positions of all the points on the three-dimensional coordinates can be known, and the three-dimensional shape of the subject Q can be obtained.
[0026]
As a subject for which the corresponding point search is easily performed, an object having a luminance difference between adjacent pixels is preferable. Moreover, the thing with a texture (pattern pattern) is better than the thing of a single color. If there is no texture, it is better to have curves and edges than to a flat one. However, when there is a curve or an edge, there is a problem that an occlusion (a blind spot) occurs depending on the degree.
[0027]
That is, when the subject Q is deep in the depth direction, the base line length, which is the distance between the optical axes of the two image input units 12 and 13 of the multi-lens input camera 5, the focal length of the lens, and the imaging elements 12b and 13b. Depending on the pixel size and the like, there is a possibility that a visible part and an invisible part appear between the images. In particular, when the gradient method is used for the corresponding point search, a step is a problem. Therefore, although there is a problem of degree, a subject having a gentle difference in brightness on the surface and a gentle inclination is a subject whose corresponding points are easily obtained.
[0028]
Now, after generating the three-dimensional data, color mapping is performed on the three-dimensional data. That is, color display is performed by pasting a color two-dimensional image on three-dimensional data. For this purpose, an image sensor capable of acquiring a color two-dimensional image is provided.
[0029]
FIG. 5 is a block diagram showing the configuration of the multi-lens input camera 5, FIG. 6 is a diagram showing an example of the pixel configuration of the imaging elements 12b and 13b, and FIG. 7 is a diagram showing extracted pixels of each color of the single-plate color CCD. .
[0030]
In FIG. 5, the multi-lens input camera 5 includes the image input units 12 and 13 and the processing circuit 15 as described above. The image input units 12 and 13 are provided with photographing lenses 12a and 13a and imaging elements 12b and 13b.
[0031]
A single-plate color CCD is used as the image sensor 12b. The two-dimensional image obtained by the image sensor 12b is used for both corresponding point search and color mapping. A monochrome CCD is used as the image sensor 13b. The two-dimensional image obtained by the image sensor 13b is used only for searching for corresponding points.
[0032]
The processing circuit 15 includes a CPU 151, memories 152 and 153, and the like. Although not shown, the shutter 155 described above, various buttons for normal operation of the multi-lens input camera 5, and other devices are also provided.
[0033]
Image data output from the image sensors 12b and 13b is temporarily stored in the memories 152 and 153. The image data stored in the memories 152 and 153 is read by the CPU 151 and subjected to necessary processing. As processing of the CPU 151, processing for generating three-dimensional data can also be performed. The image data stored in the memories 152 and 153 or the image data processed by the CPU 151 can be output to the outside.
[0034]
As shown in FIG. 6A, an image sensor 12b made of a single-plate color CCD is assigned pixels of each color of R (red), G (green), and B (blue) on one CCD surface. ing. Therefore, R, G, B image data is not obtained for all pixels, but only one color image data is obtained from one pixel.
[0035]
On the other hand, as shown in FIG. 6B, the image sensor 13b made of a monochrome CCD has the same resolution as the image sensor 12b, and image data indicating luminance is obtained from all pixels.
[0036]
As can be understood from the above description, the “pixel” of the image sensor in this specification refers to one cell of the image sensor. The “resolution” for the image sensor is a value determined by the number of cells of the image sensor.
[0037]
7A, 7 </ b> B, and 7 </ b> C, the arrangement of the image sensor 12 b with respect to R is the R pixel surface 12 b R, the arrangement with respect to G is the G pixel surface 12 b G, and the arrangement with respect to B is the B pixel surface 12 b B. , Respectively.
[0038]
As can be seen from these figures, each color has pixels without image data. Therefore, for each color, pixels without image data need to be created by complementation.
[0039]
For example, when viewing the G pixel surface 12bG, G (0, 0) can use the same data value. However, since G (1, 0) has no image data, it is necessary to interpolate using the image data of G (0, 0), G (2, 0), and G (1, 1) that are in the vicinity thereof. There is. That is,
G (1,0) = [G (0,0) + G (2,0) + G (1,1)] / 3
Asking. For G (2,1),
G (2,1) = [G (2,0) + G (1,1) + G (1,3) + G (2,2)] / 4
Asking. Thus, for pixels without image data, data values are obtained by interpolation using image data of neighboring pixels.
[0040]
For the R pixel surface 12bR and the B pixel surface 12bB, the same processing is performed for pixels without image data to obtain data values. As a result, image data of R, G, and B colors can be obtained for all pixels.
[0041]
Then, luminance and color information of each pixel is obtained based on the obtained image data. To obtain the luminance of each pixel, the data values of R, G, and B for the same pixel are added at a predetermined ratio. That is, assuming that the coordinates of a pixel are i, j, the luminance value Y i, j of the pixel is expressed by the following equation (1):
Yi, j = A1.Ri, j + B1.Gi, j + C1.Bi, j (1)
Indicated by A1, B1, and C1 are coefficients, respectively. For example, A1 = 0.2990, B1 = 0.5870, and C1 = 0.1140.
[0042]
In this manner, the corresponding point search is performed based on the luminance value of each pixel obtained from the image sensor 12b and the luminance value of each pixel obtained from the image sensor 13b. For color mapping, the image data obtained from the image sensor 12b is used as it is. In this way, a two-dimensional image for searching for corresponding points and a two-dimensional image for color mapping are obtained simultaneously by one image sensor 12b.
[0043]
However, with such an interpolation method, it is not possible to obtain accurate luminance for all pixels. As a result, compared with a case where a corresponding point search is performed using a monochrome image, there is a case where it is not possible to correspond to a pixel that should originally correspond, and there is a case where accuracy is lowered.
[0044]
However, it is possible to realize a substantially accurate three-dimensional shape formation by performing visual correction by an optimization process using an algorithm for reconstructing the three-dimensional shape.
[0045]
According to the above embodiment, it is possible to generate three-dimensional data with a simple configuration. Since the single image sensor 12b serves as both the corresponding point search image sensor and the color mapping image sensor, the configuration is simple and the correspondence between the two-dimensional image and the three-dimensional data is known, so that the alignment is performed during color mapping. Is easy. Also, the calibration for obtaining the positional relationship between the image input units 12 and 13 necessary for the reconstruction of the three-dimensional data can be reduced by the amount of the image sensor.
[Second Embodiment]
FIG. 8 is a diagram showing an example of the pixel configuration of the image sensors 12Bb and 13Bb according to the second embodiment of the present invention.
[0046]
In the second embodiment, the configurations of the multi-lens input camera 5B and the three-dimensional data generation device 1 are basically the same as those in the first embodiment. The differences between the multi-lens input camera 5B are as follows.
[0047]
That is, in the first embodiment, the image pickup devices 12b and 13b have the same resolution, although there is a difference that one is a color CCD and the other is a monochrome CCD. In contrast, in the second embodiment, the resolution of one image sensor 12Bb that is a color CCD is higher than the resolution of the other image sensor 13Bb that is a monochrome CCD. Specifically, the resolution of the image sensor 12Bb is four times the resolution of the image sensor 13Bb. However, other suitable multiple resolutions can be used.
[0048]
As shown in FIG. 8, one pixel of the monochrome CCD image sensor 13Bb corresponds to four pixels of the color CCD image sensor 12Bb. Therefore, the luminance value corresponding to the luminance value of one pixel of the monochrome CCD can be accurately obtained from the four pixels of the color CCD, that is, image data of each color of R, G, G, and B, without performing interpolation. Can do. Thereby, compared with the case of 1st Embodiment, a corresponding point search can be performed still more correctly and a reliable three-dimensional shape can be reconfigure | reconstructed.
[0049]
As a method for obtaining the luminance value of the color CCD corresponding to the luminance value of one pixel of the monochrome CCD, as shown in the first embodiment, the luminance value for each pixel of the color CCD is obtained. May be added at a predetermined ratio. Even in this case, since it is not necessary to perform interpolation, the luminance value can be obtained accurately, and the corresponding point search can be performed more accurately.
[0050]
Here, a color mapping method will be described.
FIG. 9 is a diagram showing an example of the three-dimensional shape QF1 reconstructed by the three-dimensional data generation apparatus 1, and FIGS. 10 and 11 are polygons PG forming the three-dimensional shape QF and 2 for color mapping corresponding to each point. It is a figure which shows the relationship with a dimension image (henceforth "mapping image QM"). 10 shows a case where the mapping image QM2 has a low resolution, and FIG. 11 shows a case of the present embodiment where the mapping image QM3 has a high resolution.
[0051]
In FIG. 9, each point 1, 2, 3,... Indicated by a black circle is a point constituting the three-dimensional shape QF1, that is, three-dimensional data of each point. The polygon PG is formed by any three points.
[0052]
Now, each point of the three-dimensional shape QF is obtained based on the correspondence between the two two-dimensional images input to the image sensors 12Bb and 13Bb. By using the color information of the points used at that time, the three-dimensional shape QF Color mapping can be performed for each polygon PG of the shape QF.
[0053]
In FIG. 10, the color mapping for the polygon PG2 of the three-dimensional shape QF2 will be described. The points “1”, “2”, and “3” that form the polygon PG2 correspond to the images “4”, “5”, and “6” of the mapping image QM2, respectively. Therefore, these three images are pasted on the polygon PG2 as shown in the figure.
[0054]
In FIG. 11, the color mapping for the polygon PG3 of the three-dimensional shape QF3 will be described. The points “1”, “2”, and “3” that form the polygon PG3 correspond to the images “4”, “5”, and “6” of the mapping image QM3, respectively. Each of these images consists of four pixels, and therefore the resolution of the images is high. These three images are pasted on the polygon PG3 as shown in the figure, but since the resolution of each image is high, the resolution of the image color-mapped to the polygon PG3 is higher than in the case of FIG. I understand that.
[0055]
That is, as in the present embodiment, by setting the color CCD resolution higher than the monochrome CCD resolution, a high-resolution mapping image QM can be obtained. Can be mapped. Therefore, the visual effect of the color-mapped three-dimensional shape QF is high. For example, even when the accuracy of the three-dimensional shape QF is poor, or even when the three-dimensional shape QF has a defect, it can be visually improved by the high-resolution mapping image QM.
[Third Embodiment]
FIG. 12 is a diagram showing the configuration of the image input unit 12C according to the third embodiment of the present invention, and FIG. 13 is a diagram showing the configuration of the image input unit 13C.
[0056]
In the third embodiment, the configurations of the multi-lens input camera 5C and the three-dimensional data generation device 1 are basically the same as those in the first embodiment. The difference between the multi-lens input camera 5C is as follows.
[0057]
That is, in the first embodiment, a single-plate color CCD is used as the image sensor 12b. However, in the third embodiment, a multi-plate color CCD as shown in FIG. 12 is used as the image sensor 12Cb. .
[0058]
In FIG. 12, an image input unit 12C includes a photographing lens 12Ca, a prism 121, a filter 122 that reflects an R component, a filter 123 that reflects only a G component, and imaging elements 12CbR and 12CbG for R, G, and B colors. , 12CbB.
[0059]
The light LL that has entered the prism 121 after passing through the photographing lens 12Ca is first divided in optical path by the filter 122. The R component of the light LL is reflected by the prism 121 and received by the image sensor 12CbR. The G component is transmitted through the filter 122, reflected by the filter 123, and received by the image sensor 12CbG. The B component passes through the filters 122 and 123 and is received by the image sensor 12CbB.
[0060]
In FIG. 13, the image input unit 13C includes a photographing lens 13Ca and an image sensor 13Cb that is a monochrome CCD. The configuration of the image input unit 13C is the same as the configuration of the image input units 13 and 13B of the first and second embodiments.
[0061]
The resolution of each image sensor 12CbR, 12CbG, 12CbB and the resolution of the image sensor 13Cb are the same. That is, one pixel of each color component of R, G, B in each color CCD is the same size as one pixel of the monochrome CCD.
[0062]
Therefore, in the image input unit 12C, the luminance of each pixel can be obtained from the R, G, and B image data of each pixel of the imaging elements 12CbR, 12CbG, and 12CbB by the above equation (1). That is, the luminance value corresponding to the luminance value of one pixel of the monochrome CCD can be accurately obtained without performing interpolation. Thereby, the corresponding point search can be performed more accurately, and a highly reliable three-dimensional shape can be reconstructed. Further, for the mapping image QM obtained by the image input unit 12C, complete color information can be obtained without interpolating the color information, and the visual effect is good.
[Fourth Embodiment]
FIG. 14 is a diagram showing an example of the pixel configuration of the image sensors 12Db and 13Db according to the fourth embodiment of the present invention.
[0063]
In the fourth embodiment, the configurations of the multi-lens input camera 5D and the three-dimensional data generation device 1 are basically the same as those in the first and third embodiments. The differences between the multi-view input camera 5D and the third embodiment are as follows.
[0064]
That is, in the third embodiment, the color CCD and the monochrome CCD have the same resolution. On the other hand, in the fourth embodiment, the resolutions of the image pickup devices 12DbR, 12DbG, and 12DbB that are color CCDs are higher than the resolution of the image pickup device 13Db that is a monochrome CCD.
[0065]
As shown in FIG. 14, one pixel of the image pickup device 13Db that is a monochrome CCD extends over a plurality of pixel regions (a region of four pixels in this embodiment) of the image pickup devices 12DbR, 12DbG, and 12DbB that are color CCDs. . Therefore, the luminance value by the color CCD corresponding to one pixel of the monochrome CCD can be accurately obtained from the average value of a plurality of pixels.
[0066]
Then, a high-resolution mapping image QM can be obtained by the imaging elements 12DbR, 12DbG, and 12DbB, whereby a high-resolution image can be mapped.
[Fifth Embodiment]
FIG. 15 is a block diagram showing a configuration of a multi-lens input camera 5E according to the fifth embodiment of the present invention.
[0067]
In FIG. 15, the multi-lens input camera 5E includes image input units 12E and 13E and a processing circuit 15E. The image input unit 12E is provided with a photographing lens 12Ea, a filter 125, and two imaging elements 12EbC and 12EbK. The image input unit 13E is provided with a photographing lens 13Ea and an image sensor 13Eb that is a monochrome CCD. The configuration of the image input unit 13E is the same as the configuration of the image input units 13 to 13D of the first to fourth embodiments.
[0068]
As the image sensor 12EbC, the same single-plate color CCD as the image sensor 12b shown in FIG. 6A is used. Therefore, the image pickup device 12EbC has the same resolution as the image pickup devices 12EbK and 13Eb, which are monochrome CCDs, and R, G, and B pixels are assigned to one CCD surface, and one pixel Can obtain image data of only one color. As the image pickup devices 12EbK and 13Eb, the same monochrome CCD as the image pickup device 13b shown in FIG. 6B is used.
[0069]
The filter 125 divides the light incident on the photographing lens 12Ea into two optical paths, and guides the light to the imaging element 12EbC that is a color CCD and the imaging element 12EbK that is a monochrome CCD. The two-dimensional image obtained by the image sensor 12EbC is used for color mapping, and the two-dimensional image obtained by the image sensor 12EbK is used for searching for corresponding points. The two-dimensional image obtained by the image sensor 13Eb is used for searching for corresponding points.
[0070]
That is, the image sensors 12EbK and 13Eb, which are monochrome CCDs, are used for searching for corresponding points, and separately, the image sensor 12EbC, which is a single-plate color CCD, is used for color mapping.
[0071]
The processing circuit 15E is provided with memories 152C, 152K, and 153 for storing image data output from the imaging elements 12EbC, 12EbK, and 13Eb, respectively. The CPU 151 processes the image data stored in the memories 152C, 152K, and 153, and performs processing for searching for corresponding points, generating a three-dimensional shape QF, color mapping, and the like.
[0072]
In the present embodiment, since the corresponding point search is performed by the two-dimensional images obtained by the two image pickup devices 12EbK and 13Eb having the same resolution made of the monochrome CCD, the ideal corresponding point search with high reliability is performed. It is possible to accurately reconstruct the three-dimensional shape QF.
[Sixth Embodiment]
In the sixth embodiment, the configurations of the multi-lens input camera 5F and the three-dimensional data generation device 1 are basically the same as those of the fifth embodiment. The difference between the multi-lens input camera 5F is as follows.
[0073]
That is, in the fifth embodiment, the resolution of the color CCD and the resolution of the monochrome CCD are the same, but in the sixth embodiment, the resolution of the color CCD is higher than the resolution of the monochrome CCD. Specifically, for example, there is a relationship between the image sensor 12Bb which is a color CCD and the image sensor 13Bb which is a monochrome CCD shown in FIG.
[0074]
That is, the multi-view input camera 5F of the sixth embodiment is a combination of the features of the multi-view input camera 5E of the fifth embodiment and the features of the multi-view input camera 5B of the second embodiment. it can.
[0075]
Therefore, according to the sixth embodiment, a high-resolution mapping image QM can be obtained, and a high-resolution image can be mapped to the three-dimensional shape QF. Accordingly, the visual effect of the color-mapped three-dimensional shape QF is enhanced.
[Seventh Embodiment]
In the seventh embodiment, the configurations of the multi-lens input camera 5G and the three-dimensional data generation device 1 are basically the same as those of the fifth embodiment. The differences between the multi-view input camera 5G are as follows.
[0076]
That is, a single-plate color CCD is used in the fifth embodiment, but a multi-plate color CCD as shown in FIG. 12 is used in the sixth embodiment.
[0077]
That is, the multi-view input camera 5G of the seventh embodiment is a combination of the features of the multi-view input camera 5E of the fifth embodiment and the features of the multi-view input camera 5C of the third embodiment. it can.
[0078]
Therefore, according to the seventh embodiment, it is possible to accurately obtain the luminance value corresponding to the luminance value of one pixel of the monochrome CCD without performing interpolation, and thereby the corresponding point search is performed more accurately. And a highly reliable three-dimensional shape can be reconstructed.
[Eighth Embodiment]
In the eighth embodiment, the configurations of the multi-lens input camera 5H and the three-dimensional data generation device 1 are basically the same as those in the seventh embodiment. The differences between the multi-view input camera 5H are as follows.
[0079]
That is, in the seventh embodiment, the multi-plate color CCD and the monochrome CCD have the same resolution. On the other hand, in the eighth embodiment, the resolution of the multi-plate color CCD is higher than that of the monochrome CCD.
[0080]
That is, the multiview input camera 5H of the eighth embodiment is a combination of the features of the multiview input camera 5G of the seventh embodiment and the features of the multiview input camera 5D of the fourth embodiment. it can.
[0081]
Therefore, according to the eighth embodiment, the luminance value by the multi-plate color CCD corresponding to one pixel of the monochrome CCD can be accurately obtained from the average value of a plurality of pixels. A multi-plate color CCD can obtain a high-resolution mapping image QM, and thereby a high-resolution image can be mapped.
[0082]
According to each embodiment described above, by configuring only one image input unit among the plurality of image input units so that the specifications relating to image quality are different from those of other image input units, After shooting with one shot, a two-dimensional image that can be used for both corresponding point search and color mapping can be obtained simultaneously.
[0083]
The eight embodiments have been described above. Although part of the description of the effects of the respective embodiments has been omitted, it is obvious when considering the effects of other related embodiments. Also, various combinations of these embodiments are possible.
[0084]
In the embodiment described above, the case where a still image is input as a two-dimensional image has been described, but the present invention can be similarly applied to a case where a moving image is input as a two-dimensional image. In addition, the configuration, shape, material, processing content, order, etc. of the whole or each part of the information processing apparatus 4, the multi-lens input camera 5 to 5H, or the three-dimensional data generation apparatus 1 may be appropriately changed in accordance with the spirit of the present invention. Can do.
[0085]
【The invention's effect】
According to the present invention, there is provided a multi-view data input device that can simultaneously acquire a two-dimensional image for three-dimensional data and a two-dimensional image for color mapping by one photographing, and has a simple configuration. be able to.
[0086]
  Claim 2And 6According to the present invention, the configuration is simpler.
  Claim5 to 8According to the invention, it is possible to obtain a mapping image with high resolution, and to enhance the visual effect of the color-mapped three-dimensional shape.
[0087]
  AlsoThus, the luminance value necessary for the corresponding point search can be accurately obtained without performing interpolation.AndAn ideal corresponding point search with high reliability is performed, and the three-dimensional shape can be accurately reconstructed.
[Brief description of the drawings]
FIG. 1 is a perspective view of a multi-lens input camera showing a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a configuration of a three-dimensional data generation apparatus including a multi-view input camera.
FIG. 3 is a diagram conceptually showing a state when a subject is photographed using a multi-lens input camera.
FIG. 4 is a diagram for explaining a concept of corresponding point search.
FIG. 5 is a block diagram illustrating a configuration of a multi-lens input camera.
FIG. 6 is a diagram illustrating an example of a pixel configuration of an image sensor.
FIG. 7 is a view showing pixels of each color extracted from a single-plate color CCD.
FIG. 8 is a diagram illustrating an example of a pixel configuration of an image sensor according to a second embodiment of the present invention.
FIG. 9 is a diagram illustrating an example of a three-dimensional shape reconstructed by a three-dimensional data generation apparatus.
FIG. 10 is a diagram illustrating a relationship between a polygon and a low-resolution mapping image.
FIG. 11 is a diagram illustrating a relationship between a polygon and a high-resolution mapping image.
FIG. 12 is a diagram illustrating a configuration of an image input unit according to a third embodiment of the present invention.
FIG. 13 is a diagram illustrating a configuration of an image input unit.
FIG. 14 is a diagram illustrating an example of a pixel configuration of an image sensor according to a fourth embodiment of the present invention.
FIG. 15 is a block diagram showing a configuration of a multi-lens input camera according to a fifth embodiment of the present invention.
[Explanation of symbols]
1 3D data generator
5-5H multi-lens input camera (multi-lens data input device)
12 Image input unit (one image input unit)
13 Image input unit (other image input units)
12b Image sensor (single-plate image sensor)
12Bb image sensor (high resolution single-plate image sensor)
12Cb image sensor (multi-plate image sensor)
12Db image sensor (high resolution multi-plate image sensor)
12EbC image sensor (single-plate image sensor)
12EbK image pickup device (image pickup device for obtaining a monochrome two-dimensional image)
13b, 13Bb, 13Cb, 13Db, 13Eb, 13Fb, 13Gb, 13Hb Image sensor (image sensor for acquiring a monochrome two-dimensional image)
151 CPU (calculation unit)
Q Subject
QL, QR 2D image

Claims (8)

A multi-view data input device having a plurality of image input units for inputting a two-dimensional image of a subject from a plurality of different viewpoint positions,
Into one image input unit of the plurality of image input unit, an imaging device is provided that can obtain a two-dimensional color image, other image input unit, acquires the two-dimensional image of the monochrome An image sensor is provided,
The color image sensor and the monochrome image sensor have the same resolution,
With respect to the color image sensor, by interpolating using the image data output from the neighboring pixels for the color pixels for which no image data is output among the pixels, the image data for each color is obtained for all the pixels. Obtaining the image data indicating the luminance of each pixel by adding the data values of the image data of each color for the same pixel at a predetermined ratio,
For the monochrome image sensor, image data indicating luminance is obtained from image data output from each pixel for all pixels.
A multi-view data input device. The color image sensor is a single-plate image sensor having a color filter .
The multi-view data input device according to claim 1. The color image sensor is a multi-plate image sensor having a color filter.
The multi-view data input device according to claim 1. A calculation unit that calculates three-dimensional data of the subject based on the two-dimensional image;
The multi-view data input device according to claim 1. A multi-view data input device having a plurality of image input units for inputting a two-dimensional image of a subject from a plurality of different viewpoint positions,
One image input unit among the plurality of image input units is provided with an imaging device capable of acquiring a color two-dimensional image, and the other image input unit acquires a monochrome two-dimensional image. An image sensor is provided,
The color image sensor has a higher resolution than the monochrome image sensor.
A multi-view data input device. The color image sensor is a single-plate image sensor having a color filter.
The multi-view data input device according to claim 5. The color image sensor is a multi-plate image sensor having a color filter.
The multi-view data input device according to claim 5. A calculation unit that calculates three-dimensional data of the subject based on the two-dimensional image;
The multi-view data input device according to claim 5.

Images