JP2001016621A - Multi-eye data input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simultaneously obtain three-dimensional data and two-dimensional images used for color mapping with a single photographing action and in a simple constitution by preparing a image input part that has the different specifications about the picture quality from other image input parts. SOLUTION: A single plate image pickup element 12b having a color filter is prepared at an image input part 12 to obtain a two-dimensional color image, and an image pickup elements 13b is prepared at an image input part 13 to obtain a two-dimensional monochromatic image. The two-dimensional image obtained via the element 12b is used for both corresponding point retrieval and color mapping purposes, and the two-dimensional image obtained via the element 13b is used for only a corresponding point retrieval purpose. In such a constitution, it is possible to simultaneously obtain two-dimensional images which are available for both corresponding point retrieval and color mapping purposes after the photographing is carried out by a single shot.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-view data input device for acquiring three-dimensional data.

[0002]

2. Description of the Related Art Conventionally, a multi-view data input device is known as one of passive type three-dimensional data input devices. The multi-view data input device includes a plurality of input units for obtaining a two-dimensional image, a calculating unit for calculating three-dimensional data based on the obtained two-dimensional images, and the like.

In order to generate three-dimensional data using such an apparatus, one of a plurality of two-dimensional images obtained is used.
One is set as a reference image. With respect to all coordinate points in the region indicating the subject in the reference image, corresponding coordinate points are obtained in another two-dimensional image. These corresponding points are
It can be obtained by a gradient method, a correlation method, or the like, using the shading of the pixels constituting the two-dimensional image. Based on the corresponding point, the position of the point in the three-dimensional space is determined by the principle of stereo vision. A set of position data of the obtained points is three-dimensional data of the subject.

In a conventional apparatus, an image sensor for obtaining a monochrome two-dimensional image is used in the left and right input units.

[0005]

However, when a monochrome image sensor is used, it is convenient to search for a corresponding point, but a color image cannot be obtained. And color mapping for three-dimensional data cannot be realized.

For this reason, it is necessary to separately use a camera capable of shooting a color two-dimensional image and perform shooting separately from shooting for three-dimensional data. Therefore, it takes time and effort to shoot, and the positional relationship between shooting for three-dimensional data and shooting for a two-dimensional color image is not constant, so that it is not easy to perform positioning when performing color mapping. There were also problems.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problem, and a two-dimensional image for three-dimensional data and a two-dimensional image for color mapping can be simultaneously obtained by one photographing. It is an object of the present invention to provide a multi-view data input device having a simple configuration.

[0008]

According to a first aspect of the present invention, as shown in FIGS. 1 to 5, a plurality of two-dimensional images QL and QR of a subject Q are input from a plurality of different viewpoint positions. A multi-view data input device 5 having image input units 12, 13, wherein the plurality of image input units 12, 1
One of the three image input units 12 is configured such that the specifications regarding image quality are different from those of the other image input units 13.

As shown in FIG. 5, in the apparatus according to the second aspect of the present invention, the single image input section 12 has a color filter and is a single-plate type capable of acquiring a color two-dimensional image. An image sensor 12b is provided, and the other image input unit 13 is provided with an image sensor 13b for acquiring a monochrome two-dimensional image.

In the apparatus according to the third aspect of the present invention, as shown in FIG. 8, the image pickup device 12Bb provided in the one image input section 12B is replaced with the image pickup element 13Bb provided in the other image input section 13B. Higher resolution.

As shown in FIG. 12, in the apparatus according to the fourth aspect of the present invention, the one image input section 12C has a color filter and is capable of acquiring a two-dimensional color image. An image sensor 12Cb is provided, and the other image input unit 13C is provided with an image sensor 13Cb for acquiring a monochrome two-dimensional image.

In the apparatus according to a fifth aspect of the present invention, the image pickup device provided in the one image input unit 12C has a higher resolution than the image pickup device provided in the other image input unit 13C.

As shown in FIG. 15, in the apparatus according to the sixth aspect of the present invention, the one image input section 12E has a color filter and is a single-plate type capable of acquiring a color two-dimensional image. An image sensor 12EbC and an image sensor 12EbK for acquiring a monochrome two-dimensional image are provided, and the other image input unit 13E is provided with an image sensor 13Eb for acquiring a monochrome two-dimensional image.

According to a seventh aspect of the present invention, there is provided a color image pickup device provided in the one image input section.
bC has a higher resolution than the image sensor 13Eb provided in the other image input unit 13E.

According to an eighth aspect of the present invention, the one image input section 12 has a multi-plate type image pickup device having a color filter and capable of acquiring a two-dimensional color image.
And an image sensor for acquiring a monochrome two-dimensional image is provided, and the other image input unit 13 is provided with an image sensor for acquiring a monochrome two-dimensional image.

According to a ninth aspect of the present invention, the color image pickup device provided in the one image input unit has a higher resolution than the image pickup device provided in the other image input unit.

An apparatus according to a tenth aspect of the present invention includes a calculating section 151 for calculating three-dimensional data of the subject Q based on the two-dimensional images QL and QR.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [First Embodiment] FIG. 1 is a perspective view of a multi-view input camera 5 showing a first embodiment of the present invention, and FIG. Generator 1
FIG. 3 is a diagram conceptually showing a situation when the subject Q is photographed using the multi-view input camera 5.

As shown in FIG. 1, a multi-lens input camera 5 includes a camera body 11, image input units 12 and 13 having photographing lenses 12a and 13a, respectively, and a texture projection unit 1.
6 and a shutter 155 are provided. Although not shown in FIG. 1, a processing circuit 15 is built in the multi-view input camera 5.

As shown in FIG. 2, the three-dimensional data generation device 1 includes an information processing device 4 and a multi-view input camera 5.
The information processing device 4 includes a processing device 21, a display device 22 having a display surface HG, an input device 23 including a keyboard and a mouse, and the like. The processing device 21 includes a CPU, an R
AM, ROM, other peripheral devices, interface device, hard disk device, floppy disk device, C
A D-ROM device, a modem, and other devices are built in. As such an information processing device 4, a personal computer on which an appropriate program is installed can be used.

As shown in FIG. 2, the multi-view input camera 5 displays two-dimensional images QL and QR taken.
Display devices 12c and 13c each including a liquid crystal panel are provided.

Data can be transferred between the multi-lens input camera 5 and the information processing device 4. Further, a two-dimensional image input from the multi-view input camera 5 can be displayed on the information processing device 4. The information processing device 4
It is possible to generate three-dimensional data based on the two-dimensional image input from the multi-view input camera 5 and display the generated three-dimensional data on the display surface HG.

As shown in FIG. 3, the subject Q is taken in with parallax by the two photographing lenses 12a and 13a together with the background QK, and the respective display devices 12c and
13C, the two-dimensional images QL and QR are displayed. Based on these two-dimensional images QL and QR, corresponding points are searched as pre-processing for generating three-dimensional data. That is, one of the two two-dimensional images QL and QR is used as a reference image, and for all coordinate points in the area indicating the subject Q in the reference image, corresponding points in the other two-dimensional images are obtained. . Based on the corresponding points, three-dimensional data is obtained by calculation based on the principle of stereo vision. In this specification, searching for a corresponding point or a process therefor may be referred to as “corresponding point search”.

FIG. 4 is a diagram for explaining the concept of the corresponding point search. In FIG. 4, an image of the subject Q is formed on imaging surfaces of imaging elements 12 b and 13 b provided in the image input units 12 and 13. The two-dimensional image QL formed on the left imaging element 12b is set as a reference image. In addition, the image sensor 1
The two-dimensional images QL, QR obtained by the 2b, 13b are two-dimensional images QL, QR displayed on the display devices 12c, 13c.
Is the same as The corresponding point search is performed as follows.

That is, in the corresponding point search, when an arbitrary point on the subject Q is set as the gazing point A, the point L1 indicating the gazing point A on the imaging surface of the imaging device 12b that captures the reference image.
Is the operation of determining what coordinate point corresponds to the point on the imaging surface of the other imaging element 13c. In the corresponding point search, the luminance of the two-dimensional images QL and QR is used, and a conventionally known method such as a gradient method or a correlation method can be used. By using these methods, it can be seen in FIG. 4 that the point L1 on the imaging surface of the imaging device 12b corresponds to R1 on the imaging surface of the imaging device 13c.
By performing such a corresponding point search for all points on the subject Q, the positions of all the points on the three-dimensional coordinates are known, and the three-dimensional shape of the subject Q can be obtained.

As a subject in which a corresponding point search can be easily performed,
It is preferable that there is a luminance difference between adjacent pixels. Further, those having a texture (pattern) are better than those having a single color. If there is no texture, it is better to have a curved or edged effect than a flat one. However, there is a problem that when there is a curve or an edge, occlusion (blind spot) occurs depending on the degree.

That is, when the subject Q is deep in the depth direction, the two image input units 12 and 13 of the multi-view input camera 5 are used.
According to the base length, which is the distance between the optical axes, the focal length of the lens, and the size of the pixels of the imaging devices 12b and 13b.
There may be visible and invisible parts between the images. In particular, when the gradient method is used for the corresponding point search, the step becomes a problem. Therefore, although there is a problem of the degree, a subject having a luminance difference on the surface and showing a gentle inclination is a subject whose corresponding point can be easily obtained.

After the three-dimensional data is generated, color mapping is performed on the three-dimensional data. That is, a color display is performed by pasting a color two-dimensional image on the three-dimensional data. For this purpose, an image sensor capable of acquiring a color two-dimensional image is provided.

FIG. 5 is a block diagram showing the configuration of the multi-view input camera 5, FIG. 6 is a diagram showing an example of the pixel configuration of the image pickup devices 12b and 13b, and FIG. FIG.

In FIG. 5, the multi-lens input camera 5 includes the image input units 12 and 13 and the processing circuit 1 as described above.
5 and so on. The image input units 12 and 13 are provided with photographic lenses 12a and 13a and image sensors 12b and 13b.

A single-plate color CC is used as the image pickup device 12b.
D is used. The two-dimensional image obtained by the image sensor 12b is used for both corresponding point search and color mapping. A monochrome CCD is used as the imaging element 13b. The two-dimensional image obtained by the image sensor 13b is used only for corresponding point search.

The processing circuit 15 includes a CPU 151, memories 152 and 153, and the like. Although not shown, the above-mentioned shutter 155, various buttons for normal operation of the multi-view input camera 5, and other devices are also provided.

Image data output from the image pickup devices 12b and 13b is temporarily stored in the memories 152 and 153.
The image data stored in the memories 152 and 153 is
The data is read out by U151 and required processing is performed.
As a process of the CPU 151, a process of generating three-dimensional data can be performed. Also, the memories 152 and 15
3 or the image data processed by the CPU 151 can be output to the outside.

As shown in FIG. 6A, a single-plate type color C
The imaging element 12b composed of a CD is provided on one CCD surface.
Pixels of each color of R (red), G (green), and B (blue) are assigned. Therefore, image data of each color of R, G, B is not obtained for all pixels, but image data of only one color is obtained from one pixel.

On the other hand, as shown in FIG. 6B, the image pickup device 13b composed of a monochrome CCD has the same resolution as the image pickup device 12b, and image data indicating luminance is obtained from all pixels.

As can be understood from the above description, the “pixel” of the image sensor in this specification indicates one cell of the image sensor. The “resolution” of the image sensor is a value determined by the number of cells of the image sensor.

FIGS. 7A, 7B and 7C show the image pickup device 1.
When the arrangement of R of 2b is R pixel surface 12bR, G
Is shown as a G pixel plane 12bG, and the arrangement about B is shown as a B pixel plane 12bB.

As can be seen from these figures, looking at each color, there are pixels without image data.
Therefore, for each color, pixels without image data need to be created by complementation.

For example, when looking at the G pixel surface 12bG, the data value of G (0,0) can be used as it is. However, since G (1,0) has no image data, it is necessary to interpolate using the image data of G (0,0), G (2,0), and G (1,1) in its vicinity. There is. That is, G (1,0) = [G (0,0) + G (2,0) + G
(1, 1)] / 3. For G (2,1), G (2,1) = [G (2,0) + G (1,1) + G
(1,3) + G (2,2)] / 4. As described above, for a pixel having no image data, a data value is obtained by interpolation using image data of a neighboring pixel.

With respect to the R pixel surface 12bR and the B pixel surface 12bB, the data value is obtained by performing the same processing for the pixels having no image data. Thus, image data of each color of R, G, B is obtained for all the pixels.

Then, based on the obtained image data,
Luminance and color information of each pixel is obtained. In order to obtain the luminance of each pixel, the data values of R, G, and B for the same pixel are added at a predetermined ratio. That is, assuming that the coordinates of a pixel are i, j, the luminance value Yi, j of the pixel is represented by the following equation (1): Yi, j = A1 · Ri, j + B1 · Gi, j + C1 · Bi, j (1) It is shown by. A1, B1, and C1 are coefficients, respectively.
For example, A1 = 0.2990, B1 = 0.5870, C
1 = 0.1140.

A corresponding point search is performed based on the luminance value of each pixel obtained from the image pickup device 12b and the luminance value of each pixel obtained from the image pickup device 13b. For color mapping, image data obtained from the image sensor 12b is used as it is. In this manner, a single image sensor 12b can simultaneously obtain a two-dimensional image for searching for a corresponding point and a two-dimensional image for color mapping.

However, with such an interpolation method, accurate luminance cannot be obtained for all pixels. As a result, as compared with a case where a corresponding point search is performed using a monochrome image, there is a case where a correspondence with a pixel which should originally correspond cannot be obtained, and the accuracy may be reduced.

However, it is possible to perform a visual correction by an optimization process based on an algorithm for reconstructing a three-dimensional shape, thereby realizing an almost accurate three-dimensional shape.

According to the above embodiment, three-dimensional data can be generated with a simple configuration. Since the image pickup device for searching for the corresponding point and the image pickup device for color mapping are shared by one image pickup device 12b, the configuration is simple, and the correspondence between the two-dimensional image and the three-dimensional data is known, so that the alignment at the time of color mapping is performed. Is easy. Also, in the calibration for obtaining the positional relationship between the image input units 12 and 13 required at the time of reconstructing the three-dimensional data, the trouble can be reduced by the small number of the imaging elements. [Second Embodiment] FIG. 8 is a diagram showing an example of a pixel configuration of image sensors 12Bb and 13Bb according to a second embodiment of the present invention.

In the second embodiment, the configurations of the multi-view input camera 5B and the three-dimensional data generator 1 are basically the same as those of the first embodiment. The differences of the multi-view input camera 5B are as follows.

That is, in the first embodiment, the image sensors 12b and 13b have the same resolution, although one is a color CCD and the other is a monochrome CCD. In contrast, in the second embodiment, the resolution of one image sensor 12Bb, which is a color CCD, is higher than the resolution of the other image sensor 13Bb, which is a monochrome CCD. Specifically, the resolution of the image sensor 12Bb is four times the resolution of the image sensor 13Bb. However, other suitable multiple resolutions can be used.

As shown in FIG. 8, one pixel of the monochrome CCD image sensor 13Bb corresponds to four pixels of the color CCD image sensor 12Bb. Therefore, a monochrome CCD can be obtained from the four pixels of the color CCD, that is, image data of each of R, G, G, and B without performing interpolation.
The luminance value corresponding to the luminance value of one pixel can be accurately obtained. As a result, the corresponding point search can be performed more accurately than in the case of the first embodiment,
A highly reliable three-dimensional shape can be reconstructed.

As a method of obtaining the luminance value of the color CCD corresponding to the luminance value of one pixel of the monochrome CCD, as shown in the first embodiment, the luminance value of each pixel of the color CCD is obtained. The luminance values of the four pixels may be added at a predetermined ratio. Even in this case, since it is not necessary to perform interpolation, the luminance value can be accurately obtained, and the corresponding point search can be performed more accurately.

Here, a method of color mapping will be described. FIG. 9 is a diagram showing an example of a three-dimensional shape QF1 reconstructed by the three-dimensional data generation device 1. FIGS. 10 and 11 show a polygon PG forming the three-dimensional shape QF and two color mappings corresponding to each point. FIG. 3 is a diagram illustrating a relationship with a dimensional image (hereinafter, sometimes referred to as a “mapping image QM”). FIG. 10 shows the case where the mapping image QM2 has a low resolution, and FIG. 11 shows the case where the mapping image QM3 has a high resolution.

In FIG. 9, points 1, 2, 3 indicated by black circles
... are the points constituting the three-dimensional shape QF1, that is, the three points of each point
It is dimensional data. Each of the three points forms a polygon PG.

Each point of the three-dimensional shape QF is obtained based on the correspondence between the two two-dimensional images input to the image pickup devices 12Bb and 13Bb, and is obtained by using the color information of the point used at that time. , Each polygon PG of three-dimensional shape QF
Color mapping can be performed.

Referring to FIG. 10, color mapping of the three-dimensional shape QF2 to the polygon PG2 will be described. Each point "1""2""3" forming the polygon PG2
Correspond to the images “4”, “5”, and “6” of the mapping image QM2, respectively. Therefore, the polygon PG2 has
The three images are pasted as shown in the figure.

Referring to FIG. 11, color mapping of the three-dimensional shape QF3 to the polygon PG3 will be described. Each point "1""2""3" forming the polygon PG3
Correspond to the images "4", "5", and "6" of the mapping image QM3, respectively. These images are each composed of four pixels and therefore have a high image resolution. The three images are pasted on the polygon PG3 as shown in the figure.
It can be seen that the resolution of the image color-mapped to 3 is higher than in FIG.

That is, as in the present embodiment, the color CC
By setting the resolution of D higher than the resolution of the monochrome CCD, a high-resolution mapping image QM can be obtained, whereby a high-resolution image can be mapped on the three-dimensional shape QF. Therefore, the visual effect of the color-mapped three-dimensional shape QF is high.
For example, even when the accuracy of the three-dimensional shape QF is poor or when the three-dimensional shape QF has a defect, it is possible to visually improve the high-resolution mapping image QM. [Third Embodiment] FIG. 12 is a diagram showing a configuration of an image input unit 12C according to a third embodiment of the present invention, and FIG. 13 is a diagram showing a configuration of an image input unit 13C.

In the third embodiment, the configurations of the multi-view input camera 5C and the three-dimensional data generator 1 are basically the same as those of the first embodiment. The differences of the multi-view input camera 5C are as follows.

That is, in the first embodiment, a single-chip color CCD is used as the image sensor 12b.
In the third embodiment, as the image sensor 12Cb,
A multi-plate color CCD as shown in FIG. 12 is used.

In FIG. 12, an image input section 12C includes a photographing lens 12Ca, a prism 121, a filter 122 for reflecting the R component, and a filter 12 for reflecting only the G component.
3, and the image sensor 12CbR for each color of R, G, B,
12CbG and 12CbB are provided.

After passing through the photographing lens 12Ca, the prism 1
First, the light path of the light LL incident on 21 is divided by the filter 122. The R component of the light LL is
And is received by the image sensor 12CbR. The G component transmits through the filter 122 and reflects on the filter 123, and is received by the image sensor 12CbG. The B component passes through the filters 122 and 123 and
Light is received by CbB.

In FIG. 13, an image input unit 13C includes a photographing lens 13Ca and an image pickup device 13 which is a monochrome CCD.
Cb. The configuration of the image input unit 13C is the first
The configuration is the same as that of the image input units 13 and 13B of the second embodiment.

Each image sensor 12CbR, 12CbG, 12
The resolution of CbB is the same as the resolution of the image sensor 13Cb. That is, one pixel of each color component of R, G, and B in each color CCD has the same size as one pixel of the monochrome CCD.

Therefore, in the image input unit 12C,
The brightness of each pixel is represented by the image sensors 12CbR, 12CbG, 1
It can be obtained from the R, G, B image data of each pixel of 2CbB by the above equation (1). That is, the luminance value corresponding to the luminance value of one pixel of the monochrome CCD can be accurately obtained without performing interpolation. As a result, the corresponding point search can be performed more accurately, and a highly reliable three-dimensional shape can be reconstructed. Also,
As for the mapping image QM obtained by the image input unit 12C, complete color information can be obtained without interpolation of the color information, and the visual effect is good. [Fourth Embodiment] FIG. 14 is a diagram showing an example of the pixel configuration of image sensors 12Db and 13Db according to a fourth embodiment of the present invention.

In the fourth embodiment, the configurations of the multi-view input camera 5D and the three-dimensional data generator 1 are basically the same as those of the first and third embodiments. The differences of the multi-view input camera 5D from the third embodiment are as follows.

That is, in the third embodiment, the color C
The CD and the monochrome CCD have the same resolution. On the other hand, in the fourth embodiment, the resolution of the image sensors 12DbR, 12DbG, and 12DbB, which are color CCDs, is higher than the resolution of the image sensor 13Db, which is a monochrome CCD.

As shown in FIG. 14, one pixel of the image sensor 13Db, which is a monochrome CCD, is divided into a plurality of pixel areas (four pixel areas in this embodiment) of the image sensors 12DbR, 12DbG, 12DbB, which are color CCDs. Straddles. Therefore, the luminance value of the color CCD corresponding to one pixel of the monochrome CCD can be accurately obtained from the average value of a plurality of pixels.

Then, the image pickup devices 12DbR, 12Db
With G, 12DbB, a high-resolution mapping image QM can be obtained, and thereby a high-resolution image can be mapped. [Fifth Embodiment] FIG. 15 is a block diagram showing a configuration of a multi-view input camera 5E according to a fifth embodiment of the present invention.

In FIG. 15, a multi-view input camera 5E is
It comprises image input sections 12E and 13E, a processing circuit 15E and the like. The image input unit 12E includes a photographing lens 12Ea, a filter 125, and two image sensors 12EbC and 12EbC.
EbK is provided. The image input unit 13E is provided with a photographing lens 13Ea and an image sensor 13Eb which is a monochrome CCD. The configuration of the image input unit 13E includes first to fourth
This is the same as the configuration of the image input units 13 to 13D of the embodiment.

As the image sensor 12EbC, the same single-plate color CCD as the image sensor 12b shown in FIG. 6A is used. Therefore, the image sensor 12EbC is a monochrome C
The resolution is the same as that of the imaging devices 12EbK and 13Eb, which are CDs. Pixels of each color of R, G, and B are allocated on one CCD surface, and image data of only one color is obtained from one pixel. Is obtained. Image sensor 12EbK, 13E
As b, the same monochrome CCD as the image sensor 13b shown in FIG. 6B is used.

The filter 125 divides the light incident on the photographing lens 12Ea into two optical paths, and forms an image sensor 12EbC which is a color CCD and an image sensor 1 which is a monochrome CCD.
Lead to 2EbK. 2 obtained by the imaging device 12EbC
The two-dimensional image is used for color mapping, and the image sensor 1
The two-dimensional image obtained by 2EbK is used for corresponding point search. The two-dimensional image obtained by the image sensor 13Eb is used for searching for a corresponding point.

That is, the monochrome C
The imaging devices 12EbK and 13Eb which are CDs are used,
Separately, an image sensor 12EbC, which is a single-chip color CCD, is used for color mapping.

The processing circuit 15E includes each image sensor 12Eb.
C, 12EbK, and memories 152C, 152K, and 1 for storing image data output from 13Eb, respectively.
53 are provided. The CPU 151 controls the memory 15
The image data stored in 2C, 152K, and 153 are processed, and processing for searching for corresponding points, generating a three-dimensional shape QF, and performing color mapping is performed.

In the present embodiment, two image pickup devices 12Eb which are composed of monochrome CCDs and have the same resolution
Since the corresponding point search is performed using the two-dimensional image obtained by K and 13Eb, a highly reliable ideal corresponding point search is performed, and the three-dimensional shape QF can be accurately reconstructed. [Sixth Embodiment] In the sixth embodiment, the configurations of the multi-view input camera 5F and the three-dimensional data generation device 1 are basically the same as those of the fifth embodiment. Multi-view input camera 5
The differences of F are as follows.

That is, in the fifth embodiment, the color C
Although the resolution of the CD is the same as the resolution of the monochrome CCD, in the sixth embodiment, the resolution of the color CCD is higher than the resolution of the monochrome CCD. In particular,
For example, the image sensor 12B which is a color CCD shown in FIG.
b and the image sensor 13Bb which is a monochrome CCD.

That is, the multiview input camera 5F of the sixth embodiment is a combination of the features of the multiview input camera 5E of the fifth embodiment and the features of the multiview input camera 5B of the second embodiment. It can be said.

Therefore, according to the sixth embodiment, a high-resolution mapping image QM can be obtained, and a high-resolution image can be mapped on the three-dimensional shape QF. Therefore, the visual effect of the color-mapped three-dimensional shape QF is enhanced. [Seventh Embodiment] In the seventh embodiment, the configurations of the multi-view input camera 5G and the three-dimensional data generation device 1 are basically the same as those of the fifth embodiment. Multi-view input camera 5
G is as follows.

That is, in the fifth embodiment, a single-chip color CCD is used, but in the sixth embodiment, a multi-chip color CCD as shown in FIG. 12 is used.

That is, the multiview input camera 5G of the seventh embodiment combines the features of the multiview input camera 5E of the fifth embodiment with the features of the multiview input camera 5C of the third embodiment. It can be said.

Therefore, according to the seventh embodiment, the luminance value corresponding to the luminance value of one pixel of the monochrome CCD can be accurately obtained without performing the interpolation, whereby the corresponding point search can be further performed. Can be done exactly,
A highly reliable three-dimensional shape can be reconstructed. [Eighth Embodiment] In the eighth embodiment, the configurations of the multi-view input camera 5H and the three-dimensional data generation device 1 are basically the same as those of the seventh embodiment. Multi-view input camera 5
The differences between H are as follows.

That is, in the seventh embodiment, the resolution of the multi-plate type color CCD and that of the monochrome CCD are the same.
On the other hand, in the eighth embodiment, the resolution of the multi-plate color CCD is higher than the resolution of the monochrome CCD.

That is, the multiview input camera 5H of the eighth embodiment combines the features of the multiview input camera 5G of the seventh embodiment with the features of the multiview input camera 5D of the fourth embodiment. It can be said.

Therefore, according to the eighth embodiment, the luminance value of a multi-plate color CCD corresponding to one pixel of a monochrome CCD can be accurately obtained from the average value of a plurality of pixels. Then, a high resolution mapping image QM can be obtained by the multi-plate type color CCD, and thereby a high resolution image can be mapped.

According to each of the above-described embodiments, only one of the plurality of image input units is configured so that the specifications regarding the image quality are different from those of the other image input units. With such a configuration, a two-dimensional image that can be used for both the corresponding point search and the color mapping can be obtained at the same time after one-shot shooting.

The eight embodiments have been described above.
Although a part of the description of the effects of the respective embodiments is omitted, it is apparent from the effects of other related embodiments. Also, various combinations of these embodiments are possible.

In the embodiment described above, a case where a still image is input as a two-dimensional image has been described.
The same applies to the case where a moving image is input as a two-dimensional image. In addition, the configuration, shape, material, processing content, order, and the like of the entire information processing device 4, the multi-view input cameras 5 to 5H, and the three-dimensional data generation device 1 or each unit may be appropriately changed according to the spirit of the present invention. Can be.

[0085]

According to the present invention, a two-dimensional image for three-dimensional data and a two-dimensional image for color mapping can be simultaneously acquired by one photographing, and multi-view data having a simple structure can be obtained. An input device can be provided.

According to the second aspect of the present invention, the structure is simpler. According to the third, fifth, seventh, and ninth aspects of the present invention, it is possible to obtain a high-resolution mapping image, and to enhance the visual effect of a color-mapped three-dimensional shape.

According to the third to ninth aspects of the present invention, it is possible to accurately obtain a luminance value required for a corresponding point search without performing interpolation. According to the fifth to ninth aspects of the present invention, a highly reliable ideal corresponding point search is performed, and a three-dimensional shape can be accurately reconstructed.

[Brief description of the drawings]

FIG. 1 is a perspective view of a multi-view input camera according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a configuration of a three-dimensional data generation device including a multi-view input camera.

FIG. 3 is a diagram conceptually showing a situation when a subject is photographed using a multi-view input camera.

FIG. 4 is a diagram for explaining a concept of a corresponding point search.

FIG. 5 is a block diagram illustrating a configuration of a multi-view input camera.

FIG. 6 is a diagram illustrating an example of a pixel configuration of an image sensor.

FIG. 7 is a diagram illustrating extracted pixels of each color of a single-plate color CCD.

FIG. 8 is a diagram illustrating an example of a pixel configuration of an image sensor according to a second embodiment of the present invention.

FIG. 9 shows 3 reconstructed by the three-dimensional data generating apparatus.
It is a figure showing an example of a three-dimensional shape.

FIG. 10 is a diagram illustrating a relationship between a polygon and a low-resolution mapping image.

FIG. 11 is a diagram illustrating a relationship between a polygon and a high-resolution mapping image.

FIG. 12 is a diagram illustrating a configuration of an image input unit according to a third embodiment of the present invention.

FIG. 13 is a diagram illustrating a configuration of an image input unit.

FIG. 14 is a diagram illustrating an example of a pixel configuration of an image sensor according to a fourth embodiment of the present invention.

FIG. 15 is a block diagram showing a configuration of a multi-view input camera according to a fifth embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 3D data generation apparatus 5-5H Multi-view input camera (multi-view data input apparatus) 12 Image input section (one image input section) 13 Image input section (other image input sections) 12b Image sensor (single board type) 12Bb Image sensor (high-resolution single-chip image sensor) 12Cb Image sensor (multi-chip image sensor) 12Db Image sensor (high-resolution multi-chip image sensor) 12EbC Image sensor (single-chip image sensor) 12EbK Image sensor (image sensor for acquiring monochrome two-dimensional image) 13b, 13Bb, 13Cb, 13Db, 13Eb, 1
3Fb, 13Gb, 13Hb Image sensor (image sensor for acquiring monochrome two-dimensional image) 151 CPU (calculator) Q Subject QL, QR Two-dimensional image

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) // H04N 9/09 G06F 15/64 310 5C065 (72) Inventor Satoru Hirose Azuchi-cho, Chuo-ku, Osaka-shi, Osaka No. 2-313 Osaka International Building Minolta Co., Ltd. F-term (reference) 2F065 AA04 AA06 BB05 CC16 DD06 FF05 GG10 HH02 JJ03 JJ05 JJ26 LL12 LL21 LL30 QQ24 2F112 AC06 BA05 CA08 CA12 FA03 FA21 2H059 AA08 5B0AB AB07A04 BC 5C061 AA20 AA25 AB06 AB08 5C065 AA03 BB48 CC01 CC08 DD02 DD19 EE01 EE05 EE06 GG30

Claims (10)

[Claims] 1. A multi-view data input device comprising a plurality of image input units for inputting a two-dimensional image of a subject from a plurality of different viewpoint positions, wherein one of the plurality of image input units A multi-view data input device, wherein one image input unit is configured so that specifications regarding image quality are different from those of the other image input units. 2. The image input device according to claim 1, wherein the one image input unit is provided with a single-chip image sensor having a color filter and capable of acquiring a color two-dimensional image. The multi-view data input device according to claim 1, further comprising an imaging element configured to acquire the two-dimensional image. 3. The multi-view data input device according to claim 2, wherein an image sensor provided in said one image input unit has a higher resolution than an image sensor provided in said another image input unit. 4. A multi-chip image sensor having a color filter and capable of acquiring a color two-dimensional image is provided in the one image input unit, and the other image input unit is provided with a monochrome image sensor. The multi-view data input device according to claim 1, further comprising an image sensor that acquires the two-dimensional image. 5. The multi-view data input device according to claim 4, wherein an image sensor provided in said one image input unit has a higher resolution than an image sensor provided in said another image input unit. 6. The single image input section is provided with a single-chip image sensor having a color filter and capable of acquiring a color two-dimensional image, and an image sensor for acquiring a monochrome two-dimensional image. The multi-view data input device according to claim 1, wherein the other image input unit is provided with an image sensor for acquiring a monochrome two-dimensional image. 7. The multi-view data input according to claim 6, wherein a color image sensor provided in said one image input unit has a higher resolution than an image sensor provided in said another image input unit. apparatus. 8. The one image input section is provided with a multi-plate type image sensor having a color filter and capable of acquiring a color two-dimensional image, and an image sensor for acquiring a monochrome two-dimensional image. The multi-view data input device according to claim 1, wherein the other image input unit is provided with an image sensor for acquiring a monochrome two-dimensional image. 9. The multi-view data input according to claim 8, wherein a color image sensor provided in said one image input unit has a higher resolution than an image sensor provided in said another image input unit. apparatus. 10. The multi-view data input device according to claim 1, further comprising a calculating unit that calculates three-dimensional data of the subject based on the two-dimensional image.