JPH06109449A - Plural-visual point three-dimensional image input device - Google Patents

Abstract

PURPOSE:To obtain the device characterized by that a storage capacity value is small, that a transmission time is short, that the distance accuracy of a long distance image is excellent and that there is not parallax when a line of sight is changed. CONSTITUTION:The distance images S32-1-S32-n outputted from 3D cameras 30-1-30-n are separated into a short distance image S34 and a long distance image 35 by a distance image separator 33. Further, the long distance image S35 is separated into a middle distance image S37 and a long distance image S38 by re-searching the mutual corresponding points of the 3D cameras by a corresponding point re-searching device 36. The short distance image 34 and the middle distance image 37 are selected as images effective for images having the max. numbers of subject expression pixels by a selector 39 and the selected effective distance image S40 is stored in a memory 40. The long distance image 38 is converted to a noneffective distance image 42 by an one-layer face sequence arrangement device 41 to be stored in the memory 40.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to three-dimensional image information technology such as computer vision (CV) and computer graphics (CG). The present invention relates to a multi-viewpoint three-dimensional image input device for displaying an image on a screen, and more particularly to a multi-viewpoint three-dimensional image input device capable of reducing an image storage capacity value.

[0002]

2. Description of the Related Art Conventionally, as a technique in such a field,
For example, there are those described in the following documents. Reference 1; Journal of Television Society, 45 [4] (1991).
P. 446-452 Document 2; Journal of the Television Society, 45 [4] (1991).
P. 453-460 Conventionally, a three-dimensional image input method includes a passive method (passive method) and an active method (active method). The active method irradiates the target with energy (light wave, radio wave, sound wave) that is skillfully controlled to obtain three-dimensional information and has some meaning with respect to its shape pattern, density, spectrum, etc. Refers to the method. On the other hand, the passive method means a measurement that does not use meaningful energy for measurement, even if the target is normally illuminated. Generally speaking, the active method is
The measurement is more reliable than that of the passive method. A typical passive method is a stereo image method, which is shown in FIG.

FIG. 2 shows the conventional 3 described in the above document 2.
It is explanatory drawing of the stereo image method which is one of the three-dimensional image input systems. In this stereo image method, two cameras 1 and 2 which are two-dimensional image input devices are arranged at a predetermined distance from each other,
The difference between the imaging positions of the subject 3 taken by the left and right cameras 1 and 2, that is, the phase difference is used, and the subject 3 is triangulated.
Is a way to measure the distance to.

FIG. 3 is an explanatory diagram of two images, a grayscale image of a signal and a distance image obtained by the stereo image method of FIG. The grayscale image is a color or monochrome image obtained by the cameras 1 and 2 in FIG. The distance image is an image regarding a three-dimensional position, and each pixel in the matrix data has information regarding the depth of the object (subject 3). From such a grayscale image and a range image, a stereoscopic image is displayed by a binocular fusion method using a polarization filter, or a stereoscopic image is displayed using a lenticular plate. An example of a stereoscopic image display is shown in FIG.

FIG. 4 shows the conventional 3 described in the above document 1.
It is a figure which shows the principle of the multi-lens type lenticular system which is one of the three-dimensional image display systems. The multi-lens lenticular system is a lenticular plate 10 including a plurality of kamaboko-shaped lens plates.
Is used to arrange the left and right images in stripes on the focal plane of each lens plate. A, b, in one lens board
c, ..., the portion of the f, respectively _{a 1, b 1, c 1} , ...,
f ₁ striped multiview image captured from multiple directions of 1
1 is displayed. By the action of the lens plate, the striped multi-view image 11 in each direction enters the left and right eyes 12 and 13 separately, and if the viewpoint is moved, a horizontal stereoscopic image can be viewed.

[0006]

However, the apparatus having the above structure has the following problems. (1) Lenticular plate 1 as a three-dimensional image display system
When 0 is used, a two-dimensional image can be viewed three-dimensionally,
When the observer's line of sight is changed, the appearance of the object is as narrow as 5 to 10 cm in the left-right direction and ± 30 cm in the front-back direction when viewed at a distance of about 5 m as a stereoscopic viewing area. Also,
In the binocular fusion method, there is a problem that the image itself does not change even if the line of sight is changed, only with a stereoscopic representation of a two-dimensional image. (2) Therefore, in order to solve the above-mentioned problems such as the observation field of view being narrow, and the image not changing even if the line of sight is changed, Japanese Patent Application No. 4-192272 (the prior proposal) ) Proposed a three-dimensional image input device. In this proposal, a three-dimensional camera that can output two images, a grayscale image and a range image (hereinafter referred to as 3D camera)
At least two three-dimensional image input devices each composed of are arranged so that a display image can be synthesized from a grayscale image and a distance image respectively output from each of the 3D cameras to display a three-dimensional image. I have to. (3) However, when two 3D cameras are used as in the above proposal, there is a problem that the subjects that can be imaged are limited, and blind spots occur especially in subjects having concave portions. In order to compensate for this blind spot, as shown in FIG.
A method of increasing the number of D cameras can be considered. Figure 3
It is a schematic block diagram of a multi-viewpoint three-dimensional image input device previously proposed by the present inventor when the number of D cameras is increased from two to five. This multi-viewpoint three-dimensional image input device includes five 3D cameras 20-1 to 20-5, and their optical axes H20-1 to H20-5 are arranged so as to gather at an intersection K. Each 3D camera 20-1 to 20-5
Is a grayscale image 20-1v to 20-5v and a distance image 20 which express an image of a subject (not shown).
It has a function of outputting signals of -1r to 20-5r. In this kind of multi-viewpoint three-dimensional image input device,
An image of a subject is input by each of the 3D cameras 20-1 to 20-5, grayscale images 20-1v to 20-5v and distance images 20-1r to 20-5r are output, and stored in a storage device (memory). . These stored grayscale images 20-
The 1v to 20-5v and the distance images 20-1r to 20-5r are image-synthesized to display a three-dimensional image. In particular, since the number of 3D cameras 20-1 to 20-5 is increased, it is possible to reduce the blind spot of an object having a concave portion, and it is possible to completely prevent the blind spot if the number of 3D cameras is further increased. (4) However, in the device of FIG. 5, each 3D camera 20-
Since a grayscale image memory and a range image memory are provided for each 1 to 20-5, an enormous storage capacity value can be obtained when, for example, data is recorded and stored, or when data is transmitted using a line or the like. There is a problem in that it is necessary or requires a large amount of transmission time. Also, each 3D camera 20-1 to 20-
Reference numeral 5 has at least two two-dimensional image input devices each including a solid-state image sensor such as a charge-coupled device (hereinafter referred to as CCD), which are arranged at narrow intervals. Therefore, as the range images 20-1r to 20-5r output from the respective 3D cameras 20-1 to 20-5, the results of the two-dimensional image input devices arranged at narrow intervals are used, and therefore, in the near range image. Function sufficiently, but in long-distance images, the distance accuracy is poor, and the 3D cameras are arranged at wide intervals, which causes a parallax problem, which is still technically sufficiently satisfactory. It has been difficult to provide a multi-viewpoint three-dimensional image input device.
The present invention has solved the problems that the above-mentioned conventional art has such as a large storage capacity value, a long transmission time, and parallax when changing the line of sight caused by poor distance accuracy of a long-distance image. A multi-viewpoint three-dimensional image input device is provided.

[0007]

In order to solve the above-mentioned problems, the first invention inputs an image of an illuminated object and outputs signals of a grayscale image and a distance image representing the object, respectively. In a multi-viewpoint three-dimensional image input device in which a plurality of three-dimensional image input devices are arranged so that their optical axes intersect at one point, a first separating means and a second separating means are provided. The first separating means has a function of separating the short distance image having a desired phase difference or more and the long distance image having a desired phase difference or less based on the distance images from the plurality of three-dimensional image input devices. is doing. The second separating means performs a corresponding point search again on the long-distance image by using the grayscale images of the plurality of three-dimensional image input devices, and extracts a middle-distance image and a non-phase-difference image which are phase difference images. It has a function of separating it into a certain long-distance image. In the second invention, the long-distance image of the first invention is stored in a storage unit different from the short-distance image and the medium-distance image. In the third invention,
In the second invention, the long-distance image is provided with an arrangement means for sequentially arranging it in a predetermined storage means address according to the arrangement of the plurality of three-dimensional image input devices. In the fourth invention, the arrangement angle and the field angle of each of the three-dimensional image input devices of the second invention are kept constant. In a fifth aspect based on the second aspect, a setting means for keeping the arrangement angle and the field angle of each of the three-dimensional image input devices in a constant relationship and for identifying that an image is captured at a certain distance or more; Selecting means for selecting, as an effective distance image, an image having the largest number of pixels expressing the subject from the short distance image and the medium distance image. In a sixth aspect based on the second aspect, the storage capacity values of the short distance image and the middle distance image are made larger than the storage capacity values of the long distance image. In the seventh invention,
In the third invention, the long-distance images sequentially arranged are provided with conversion means for converting the pixel density into a predetermined number of screens so as to be matched with a predetermined storage capacity value. According to an eighth invention, in the sixth or seventh invention, a means for operating the three-dimensional image input device in conjunction with each other so that the arrangement angle and the angle of view are matched with the storage capacity value of the long-distance image, It is provided.

[0008]

According to the first aspect of the invention, as described above, the plural viewpoints 3
Since the three-dimensional image input device is configured, the first separating means is
The distance image of each three-dimensional image input device is separated into a short distance image and a long distance image. The second separating means performs a corresponding point search again on the separated long-distance image by using the grayscale images of the plurality of three-dimensional image input devices, and separates the medium-distance image and the long-distance image. In this way, by separating the medium-distance image and the long-distance image from the long-distance image and obtaining the medium-distance image, it is possible to prevent the occurrence of parallax. According to the second invention,
The long-distance image is stored in a storage unit different from the short-distance image and the medium-distance image. Then, by properly setting the capacity of the storage means for the long-distance image and the capacity of the storage means for the short-distance image and the medium-distance image, it is possible to eliminate waste of the storage capacity value. Furthermore, when the display is made in line with the line of sight of the observer, the display can be performed by simply specifying the address without performing complicated calculation. According to the third aspect of the invention, the arranging means sequentially arranges the long-distance images at predetermined storage means addresses, and has a storage capacity having a storage capacity determined by, for example, the angle of view and the arrangement angle of the three-dimensional image input device. To memorize. As a result, even if the number of three-dimensional image input devices increases, it is possible to store a medium-distance image in which the storage capacity value is not wasted and parallax does not occur. According to the fourth aspect, the arrangement angle and the angle of view of each three-dimensional image input device are kept within a certain allowable value, and the storage capacity value in the storage unit of the long distance image can be reduced. According to the fifth aspect, the setting means keeps the arrangement angle and the field angle of each three-dimensional image input device in a constant relationship and identifies that the image is captured over a certain distance. The selecting means extracts only effective pixel signals from the short-range image and the medium-range image and stores them in the storage means according to the desired shortest imaging distance by the output of the setting means. As a result, the storage capacity value of the storage means can be reduced. According to the sixth aspect, since the storage capacity values of the short-distance image and the medium-distance image are larger than the storage capacity value of the long-distance image, the imaging of the subject corresponding to the distance up to the large storage capacity value is performed. Is possible. According to the seventh aspect, the conversion means sets the predetermined number of screens so that the long-distance image can be matched with the predetermined storage capacity value determined by the angle of view and the arrangement angle of the three-dimensional image input device. The pixel density is converted to. As a result, a margin is created in the initially set minimum object distance, and the degree of freedom in installing the multi-viewpoint three-dimensional image input device at the time of image pickup is improved.
According to the eighth aspect, when the zoom ratio of a long-distance subject is changed and an image is taken, the angle of view of the three-dimensional image input device is set so as not to exceed the storage capacity value of the storage unit that stores the long-distance image. The arrangement angle changes in conjunction with each other, and the occurrence of omission of a long-distance image due to a change in zoom ratio is prevented. Therefore, the above problem can be solved.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment FIG. 1 is a block diagram showing the configuration of a multi-viewpoint three-dimensional image input device showing a first embodiment of the present invention. This device includes a plurality of three-dimensional image input devices (for example, 3D cameras) 30-
1-30-n and their optical axes H30-1 to H30
-N is arranged so as to intersect at the intersection point k. Each of the 3D cameras 30-1 to 30-n has a function of inputting an image of a subject and outputting signals of grayscale images S31-1 to S31-n and distance images S32-1 to S32-n expressing the subject. And their grayscale images S31-1 to S31
-N and range images S32-1 to S32-n are normally grayscale image memories 31-1 to 31-n and range image memory 32-
1 to 32-n are respectively stored.
At the output side of the grayscale image memories 31-1 to 31-n and the range image memories 32-1 to 32-n, the range image separation device 3 is provided.
3 is connected to the output side of the short-distance image memory 3
4 and the long-distance image memory 35 are connected. The range image separation device 33 uses the plurality of grayscale images S31-1 to S31.
-N and range images S32-1 to S32-n are separated into a range image (short range image) S34 having a desired phase difference or more and a range image (long range image) S35 having a desired phase difference or less, and these are short range. The image memory 34 and the long-distance image memory 35 each have a function of storing. Long-distance image memory 35
The intermediate distance image memory 37 and the long distance image memory 38 are connected to the output side of the through the corresponding point re-search device 36, and the subject expression maximum pixel number selection device 39 is further provided on the output side of the short distance image memory 34. It is connected. The corresponding point re-search device 36 performs a corresponding point search again on the distance image S35 by using a plurality of grayscale images S31-1 to S31-n in order to make the long distance image S35 a highly accurate distance image again. The intermediate distance image S37, which is a phase difference image when there is a phase difference shift, and the long distance image S38, which is a non-phase difference image with no phase difference shift, are separated from the medium distance image memory 37. The distance image memory 38 has a function of storing each. Object representation maximum pixel number selection device 39
Is from the short-range image S34 and the medium-range image S37,
It has a function of selecting the image having the largest number of pixels representing the subject as the effective distance image S40. The distance image memory 40 is connected to the output side of the object expression maximum pixel number selection device 39, and the non-distance image memory 42 is connected to the output side of the long distance image memory 38 via the one-layer screen sequential arrangement device 41. Has been done. Distance image memory 40
Stores the distance image S40 output from the subject expression maximum pixel number selection device 39 according to a certain rule,
For example, it is a memory having a five-layer structure. The one-layer screen sequential arrangement device 41 stores the long-distance image memory 38 of each of the 3D cameras 30-1 to 30-n stored in the long-distance image memory 38 into the long-distance image memory 38.
The non-distance image S4 is obtained by sequentially arranging the D cameras 30-1 to 30-n on the screen one by one according to the arrangement angle and the angle of view.
2 is generated and stored in the one-layer structure non-distance image memory 42 having a desired storage capacity.

Next, the operation of the apparatus shown in FIG. 1 will be described with reference to FIGS. FIG. 6 is an operation explanatory diagram of FIG. 1, and the pattern of processing of each image in the apparatus of FIG. 1 will be described with images of two 3D cameras with reference to this figure. Figure 6
, The i-th 3D camera 30-i and the j-th 3D
Distance images S32-i and S32-j are output from the camera 30-j, respectively. Each distance image S32-i, S32-
A grayscale image (not shown) is located behind them corresponding to j. The distance images S32-i and S32-j are
The vertical dimension is V and the horizontal dimension is H. Among them, a short distance object 51 and long distance objects 52 and 53 are included.
Exists. The range images S32-i and S32-j are
The distance image separation device 33 of FIG.
i, S34j and the long-distance image S35 are separated. The threshold value of the distance value to be separated is changed according to the processing capability of the corresponding point re-search device 36 in FIG.
Minimum distance resolution at 30-j (ie phase difference of 1 pixel)
Is set as a threshold value, and a distance shorter than the threshold value
34i, S34j, the long distance is allocated to the long distance image S35. In this example, since the object 51 has a short distance value, the objects 52 and 53 have a long distance value, and thus the object 51 and the object 52 are allocated to the long distance image S35.

Next, the long-distance image S35 is converted into a high-precision distance image again by the corresponding point re-search device 36 in FIG. 1, and separated into a medium-distance image S37 and a long-distance image S38. The reason for the conversion by the corresponding point re-search device 36 is as follows. That is, the distance image S32-i or S32-
j and the like are obtained by the 3D cameras 30-i, 30-j, ... Which are configured by arranging at least two two-dimensional image input devices. However, at this time, since the two-dimensional image input device is arranged at a narrow interval, a subject at a certain distance or more becomes a distance image in a device that cannot be decomposed any further. On the other hand, in the multi-viewpoint three-dimensional image input device in which the respective 3D cameras 30-i, 30-j, ... Are arranged, the 3D cameras 30-i,
Since the arrangement interval of 30-j, ... Is sufficiently wider than the arrangement interval of one 3D camera, parallax occurs even at a distance that cannot be resolved by the distance image S32-i or S32-j in this device. Therefore, by the corresponding point re-search device 36 of FIG.
Further, processing for converting to a more accurate range image is performed. However, the corresponding point re-search in this case is performed by the 3D camera 30.
-I, 30-j arrangement, the 3D cameras 30-i, 30-
Since the arrangement interval of j and the arrangement interval of the two-dimensional image input device are known, the corresponding point re-search range is limited.
Therefore, the process is easier than the conventional corresponding point search as shown in FIG. In the medium-distance image S37 and the long-distance image S38 separated by the corresponding point re-search device 36, the subject 52 is divided into the medium-distance image S37 and the subject 53 is divided into the long-distance image S38. The standard of this distribution is as follows, for example. That is, the image in which the phase difference is detected is inserted into the medium distance image S37, and the image with the distance resolution or less (the phase difference is 1 pixel or less) is inserted into the far distance image S38.

The long-distance image S38 is converted into a non-distance image S42 by the single-layer screen sequential arrangement device 41 of FIG. 1 and stored in the non-distance image memory 42 of FIG. How to store at this time is to know which address (horizontal direction) on the screen of the non-distance image S42 corresponds to the first pixel of the long-distance image S38 from the arrangement of the 3D cameras 30-i and 30-j. Therefore, for example, it is stored in the corresponding portion indicated by the broken line in the non-distance image S42. Other 3D camera 30
The long-distance image of -j is also stored at a predetermined position, as described above. In this case, a long-distance image of a certain 3D camera and a long-distance image of a different 3D camera may overlap, but in that case, priority is given to the non-distance image memory 42 by the arrangement of the 3D cameras, and the priority image is stored. is doing.
At the boundary of prioritization, the method shown in FIGS. 7 and 8 is used.

FIG. 7 is an explanatory view of the overlap of the angle of view of the 3D camera at a long distance. FIG. 8 shows the non-distance image memory 4
FIG. 3 is an explanatory diagram of an example of how to write data in 2; 7 is 3D
The arrangement angle of the cameras 30-i and 30-j, Δδ, is the arrangement angle difference, and Δδ / (δ / m) ≠ M (where M is an integer). An enlarged view of the portion A of FIG. 7 is shown in FIG. As shown in FIG. 7, the difference Δδ in the arrangement angle δ is 1
If it is divisible by the angle expressing the pixel, the respective prioritized image data is written in the non-distance image memory 42. However, if it is not divisible, the overlapping degree of both is compared, and the one with the larger overlap is the boundary. I try to write at the address. Although an image shift occurs at this seam, the seam position in the horizontal direction is randomly changed depending on the position in the vertical direction V, so that the deterioration of the image quality can be almost ignored. All of these functions are provided in the one-layer screen sequential arrangement device 41 of FIG.

On the other hand, the short-distance image S34i and the medium-distance image S37 shown in FIG. 6 are selected by the subject expression maximum pixel number selection device 39 shown in FIG. 1 from the images of the respective 3D cameras 30-i and 30-j. Processing is performed in which only the signals of the 3D camera having a large number of pixels to be expressed are validated and the signals of the remaining 3D cameras of the subject are discarded. The distance image S40 selected by the object expression maximum pixel number selection device 39 is stored in the distance image memory 40 of FIG.
The image remaining on each of the 3D cameras 30-i and 30-j becomes an effective image S39. In the case of this example, the short-distance image S34
The subject 51 in j and the subject 52 in the medium-distance image S37 are consequently stored in the distance image memory 40. Distance image memory 40 for storing this distance image S40
Has a configuration as shown in FIG.

FIG. 9 is an illustration of an example of how to write to the distance image memory 40. As shown in this figure, the distance image memory 40 sequentially stores the distance images S40 in the images of the respective 3D cameras 30-i and 30-j. Values, the horizontal (X) and vertical (Y) addresses of those image values, and the 3D camera NO. (That is, the angle of view δ and 3
The arrangement angle θi) at which the D camera looks at the intersection K is stored in layers. It should be noted that, although the layered structure is described here, any arrangement may be used as long as the above values form one set. Next, FIGS. 10 (a), (b), and FIG.
With reference to (a), (b), and FIG. 12, the distance image memory 40 storing the distance image S40 and the non-distance image S are stored.
The storage capacity of the non-distance image memory 42 for storing 42 will be described. 10A and 10B show the non-distance image S4.
2A is an explanatory diagram of FIG. 2A, in which FIG.
The explanatory diagram in which the pixel cannot be decomposed and FIG. 14B is an explanatory diagram when the non-distance image S42 cannot be decomposed in one pixel, and shows an example of calculating the distance when the image is directly in front. In FIG. 10A, δ is the angle of view, m is the number of pixels, and k ₁ is an arbitrary coefficient. Figure 11
(A), (b) is a figure explaining the example of calculation of the required number of screens of non-distance image S42, the figure (a) is an example without a gap,
And FIG. 6B is a diagram showing an example with a gap. 12
[Fig. 7] is an explanatory view of a minimum value of a view angle.

First, regarding the non-distance image S42 stored in the non-distance image memory 42, the image S42 is a subject at a distant distance that does not cause an error even when viewed between the 3D cameras 30-1 to 30-n. The information is stored.
Explaining with reference to FIGS. 10A and 10B, this figure shows an example in which a semicircle is drawn at the center point O ₀ of the 3D camera arrangement according to the present embodiment and the subject is viewed by the 3D cameras at both ends. The distance at which the points a ₁ and a ₂ on the semicircle cannot be separated as one pixel is the far distance l described above, and is represented by the equation (4) based on the following equations (1), (2) and (3). It

[0017]

[Equation 1] In FIG. 10B, the inseparable angle x is the largest angle in front of the 3D camera arrangement direction. Therefore, if this angle x is smaller than the value obtained by dividing the angle of view δ by the number of horizontal pixels m. Well, the result is equation (4).
Although the coefficient k ₁ is arbitrary in the equation (4), the coefficient k ₁ may be larger than ₁ if the spatial frequency of the distant object is large.

Distance images S32-i, S as shown in FIG.
Since 32-j is an image that cannot be separated into one pixel, there is no problem in handling that image as if it was recorded on a semicircle as shown in FIG. Therefore, point c on this semicircle
It can be understood that the storage capacity value (the number of stored pixels) of the non-distance image memory 42 may be set with an angle representing from the point to the point d. From the relationship between the angle of view δ and the 3D camera arrangement angle θ, the angles θ1 and θ2 necessary for expressing them are found in FIG.
(A) and (b). FIG. 11 (a) shows 3D at both ends of a line perpendicular to the straight line of the semicircle (center line of view of this device).
This is an example in which the line outside the angle of view of the camera is open (larger than 2 / π), and FIG. 7B is an explanatory diagram of a small example. In the case of FIG. 11B, the lines outside the angle of view of the 3D cameras at both ends intersect at the point Q.

In both FIGS. 11A and 11B, the required angle θ
1 and θ2 are represented by the following equations (5) and (6) and have the same equation. [theta] 1 = 2 [delta]-([delta]-[pi] +2 [theta]) = [delta] + [pi] -2 [theta] ... (5) [theta] 2 = 2 [delta] + ([pi] [delta] -2 [theta]) = [delta] + [pi] -2 [theta] ... It can be considered that the object has converged to the point 0, and equations (5) and (6)
Is represented. Therefore, the required number of pixels M is obtained by the following equation (7), and the product of the number of pixels H of one 3D camera and the number of pixels of the equation (7) is the horizontal in the non-distance image memory 42 as shown in FIG. This is the number of pixels stored in the direction (m ₁ H). The vertical direction is the number V of pixels in the vertical direction of one 3D camera.

[0020]

[Equation 2] Further, in the example of FIG. 11 (b), since an insensitive part occurs farther than the point Q, in the case of the usage like this example, two 3D units are used.
The camera alone is not enough, other 3D that complements the blind spots
It is necessary to use a camera. In order to complement this insensitive portion, it is sufficient to satisfy the condition of Expression (8) as shown in FIG.

[0021]

[Equation 3] In FIGS. 12 and (8), θ is the camera arrangement angle, and δ
Is the angle of view of one 3D camera, n is the number of 3D cameras to be arranged, and δ _min is the minimum angle of view that can be taken by the angle of view of one 3D camera. As is clear from the equation (8), the minimum angle of view δ
_{When min} is used, n number of screens is required in the non-distance image memory 42. Next, with reference to FIG. 13, a distance image memory 40 for storing the distance image S40.
The storage capacity of will be described. FIG. 13 is an explanatory diagram of the required number of screens of the distance image memory 40 at a short distance.
In this figure, for example, five 3D cameras 30-1 to 30-
5 is a diagram in which optical axes H30-1 to H30-5 of No. 5 are arranged at equal intervals on an arc centering on an intersection K. FIG. O _{1 to} O ₅ are the center points of the 3D cameras 30-1 to 30-5, and the distance between the center points O ₁ and O ₅ at both ends is L ₀ . The distance L is calculated from the following equation (9) when measured from the center point O ₃ .

[0022]

[Equation 4] This means that the number of screens required increases when viewing a subject at a short distance, but the number of screens can be reduced when viewing a subject at a certain distance or more. In FIG. 13, two sides, three sides,
An example of the distance is shown as 5 planes. Figure 11
Up to the distance of the point Q in (b), the storage capacity (m ₀ H) of the distance image memory 40 shown in FIG. 6 can be determined by the relationship of the equation (9). In the case of points Q and beyond (parts other than the triangle surrounded by points QC ₁ C ₂ ) shown in FIG. 11B, the long distance image S
Since the medium distance image S37 of 35 is also included, the distance image memory 40 must have at least the storage capacity of the non-distance image memory 42 to store all the distance images S40. Since the distance value of a subject is generally indefinite, in order to store all of the ranged image S40 as described above, the storage capacity of the ranged image memory 40 is lower than the storage capacity of the non-ranged image memory 42. It will not happen. Note that the meaning of reducing the number of screens n in equation (9) is
This is because the number of pixels for spatially sampling the surface depending on the distance can be represented by the number of screens.

As described above, the 3D cameras 30-1 to 30
When the angle of view δ of −5, the arrangement angle θ of the 3D cameras 30-1 to 30-5, and the minimum imaging distance are determined, the distance image S40 and the non-distance image S42 of FIG. It is possible to reduce the storage capacity. When the distance is shorter than the minimum distance, a flag is set on the short distance image S34 in FIG. 1, the flag is sent to the distance image memory 40, and that effect is stored and displayed. This is 3D camera 3
When the number of 0-1 to 30-5 is further increased, the reduction ratio can be increased.

The present embodiment has the following advantages. (1) The distance image separation device 3 converts the distance images S32-1 to S32-n output from the 3D cameras 30-1 to 30-n.
In 3 the short distance image S34 and the long distance image S35 are separated,
Furthermore, only the long-distance image S35 is processed by the corresponding point re-search device 3
Since the corresponding points are re-searched between the three-dimensional cameras by 6 to separate the medium-distance image S37 and the long-distance image S38, an accurate distance image can be obtained in a shorter time than the conventional one. (2) The long-distance image S38 is displayed on the one-layer screen sequential arrangement device 41.
Since the non-distance image S42 is converted into the non-distance image S42 and the non-distance image S42 is handled separately from the ranged image S40, a simple calculation can be performed without performing a complicated calculation when displaying the image in accordance with the line of sight of the observer. Since it can be displayed only by specifying the address, the calculation time before displaying can be shortened.

Second Embodiment FIG. 14 is a block diagram of a multi-viewpoint three-dimensional image input apparatus showing a second embodiment of the present invention, which is common to the elements in FIG. 1 showing the first embodiment. Elements are given common reference numerals. In this multi-viewpoint three-dimensional image input device, the device of FIG.
D camera setting instruction device 62 and pixel density conversion device 63
Is added, and the other structure is the same as that of the apparatus of FIG. The 3D camera placement device 60 is a device that sets the placement angle θ and the field angle δ of the plurality of 3D cameras 30-1 to 30-n to desired values. The distance determination device 61 is a device that is connected to the output side of the short-distance image S34 and determines what is the closest distance value in the subject.
A camera setting instruction device 62 is connected. The 3D camera setting instruction device 62 instructs the 3D camera arrangement device 60 about desired values of the angle of view δ and the arrangement angle θ of each of the 3D cameras 30-1 to 30-n, and holds the desired number of screens. It is a device that operates and gives an instruction to process data according to the pixel density in the screen. The 3D camera setting instruction device 62 has a memory such as a read-only memory (hereinafter referred to as a ROM) for holding the number of screens, a limiter, and a function for manually changing the zoom ratio (angle of view δ). Have Pixel density converter 63
Is connected between the output side of the one-layer screen sequential arrangement device 41 and the input side of the non-distance image memory 42, and is based on the instruction of the 3D camera setting instruction device 62, the number of screens of the long-distance image S42 currently obtained. Is a device for converting the pixel density from a predetermined number of screens.

Next, the operation of the apparatus shown in FIG. 14 will be described. In the multi-viewpoint three-dimensional image input device of this embodiment, the distance image S40 and the non-distance image S42 are obtained according to the same basic process as in the first embodiment. The operation of the function added to the first embodiment will be described below. First, the distance image S40
And the number of screens of the non-distance image S42 is set to a predetermined number in advance. For example, if the number of screens is 3 and 2 respectively,
The number of screens in the horizontal direction is three times and twice the number of screens of one 3D camera. Therefore, the angle of view δ and the arrangement angle θ are initialized so that the number of screens of the non-distance image S42 is two. Of course, from the arrangement angle θ, the angle 3D cameras 30-1 to 30-30
Of the intersection K where the optical axes H30-1 to H30-n of −n intersect,
Since the distance from the 3D cameras 30-1 to 30-n changes, the arrangement angle θi of each 3D camera 30-1 to 30-n also becomes
The intersection K and the optical axes H30-1 to H30-n are arranged so as to intersect with each other. In this case, from the relationship of the equation (9), when the subject is separated from the 3D cameras 30-1 to 30-n by a certain distance or more, the distance determining device 61 causes the entire image to enter a predetermined number of screens. Give a good signal. This allows
Since the 3D camera setting instruction device 62 holds the current instruction, the pixel density conversion ratio of the pixel density conversion device 63 is also 1. Therefore, the operation is the same as that of the first embodiment.

Next, when the subject is present at a distance shorter than the distance corresponding to three screens, the following operation is performed. Since the distance that is too close is included in the short-distance image S34, the distance determination device 61 uses the 3D camera setting instruction device 62.
Generates a signal indicating that the value is different from the allowable value according to the instruction. Receiving this signal, the 3D camera setting instruction device 62 receives the shortest distance value and gives an instruction to change the angle of view δ so as to satisfy the number of screens of the distance image S40 or change the arrangement angle θ. This instruction value is set in advance in the ROM or the like in the 3D camera setting instruction device 62 based on the angle of view δ and the arrangement angle θ at the time of initial setting, the values of the angle of view δ and the arrangement angle θ corresponding to the distance value. Has been done. To change the angle of view δ, a zoom lens is used and the zoom ratio is changed by the 3D camera setting instruction device 62. In the present embodiment, when the view angle δ can be dealt with, the view angle δ is dealt with, and at other times, the arrangement angle θ is changed. The angle of view δ moves so that δ expands, and the arrangement angle θ moves so that θ decreases.

In this way, when the 3D camera placement device 60 is set so that the subject can be photographed, the distance determination device 61
Will output a signal with no problem. Each 3D camera 30-1
When the next screen (the screen after the rearrangement by the 3D camera placement device 60) next to 30-n comes in, a signal without any problem is output from the distance determination device 61, and the subsequent operation is the first implementation. Same as the example. Then, the 3D camera NO. For the angle of view δ and the arrangement angle θ corresponding to, the respective values after rearrangement are stored in the ROM or the like in the 3D camera setting instruction device 62. For the signals before that, the angle of view δ and the angle of arrangement θ before rearrangement are stored in the 3D camera setting instruction device 62, and the shortage of the number of screens is truncated.

On the other hand, in the case of the long-distance image S38, the angle of view δ
Is widened or the arrangement angle θ is reduced, the number of required screens is increased. Therefore, the pixel density conversion device 63
Then, the one-layer screen sequential arrangement device 6 is arranged so that the predetermined number of screens is obtained.
The pixel number conversion is performed on the image output from No. 1 and the non-distance image S42 is stored in the non-distance image memory 42. There are various methods for pixel density conversion in the pixel density conversion device 63, but here, it is simply handled by thinning out pixels randomly from the required screen according to the pixel number conversion ratio. At this time, a limiter is provided in the 3D camera setting instruction device 62 so that the required number of screens does not exceed the number of ranged image screens as an operation range.

In the next case, the subject is the 3D camera 3
A case will be described in which it is located far from the initial arrangement relationship of 0-1 to 30-n. Even when the subject is located far away in this way, the shortest distance value of the subject can be known by the distance determination device 61, and the value is known by the display device provided in the 3D camera setting instruction device 62. Is able to. However, in the long-distance image S38, for example, the ground and the like are also reflected, so that the observer can designate a subject of interest by a window frame or the like in the screen. Therefore, when the observer wants to zoom in and see the subject, the following operation can be dealt with. Here, the zoom-up is configured to be manually operated by the 3D camera setting instruction device 62.

If the zoom ratio (angle of view δ) is changed by manually operating the 3D camera setting instruction device 62, the subject can be seen larger, but as a result, the angle of view δ becomes smaller. Therefore, the required number of screens of the non-distance image S42 increases, so that the predetermined number (for example, 2
The 3D camera setting instruction device 62 issues an instruction to change the arrangement angle θ so that the number of screens is within a predetermined number. Here, even if the arrangement angle θ is changed, if the arrangement angle θ cannot be changed to more than two, the limiter that limits the view angle δ up to the number of screens 3 of the distance image S40 as the minimum view angle, It is provided in the 3D camera setting instruction device 62. At this time, the number of pixels is converted by the pixel density conversion device 63 in the same manner as described above and stored in the non-distance image memory 42.

After that, it is possible to perform the conventional image compression on the distance image S40 and the non-distance image S42. The distance image S40 is preferably reversible image compression / expansion, but the non-distance image S42 may be irreversible. The present embodiment has the following advantages in addition to the advantages substantially similar to those of the first embodiment. (I) Since the arrangement angle θ and the angle of view δ of each of the 3D cameras 30-1 to 30-n are kept within a certain allowable value by the 3D camera setting instruction device 62, the number of screens of the non-distance image S42 (that is, The storage capacity value of the non-distance image memory 42) is set to 3D
It can be reduced to a desired value regardless of the number of cameras 30-1 to 30-n. (Ii) The arrangement angle θ and the angle of view δ of each of the 3D cameras 30-1 to 30-n are kept within a certain allowable value by the 3D camera setting instruction device 62 so as to capture an image at a certain distance or more, and the maximum pixel representing the subject is represented. The number selection device 39 selects an image having the maximum number of pixels for subject expression from each of the 3D cameras 30-1 to 30-n as an effective image and discards the others. Therefore, the storage capacity value of the distance image memory 40 can be reduced to a desired value regardless of the number of 3D cameras 30-1 to 30-n. (Iii) The storage capacity value of the distance image memory 40 is set larger than the storage capacity value of the non-distance image memory 42, and the pixel density conversion device 63 can perform pixel density conversion for the non-distance image S42. ing. Therefore, a subject corresponding to the distance up to the storage capacity value of the distance image memory 40 can be imaged, a margin is created in the initially set minimum object distance, and the degree of freedom in setting the present device at the time of image pickup is increased. (Iv) When the long-distance subject is imaged by the 3D camera setting instruction device 62 while changing the zoom ratio, the zoom ratio (angle of view δ) and the arrangement angle θ are set so as not to exceed the storage capacity value of the non-distance image memory 42. Is linked with each other, stable imaging can be performed without omission of the long-distance image S38 due to the zoom ratio change. (V) When the long-distance subject is imaged by the 3D camera setting instruction device 62 while changing the zoom ratio, even if the arrangement angle θ of the 3D cameras 30-1 to 30-n is the maximum changeable limit value, it is present. Since the pixel density conversion device 63 handles the pixel density conversion up to the storage capacity value of the distance image memory 40, the zoom ratio can be increased accordingly and the object can be enlarged.

[0033]

As described in detail above, according to the first invention, the first separation means separates the short-distance image and the long-distance image, and further only the long-distance image is separated into the second separation image. By performing the corresponding point re-search by the means, the medium-distance image and the long-distance image are separated, so that a highly accurate distance image can be obtained in a shorter time than the conventional one. According to the second and third aspects, the long-distance image is treated as a non-distance image separately from the ranged image, so that complicated display is not required when displaying the image in line with the line of sight of the observer. , It can be displayed by simply specifying the address, and the calculation time until display can be shortened. According to the fourth aspect, the arrangement angle and the angle of view of each three-dimensional image input device are kept within a certain allowable value.
The storage capacity value of a long-distance image can be reduced to a desired value regardless of the number of three-dimensional image input devices. According to the fifth invention,
The arrangement angle and the angle of view of each three-dimensional image input device are maintained at a certain permissible value so as to capture an image at a certain distance or more, and an image having the maximum number of pixels for expressing a subject is selected from each three-dimensional image input device as an effective image. And choose to discard the others. Therefore, the storage capacity values of the short distance image and the medium distance image can be reduced to desired values regardless of the number of three-dimensional image input devices. According to the sixth and seventh inventions, the storage capacity value of the short-distance image and the medium-distance image is made larger than the storage capacity value of the long-distance image, and further, the pixel density conversion can be performed in the long-distance image. Therefore, the object corresponding to the distance up to the storage capacity value of the short-distance image and the medium-distance image can be imaged, a margin is generated in the initial maximum object distance, and the multi-viewpoint three-dimensional image input device at the time of imaging The degree of freedom of installation is increased. According to the eighth aspect of the invention, the arrangement angle and the angle of view of each three-dimensional image input device are interlocked so that the storage capacity value of the long-distance image is not exceeded when a long-distance subject is imaged with a different zoom ratio. Therefore, stable imaging can be performed without causing omission of a long-distance image due to a change in zoom ratio. Furthermore, when a long-distance object is imaged with a different zoom ratio, even if the arrangement angle of the three-dimensional image input device is changed to the maximum limit value, pixels are stored up to the storage capacity values of the short-distance image memory and the medium-distance image memory. Since the density is converted and dealt with, the zoom ratio can be increased correspondingly, and the object can be enlarged.

[Brief description of drawings]

FIG. 1 is a configuration block diagram of a multi-viewpoint three-dimensional image input device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of a stereo image method which is one of conventional 3D image input methods.

FIG. 3 is an explanatory diagram of a grayscale image and a distance image of a signal obtained by the stereo image method of FIG.

FIG. 4 is a principle diagram of a multi-lens lenticular system, which is one of conventional 3D image display systems.

FIG. 5 is a schematic block diagram of the previously proposed multi-viewpoint three-dimensional image input device.

FIG. 6 is an operation explanatory diagram of FIG. 1.

FIG. 7 is an explanatory diagram of the overlap of the angle of view of the 3D camera at a long distance of the device of FIG.

8 is an explanatory diagram of an example of how to write into the long-distance image memory of the apparatus of FIG.

FIG. 9 is an explanatory diagram of an example of how to write in the distance image memory of the apparatus of FIG.

10A and 10B are an explanatory diagram showing that a non-distance image of the apparatus of FIG. 1 cannot be decomposed with one pixel and an example of calculating a distance when the image is directly in front of the image.

11 is an explanatory diagram of a calculation example of a required number of screens in the apparatus of FIG. 1 when there is no gap in a non-distance image and when there is a gap.

12 is an explanatory diagram of the minimum value of the angle of view of the apparatus of FIG.

13 is an explanatory diagram of a required number of screens at a short distance of the device of FIG.

FIG. 14 is a configuration block diagram of a multi-viewpoint three-dimensional image input device according to a second embodiment of the present invention.

[Explanation of symbols]

30-1 to 30-n 3D camera 31-1 to 31-n grayscale image memory 32-1 to 32-n range image memory 33 range image separation device 34 short range image memory 35 long range image memory 36 corresponding point re-search device 37 Medium Distance Image Memory 38 Long Distance Image Memory 39 Subject Expression Maximum Number of Pixels Selection Device 40 Distance Image Memory 41 Single-Layer Screen Sequential Arrangement Device 42 Distanceless Image Memory 60 3D Camera Arrangement Device 61 Distance Judgment Device 62 3D Camera Setting Indication Device 63 Pixel density conversion device S31-1 to S31-n grayscale image S32-1 to S32-n distance image S34 short distance image S35 long distance image S37 medium distance image S38 long distance image S40 distance image S42 non-distance image

Claims (8)

[Claims] 1. A plurality of three-dimensional image input devices for inputting an image of an illuminated subject and outputting signals of a grayscale image and a range image representing the subject, such that their optical axes intersect at one point. In the multi-viewpoint three-dimensional image input device arranged in, the distance image from the plurality of three-dimensional image input devices is separated into a short-distance image having a desired phase difference or more and a long-distance image having a desired phase difference or less. The first separating means, and the long-distance image, the gray-scale images of the plurality of three-dimensional image input devices are used to perform the corresponding point search again, and the middle-distance image and the non-phase-difference image that are phase-difference images. A multi-viewpoint three-dimensional image input device, characterized in that a second separation means for separating the long-distance image is provided. 2. The multi-viewpoint three-dimensional image input device according to claim 1, wherein the long-distance image is stored in a storage unit different from the short-distance image and the medium-distance image. 3. A plurality of arranging means according to claim 2, further comprising arranging means for sequentially arranging the long-distance images in a predetermined storage means address according to the arrangement of the plurality of three-dimensional image input devices. Perspective 3D image input device. 4. The multi-viewpoint three-dimensional image input device according to claim 2, wherein the arrangement angle and the angle of view of each of the three-dimensional image input devices are kept in a constant relationship. 5. Setting means for maintaining that the arrangement angle and the angle of view of each of the three-dimensional image input devices are in a fixed relationship, and for identifying that the image is taken over a certain distance, the short-distance image and the medium-distance image The multi-viewpoint three-dimensional image input device according to claim 2, further comprising: a selection unit that selects an image having the largest number of pixels representing the subject as an effective distance image from among the above. 6. The multi-viewpoint three-dimensional image input device according to claim 2, wherein the storage capacity values of the short-distance image and the medium-distance image are larger than the storage capacity value of the long-distance image. 7. The conversion means for converting the pixel density into a predetermined number of screens is provided so that the long-distance images sequentially arranged are matched with a predetermined storage capacity value. Multi-view 3D image input device. 8. A means for interlocking the arrangement angle and the angle of view of each of the three-dimensional image input devices so as to match the storage capacity value of the long-distance image is provided. Alternatively, the multi-viewpoint three-dimensional image input device according to item 7.