WO2006075528A1 - Three-dimensional object measuring device - Google Patents

Abstract

A three-dimensional object measuring device comprises a base on which a three-dimensional object to be imaged is placed, a mirror surface cylinder the inner wall of which surrounds the base and having a mirror surface, a fish-eye lens which is opposed to the base and the optical axis of which is aligned with the center axis of the mirror surface cylinder, and a camera having imaging/recording means for recoding the image formed by the fish-eye lens. The imaging/recording means of the camera captures a direct image of when the three-dimensional object placed on the base is viewed and a reflection image of the three-dimensional object on the base reflected at the mirror surface of the inner wall of the mirror surface cylinder. With this, images covering the top and bottom, the left and right, and the whole circumference of the three-dimensional object used for stereoscopic measurement can be obtained from one captured image, correspondence between the feature points of the images can be easily made, a multipoint-of-view image of the object from all the directions only by one imaging can be obtained, and a three-dimensional circumferential shape of the object can be measured from one image by reflection refraction stereoscopy.

Description

 3D object measuring device

 Technical field

 [0001] The present invention captures a three-dimensional object from a plurality of angles, and based on the correspondence between a feature point in one image and a feature point (corresponding feature point) in another image corresponding to the feature point. The present invention also relates to a 3D object measuring apparatus for performing 3D object stereo measurement processing using a plurality of images and generating 3D image data of the 3D object. Background art

 [0002] In recent years, computerization promotion plans for school education have been promoted in elementary and junior high school education, and measures have been promoted so that computers can be used in classes at all elementary and junior high schools. . In addition, broadband networks have become widespread at home, and it has become possible to easily search and browse various data on the Internet.

 In this way, the importance of enriching educational content that can be easily used has been pointed out as hardware resources and network environments are being developed in the field of education. One of the typical educational contents is an electronic picture book of insects and animals. There are still electronic pictorial books that digitize information on insects and animals, which can be browsed via the Internet, but some of the current electronic pictorial books are two-dimensional planes published in existing pictorial books. There are many electronic pictorial books that only take photographs etc. as digital data and make them into a database that can be searched and browsed. In particular, in the future, 3D electronic pictorial books will be able to see the structure and movement of insects and animals. Development of various electronic pictorial books is desired.

[0003] Now, capturing the three-dimensional shapes of insects and animals is a force that requires a great deal of labor and cost by specialists S. Dynamic development For production, it is desirable that school teachers can easily edit electronic picture books without going through specialists. In other words, ordinary people such as school teachers can easily measure the shape and movement of 3D objects, import them as 3D data, create electronic picture books based on these 3D data, and add content Development of devices that can is needed.

 [0004] The technology for measuring the shape of a 3D object and importing it as 3D data has been researched mainly in the field of computer vision, and as a method for measuring the shape of a 3D object, an active measurement method and a passive method are used. Two measurement methods are known (for example, Patent Document 1).

) o

 The former active measurement method includes a direct measurement method using a laser or an ultrasonic wave, and a pattern projection method in which a pattern is projected by a projector and the surface shape is measured by distortion of the pattern. The latter passive measurement method includes a stereo measurement method that uses two or more cameras for stereo measurement. A method for stereo measurement using a single camera using a mirror has also been proposed.

[0005] Patent Document 1: JP 2001-241928 A

 Disclosure of the invention

 Problems to be solved by the invention

[0006] First, the conventional active measurement method has a problem that a measurement range is narrow as a first problem. In direct measurement methods using lasers and ultrasonic waves, and in pattern projection methods in which patterns are projected by projectors, etc., the range that is directly exposed to the irradiation light and the range that can be observed by a measuring instrument such as a camera The problem that only measurement is possible.

 [0007] In addition, the conventional active measurement method has a problem that image data of patterns and colors cannot be obtained as a second problem. The conventional active measurement method is for measuring the external shape of a three-dimensional object, and since it is impossible to obtain pattern and color data directly, it is necessary to prepare other patterns and color data. In other words, this active measurement method is mainly a method for generating 3D wireframes and 3D polygons from 3D objects, and patterns and color data need to be prepared separately as texture images and developed images. In order to generate 3D image data, many processes and a lot of labor are required.

[0008] Next, the conventional passive measurement method has a problem that the scale of the apparatus increases as a first problem. 3 based on the captured image of a 3D object that is essentially a 2D image In order to obtain 3D image data, it is necessary to have multiple 2D images of 3D objects taken from multiple viewpoints. It is necessary to prepare a large number of cameras around and to photograph the entire circumference of the 3D object up, down, left and right. In this way, there is a problem that the scale of the apparatus becomes large in order to obtain 3D image data that can be viewed as an electronic picture book.

 [0009] In addition, the conventional passive measurement method has a problem that when a large number of images are prepared from a large number of viewpoints, it is difficult to match feature points. In the stereo measurement method, it is necessary to perform stereo measurement between a large number of images taken from a large number of viewpoints. In the process, feature points such as singular points and edge portions are selected, and Matching is performed using the feature points as clues. However, 3D objects with complicated shapes such as insects and animals have many feature points, and as the number of feature points increases, the amount of computation for matching them increases rapidly. It becomes difficult to take

 As a device to reduce the number of cameras in order to reduce the size of the device, there is a device that uses a plane mirror, but according to this device, the first problem of the passive measurement method is to alleviate the force to some extent. In order to obtain an image that covers the entire circumference of the top, bottom, left, and right, multiple cameras are required. In addition, the second problem of passive measurement cannot be alleviated with this device! / Remains a problem.

 [0010] Further, for the purpose of simplifying the system configuration, a catadioptric stereo vision system that measures the three-dimensional shape of an object with a single camera using a reflection image or a refraction image by a mirror or a prism is known. ing. A catadioptric stereo vision system is a system in which multiple lights traveling on different paths starting from the same point on the surface of an object are incident on a single camera using optical equipment, and images observed by the camera from multiple viewpoints. This system realizes stereo vision with a single camera by taking equivalent images.

This catadioptric stereo vision system has the advantage that a camera with parallax can be obtained with a single camera, so that no camera parameter correction processing is required, and images of each viewpoint are photographed synchronously. Also, arrange the camera and mirror so that the epipolar line is the same as the scanning line. As in the stereo vision system using a plurality of cameras, it is possible to limit the range where the corresponding points exist.

 However, the object circumference measurement method based on the catadioptric stereo vision system has a problem that it is difficult to obtain a correct circumference due to the occurrence of irregular reflection by a mirror and the expansion of the corresponding point search range. .

 [0011] In view of the above problems, the present invention covers the entire upper, lower, left, and right circumferences of a three-dimensional object with a small number of captured images captured by a small number of cameras (one or two) with a small apparatus size. Images can be obtained, and correspondence between feature points between images can be easily taken. Multi-viewpoint images from all directions of an object can be obtained with a single shooting. The object is to provide a 3D object measuring device that can measure the 3D circumference of an object.

 Means for solving the problem

[0012] In order to achieve the above object, a three-dimensional object measuring apparatus of the present invention surrounds a pedestal on which a three-dimensional object to be imaged is placed, an inner wall surface of the cylinder, and the inner wall surface of the cylinder is a mirror surface. A specular cylindrical body, a fisheye lens facing the pedestal and arranged so that a lens optical axis coincides with a cylindrical central axis of the specular cylindrical body, and a photographic recording for recording an image obtained through the fisheye lens A direct image obtained by directly viewing the three-dimensional object on the pedestal, and the mirror image of the inner wall of the mirror cylinder on the pedestal. This is taken together with the reflection image of the 3D object.

 According to the above configuration, a fish-eye lens can shoot an image with a full viewing angle radially centered on the lens optical axis, a direct image obtained by directly photographing a three-dimensional object placed on a pedestal facing the fish-eye lens, and the side surface of the mirror cylinder It is possible to take a picture together with the reflected image outside the 3D object reflected on the mirror of the inner wall. Due to the nature of the cylindrical mirror surface, this reflection image is a reflection of the front image seen from the omnidirectional viewpoint that surrounds the periphery of the 3D object. The front image can be obtained at once.

In addition to placing the 3D object to be photographed on the pedestal, It may be fixedly supported by piercing with means for fixing and supporting, for example, a wire. The tabletop may also be used as a pedestal.

Here, there may be a plurality of reflection images of the three-dimensional object reflected on the mirror surface of the inner wall of the side surface of the mirror cylindrical body. In other words, the mirror image is reflected once and captured by the fisheye lens, or the so-called twice reflected image reflected twice by the mirror surface and captured by the fisheye lens, such as n times (n is a natural number) There can be a reflection image that is reflected n times by the fisheye lens.

 Although the above is an explanation of still images, if still images that are continuous in time series are obtained, they can be handled as moving images, and 3D moving image data relating to the movement of the target object can also be obtained. Become.

 [0014] Next, in the three-dimensional object measuring apparatus of the present invention, the feature point extracting means for extracting and determining the feature points on the photographed image photographed by the photographing recording means of the camera, and the feature point extracting means A corresponding feature point on the reflection image corresponding to the extracted feature point on the direct image is searched and determined on the reflection image, or on the reflection image extracted by the feature point extraction means. A corresponding feature point search means for searching and determining a corresponding feature point on the direct image corresponding to the feature point of the direct image, and the direct relationship based on the correspondence between the feature point and the corresponding feature point. Stereo measurement processing of the image and the reflected image is performed! 3D image data generating means for generating 3D image data of the 3D object from the direct image and the reflected image is provided, Corresponding feature point searching means searches for the corresponding feature point by searching for an extension line connecting the center of the image and the feature point to the photographed image. Is preferred.

 With the above configuration, it is possible to easily match a feature point and a corresponding feature point corresponding to the feature point between the direct image and the reflected image through stereo measurement processing. In other words, in the three-dimensional object measuring apparatus of the present invention, the reflected image is an image developed radially from the center of the pedestal (cylindrical center), and the corresponding feature points are always on the same straight line. Therefore, there is an advantage that the corresponding point search is simplified.

[0015] Next, in the three-dimensional object measuring apparatus of the present invention, the pedestal is formed of a transparent material. The object is visible from below the pedestal, and the camera has two cameras, a first force camera and a second camera, and the fisheye lens of the first camera and the second force camera It is preferable to arrange so that the fish-eye lenses face each other across the pedestal. With the above configuration, it is possible to simultaneously capture an upper image and a lower image of a three-dimensional object by photographing with a fish-eye lens from the two directions of the upper and lower surfaces of the mirror cylinder.

 Next, in the three-dimensional object measuring apparatus of the present invention, the camera can move along the cylindrical central axis with respect to the mirror cylinder, and the distance between the pedestal and the fisheye lens It is preferable to make it variable.

 The position at which the reflection image of the 3D object is reflected on the mirror surface of the side wall of the mirror cylinder varies depending on the size (height, thickness) and shape of the 3D object to be measured. It is preferable that the position of the fisheye lens (the distance between the pedestal and the fisheye lens) can be adjusted in order to appropriately capture the direct image and the reflected image. Therefore, the distance between the pedestal and the fisheye lens is variable.

 Next, in the three-dimensional object measuring apparatus of the present invention, the pedestal, the specular cylindrical body and the camera can be moved relative to each other, and the center of the pedestal is rotationally moved with the pedestal fixed. It is preferable that the mirror cylinder and the camera can be freely rotated and moved as a center in a three-dimensional space.

With the above configuration, even when the 3D object has a sharp side shape or when the 3D object has a concave shape, an image necessary for stereo measurement can be obtained for that portion. For example, if a three-dimensional object has a sharp side shape, a fisheye lens cannot obtain an image directly with the mirror cylinder and pedestal fixed. If the 3D object has a slightly deep concave shape on the top surface, an image can be obtained directly with the mirror cylinder and camera fixed, but the concave part of the reflected image is You can't get it behind the wall. Therefore, with the pedestal fixed, the center of the pedestal is the center of rotational movement, and the mirror cylinder and the camera can be freely rotated and moved together in the three-dimensional space. Even if it is a shape or the latter concave shape, Therefore, it is possible to adjust the arrangement of the mirror cylinder and the camera so that the direct image and the reflected image of this part can be taken at different angles.

 The invention's effect

 [0018] According to the three-dimensional object measuring apparatus of the present invention, a fish-eye lens can shoot images of all viewing angles radially with the lens optical axis at the center, and a three-dimensional object placed on a pedestal in combination with a mirror cylindrical body. By acquiring a captured image, a direct image obtained by directly photographing a 3D object placed on a pedestal facing the fisheye lens, and a reflected image of the 3D object reflected on the mirror surface of the side wall of the mirror cylinder You can shoot at the same time. Further, according to the three-dimensional object measuring apparatus of the present invention, it is possible to easily match a feature point and a corresponding feature point corresponding to the feature point between the direct image and the reflected image in the stereo measurement process.

 BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the three-dimensional object measurement apparatus of the present invention will be described with reference to the drawings. However, the technical scope of the present invention is not limited to the specific uses and shape-dimensions shown in the following examples.

 In measuring a three-dimensional object, any means for supporting the three-dimensional object may be used as long as it can support the object, for example, a pedestal, a hanging thread, a wire, and a needle. Here, the pedestal is not limited to a small plate-like one incorporated in the apparatus casing, but includes a large one such as a desk or a table. When a desk is used as the support means, a mirror cylinder, which will be described later, is placed on a desk as a pedestal as the support means.

 In the embodiment described below, the support means for the three-dimensional object will be described as a pedestal that is a small disk-like plate.

 Example 1

 [0020] An example of a three-dimensional object measuring apparatus of the present invention according to Example 1 is shown.

 FIG. 1 is a diagram schematically illustrating the basic configuration of the three-dimensional object measuring apparatus 100 according to the first embodiment.

[0021] Reference numeral 10 denotes a pedestal on which a three-dimensional object to be photographed is placed. The shape of the pedestal 10 is not limited, but is, for example, a disk shape. The size of the pedestal 10 is the 3D object to be photographed. It should be large enough to place a jet.

 [0022] Reference numeral 20 denotes a mirror cylindrical body. The mirror surface cylindrical body 20 has a cylindrical shape with a perfect cross section, and its inner wall surface is a mirror surface. That is, the mirror surface cylindrical body referred to in the present invention refers to a cylindrical body whose inner wall surface is cylindrical and mirrored. The outer wall surface of the cylinder does not have to be a force and does not have to be cylindrical or mirrored. For example, the outer shape is a square pillar, which is a plastic material, and includes a cylindrical hollow on the inside and a mirror surface on the inner wall.

 The inner diameter of the mirror cylindrical body 20 is large enough to accommodate the pedestal 10 therein. In this example, the pedestal 10 is accommodated in the bottom surface portion of the mirror cylindrical body 20.

 The height of the mirror-cylindrical body 20 may be at least L from the base 10 that is the base, assuming that the height of the 3D object to be imaged is L. For example, the height of the base 10 from 2L to 10L is sufficient. Can be height.

[0023] Reference numeral 30 denotes a camera. The camera 30 includes a fisheye lens 31 and a photographing recording means 32.

 The fisheye lens 31 faces the pedestal 10 and is arranged so that the lens optical axis coincides with the cylindrical central axis of the mirror cylindrical body 20. In this example, the fisheye lens 31 is disposed on the upper surface of the mirror cylinder 20 and is installed downward so as to face the pedestal 10. The central axis (lens optical axis) of the fisheye lens 31 is arranged so as to coincide with the central axis (cylindrical central axis) of the cylindrical mirror body 20.

 The photographing and recording means 32 is not particularly limited as long as it can record the image obtained through the fisheye lens 31, and may be a so-called analog image recording means for exposure recording on an analog film, and a light receiving element such as a CCD (Charge Coupled Device). The so-called digital recording means can accept the light and record it as digital data. As an example, here, the photographing and recording means 32 is a digital image recording means.

 The image data processing device 40 receives a two-dimensional image taken by the camera 30, and generates three-dimensional image data by stereo measurement processing. The image data processing device 40 includes a feature point extracting unit 41, a corresponding feature point searching unit 42, and a three-dimensional image data generating unit 43 (not shown in FIG. 1), which will be described later.

[0025] Next, in the configuration example of the three-dimensional object measuring apparatus 100 shown in FIG. We will explain the principle and procedure for obtaining 3D image data by stereo measurement by shooting a mounted object. Here, as an example, an example of reading a three-dimensional object (object 1) having a cone shape (cone shape) will be described.

 To obtain 3D image data from 2D image data based on the stereo measurement method, as step 1, obtain a 2D image of the target 3D object from multiple viewpoints (each at different angles). Procedure, procedure 2 is a matching process between feature points between two or more acquired 2D images, and each element of a 3D object is matched, and procedure 3 is a difference calculation or inference based on stereo measurement processing. It is necessary to follow the procedure for calculating 3D image data.

 [0026] First, as procedure 1, in the three-dimensional object measuring apparatus 100 of the present invention, at least two images, that is, a direct image and a reflected image, are obtained by one shooting. Here, the direct image refers to an image obtained by receiving a light ray directly incident on the fisheye lens 31 from a 3D object on the pedestal 10, that is, an image of a 3D object in which the visual power of the fisheye lens 31 can be directly seen. Yeah. A reflected image is an image obtained by receiving a light beam reflected from the three-dimensional object on the base 10 and reflected on the inner wall mirror surface of the mirror cylinder 20 and entering the fisheye lens 31, that is, from the viewpoint of the fisheye lens 31. A reflection image of a three-dimensional object reflected on the inner wall mirror of the mirror cylinder 20. The reflected image is reflected once on the inner wall mirror surface of the mirror cylinder 20 and incident on the fisheye lens 31. The reflected image R1 is reflected twice on the inner wall mirror surface of the mirror cylinder 20 and incident on the fisheye lens 31. Includes images obtained by multiple reflections such as R2 reflection images. That is, what is reflected n times (n is a natural number) on the mirror surface of the inner wall of the mirror cylindrical body 20 and enters the fisheye lens 31 is an n-time reflected image Rn.

FIG. 2 is a diagram schematically showing a state of light reception of the direct image and the reflected image in the camera 30 in the longitudinal section of the three-dimensional object measuring apparatus 100. In Fig. 2, we focus only on point A where 3D object 1 exists. Light emitted from a point A of the three-dimensional object 1 placed on the pedestal 10 and directly incident on the fisheye lens 31 is an optical path AO that directly forms an image. Light that is emitted from the point A, reflected once on the inner wall mirror surface of the mirror cylindrical body 20, and incident on the fisheye lens 31 is an optical path A1 that forms a once reflected image. Light emitted from point A, reflected twice on the inner wall mirror surface of the mirror cylinder 20 and incident on the fisheye lens 31 is reflected twice. This is an optical path A2 that forms a reflected image.

 Although there are optical paths that reflect three or more times, FIG. 2 shows the optical path A2 that reflects twice, and illustration of the other optical paths is omitted. FIG. 2 shows an optical path focusing only on point A. Light emitted from all points on the surface of the object 1 enters the fisheye lens 31 according to the same principle as in FIG.

 FIG. 3 is a diagram schematically showing an image obtained by photographing the object 1 having a cone shape placed on the pedestal 10 by the camera 30. A point A on object 1 is shown as a small dot with a white circle for reference. The image power seen in the center in Fig. 3 is a direct image DO of object 1 and shows the force directly above the cone shape. In Fig. 3, the circle directly surrounding the image DO is the pedestal 10 and the edge E of the mirror cylinder 20. Next, the outer ring surrounding the edge E is a one-time reflected image R1 of the object 1 reflected on the inner wall mirror surface of the mirror cylinder 20. The cone-shaped side surface is reflected around the inner wall mirror surface of the surface cylindrical body 20. Next, the outer ring-shaped object surrounding the once-reflected image R1 is the twice-reflected image R2 of the object 1 reflected on the inner wall mirror surface of the mirror cylinder 20. Similar to the one-time reflection image R1, the cone-shaped side surface is reflected around the inner wall mirror surface of the surface cylindrical body 20, but in the two-time reflection image R2, the image is a point-symmetric image. . In other words, the image of a point A on object 1 is on the right side in the direct image DO (image point GO), and on the right side in the single reflection image R1 (image point G1), but on the left side in the double reflection image R2. Reflected (image point G2).

 In FIG. 3, the reflection images (reflection image R3, etc.) of three or more reflections are not shown. In the three-dimensional object measuring apparatus 100 of the present invention, a plurality of images, that is, a direct image and a reflected image are obtained by one shooting as described above.

 Here, the reason why the direct image obtained by the three-dimensional object measurement apparatus 100 of the present invention and the reflected image force can be used for the three-dimensional stereo measurement as an image obtained by photographing a plurality of viewpoint forces at different angles.

[0030] FIG. 4 is equivalent to the photographing viewpoints of the optical path A0 that directly forms an image with respect to the point A of the object 1 described in FIG. 2, the optical path A1 that forms a once-reflection image, and the optical path A2 that forms a twice-reflection image. It is the figure which showed the mode that it expanded automatically. The actual fisheye lens 31 Although it is the point FO, the optical path Al corresponding to the once reflected image Rl is equivalent to the optical path A1 ′ incident on the photographing viewpoint Fl. Similarly, the optical path A2 corresponding to the twice reflected image R2 is equivalent to the optical path A2 ′ incident on the photographing viewpoint F2. In other words, the point A in the once reflected image R1 is equivalent to the point A in the image that would be obtained if the object 1 was taken at the shooting viewpoint F1 and the virtual camera 30 '. It can be said that point A in the reflected image R2 is equivalent to point A in the image that would be obtained if the object 1 was photographed by the virtual camera 30 'at the photographing viewpoint F2. If the inner diameter of the mirror cylinder 20 is!:, The photographing viewpoint F1 is a position 2r away from the fish-eye lens 31, and the photographing viewpoint F2 is a position 4r away from the fish-eye lens 31. The above description using FIG. 4 has focused on only the point A on the right side of the object 1, but in reality, all the points on the cone-shaped surface are between the inner wall mirror surface of the facing mirror cylinder. Therefore, the one-time reflected image R1 shown on the mirror surface of the inner wall 20 of the mirror cylinder 20 shown in Fig. 3 is 360 ° around the circumference of the radius 2r with the center axis of the cylinder as the center of rotation. An object equivalent to an image obtained by capturing object 1 while moving virtual camera 30 ', and collecting only the central image of each captured image (the front image that is directly opposite to the shooting viewpoint in the cone shape) It has become. Similarly, the twice-reflection image R2 is taken for object 1 while moving the virtual camera 30 "around the circumference of the radius 4r and moving the virtual camera 30" around the center axis of the cylinder, and only the center image of each captured image In other words, the direct image DO, the reflected image Rl, and the reflected image R2 obtained by the three-dimensional object measuring apparatus 100 of the present invention are the actual images where the fisheye lens 31 is arranged. Taken from countless viewpoints (number of resolutions) arranged so as to surround a direct image from the viewpoint F0 and surround the circumference 2r from the center axis of the cylinder and the circumference 4r from the circumference. However, although the number of shooting viewpoints is innumerable, the reflected image is a direct image of each shooting viewpoint, that is, a central image that is directly in front of the images shot from each shooting viewpoint. Equivalent to a collection of only images Therefore, if you focus on one point on object 1 (for example, point A), the number of image data that can be used for stereo measurement is not innumerable. (E.g., image point GO), a single reflection image R1 captured from a viewpoint with radius 2r (e.g. image point G1), and a double reflection image R2 captured from a viewpoint with radius 4r The image of the point shown in There are only three image data of image point G2). In other words, in this example, the image data used for the stereo measurement process has the same effect as that obtained when two-dimensional images from three viewpoints are obtained at one time.

 [0032] Here, a method for determining the parameter of the ray angle incident on the camera from the point through the fisheye lens in the stereo vision measurement of the three-dimensional object measurement apparatus of the present invention will be described. The fisheye lens can express the relationship between the angle φ formed by the ray incident on the fisheye lens with the optical axis and the distance d from the image center on the image plane onto which the ray is projected, as shown in Equation 1 below.

[0033] [Equation 1]

FIG. 9 shows a relationship diagram between the angle φ and the distance d from the image center. Where f is the focal length. The function g (φ) is the projection characteristic of the fisheye lens, and the characteristic varies from lens to lens.

 By having the projection characteristics of the fisheye lens as a database, the angle of the ray incident on the camera can be determined by the distance of the image center force at the point of interest in the captured image.

 Next, procedure 2 for stereo measurement in the three-dimensional object measurement apparatus 100 of the present invention will be described.

 In step 2, stereo measurement processing is performed between multiple acquired 2D images, and each 3D object is associated with each element, but it is not necessary to perform stereo measurement processing for all pixels. Since the cost becomes enormous, the image data is usually converted to the frequency domain by DCT transform or Fourier transform, and singular points and edges in the image are extracted and their representative ones are selected as feature points. Then, the feature points in the image are searched and matched. Here, feature points in other images corresponding to feature points in one image are called corresponding feature points.

[0036] In the case of a simple matching algorithm in the prior art, all feature points between images are searched and matched by trial and error. However, when the shape of an object becomes complicated, a large number of images are included in the image. There are singular points and edges, and many feature points are selected. Thus, the calculation cost of the matching process becomes enormous.

 However, when the 3D object measuring apparatus 100 of the present invention is used, the matching process between feature points can be executed using the geometric relationship between the direct image and the reflected image, which greatly increases the calculation cost. Can be reduced. The principle can be explained as follows. As described with reference to FIGS. 3 and 4 above, the image captured by the camera 30 includes the direct image DO and the reflected image (here, the once reflected image R1 and the twice reflected image R2). However, there is an important geometric relationship between the direct image DO and the reflected image (one-time reflection image R1 and two-time reflection image R2). The geometrical relationship mentioned here is a relationship in which corresponding feature points exist on the same straight line drawn through the center of the image.

 [0037] An image of a certain point on the object is reflected on the inner wall mirror of the mirror cylinder facing the object. In the case of a twice-reflection image, it is point-symmetrical, but since it is a position folded around the center point, the feature point on the reflection image corresponding to a certain feature point on the direct image is eventually the corresponding point on the direct image. It is on the extended line of the straight line connecting the feature point and the center point of the image. As shown in Fig. 3, when searching for a corresponding feature point for image point GO, which is a certain feature point directly on image DO, searching for an extension line of a straight line connecting image center C and image point GO, An image point G1, which is a corresponding feature point in the twice reflected image R1, and an image point G2, which is a corresponding feature point in the twice reflected image R2, are found. If this geometric relationship is incorporated in the matching algorithm for feature point search, the calculation cost can be greatly reduced.

 With the above procedure, the corresponding feature points corresponding to the feature points are efficiently searched using the geometric relationship between the feature points.

 For reference, an example of an image obtained by photographing a pyramid-shaped object in the prototype three-dimensional object measuring apparatus 100 of the present invention is shown.

Fig. 5 shows an image of a pyramid-shaped object placed at the center of the pedestal, taken directly from the image, and a reflection image reflected on the inner wall mirror of the mirror cylinder. The radiation drawn in the image is a line added to the image by post-processing so as to visually check the correspondence between the direct image and the reflected image, and such radiation does not reflect the initial force. Absent. It should be noted that the geometric relationship between the feature points described above does not change even if the object is placed at a position where the central force of the pedestal is shifted. Figure 6 shows that the geometric relationship between the feature points is maintained even when the object is placed at a position where the central force of the pedestal is also deviated. Compared to Fig. 5, the object has a central force of the pedestal. The reflection image reflected on the inner wall mirror surface of the mirror cylindrical body is placed at a position biased to the upper right. The force is greatly distorted compared to Fig. 5. The feature points may exist on a straight line passing through the center of the image. I understand. In Fig. 6, we focus on two feature points in the image directly, and draw two additional straight lines connecting the feature points and the center of the image in post-processing so that they can be easily divided. It can be seen that there are corresponding feature points in the once reflected image and the twice reflected image on the straight line.

 The reason for this will be described with reference to FIG. Figure 7 shows an example of the trajectory of a ray incident on the camera after the point force of the object is reflected twice by the mirror. Figure 7 (a) is observed from the side, and (b) is observed from directly above. When the optical axis of the camera and the central axis of the cylindrical mirror coincide, in order for a light ray to enter the camera, the light beam must pass through the central axis of the cylindrical mirror. When the 3D object measuring apparatus of the present invention is viewed from above as shown in Fig. 7 (b), the two tangents of the cylindrical mirrors facing each other across the central axis of the cylinder are as shown in Fig. 7 (b). Always parallel. Therefore, the light beam that passes through the center of the cylinder always enters the mirror surface with parallel tangents perpendicularly when an upward force is also observed. Since rays starting from the same point always exist on the same straight line, the epipolar line of the virtual camera is a straight line passing through the center of the cylinder.

 FIG. 8 shows a simulation image obtained by photographing a virtual object.

 An image can be obtained in which the image directly visible from the camera is in the center of the captured image and the image reflected by the cylindrical mirror is present around it. In addition, the reflected images are arranged concentrically in order of decreasing image center force.

Here, when the object in FIG. 8 (a) has a conical shape, the light ray incident on the camera from the point A on the object surface moves only on the broken line connecting the image center and the point A. For this reason, the search range of the set of corresponding points existing in the captured image can be limited to a straight line passing through the center of the image, so that the amount of calculation can be reduced and false detection of the corresponding points can be reduced. Similarly, if the object in Fig. 8 (b) has a quadrangular pyramid shape, the ray incident on the point B camera on the object surface moves only on the broken line connecting the image center and point B. It's a little bit.

 Even if the object in Fig. 8 (c) has a conical shape and the image is taken with the position shifted from the center force, the light ray incident on the camera from point C on the object surface is It moves only on the broken line connecting the point C and the point C.

 Next, procedure 3 for generating 3D image data based on stereo measurement processing in the 3D object measuring apparatus 100 of the present invention will be described.

 In step 3, stereo measurement processing of the direct image and the reflected image is performed based on the correspondence between the feature points obtained in step 2 and the corresponding feature points. Generate 3D image data of Jetato. For example, it is performed as 3D image data by direct calculation or inference calculation between direct image data and reflected image data. In the three-dimensional object measuring apparatus 100 of the present invention, the procedure 3 is not particularly limited, and any algorithm that performs stereo measurement processing can be widely applied.

 FIG. 12 is a block diagram showing components of the 3D image data processing device 40 in the 3D object measuring device 100 of the present invention. Note that the above algorithm, which can be used to concretely realize 3D image data generation processing based on stereo measurement methods using an information processing organization combined with general-purpose personal computer resources and software modules, is a semiconductor. It can be realized using dedicated hardware coded by a circuit.

[0044] 41 is a feature point extracting means for extracting and determining feature points on a photographed image photographed by the photographing recording means 32 of the camera 30. Any effective feature point extraction algorithm is not particularly limited and can be widely applied.For example, DCT transform converts image data into the frequency domain, and extracts singular points and edges in the image. Use an algorithm to select

[0045] Reference numeral 42 denotes corresponding feature point searching means for searching and determining corresponding feature points on the reflected image corresponding to the feature points on the direct image extracted by the feature point extracting means 41 on the reflected image. Or a feature on the reflection image extracted by the feature point extraction means 41. This is means for searching for and determining corresponding feature points on the direct image corresponding to the scoring points on the direct image. The corresponding feature point search means 42 incorporates an algorithm for performing a search using the above-described geometric relationship, that is, searching for a corresponding feature point by searching on an extension line connecting the center of the image and the feature point. Leave it in.

 [0046] 43 is a three-dimensional image data generation means, which performs stereo measurement processing between the direct image and the reflected image based on the correspondence between the feature points obtained by the corresponding feature point search means 42 and the corresponding feature points. This is the part that generates the 3D image data of the object such as the direct image and the reflection image that are 2D images.

 [0047] Here, as an example of the algorithm of the corresponding point search means 42, details of a method using SSD (Sum of Squared Difference) will be described. As described above, the corresponding point search means 42 can apply an existing stereo measurement algorithm, and can apply a wide range of algorithms such as DP matching, which is a nonlinear matching technique, in addition to SSD. .

 First, in the algorithm of the corresponding feature point search means 42, the captured image is expanded in polar coordinates for easy processing, and then, from the captured image expanded in polar coordinates, a small area centered on one point is searched as the search source area. And a process for searching for an area considered to be most similar to the search source area is performed.

 In this embodiment, an SSD (Sum of Squared Difference) is calculated between two areas, and it is determined that the areas are more similar as the SSD value is smaller. However, since the image that is reflected by the cylindrical mirror and enters the camera contains distortion in the radial direction, the distortion of the reflected image should be corrected when determining the similarity between regions. However, since the magnitude of the distortion of the reflected image depends on the normal direction of the object plane, which is an unknown parameter, it is impossible to correct the distortion analytically. Therefore, in this embodiment, in addition to the movement of the search area, the most similar area is calculated by calculating the SSD value while dynamically scaling the reflected image for examining the similarity in the radial direction. The set of the center points is the thread of the corresponding point.

In the following, the algorithm of the corresponding feature point search means 42 is described as follows: (1) polar coordinate expansion processing of the captured image, (2) size normalization processing of the search target region, (3) limitation of the search target region Processing will be described separately in (4) SSD value calculation processing.

[0048] (1) Polar coordinate expansion processing of captured image

 As preprocessing to simplify the calculation of SSD, polar coordinate expansion processing of the captured image is performed. Figure 10 (a) shows an example of the image that is the result of polar coordinate expansion. As described above, a set of corresponding points always exists on the same straight line passing through the center of the image taken by the three-dimensional object measurement apparatus of the present invention. Therefore, the range when searching for a point corresponding to a point in the image by expanding the captured image into a polar coordinate image with the vertical axis being the distance from the image center and the horizontal axis being the angle is 10 If the coordinate value in the image before polar coordinate expansion is (u, V) and the coordinate value after polar coordinate expansion is (t, w), it will be limited to the area of the broken line shown in (b). The correspondence between the values is expressed by the following formulas 2 and 3. Where t and w are integer values. U represents the number of pixels in the u and V axis directions in the image before polar coordinate expansion, and T and W represent the number of pixels in the t and w axis directions in the image after polar coordinate expansion, respectively. This image before polar coordinate development is taken so that the central axis of the cylinder exists at the center of the image.

 It should be noted that in the image before the polar coordinate expansion, the coordinate value (u, V) that does not satisfy the condition of the following Equation 4 is a conversion target.

[0049] [Equation 2]

[0050] [Equation 3]

[0051] [Equation 4]

[0052] (2) Normalization processing of search target area size

 Next, the normality process for the size of the search target area will be described. As described above, the size of the reflected image in the captured image in the w-axis direction varies depending on the incident angle of the light beam and the normal direction of the object plane. For example, in Figs. 10 (a) and (b), the size of the w-axis direction of the image that is reflected twice by the mirror and incident on the camera is smaller than the image that is reflected once and incident on the camera. The However, since the normal direction of the object plane is unknown at the measurement stage, it is impossible to analytically correct the change in the image size in the w-axis direction. Therefore, in the measurement apparatus outside the three-dimensional Obzier of the present invention, the similarity is examined, and the SSD value is calculated while dynamically expanding and reducing the local region in the w-axis direction. Here, to calculate the SSD value, the size of the reference area (Source) and the evaluation target area (Target) must be the same during the search.

 In general, changes in the image size due to the incident angle of the light beam and the normal direction of the object surface are non-linear, but even if the distortion of the minute image in the local region is approximated by linear scaling, the error is small. Therefore, in this 3D object measurement device, the local area is enlarged or reduced by linear interpolation for easy processing.

 The color of each pixel is expressed in the RGB color system. The luminance values l (t, w) (i = r, g, b) of R, G, and B at the point, w) in the normalized region are the luminance values of the point before normalization. (I = r, g, b) and the target scale s when Source is set to 1.0, it can be calculated from Equation 5 below. Where trunc (x) is a function that rounds off the decimal part of x.

[0053] [Equation 5] k it, W) = (l-A) 0 _l (t, U ' _int ) + Ao t, U'int, + 1

 [0054] (3) Search target area limiting process

 Next, the search target area limiting process will be described. An area centered at a point (t, w) in an nth-order reflection image of an image taken by the 3D object measuring device of the present invention Source

 S S

In the region corresponding to, when occlusion does not occur, the spatial continuity of the image is maintained even if it is reflected by the mirror.Therefore, in the vicinity of the point (t, w ⁿ ) obtained using Equation 6 below.

Suppose that it exists beside g. Where w ⁿ and w ⁿ each have an nth reflection image

 mm max

W ^S and w ^S are the upper and lower values where the Source exists.

 min max

 The lower end value is shown.

[0055] [Equation 6]

[0056] Here, the point w ⁿ obtained by Equation 6 above represents the linear ratio of the position of Target corresponding to Source.

 g

 This is an estimate. As described above, the distortion of the reflected image in the W-axis direction is generally non-linear. This is because the error when approximating non-linear distortion with a linear ratio is small locally.

 Therefore, when searching for the region corresponding to the region Source centered at the point (t, w), the search pair

 S S

Search by limiting the range in the w-axis direction that is an elephant to a certain range centered on w ⁿ

 g

 This is intended to reduce the area.

 Searches the n-th reflection image for the target corresponding to the region Source whose center is the point (t, w)

 S S

When, Table the upper w ⁿ and lower w ⁿ in the range of w to the center point of the search by the following Equation 7

 gmin gmax

The Further, when an image reflected in the mirror and Source region, the upper and lower ends w ⁿ

gmm gmax This is expressed by Equation 8 below. Here, a is a constant that represents the size of the search range. The search range becomes wider c

 [Equation 7] 5

₂ _ " ^-ll nn

tr ^s in x _ ", ^s .

 ii '-',

ί 川 '~ ^(ι j'["mx

 II:

[0058] [Equation 8]

5 7

ί ¹ + (ir

[0059] (4) SSD value calculation processing

Next, the SSD value calculation process will be described. In order to evaluate the similarity between the source and the target that has been normalized by linear interpolation, this 3D object measurement device uses the SSD value between the source and target areas as the evaluation amount. The SSD value d between the source area S and the target area T is obtained by the following formula 9. In the following formula, I ^s and 1 ^T

 ST i i

(t, w) (i = r, g, b) is the luminance of each pixel in Source and Target, respectively. In addition, the range of values that 1 S and 1 ^T (t, w) (i = r, g, b) can take are both 0 or more and 255 or less.

[0060] [Equation 9]

 S 2

9

[0061] Here, the flow of searching for a target corresponding to a certain source will be described with reference to FIG. Figure 11 shows an example in which the image directly observed from the camera is the entire source set S.

 all

 Yes. S force The search candidate area corresponding to the source area Sa (Sa belongs to the set of S) all all cut out with an arbitrary window size can be limited to the area indicated by the broken line in FIG. The Target corresponding to the Source is considered to exist in the local area described above, and the SSD value with Sa is calculated by cutting out the area while changing the window size in the search candidate area. Then, the region Ta having the smallest SSD value is selected as the region corresponding to Sa, and the center point of each is set as a pair of corresponding points used for stereo viewing. Similarly, all images contained in S

 all

 The corresponding points are searched for the prime.

As described above, according to the three-dimensional object measuring apparatus 100 of the present invention shown in the first embodiment, a direct image obtained by directly photographing a three-dimensional object placed on a pedestal facing the fisheye lens, and a mirror cylindrical body It is possible to shoot the reflection image of the 3D object reflected on the mirror of the inner wall of the side at the same time, and easily find the feature point and the corresponding feature point corresponding to the feature point between the direct image and the reflection image. 3D image data can be generated from 2D image data based on stereo measurement processing.

 Example 2

 [0063] An example of a three-dimensional object measuring apparatus 100a of the present invention according to Example 2 is shown.

 The three-dimensional object measuring apparatus 100a of the second embodiment is different from the three-dimensional object measuring apparatus 100 of the first embodiment in that the pedestal 10 is formed of a transparent material, and as shown in FIG. The second camera 30b is provided with two cameras, and the fisheye lens 3la of the first camera 30a and the fisheye lens 31b of the second camera 30b are arranged so as to face each other with the pedestal 10 in between.

 This is intended to capture multiple images from different viewpoints at the same time for the lower image as well as the upper image of object 1.

[0064] In the configuration in which one camera 30 is provided on the upper part of the specular cylindrical body 20 shown in the first embodiment, what is obtained as a captured image is a direct image D0 on the upper surface of the object 1 and a reflection on the upper surface of the object 1. It is image Rn, and the direct image D0 on the lower surface of object 1 and the reflected image Rn on the lower surface of object 1 cannot be taken together. [0065] Therefore, the three-dimensional object measuring apparatus 100a of the second embodiment is configured to include not only the camera 30a above the mirror cylindrical body 20 but also the camera 30b below the mirror cylindrical body 20, so that the object 1 In addition to the upper image, the lower image as well as the direct image DO and the reflection image Rn are taken at the same time. The pedestal 10 is made of a transparent material such as a glass plate so that the object 1 can be photographed from below, and the mirror cylindrical body 20 needs to be extended below the pedestal 10.

 Since the imaging principle of the direct image DO and reflection image Rn of the lower surface of the object 1 using the lower camera 30b and the principle of the stereo measurement processing are the same as those in the first embodiment, the description thereof is omitted here.

 Example 3

 An example of the three-dimensional object measurement apparatus 100b according to the present invention according to the third embodiment is shown.

 The three-dimensional object measuring apparatus 100b according to the third embodiment has a camera 30 that is movable with respect to the mirror cylindrical body 20 with respect to the three-dimensional object measuring apparatus 100 according to the first embodiment. The camera 30 can move up and down along the central axis of the mirror cylinder 20, and the distance between the base 10 and the fisheye lens 31 is variable.

 The camera 30 may be moved manually by the user. However, the camera 30 can be moved by hand in combination with a power camera casing and a stepping motor mechanism. If the movement can be controlled, the distance between the fisheye lens 31 and the object on the pedestal 10 can be adjusted accurately.

Considering the distance between the base 10 and the fisheye lens 31, the following can be said.

 When the distance between the pedestal 10 and the fisheye lens 31 is small (when the distance between the object 1 and the camera 30 is short), there are advantages and disadvantages. Conversely, when the distance between the pedestal 10 and the fisheye lens 31 is large (object 1) (If the distance between the camera 30 and the camera 30 is far), there are advantages and disadvantages, and it is necessary to adjust the distance between the pedestal 10 and the fisheye lens 31 in combination.

FIG. 15 is a diagram schematically illustrating advantages and disadvantages when the distance between the base 10 and the fisheye lens 31 is small.

When the distance between the pedestal 10 and the fisheye lens 31 is reduced, the object can be enlarged by the camera 30, and a detailed direct image DO of the object surface can be obtained. The In other words, the resolution of the image directly increases. This is a merit.

 On the other hand, the incident angle to the fisheye lens 31 of the optical path A1 that forms the one-time reflected image R1 of the object reflected on the mirror surface of the inner wall of the mirror cylindrical body 20 is increased. That is, the height of the shooting viewpoint F1 is lowered, and the once reflected image R1 corresponds to an image obtained by shooting the object from a viewpoint at a low position in the oblique direction. The angle of incidence on the fisheye lens 31 of the optical path A2 that forms the twice reflected image R2 of the object reflected on the inner wall mirror surface of the mirror cylinder 20 becomes larger, and the twice reflected image R2 shows the object at a lower position in the oblique direction. This is equivalent to the image taken by the photographer, but the difference between the twice reflected image R2 and the one-time reflected image R1 since the once-reflected image R1 originally corresponds to the image photographed from the low-angled position in the diagonal direction. However, the amount of information is small as a two-dimensional image used for stereo measurement. This is a disadvantage.

 Next, FIG. 16 is a diagram schematically illustrating merits and demerits when the distance between the base 10 and the fisheye lens 31 is large.

 When the distance between the pedestal 10 and the fisheye lens 31 is increased, the object cannot be taken close-up by the camera 30 as compared with the case of FIG. 15, and the resolution of the direct image DO on the object surface is lowered. This is a disadvantage.

 On the other hand, the incident angle to the fisheye lens 31 of the optical path A1 forming the one-time reflected image R1 of the object reflected on the mirror surface of the inner wall of the mirror cylindrical body 20 is smaller than that in the case of FIG. In other words, the height of the photographing viewpoint F1 is increased, and the once reflected image R1 corresponds to an image obtained by photographing an object with a high viewpoint power in an oblique direction. The angle of incidence on the fisheye lens 31 of the optical path A2 forming the R2 reflection image R2 of the object reflected on the mirror surface of the inner wall mirror 20 of the mirror cylinder 20 is large, and the twice reflection image R2 shows the object from the viewpoint of a low position in the diagonal direction. Although it corresponds to the captured image, the change in the incident angle of the once reflected image R1 and the incident angle of the twice reflected image is larger than in the case of FIG. In other words, the difference between the two-time reflection image R2 and the one-time reflection image R1 is larger than that in the case of FIG. 15, and the amount of information for the two-dimensional image used for stereo measurement is larger than that in the case of FIG. . This is a merit.

[0072] Thus, the advantages and disadvantages that occur when the distance between the pedestal 10 and the fisheye lens 31 is large, and the advantages and disadvantages that occur when the distance between the pedestal 10 and the fisheye lens 31 is small are in a trade-off relationship. is there. Furthermore, as a variable amount, it is actually a 3D measurement. Since the size (height) of the bougietat affects, it is necessary to consider the distance between the fisheye lens 31 and the object 1, not the distance between the base 10 and the fisheye lens 31.

 Therefore, the three-dimensional object measuring apparatus 100b of the third embodiment is such that the camera 30 is movable up and down with respect to the mirror cylinder 20, and the camera 30 moves up and down along the central axis of the mirror cylinder 20. The distance between the base 10 and the fisheye lens 31 is variable.

 [0073] Note that instead of making the camera 30 movable, considering the configuration in which the position of the camera 30 is fixed and the camera 30 includes a zoom mechanism, the size of the image DO, in particular, the optical zoom mechanism can be directly mounted. For example, the resolution of the image DO can be directly manipulated, but the height of the photographing viewpoint F1 of the reflected image R1 and the photographing viewpoint F2 of the reflected image R2 cannot be manipulated. In addition, when the zoom mechanism is used, the surrounding image cannot be captured instead of the center image appearing large, and the higher-order reflected image Rn, and in some cases the twice-reflected image R2 and the once-reflected image R1 are captured in the captured image. There is also a risk of being lost. Therefore, the three-dimensional object measuring apparatus 100b of the third embodiment attaches importance to manipulating the height of the shooting viewpoint F1 of the reflected image R1 and the shooting viewpoint F2 of the reflected image R2 while considering the height of the object 1, The camera 30 can move up and down along the central axis of the mirror cylindrical body 20, and the distance between the base 10 and the fisheye lens 31 is variable.

 However, the above description is not intended to exclude the camera 30 equipped with an optical zoom mechanism. The camera 30 of the three-dimensional object measuring apparatus 100 of the present invention is not equipped with an optical zoom mechanism or a digital zoom mechanism. Of course, it is possible to perform zoom shooting within the range that the reflected image that is the surrounding image can be captured in the image.

 Example 4

 [0074] An example of a three-dimensional object measurement device 100c according to the present invention according to a fourth embodiment will be described.

The three-dimensional object measurement apparatus 100c of the fourth embodiment enables relative movement of the mirror cylindrical body 20 and the camera 30 with respect to the base 10 with respect to the three-dimensional object measurement apparatus 100 of the first embodiment. As shown in Fig. 3, with the pedestal 10 fixed, the center of the pedestal 10 is the center of rotational movement, and the mirror cylinder 20 and the camera 30 are integrated in a three-dimensional space. It enables free rotational movement.

 The mirror cylinder 20 may be rotated by the user by hand, but the mirror cylinder 20 may be rotated by hand, but the mirror cylinder 20 is combined with the stepping motor mechanism, and the mirror cylinder 20 according to the user's operation input. If the 20 movements can be controlled, the direction of the photographing viewpoint of the camera 30 can be accurately adjusted so as to obtain an angle that enables photographing of a powerful part that cannot be photographed in the basic posture.

 [0075] The three-dimensional object measuring apparatus 100c according to the fourth embodiment has a blind spot depending on the shape and direction of the object 1 to be three-dimensionally measured in the basic posture, and even when a portion where the direct image DO or the reflected image Rn cannot be obtained occurs. The mirror cylindrical body 20 and the camera 30 are integrated to change the angle with respect to the pedestal 10 to make another angle, and a direct image DO or reflected image Rn can be obtained even for a part that is a blind spot and a captured image cannot be obtained. Is given.

 [0076] The following three cases are generally assumed when there is a blind spot depending on the shape and direction of the object 1 and a portion where the direct image DO or the reflected image Rn cannot be obtained.

 [0077] The first is a case where the object 1 has a slightly deep concave shape. For example, if the concave shape is on the top surface, the image DO can be obtained directly for the part, but the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and photographing viewpoint F2 have a concave bottom surface portion. May not appear in the blind spot at the periphery. In addition, for example, when the concave shape is on the side, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and photographing viewpoint F2 facing directly to the part can be obtained, but the direct image DO In some cases, however, the bottom surface of the concave shape will not appear as a blind spot at the periphery.

 [0078] The second is a case where the side surface of the object 1 is nearly vertical. In this case, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and the photographing viewpoint F2 facing the part can be obtained, but the direct image DO has an angle that is too large and an effective image can be obtained. There may be cases. Even if multiple reflection images Rn can be obtained, direct image DO is important information in 3D stereo measurement, so it is recommended to devise the shooting direction to obtain direct image DO. Don't say anything.

 The third is a case where the shape of the object 1 is complicated and a part thereof is behind other parts.

Typically, it is a site like the base of an ankle joint in insects. For example, Thus, although the image DO can be obtained directly, the reflected image R1 and the reflected image R2 obtained from the oblique photographing viewpoint F1 and the photographing viewpoint F2 may not be reflected behind other parts. The 3D object measuring device 1 OOc of the fourth embodiment integrates the mirror cylindrical body 20 and the camera 30 even when a part is generated because a blind spot is formed depending on the shape and direction of the object 1 and a direct image DO or reflection image Rn is not obtained. As a result, the angle with respect to the base 10 is changed, and an image of another angle is obtained for obtaining the necessary direct image DO or reflection image Rn for the part.

[0080] Here, as an example, a case where the side surface of the object 1 is nearly vertical will be described.

 FIG. 18 is a diagram schematically showing light reception of the direct image and the reflected image in the camera 30 in the longitudinal section of the three-dimensional object measuring apparatus 100c, as in FIG. As shown in FIG. 18, the optical path B 0 corresponding to the direct image DO cannot directly enter the fish-eye lens 31 with respect to the point B on the side surface portion of the object 1. In other words, the direct image DO does not include the image of the point B on the side surface portion, or even if it is included, effective information cannot be obtained because the shooting angle is extremely shallow. On the other hand, the once reflected optical path B1 corresponding to the reflected image R1 is incident on the fisheye lens 31, and the image of the point B on the side surface portion is included in the once reflected image R1. Similarly, the twice-reflection optical path B2 corresponding to the reflection image R2 is incident on the fish-eye lens 31, and the image of the point B on the side surface portion is included in the twice-reflection image R2. As described above, the plurality of reflection images R1 and R2 include image information regarding the point B, but information regarding the point B is not directly obtained in the image D0.

Next, FIG. 19 shows the basic posture of FIG. 18, in the state where the pedestal 10 is fixed, the center of the pedestal 10 is the center of the rotational movement, and the mirror cylinder 20 and the camera 30 are integrally rotated clockwise in the vertical plane. Shows a state of rotational movement. As shown in FIG. 19, the point B on the side surface portion of the object 1 is a position that can be seen well from the photographing viewpoint of the camera 30, and the optical path B0 corresponding to the direct image D0 directly enters the fisheye lens 31. That is, the image of the point B on the side surface portion is effectively included in the direct image D0. The once-reflected optical path B1 corresponding to the reflected image R1 is also directly incident on the fisheye lens 31. The image of point B is included. Similarly, the image of the point B on the side portion is also included in the twice reflected image R2.

 As described above, sufficient image information cannot be obtained in the direct image DO in the basic posture in FIG. 19, and the image B sufficient in the direct image DO in the posture in FIG. Information will be obtained.

Note that the posture in FIG. 19 is an example of rotation for obtaining a direct image DO of the point A on the right side of the object 1 that is sharp, but the left side of the object 1, the front (front side), To obtain a direct image DO of the other side such as the back (back side), separately rotate the mirror cylinder 20 and the camera 30 so that the target side faces the fisheye lens 31. ,. Here, the rotation control means is not limited! However, for example, a stepping motor or the like may be used.

 The object of the 3D object measurement apparatus of the present invention is to obtain a plurality of images used for 3D stereo measurement with a small number of imaging operations. In addition, the image is taken one more time at different angles, but multiple images can be obtained by taking two images of a part that is affected by the shape and posture of the object and is difficult to shoot. However, it can be said that it is in line with the object of the present invention to obtain a plurality of images used for three-dimensional stereo measurement with a small number of photographing.

 While the preferred embodiment of the invention has been illustrated and described, it will be appreciated that various changes can be made without departing from the scope of the invention.

 Example 5

In Example 5, a prototype of the three-dimensional object measurement apparatus of the present invention is manufactured and the shape of the actual object is measured. The object measurement environment of the fifth embodiment showing the result of shape measurement data using the actual measurement image will be described below. The camera used for the measurement was a DepictDlE manufactured by Opteon, and used a 1024 X 1024 (pixel s) image cut out from an image taken at I ³⁹² X 1 (Μ0 (pixels). A ring light was used as the light source. The cylindrical mirror has an inner diameter of 90 (mm) and a height of 100 (mm), but the camera used in this example can only obtain a grayscale image. Therefore, in Equation 9 above L ^S. (T, w) and 1 ^T (t, w) (i = r, g, b) are expressed by Equation 10 below. Here, I ^s (t, w) and 1 ^T (t, w) are obtained from the source and target, respectively.

 This is the brightness value of the captured image. The range of possible values is 0 or more and 255 or less for both.

 [Equation 10]

In this embodiment, a conical object shown in FIG. 20 is used as a measurement target.

 The cone-shaped object has a bottom diameter of 56 (mm) and a height of 34 (mm), and the cone surface has a grayscale scene image as a texture.

 When the cone-shaped object shown in FIG. 20 was photographed with the three-dimensional object measuring apparatus of the present invention, the photographed image shown in FIG. 21 was obtained. In this example, the shape of the cone object was measured from this image.

 Here, the SSD window size in this embodiment was set to 5 × 5 (pixels). In addition, the scale s in Equation 5 above was changed by 0.1 from 0.5 to 2.0, and the shape was measured by stereo vision using the image directly observed by the camera and the image reflected once by the cylindrical mirror. FIG. 22 shows the result of cone shape measurement by the three-dimensional object measurement apparatus of the present invention.

 FIG. 23 shows a scatter diagram in which the horizontal axis represents the distance from the center of the image and the vertical axis represents the height of the cone shape measured by the three-dimensional object measuring apparatus of the present invention.

[0086] As described above, as a result of an attempt to measure the shape of an actual object, it has been shown that the entire circumference of an actual object can be measured with a single image using the three-dimensional object measurement device of the present invention. According to the fifth embodiment, this 3D object measurement device can measure the entire shape of an object with a simple device configuration consisting of only a camera and a cylindrical mirror and a simple shooting process of shooting only one image. It has been shown that it is useful for the measurement of the entire circumference. Industrial applicability [0087] The three-dimensional object measurement apparatus of the present invention can obtain a multi-viewpoint image from all directions of an object by one shooting by putting an object in a cylindrical mirror and shooting the object from above with a camera. Because it is possible to measure the three-dimensional shape of a single object with reflection / refraction stereo vision, it can be used for content creation in educational fields such as electronic pictorial books for insects and small animals. By using this content, it is possible to enhance the description of the video. This 3D object measuring device can be used not only in the education field, but also in the medical field and academic research field.

 The 3D object measuring device of the present invention allows the user to easily measure the shape of personal belongings or to efficiently measure the entire circumference of many objects because of the simplicity of the device configuration and imaging process. It is useful for the application. In addition, since the entire shape of the object can be measured from a single image, it is possible to record the entire shape of the object along with the movement of the animal if images are taken continuously. Can be improved.

 Brief Description of Drawings

 [0088] [FIG. 1] A diagram schematically showing the basic configuration of the three-dimensional object measuring apparatus according to the first embodiment. [FIG. 2] A longitudinal section of a direct image and a reflected image received by a camera. Schematic representation in

 [FIG. 3] A diagram schematically showing an image obtained by photographing a cone-shaped object placed on a pedestal with a camera.

 [Figure 4] Direct image DO for object point A, single reflection image Rl, double reflection image R

The figure which showed the mode that the photography viewpoint of 2 was developed equivalently

 [Fig.5] Diagram showing a direct image of a pyramid-shaped object placed at the center of the pedestal and a reflection image reflected on the mirror surface of the inner wall of the mirror cylinder

 [Fig. 6] Diagram showing that the geometric relationship of feature points is maintained even when the object is placed at a position deviated from the center of the pedestal.

 [Fig.7] A diagram showing an example of the trajectory of a ray incident on the camera after the object's single-point force is reflected twice by the mirror.

 [Figure 8] Figure showing a simulation image of a virtual object

[Figure 9] Relationship between angle φ and distance d of image center force 圆 10] Image taken as a result of polar coordinate expansion

[Fig.11] Diagram showing search area

[12] Block diagram showing components for executing stereo measurement processing in the three-dimensional object measurement apparatus of the present invention

圆 13] Diagram showing the basic configuration of the 3D object measurement device according to Example 2 圆 14] Diagram showing the basic configuration of the 3D object measurement device according to Example 3 圆 15 ] Diagram explaining the advantages and disadvantages of a small distance between the pedestal and the fisheye lens

圆 16] Diagram explaining the advantages and disadvantages of a large distance between the pedestal and the fisheye lens

圆 17] Diagram showing the basic configuration of the 3D object measurement device according to Example 4 圆 18] The optical path of the direct image and the reflected image in the basic posture in the vertical section of the 3D object measurement device. Figure expressed in

 [Figure 19] Diagram showing a state where the mirror cylinder and the camera are rotated 45 degrees clockwise in the vertical plane with the center of the pedestal as the center of rotational movement.

 [Figure 20] Diagram showing a real object with a conical shape used as a measurement target

 [Fig.21] Photographed image of real object with conical shape used as measurement target

圆 22] Diagram showing cone shape measurement results using measured images

[23] Figure showing the relationship between the distance from the center of the cone and the height

Explanation of symbols

 1 object

 10 pedestal

 20 Specular cylinder

 30, 30a, 30b camera

 31 Fisheye lens

 32 Shooting and recording means

 40 Image data processor

41 Feature point extraction means Corresponding feature point search means

3D image data generation means

, 100a, 100b, 100c 3D object measuring device

Claims

The scope of the claims

 [1] A pedestal on which a 3D object to be photographed is placed,

 The pedestal is surrounded by the inner wall surface of the cylinder, the mirror surface cylindrical body having the inner wall surface of the cylinder as a mirror surface, and the lens optic axis is arranged so as to coincide with the cylindrical central axis of the mirror surface cylindrical body. A camera having a fisheye lens and a photographing recording means for recording an image obtained through the fisheye lens;

 A direct image obtained by directly viewing the three-dimensional object on the pedestal by the photographing recording means of the camera, and a reflected image of the three-dimensional object on the pedestal reflected on the mirror surface of the inner wall of the specular cylindrical body, A three-dimensional object measurement device that captures images.

[2] feature point extraction means for extracting and determining feature points on a photographed image taken by the camera recording means;

 The corresponding feature point on the reflection image corresponding to the feature point on the direct image extracted by the feature point extraction unit is searched and determined on the reflection image, or the feature point extraction unit Corresponding feature point search means for searching and determining the corresponding feature point on the direct image corresponding to the feature point on the reflection image extracted by the direct image;

 Based on the correspondence between the feature points and the corresponding feature points, stereo measurement processing of the direct image and the reflection image is performed 1, and the three-dimensional object of the three-dimensional object is obtained from the direct image and the reflection image. 3D image data generating means for generating image data is provided,

 The corresponding feature point searching means searches for the corresponding feature point by searching the extension line connecting the center of the image and the feature point to the photographed image. The three-dimensional object measurement device according to claim 1.

[3] The pedestal is made of a transparent material, and the object is visible from below the pedestal,

 The camera has two cameras, a first camera and a second camera,

3. The three-dimensional object measurement apparatus according to claim 1, wherein the fisheye lens of the first camera and the fisheye lens of the second camera are arranged so as to face each other with the pedestal interposed therebetween.

[4] The camera according to any one of claims 1 to 3, wherein the camera is capable of moving along the cylindrical central axis with respect to the mirror cylindrical body, and the distance between the pedestal and the fisheye lens is variable. 3D object measuring device.

 [5] Enabling relative movement between the pedestal, the mirror cylinder, and the camera,

 5. The device according to claim 1, wherein the pedestal is fixed, and the center of the pedestal is used as a center of rotational motion to freely rotate the mirror cylindrical body and the camera integrally in a three-dimensional space. A three-dimensional object measuring device according to the above.