JP6237032B2 - Color and three-dimensional shape measuring method and apparatus - Google Patents

Description

  The present invention relates to a method, apparatus, and program for measuring the color and three-dimensional shape of a target object from a group of images taken by sequentially illuminating the object with a plurality of illumination light sources.

  Measurement of the color and three-dimensional shape of an object plays an important role in fields such as computer graphics (CG), digital archiving, and quality inspection. For example, in video production such as movies, in order to make a desired real object appear in a live-action scene or a scene created by CG, the information on the color and three-dimensional shape of the target object is measured and digitized, A technique is used in which lighting is virtually performed on the CG in accordance with the scene and an image is synthesized. In digital archives of art and cultural properties, it is required to digitize the color and three-dimensional shape of the target material with high accuracy as a means of researching the material, recording the current status, or disclosing it. Further, in quality inspection on a product production line, information on color and three-dimensional shape may be used to determine whether a product is non-defective or defective. There are two types of measurement methods, contact and non-contact, but non-contact measurement technology can be used in a wide range because measurement can be performed in a shorter time than contact method without damaging the target object. ing.

  There are two methods for measuring the three-dimensional shape of an object in a non-contact manner: an active method that irradiates a target object with light or radio waves to assist measurement, and a passive method that does not require such assistance. Separated. Among them, there are a pattern projection method and an illuminance difference stereo method as an active method and a stereo method as a passive method that can measure color information simultaneously with the three-dimensional shape of an object.

  The pattern projection method projects a plurality of pattern lights (such as a fringe pattern and a sine wave pattern) from a projector such as a digital projector onto a target object, and shoots with an imaging device such as a digital camera. This technique restores the three-dimensional shape of a target object based on the principle of triangulation by photographing the target object from two or more different positions with an imaging device such as a digital camera. In these techniques, if, for example, a digital camera or the like capable of obtaining an RGB three-channel image is used as the imaging device, approximate color information of the target object can be obtained.

  However, with these methods, the color information of the target object itself, in particular, brightness and darkness cannot be obtained correctly. In general, the reflected light of an object surface has a diffuse reflection component and a specular reflection component, and the color information of the object appears in the diffuse reflection light. Diffuse reflected light is composed of the interaction of three elements: diffuse albedo, which is color information of the object itself, the orientation of the surface of the object, and the angle formed by the illumination direction. In pattern projection and stereo methods, The reason is that since the direction of the illumination light is not defined at the time of photographing for obtaining color information, these elements cannot be separated.

  The illuminance-difference stereo method irradiates a target object with light from a plurality of different directions, captures the target object for each illumination by an imaging device such as a digital camera, and performs color information (of the imaging device) on the target object in units of image pixels. Determine the diffusion albedo for each channel and the orientation of the surface (normal vector). This method is characterized in that a diffusion albedo, which is color information of the target object itself, can be obtained without depending on the direction of the surface of the target object and the direction of illumination. Further, the three-dimensional shape of the target object can be restored by connecting the orientations of the surfaces obtained in units of pixels.

  In order to perform illuminance difference stereo, it is a precondition that the direction and intensity of illumination light with respect to each position on the target object surface are known. For example, Patent Document 1 is characterized in that a reflectance image is generated by illuminance difference stereo. However, in constructing the apparatus, prior calibration of the light source position and input to the system are required. In such a system that requires pre-calibration of the position and intensity of illumination, if the size of the target object is always within a certain range, the calibration operation can be performed only once and the result can be applied to all imaging. There is an advantage. On the other hand, the system cannot be used for target objects that exceed the size allowed by the system, or the light source must be relocated and recalibrated to use the system. There are disadvantages such as.

  On the other hand, a technique for performing illuminance difference stereo is proposed even when the position of the light source is unknown.

  For example, in Non-Patent Documents 1 to 5, when the illumination light used at the time of photographing is parallel light (when there is no problem assuming that the light source is located at infinity with respect to the target object, or using an optical system such as a lens) Describes a method for realizing illuminance difference stereo even when the direction of illumination is unknown. When illuminance difference stereo is performed using parallel light whose direction is unknown, a problem called generalized bas-relief indefiniteness (GBR indefiniteness) occurs. This is a problem of non-uniqueness in which all normal vectors obtained for each pixel of the image are given the same rotation, and the solution is established. On the other hand, Non-Patent Document 1 solves the GBR ambiguity by using information on gloss generated on the subject. Non-Patent Document 2 solves the GBR ambiguity by using ring illumination in which a plurality of light sources are arranged on the circumference of a circle that is centered on the optical axis of the imaging system and is perpendicular to the optical axis. In Non-Patent Document 3, GBR ambiguity is solved using subject color information. In Non-Patent Document 4, GBR ambiguity is solved by using a three-dimensional shape of a target object acquired by a laser range sensor. In Non-Patent Document 5, GBR ambiguity is solved by using pixel information that maximizes diffuse reflected light among a plurality of images having different illumination directions.

  In these technologies, it is an object to solve the GBR ambiguity caused by the illumination being parallel light. However, in photographing in a limited space such as indoors, it is often difficult to take a distance between the target object and the light source so that incident light on the target object can be regarded as parallel light. Although it is conceivable to use non-parallel light emitted from a light source located not far from the target object as illumination light, a dedicated optical system device or the like is required.

  In order to deal with these problems, a method for realizing illuminance difference stereo in shooting under the condition that the distance between the illumination light source and the target object is close and the position of the illumination light source is unknown has been proposed as a more realistic situation. ing.

  For example, in Non-Patent Document 6, the position of one point light source is fixed with respect to the camera, and the subject is photographed while changing the position and orientation of the camera, thereby using multi-viewpoint stereo technology and illuminance difference stereo technology. To restore the diffuse albedo and shape of the subject. However, in this method, the photographer must perform a large number of photographing while moving the camera. Another problem is that it is difficult to align each image for an object with little change in color and undulation.

  Further, in Non-Patent Document 7, illuminance difference stereo in the case where the light source position is unknown is realized by using a proximity light source that is relatively close to the target object. However, what is described here is a method for solving the above-mentioned GBR ambiguity by using a proximity light source, and as a precondition, an image photographed using parallel light in three or more directions is required.

JP 2006-285663 A

O. Drbohlav and R. Sara, Specularities Reduce ambiguity of Uncalibrated Photometric Stereo, Proc. 7th European Conference on Computer Vision, pp. 46-60 (2002) Z. Zhou and P. Tan, Ring-light Photometric Stereo, Proc. 11th European Conference on Computer Vision, pp. 265-279 (2010) B. Shi, Y. Matsushita, Y. Wei, C. Xu, and P. Tan, Self-calibrating Photometric Stereo, Proc.IEEE Conference on Computer Vision and Pattern Recognition, pp. 1118-1125 (2010) Kamikawa, Miyazaki, Baba, Furukawa, Aoyama, Hiura, and Asada, “High-definition 3D shape using photometric stereo method under unknown light source orientation”, Image Recognition and Understanding Symposium (2012) P. Favaro and T. Papadhimitri, A Closed-Form Solution to Uncalibrated Photometric Stereo via Diffuse Maxima, Proc.IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 821-828 (2012) T. Higo, Y. Matsushita, N. Joshi and K. Ikeuchi, A Hand-held Photometric Stereo Camera for 3-D Modeling, Proc.IEEE Conference on Computer Vision (2009) Okabe and Sato, “Do near-point light sources solve the indeterminacy of shape restoration in uncalibrated illuminance stereo?”, Image Recognition and Understanding Symposium, 2007 D. Nehab, S. Rusinkiewicz, J. Davis, R. Ramamoorthi, Efficiently Combining Positions and Normals for Precise 3D Geometry, Proc. SIGGRAPH (2005)

  The present invention has been made in view of the above-described problems, and in the illuminance difference stereo method using an image group obtained by sequentially illuminating an object with a plurality of illumination light sources, the position of the illumination light source at the time of shooting is the object to be imaged. It is an object of the present invention to provide a method, an apparatus, and a program for obtaining color information and a three-dimensional shape of a target object with high accuracy without performing preliminary calibration of the light source position under close proximity light source conditions.

A first aspect of the present invention provided to solve the above-described problem is a measuring apparatus that measures the color and three-dimensional shape of an object, and measures the rough shape of a rough three-dimensional shape of an object to be measured. Means, an image pickup means for picking up an image of an object to be measured with three or more channels, an illumination means comprising four or more illumination light sources used for picking up an object, and four or more of the illumination means The illumination light source is exclusively turned on, and an image input unit that sequentially captures an object by the imaging unit and obtains a captured image for each illumination light source, and a plurality of captured images and a rough three-dimensional shape, includes four or more Light source position estimation means for estimating the position at the time of imaging for each of the illumination light sources, and object restoration means for calculating the color information and three-dimensional shape information of the object based on the positions of the respective illumination light sources obtained by the light source position estimation means When Equipped with a measuring device.

  In the first aspect, at least one of the four or more illumination light sources may be fixed in a relative positional relationship with the other one or more illumination light sources.

  In addition, the light source position estimation unit in the first aspect includes a sample pixel extraction unit that selects a plurality of sample pixels from a region including an object to be measured in the captured image, and a three-dimensional position and orientation of the illumination light source. A light source attitude calculation means for calculating the position and orientation of each of four or more illumination light sources from the rotation parameter and the movement parameter, and the position and orientation of the light source calculated by the light source attitude calculation means. Incident light calculation means for calculating the direction of each light source and the radiance of light emitted from each light source at each point on the surface of the measurement target object corresponding to the sample pixel, using rough three-dimensional shape information of the measurement target object All sample pixels using the direction of each light source calculated by the incident light calculation means, the radiance of the light emitted from each light source, and the sensor response at the sample pixel. About the surface calculation means for calculating the normal vector and the diffusion albedo of the measurement object, the normal vector and the diffusion albedo calculated by the surface calculation means, and the direction of each light source and each light source calculated by the incident light calculation means Sensor response model value calculating means for calculating a sensor response model value for all sample pixels and all light sources using the radiance of light emitted from the sensor, and a sensor calculated by the sensor response model value calculating means An error calculation unit that calculates an error between the response model value and the sensor response of the captured image, and a rotation parameter and a movement parameter that represent the position and orientation of the illumination light source that minimize the error calculated by the error calculation unit are nonlinearly optimized. And a light source attitude calculating means determined by conversion.

  The light source position estimation means including each of the above means illuminates an object by illuminating four or more illumination light sources exclusively, and illuminates an image captured for each illumination light source obtained by sequentially imaging the object with an imaging device. It can be configured as a light source attitude estimation device that estimates the position and orientation of a light source.

According to a second aspect of the present invention, there is provided a measuring method for measuring the color and three-dimensional shape of an object, the rough shape measuring step for measuring a rough three-dimensional shape of an object to be measured, and four or more illumination light sources. The image input step of sequentially capturing an object with an imaging device having three or more channels and obtaining a captured image for each illumination light source, a plurality of captured images, and a rough three-dimensional shape , 4 Based on the light source position estimation step for estimating the position at the time of imaging for each of the plurality of illumination light sources and the position of each illumination light source obtained by the light source position estimation step, the color information and three-dimensional shape information of the object are calculated. A measurement method comprising an object restoration step.

  The light source position estimation step in the second aspect includes a sample pixel extraction step for selecting a plurality of sample pixels from a region including an object to be measured in the captured image, and a position and orientation of the illumination light source as a three-dimensional object. Rotation and movement of the light source orientation calculation step for calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter, the position and orientation of the light source obtained in the light source orientation calculation step, and measurement Incident light calculation step for calculating the direction of each light source at each point on the surface of the measurement target object corresponding to the sample pixel and the radiance of the light emitted from each light source using the rough three-dimensional shape information of the target object; Using the direction of each light source obtained in the incident light calculation step, the radiance of the light emitted from each light source, and the sensor response at the sample pixel For all sample pixels, the surface calculation step for calculating the normal vector and the diffusion albedo of the measurement target object, the normal vector and the diffusion albedo obtained in the surface calculation step, and the respective light sources obtained in the incident light calculation step Sensor response model value calculation step for calculating sensor response model values for all sample pixels and all light sources using the direction and radiance of light emitted from each light source, and sensor response model value calculation step An error calculation step for calculating an error between the obtained sensor response model value and the sensor response of the captured image, and a rotation parameter and a movement parameter indicating the position and orientation of the illumination light source that minimize the error obtained in the error calculation step. And a light source attitude calculation step determined by nonlinear optimization.

  In addition, the light source position estimation step including the above steps illuminates an object by illuminating an object by exclusively turning on four or more illumination light sources, and illuminating an image captured for each illumination light source obtained by sequentially imaging the object with an imaging device. It can be configured as a light source posture estimation method for estimating the position and orientation of the light source.

  Typically, the measurement method and the light source attitude estimation method in the second aspect described above are realized as a measurement program that is executed by a computer of a measurement device that measures the color and three-dimensional shape of an object, and is recorded on a recording medium or the like. obtain.

  As described above, according to the present invention, four or more illumination light sources are exclusively turned on, and a captured image for each illumination light source obtained by sequentially imaging an object with an imaging device having three or more channels. From this, it is possible to estimate the position and orientation of the illumination light source at the time of photographing, and use the information to derive color information and three-dimensional shape information of the target object. As a result, an apparatus and a method for deriving the color information and three-dimensional shape information of an object without performing prior calibration of the light source position as described above can be provided.

Explanatory drawing showing the 1st apparatus structural example of the imaging | photography system which concerns on this invention. Explanatory drawing showing the 2nd apparatus structural example of the imaging | photography system which concerns on this invention. Explanatory drawing of the light source set in the 2nd apparatus structural example Explanatory drawing of the nonlinear optimization process of the light source position estimation which concerns on this invention Explanatory drawing of evaluation function processing in the above nonlinear optimization Explanatory drawing of preprocessing in the above nonlinear optimization Explanatory drawing showing an example of a plane defining method for acquiring a rough three-dimensional shape of a target object

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

<Principle of photometric stereo>
First, the principle of illuminance difference stereo will be described. In illuminance difference stereo, the object is irradiated with light from three or more directions, each of which is imaged, and the diffusion albedo and normal vector of the target object are derived in pixel units from the obtained sensor response.

When the reflection of light on the surface of an object follows the Lambert model, the target object is illuminated with parallel light, or illuminated with a minute light source that can be regarded as a point light source, and captured by an imaging device such as a digital camera. The sensor response I in the pixel corresponding to a certain minute area on the object surface is formulated by the equation (1) (for the sake of simplicity, the notation related to the pixel position and the wavelength of light is omitted).

  In equation (1), ρ is a diffusion albedo on the surface of the target object, n is a normal vector (unit length) of 3 rows and 1 column indicating the orientation of the surface of the target object surface, and l is 3 from the target object surface toward the light source direction. A light source vector (unit length) in row 1 column, R is the radiance of incident light on the surface of the target object, α is a normalization coefficient based on the ratio of the radiance of reflected light on the surface of the target object and the sensor response, T Represents transposition of vectors and matrices.

When the target object is irradiated with light from N different directions and the respective objects are photographed, I, l, and R with respect to the i-th illumination direction (i = 1, 2,..., N) are respectively represented by I _i. , L _i , R _i , equation (2) holds.

光源ベクトルｌ_ｉ及び入射光の放射輝度Ｒ_ｉが全ての光源について既知であり、かつ、Ｎ≧３のとき、式（５）により拡散アルベドρと法線ベクトルｎの転置の積を求めることができる。

Therefore, when defined as Equation (3) and Equation (4),

When the light source vector l _i and the radiance R _{i of the} incident light are known for all the light sources and N ≧ 3, the product of the transposition of the diffusion albedo ρ and the normal vector n can be obtained by Equation (5). it can.

Here, since the normal vector n has a unit length, ρn ^T obtained by Expression (5) can be uniquely separated into the diffusion albedo ρ and the normal vector n. That is, the length of the vector ρn ^T is the diffusion albedo ρ, and the normalization of the normal vector n is obtained by normalizing the vector ρn ^T to the unit length.

  Since the normal vector represents the orientation of a surface in a minute area on the surface of the target object, the shape of the entire target object can be restored by connecting the surfaces with adjacent pixels. In the above description, for the sake of simplicity, the notation regarding the wavelength of light is omitted. However, if a digital camera or the like that can obtain an RGB three-channel image is used as the imaging device, a diffusion albedo for each of the RGB channels in each pixel is used. Can be obtained. Therefore, the illuminance difference stereo is characterized in that not only shape information such as a laser range finder but also color information of a target object can be measured as well as shape information.

  In illuminance-difference stereo using an actual imaging device, an error may be included in a derived normal vector due to an error included in the light source vector l or incident light radiance R, an image noise, or the like. In this case, if the shape is restored by connecting the surfaces obtained in units of pixels, there is a possibility that the accuracy of shape restoration is reduced due to accumulation of errors. For this reason, a method has been proposed in which a rough three-dimensional shape of the target object is acquired by another method, and this is combined with a normal vector obtained by illuminance difference stereo to process the high-definition three-dimensional shape. (For example, Non-Patent Document 8).

<Self-calibrated illuminance difference stereo>
Next, when the light source position at the time of photographing is unknown, that is, the illuminance difference stereo when the light source vector l _i and the radiance R _{i of the} incident light are unknown at each point on the target object surface, the self according to the present invention. The apparatus configuration and processing method of calibration illuminance difference stereo will be described.

[Device configuration example 1]
First, a first apparatus configuration example of the photographing system according to the present invention will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating a first apparatus configuration example of a photographing system according to the present invention.
In FIG. 1, the first device includes a digital camera 11, LED lamps 12 to 15, a computer 16, a keyboard 17, and a monitor 18. The digital camera 11 is a digital camera for photographing digital images of three channels of R, G, and B, and is equipped with an appropriate lens for photographing a target object. The LED lamps 12 to 15 are used as illumination light sources when the target object is photographed by the digital camera 11. The computer 16 includes a hard disk as an external storage device, a memory as a temporary storage device, a CPU as a calculation device, etc., controls the digital camera 11 to capture a photographed image, and turns on / off the LED lamps 12 to 15. To control. In addition, the computer 16 performs a process of calculating the three-dimensional shape and color information of the target object using the captured image. However, the computer that captures the captured image and the computer for data processing may be different computers. The keyboard 17 is an input interface for the user to operate the computer 16. The monitor 18 is an output interface for displaying the processing result of the computer 16.

  First, the user adjusts the position and orientation of the digital camera 11 so that the target object is appropriately included in the captured image of the digital camera 11, and focuses. Next, the LED lamp 12 is installed at an appropriate position and orientation that can illuminate the target object. Then, only the LED lamp 12 is turned on, and the digital camera 11 performs photographing with an appropriate exposure amount. After the photographing is completed, the LED lamp 12 is turned off. Thereafter, all the remaining LED lamps 13 to 15 are similarly photographed. During shooting, it is preferable that light other than the illumination light from the LED lamps 12 to 15 does not strike the target object. However, if this is not possible, the target object is shot with all the LED lamps 12 to 15 turned off. Then, by taking the difference from each image captured by lighting each of the LED lamps 12-15, an image equivalent to a captured image in a state where no light other than the LED lamps 12-15 exists can be obtained. .

  In addition, in FIG. 1, the form which uses four LED lamps 12-15 is shown, and the number of illumination light sources may be four on the principle of self-calibration illuminance difference stereo. However, depending on the shape of the target object, illumination from a specific direction may be shielded against a part of the target object, so that it is also preferable to use five or more illumination light sources.

  The photographing system according to the present invention is not limited to the configuration shown in FIG. 1. For example, the digital camera 11 can be directly operated by the user without being controlled by the computer 16. Similarly, the LED lamps 12 to 15 may be configured to be turned on or off directly by the user without being controlled from the computer 16. The illumination light source is not limited to the LED lamp, but is preferably sufficiently small with respect to the size of the target object. In FIG. 1, the LED lamps 12 to 15 are illustrated as individual lamps, but it is also possible to sequentially photograph the target object while changing the position using one LED lamp.

[Device configuration example 2]
Next, a second apparatus configuration example of the photographing system according to the present invention will be described with reference to FIGS. FIG. 2 is an explanatory diagram illustrating a second apparatus configuration example of the imaging system according to the present invention. FIG. 3 is a diagram for explaining an example of the structure of a light source set used in the second apparatus configuration of FIG.

  In FIG. 2, the second apparatus includes a digital camera 21, light source sets 22 and 23, a computer 26, a keyboard 27, and a monitor 28. The digital camera 21 is a digital camera for photographing digital images of three channels of R, G, and B, and is equipped with an appropriate lens for photographing a target object. As shown in FIG. 3, the light source sets 22 and 23 have a structure in which a plurality of LED lamps 31 to 34 used as illumination light sources are fixed on a frame. The positional relationship between the LED lamps 31 to 34 is known. The computer 26 includes a hard disk as an external storage device, a memory as a temporary storage device, a CPU as an arithmetic device, and the like, controls the digital camera 21 to capture a photographed image, and LED lamps on the light source sets 22 and 23. The lighting / extinguishing of 31-34 is controlled. Further, the computer 26 performs a process of calculating the three-dimensional shape and color information of the target object using the captured image. However, the computer that captures the captured image and the computer for data processing may be different computers. The keyboard 27 is an input interface for the user to operate the computer 26. The monitor 28 is an output interface for displaying the processing result of the computer 26.

  First, the user adjusts the position and orientation of the digital camera 21 so that the target object is appropriately included in the captured image of the digital camera 21, and focuses. Next, the light source set 22 is installed at an appropriate position and orientation in which the LED lamps 31 to 34 on the light source set 22 can illuminate the target object. Then, only the LED lamp 31 on the light source set 22 is turned on, and photographing is performed by the digital camera 21 with an appropriate exposure amount. After the photographing is completed, the LED lamp 31 on the light source set 22 is turned off. Thereafter, all the remaining LED lamps 32 to 34 on the light source set 22 are imaged in the same manner. Similarly to the light source set 22, the light source set 23 is installed at an appropriate position and orientation, and the lamp is turned on, photographed, and turned off sequentially. During shooting, it is preferable that light other than illumination light from the LED lamps 31 to 34 is not applied to the target object. However, if this is not possible, the target object is shot with all the LED lamps 31 to 34 turned off. Then, by taking the difference from each image captured by turning on each of the LED lamps 31 to 34, an image equivalent to a captured image in a state where there is no light other than the LED lamps 31 to 34 can be obtained. .

  FIG. 2 shows a form in which two light source sets 22 and 23 in which four LED lamps 31 to 34 are arranged on a frame are used. However, on the principle of self-calibration illuminance difference stereo, The total number may be four. However, depending on the shape of the target object, illumination from a specific direction may be shielded against a part of the target object, so that it is also preferable to use five or more illumination light sources. For example, a form in which only one light source set with four LED lamps is used, a form in which two light source sets with three LED lamps are used, or a plurality of light source sets each having a different number of LEDs A form having a lamp may be used.

  The photographing system according to the present invention is not limited to the configuration shown in FIG. 2. For example, the digital camera 21 may be directly operated by the user without being controlled by the computer 26. Similarly, the LED lamps 31 to 34 are not controlled from the computer 26, and the user can directly turn on or off the LED lamps 31 to 34. The illumination light source is not limited to the LED lamp, but is preferably sufficiently small with respect to the size of the target object. In FIG. 2, the light source sets 22 and 23 are illustrated as individual illumination devices, but it is also possible to sequentially photograph the target object while changing the position using one light source set.

[Processing method]
Next, a processing method for obtaining the color information and three-dimensional shape of the target object from the image group obtained by the photographing system according to the present invention will be described with reference to FIGS. 4, 5, and 6. FIG. The first apparatus configuration example of the present invention shown in FIG. 1 is regarded as equivalent to the case where the number of LED lamps arranged on the light source set is one in the first apparatus configuration example of the present invention of FIG. In the following, the processing method will be described based on the first apparatus configuration example of the present invention in FIG. In consideration of generality, the number of light source sets to be used is M, and the number of LED lamps installed in the jth light source set (j = 1, 2,..., M) is D _j . To do. Hereinafter, the j-th light source set is denoted as light source set j, and the k-th light source (k = 1, 2,..., D _j ) in light source set j is denoted as light source j _k .

  First, the outline of the data processing method according to the present invention will be described. As described above, in order to derive the normal vector and the diffusion albedo by the illuminance difference stereo, the light source orientation (position and orientation) needs to be estimated because the light source vector and the incident light radiance at each point of the target object are used. Since an arbitrary posture of the object in the three-dimensional space can be represented by rotation and movement with respect to the reference posture of the object, the estimation of the light source position can be regarded as estimation of the rotation parameter and the movement parameter for each light source set. In the processing according to the present invention, these parameters are obtained by nonlinear optimization using a group of captured images. FIG. 4 is an explanatory diagram of nonlinear optimization processing. The nonlinear optimization module provides the evaluation function with rotation and movement parameters for all light source sets. The evaluation function calculates an evaluation value E based on the given parameters. The nonlinear optimization module repeatedly performs parameter update and call of the evaluation function until the evaluation value E converges. The rotation parameter and the movement parameter at the time of convergence are parameters for specifying the posture of each light source set, and the postures of all the light sources are determined.

ここで、ｑ_ｊ，ｋは、光源セットｊの基準姿勢における光源ｊ_ｋの三次元座標を表す３行１列のベクトル、Ｒ_ｊは三次元空間における光源セットｊの回転を表す３行３列の回転行列、ｔ_ｊは三次元空間における光源セットｊの移動を表す３行１列の移動ベクトル、ｐ_ｊ，ｋは光源セットｊを回転、移動した後の光源ｊ_ｋの三次元座標を表す３行１列のベクトルである。 The relationship between the light source set and the posture of the light source installed in the light source set, the rotation parameter, and the movement parameter is formulated as the following equation (6).

Here, q _{j, k} is a 3-row 1-column vector representing the three-dimensional coordinates of the light source j _k in the reference posture of the light source set j, and R _j is a 3-row 3-column representing the rotation of the light source set j in the three-dimensional space. T _j is a 3-by-1 movement vector representing the movement of the light source set j in the three-dimensional space, and p _{j, k} represents the three-dimensional coordinates of the light source j _k after rotating and moving the light source set j. It is a vector of 3 rows and 1 column.

Since the positional relationship of each light source in the light source set j is known, the three-dimensional coordinates of the light source j _k in the reference posture of the light source set j can be defined in an arbitrary Euclidean space. For example, the light sources are arranged on a plane. The coordinates of the light source j ₁ are (0, 0, 0), the coordinates of the light source j ₂ are (X _{j, 2} , Y _{j, 2} , 0), and the coordinates of the light source j ₃ are (X _{j, 3} , Y _{j, 3} , 0).

The rotation matrix R _j in the three-dimensional space has three parameters, for example, a rotation angle θ _{X; j with} the X axis as the rotation axis, a rotation angle θ _Y with the Y axis as the rotation axis, _j , and the Z axis as the rotation axis. The rotation angle θ _Z; If the three components of the movement vector t _j are expressed as t _{X; j} , t _{Y; j} , t _{Z; j} , the arbitrary posture of the light source set j is θ _{X; j} , θ _{Y; j} , θ _{Z; j} , TX _{; j} , tY _{; j} , tZ _{; j} . In the above formula (6), the position of the light source j _k is formulated, but the direction of the light source j _k can also be grasped by the rotation matrix R _j .

  When the nonlinear optimization module starts optimizing parameters, initial values of parameters given to the evaluation function are required. There is no problem if the initial value of the parameter does not deviate significantly from the posture of the light source set at the time of actual shooting. For example, the user can determine the value sensuously or use a magnetic sensor or the like at the time of shooting. You can grasp the approximate position and enter it. Further, the algorithm of the nonlinear optimization module need not be limited to a specific one, and for example, a Nelder-Mead downhill simplex method may be used.

Next, details of the evaluation function used in the nonlinear optimization process will be described. FIG. 5 illustrates the processing performed by the evaluation function in FIG. 4 and includes five steps S01 to S05. The inputs to the evaluation function are θ _{X; j} , θ _{Y; j} , θ _{Z; j} , t _{X; j} , t _{Y; j} , j = _{Z; j} (j = 1, 2,...) For all light source sets. , M). FIG. 6 is an explanatory diagram of pre-processing that is performed only once prior to nonlinear optimization processing, and includes four steps P01 to P04.

First, the preprocessing shown in FIG. 6 will be described.
[Step P01]
In the captured image, G pixels are extracted from the image area corresponding to the target object. Hereinafter, these pixels are referred to as sample pixels. The extraction method is not particularly limited. For example, a method of selecting an appropriate pixel using a mouse while a user views an image displayed on a monitor or a method of using a program for randomly selecting a pixel. Use it.

[Step P02]
For all the sample pixels selected in step P01, sensor responses of images taken under each light source are extracted. This is equivalent to acquiring S in the above equation (3) for G pixels. The extracted sensor response is retained for use in the evaluation function.

[Step P03]
Calculate the rough three-dimensional shape of the target object. Although a high-definition three-dimensional shape is finally obtained based on the illuminance difference stereo method, since information on the rough three-dimensional shape of the target object is required in step S02 described later, the three-dimensional shape is obtained by an appropriate method. Calculate it. As a means for acquiring a rough three-dimensional shape, for example, the above-described pattern projection method can be applied. Alternatively, if the target object is close to a plane, as shown in FIG. 7, it is also possible to take a method of photographing a marker whose positional relationship is known at the same time as the target object. In the example of FIG. 7, the markers 41 to 44 are cross-intersection lines on the same plane, and the three-dimensional coordinates of the intersection of each marker can be defined by setting the Z component to zero. These markers 41 to 44 are installed on the same plane as the imaging target surface. By extracting the pixel position corresponding to the intersection of each marker in the photographed image, it is possible to obtain a projective transformation for the photographed image from the plane indicated by the four markers 41 to 44. The approximate three-dimensional coordinates of the point on the target object corresponding to the pixel can be specified.
[Step P04]

  For all the sample pixels selected in step P01, the approximate three-dimensional coordinates of the points on the corresponding target object are extracted based on the result of step P03. The extracted three-dimensional coordinates are retained for use in the evaluation function.

Next, the evaluation function shown in FIG. 5 will be described.
[Step S01]
With respect to the light source set j, the posture by the above equation (6) using the rotation parameter θ _{X; j} , θ _{Y; j} , θ _{Z; j} and the movement parameter t _{X; j} , t _{Y; j} , t _{Z; j.} Conversion is performed, and the positions of all the light sources installed in the light source set j are calculated. When the light emitted from the light source has directivity, the direction of the light source is also calculated from the rotation parameter. This is done for all j.

[Step S02]
For all sample pixels selected in step P01, the light source vector and the incident light radiance for each light source are calculated.

G th pixel among all samples pixels (g = 1,2, ..., G ) and a vector r _g of 3 rows and one column a generally three-dimensional coordinates of points on the object surface corresponding to, If the vector v _{g, j, k} heading from there to the light source j _k is defined as equation (7), the light source vector l _{g, j, k} is expressed by equation (8). However, the double line represents the norm of the vector.

ただし、βは光源からの距離が１の点における放射輝度であり、事前に光源の放射輝度キャリブレーションを行うことで決定される。また、光源が指向性を持つ場合には、βは光源の向きの関数となる。 Further, the radiance R _{g, j, k} of the incident light from the light source j _k to the position r _g is expressed by Expression (9).

However, β is a radiance at a point whose distance from the light source is 1, and is determined by performing a radiance calibration of the light source in advance. When the light source has directivity, β is a function of the direction of the light source.

  The equation (9) is configured based on the principle that when the size of the light source is sufficiently small, the radiance of the light emitted from the light source is inversely proportional to the square of the distance from the light source. If there is a discrepancy with the actual light source, it may be replaced with an appropriate function.

  The calculations of Equation (8) and Equation (9) are performed for all light source and all sample pixel combinations.

[Step S03]
Using the position (and orientation) of the light source obtained at step S01, the light source vector and incident light radiance obtained at step S02, and the sensor response obtained at step P02, A normal vector and a diffusion albedo are obtained for the sample pixel.

  In formula (5), the description relating to the camera channel is omitted for the sake of simplicity, but actually, it is necessary to obtain a diffusion albedo for each camera channel used for photographing. For example, when the camera has three channels of R, G, and B, the normal vector and the diffusion albedo are calculated based on the equation (5) using only the sensor response of the G channel, and then the G channel. The obtained normal vector is substituted into equation (2), and the diffusion albedo of the R channel and the B channel may be obtained using the least square method. Alternatively, for each of the R, G, and B channels, a normal vector and a diffusion albedo are obtained based on Equation (5), and the obtained three normal vectors are averaged to form a single normal vector. It can also be used. In these processes, the radiance of incident light also needs to be decomposed into three components of R, G, and B. For example, from the sensor response of an image taken by illuminating a complete diffuser with each light source, Calibration may be performed in advance.

  Further, in the derivation of the normal vector and the diffusion albedo according to the equation (5), it is assumed that the light reflection of the target object follows the Lambert model, but actually, the light reflected from the object has a specular reflection component. May be included. Therefore, in this step, the sensor response obtained in step P02 is referred to, and if the sensor response is somewhat large, it is considered that the specular reflection component is included, and the sensor response corresponding to the corresponding light source is calculated by the method according to the equation (5). It is also preferred not to use it to derive line vectors and diffusion albedo. Similarly, when the sensor response obtained in step P02 is somewhat small and it is considered that the sensor response is affected by light shielding according to the shape of the target object, it is preferable not to use the sensor response corresponding to the corresponding light source.

[Step S04]
Using the normal vector and the diffusion albedo obtained in step S03, the light source vector and the incident light radiance obtained in step S02, the combinations of all the sample pixels and all the light sources are expressed in Equation (1). Based on this, the sensor response in the Lambert model assumption is calculated. This calculation is also performed for each of the R, G, and B channels as in step S03.

ただし、Ｉ_{ｊ,ｋ；ｇ；ｃ}は、光源ｊ_ｋを点灯させて撮影したときのｇ番目のサンプル画素におけるｃチャンネルのセンサ応答の実測値、Ｉ_{ｊ,ｋ；ｇ；ｃ}は、対応するセンサ応答のランバートモデル値である。 [Step S05]
The difference between the Lambert model value of the sensor response calculated in step S04 and the sensor response (actually measured value) extracted from the actual captured image obtained in step P02 is calculated by the equation (10), and is set as an evaluation value E.

However, I _{j, k; g; c} corresponds to an actually measured value of the sensor response of the c-channel in the g-th sample pixel when the light source j _k is turned on, and I _{j, k; g; c} corresponds to This is the Lambert model value of the sensor response.

In the processing of the above steps S01 to S05, the parameters θ _X given to the evaluation function θ, _j , θ _{Y; j} , θ _{Z; j} , t _{X; j} , t _{Y; j} , t _{Z; j} (j = 1, 2,..., M) are closer to the actual shooting state, the smaller the error in the calculation process, and the smaller the evaluation value E inevitably. Therefore, the nonlinear optimization module updates the parameters so that the evaluation value E calculated by the evaluation function becomes small, so that the posture of each light source set, that is, the position and orientation of each light source is changed to the actual shooting state. You can get closer. The nonlinear optimization module completes the estimation of the position and orientation of the light source when the evaluation value E converges.

  If the position and orientation of the light source are determined, the normal vector and the diffusion albedo of each point on the target object surface can be calculated from the captured image group based on the principle of ordinary illuminance difference stereo. Furthermore, since the normal vector represents the orientation of a surface in a minute region on the surface of the target object, the shape of the entire target object can be restored by connecting the surfaces with adjacent pixels. Further, by combining the rough three-dimensional shape of the target object obtained in step P03 and the normal vector obtained by illuminance difference stereo, for example, using the technique disclosed in Non-Patent Document 8, three-dimensional with higher accuracy. It is also preferable to restore the shape.

In the non-linear optimization process, the number of parameters to be optimized is such that the parameters for each light source set are θ _{X; j} , θ _{Y; j} , θ _{Z; j} , t _{X; j} , t _{Y; j} , t _{Z ; J} is 6 and the number of the light source sets is M, so the total is 6M. In order to perform the nonlinear optimization process at high speed and stably, it is preferable that the number of parameters is as small as possible. On the other hand, in the derivation of the normal vector and the diffusion albedo by the illuminance difference stereo, it is preferable to use a certain number of light sources in order to mitigate the influence of accuracy reduction due to image noise or the like. In the present invention, a plurality of light sources are installed in a light source set, and with these light sources, rotation parameters θ _{X; j} , θ _{Y; j} , θ _{Z; j} and movement parameters t _{X; j} , t _{Y; j} , t _{Z By making j} common, the total number of parameters to be optimized can be reduced. For example, assuming that shooting is performed with eight light sources, when each light source is installed independently (when the number of light sources in the light source set is one), the total number of parameters to be optimized is 48. However, when four light source sets including two light sources are used, the total number of parameters is 24, and when two light source sets including four light sources are used, the total number of parameters is twelve. Furthermore, when only one light source set including eight light sources is used, the total number of parameters is six, and as the number of light sources installed in one light source set is increased, the parameter to be optimized is increased. The total number of can be reduced. Actually, if the number of light sources installed in one light source set is increased, the light source set becomes larger and may be difficult to handle. Therefore, an appropriate design should be made in consideration of the size of the target object and the shooting environment. Thus, it is preferable to balance the securing of a certain number of light sources and the reduction of the number of optimization parameters.

これを変形すると、式（１２）が得られる。

In the nonlinear optimization processing according to the present invention, the position of the light source set is optimized while deriving the normal vector and the diffusion albedo for the G pixels based on the principle of the illuminance difference stereo method. The number of unknown variables in the optimization is that the normal vector and the diffusion albedo in each of the G pixels have 3 degrees of freedom (the product of the normal vector and the diffusion albedo is derived in equation (5)). Equivalently), since the rotation parameter and movement parameter of the light source set have 6M degrees of freedom, the total is 3G + 6M. On the other hand, the known information is the sensor response obtained for each light source in each of the G pixels. If the number of light sources is F, the total number is GF. In order to achieve nonlinear optimization, known information more than the number of unknown variables is required, and the condition is expressed by Expression (11).

When this is transformed, equation (12) is obtained.

  Therefore, the number G of pixels to be selected in step P01 needs to be determined based on Expression (12) according to the total number of light sources used for photographing.

11, 21 Digital camera 12-15, 31-34 LED lamp 16, 26 Computer 17, 27 Keyboard 18, 28 Monitor 22, 23 Light source set 41-44 Marker

Claims (10)

A measuring device that measures the color and three-dimensional shape of an object,
A rough shape measuring means for measuring a rough three-dimensional shape of an object to be measured;
Imaging means for imaging an image of an object to be measured with three or more channels;
Illuminating means comprising four or more illumination light sources used for imaging the object;
An image input means for exclusively turning on the four or more illumination light sources in the illumination means, sequentially imaging the object by the imaging means, and obtaining a captured image for each illumination light source;
Light source position estimating means for estimating a position at the time of imaging for each of the four or more illumination light sources from the plurality of captured images and the rough three-dimensional shape ;
A measurement apparatus comprising: object restoration means for calculating color information and three-dimensional shape information of the object based on the position of each illumination light source obtained by the light source position estimation means.   The measurement apparatus according to claim 1, wherein at least one of the four or more illumination light sources is fixed in a relative positional relationship with another one or more illumination light sources. The light source position estimating means includes
Sample pixel extraction means for selecting a plurality of sample pixels from an area including an object to be measured in the captured image;
A light source attitude calculation means for expressing the position and orientation of the illumination light source by rotation and movement of a three-dimensional object, and calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source calculated by the light source orientation calculation means and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel Incident light calculation means for calculating the radiance of light emitted from the light source;
Using the direction of each light source calculated by the incident light calculation means, the radiance of the light emitted from each light source, and the sensor response in the sample pixel, the normal vector of the measurement target object for all the sample pixels And surface calculation means for calculating the diffusion albedo;
All of the sample pixels using the normal vector and diffusion albedo calculated by the surface calculation means, and the direction of each light source calculated by the incident light calculation means and the radiance of light emitted from each light source. And sensor response model value calculation means for calculating a model value of sensor response for all light sources,
Error calculation means for calculating an error between the sensor response model value calculated by the sensor response model value calculation means and the sensor response of the captured image;
2. The measurement according to claim 1, further comprising: a light source attitude calculation unit that determines a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimizes the error calculated by the error calculation unit by nonlinear optimization. apparatus. A light source attitude estimation device that estimates the position and orientation of an illumination light source from captured images of each illumination light source obtained by illuminating an object by exclusively lighting four or more illumination light sources and sequentially imaging the object with an imaging device Because
Sample pixel extraction means for selecting a plurality of sample pixels from an area including an object to be measured in the captured image;
A light source attitude calculation means for expressing the position and orientation of the illumination light source by rotation and movement of a three-dimensional object, and calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source calculated by the light source orientation calculation means and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel Incident light calculation means for calculating the radiance of light emitted from the light source;
Using the direction of each light source calculated by the incident light calculation means, the radiance of the light emitted from each light source, and the sensor response in the sample pixel, the normal vector of the measurement target object for all the sample pixels And surface calculation means for calculating the diffusion albedo;
All of the sample pixels using the normal vector and diffusion albedo calculated by the surface calculation means, and the direction of each light source calculated by the incident light calculation means and the radiance of light emitted from each light source. And sensor response model value calculation means for calculating a model value of sensor response for all light sources,
Error calculation means for calculating an error between the sensor response model value calculated by the sensor response model value calculation means and the sensor response of the captured image;
A light source attitude estimation device comprising: a light source attitude calculation means that determines a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimizes the error calculated by the error calculation means by nonlinear optimization. A measurement method for measuring the color and three-dimensional shape of an object,
A rough shape measuring step for measuring a rough three-dimensional shape of an object to be measured;
An image input step of exclusively lighting four or more illumination light sources, sequentially imaging the object by an imaging device having three or more channels, and obtaining a captured image for each illumination light source;
A light source position estimation step of estimating a position at the time of imaging for each of the four or more illumination light sources from the plurality of captured images and the rough three-dimensional shape ;
A measurement method comprising: an object restoration step of calculating color information and three-dimensional shape information of the object based on the position of each illumination light source obtained by the light source position estimation step. The light source position estimating step includes:
A sample pixel extraction step of selecting a plurality of sample pixels from a region including an object to be measured in the captured image;
A position and orientation of the illumination light source represented by rotation and movement of a three-dimensional object, and a light source attitude calculation step of calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source obtained in the light source orientation calculation step and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel An incident light calculation step for calculating the radiance of light emitted from the light source;
Using the direction of each light source obtained in the incident light calculation step, the radiance of light emitted from each light source, and the sensor response in the sample pixel, the normal line of the measurement target object for all the sample pixels A surface calculation step for calculating vectors and diffusion albedo;
Using the normal vector and diffusion albedo obtained in the surface calculation step, and the direction of each light source obtained in the incident light calculation step and the radiance of light emitted from each light source, all the sample pixels and A sensor response model value calculating step for calculating a sensor response model value for all light sources;
An error calculation step of calculating an error between the sensor response model value obtained in the sensor response model value calculation step and the sensor response of the captured image;
The measurement according to claim 5, further comprising: a light source attitude calculation step that determines a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimizes the error obtained in the error calculation step by nonlinear optimization. Method. A light source attitude estimation method for estimating the position and orientation of an illumination light source from captured images of each illumination light source obtained by illuminating an object by exclusively turning on four or more illumination light sources and sequentially imaging the object with an imaging device Because
A sample pixel extraction step of selecting a plurality of sample pixels from a region including an object to be measured in the captured image;
A position and orientation of the illumination light source represented by rotation and movement of a three-dimensional object, and a light source attitude calculation step of calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source obtained in the light source orientation calculation step and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel An incident light calculation step for calculating the radiance of light emitted from the light source;
Using the direction of each light source obtained in the incident light calculation step, the radiance of light emitted from each light source, and the sensor response in the sample pixel, the normal line of the measurement target object for all the sample pixels A surface calculation step for calculating vectors and diffusion albedo;
Using the normal vector and diffusion albedo obtained in the surface calculation step, and the direction of each light source obtained in the incident light calculation step and the radiance of light emitted from each light source, all the sample pixels and A sensor response model value calculating step for calculating a sensor response model value for all light sources;
An error calculation step of calculating an error between the sensor response model value obtained in the sensor response model value calculation step and the sensor response of the captured image;
A light source posture estimation method comprising: a light source posture calculation step that determines a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimize the error obtained in the error calculation step by nonlinear optimization. A measurement program that is executed by a computer of a measurement device that measures the color and three-dimensional shape of an object,
A rough shape measurement process for measuring a rough three-dimensional shape of an object to be measured;
4 or more illumination light sources are exclusively turned on, and the object is sequentially imaged by an imaging device having 3 or more channels, and an image input process for obtaining a captured image for each illumination light source;
A light source position estimation process for estimating a position at the time of imaging for each of the four or more illumination light sources from the plurality of captured images and the rough three-dimensional shape ;
A measurement program for causing a computer to execute color reconstruction of an object and object restoration processing for calculating three-dimensional shape information based on the position of each illumination light source obtained by the light source position estimation processing. The light source position estimation process includes:
Sample pixel extraction processing for selecting a plurality of sample pixels from a region including an object to be measured in the captured image;
A position and orientation of the illumination light source represented by rotation and movement of a three-dimensional object, and a light source orientation calculation process for calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source obtained by the light source orientation calculation process and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel Incident light calculation processing for calculating the radiance of light emitted from the light source;
Using the direction of each light source obtained in the incident light calculation process, the radiance of the light emitted from each light source, and the sensor response in the sample pixel, the normal of the measurement target object for all the sample pixels Surface calculation processing to calculate vector and diffusion albedo;
Using the normal vector and diffusion albedo obtained in the surface calculation process, and the direction of each light source and the radiance of light emitted from each light source obtained in the incident light calculation process, all the sample pixels and Sensor response model value calculation processing for calculating a sensor response model value for all light sources;
An error calculation process for calculating an error between the sensor response model value obtained by the sensor response model value calculation process and the sensor response of the captured image;
The measurement according to claim 8, further comprising: a light source attitude calculation process that determines a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimizes the error obtained by the error calculation process by nonlinear optimization. program. A light source attitude estimation device that estimates the position and orientation of an illumination light source from captured images of each illumination light source obtained by illuminating an object by exclusively lighting four or more illumination light sources and sequentially imaging the object with an imaging device A light source attitude estimation program to be executed by a computer of
Sample pixel extraction processing for selecting a plurality of sample pixels from a region including an object to be measured in the captured image;
A position and orientation of the illumination light source represented by rotation and movement of a three-dimensional object, and a light source orientation calculation process for calculating the position and orientation of each of the four or more illumination light sources from the rotation parameter and the movement parameter;
Using the position and orientation of the light source obtained by the light source orientation calculation process and the rough three-dimensional shape information of the measurement target object, the direction and each of the light sources at each point on the measurement target object surface corresponding to the sample pixel Incident light calculation processing for calculating the radiance of light emitted from the light source;
Using the direction of each light source obtained in the incident light calculation process, the radiance of the light emitted from each light source, and the sensor response in the sample pixel, the normal of the measurement target object for all the sample pixels Surface calculation processing to calculate vector and diffusion albedo;
Using the normal vector and diffusion albedo obtained in the surface calculation process, and the direction of each light source and the radiance of light emitted from each light source obtained in the incident light calculation process, all the sample pixels and Sensor response model value calculation processing for calculating a sensor response model value for all light sources;
An error calculation process for calculating an error between the sensor response model value obtained by the sensor response model value calculation process and the sensor response of the captured image;
A light source for causing a computer to execute a light source attitude calculation process for determining a rotation parameter and a movement parameter representing the position and orientation of the illumination light source that minimize the error obtained by the error calculation process by nonlinear optimization. Attitude estimation program.

Images