KR101342393B1 - Georeferencing Method of Indoor Omni-Directional Images Acquired by Rotating Line Camera - Google Patents

Abstract

The present invention defines an object coordinate system as a reference in the room for an indoor omnidirectional image obtained by a rotary line camera, secures an object reference point, estimates an external expression element of an indoor omnidirectional image obtained by a rotary line camera, and is the most suitable projection method. The present invention relates to a georeferencing method capable of providing a high quality omnidirectional image.

Description

Georeferencing Method of Indoor Omni-Directional Images Acquired by Rotating Line Camera}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a georeferenced method, and to a georeferenced method capable of providing a high quality omnidirectional image service to an indoor omnidirectional image obtained by a rotary line camera.

Recently, service development using omnidirectional video such as Google's Street View and Daum's Road View has been actively conducted. Omni-directional video is an image taken in all directions from the shooting location, and the user can see all around the user at 360 °. Therefore, services using the omnidirectional image can more accurately deliver information on places that have never been visited to users. For example, it is difficult to find a direction at the place of first visit using the existing map service, but the omnidirectional video service includes information of all directions based on the current location, so the direction can be easily found.

The omnidirectional image is classified into a single-view omnidirectional image having a large projection center and a multi-view omnidirectional image having multiple projection centers according to the acquired method. The short-term omnidirectional image is a method of generating an omnidirectional image using a frame camera using a mirror. This method causes severe distortion and is not easy to correct. In addition, it is difficult to obtain an image having a high and uniform resolution. Most of the omnidirectional video services currently provided utilize multi-view omnidirectional video. Multi-view omnidirectional images are generally created by joining images taken by multiple frame cameras simultaneously. Although relatively uniform high resolution image information is provided, in particular, when the individual images are joined, a distortion occurs in the overlapping area, that is, the farther away from the center of one individual image. Therefore, by using a large number of cameras to reduce the field of view (individual image) of the individual image it is possible to generate a multi-view omnidirectional image with less distortion. However, even in this case, since the distance to the object for each individual pixel of the individual image is not accurately known, complete distortion correction is difficult.

To overcome this disadvantage, a rotary line camera capable of acquiring omnidirectional images with a single camera having a small viewing angle may be used. The rotary line camera sequentially joins line images taken while rotating one line camera 360 ° to generate an omnidirectional image. Unlike the conventional method of joining and generating images acquired by a plurality of frame cameras at the same time, there is an advantage in that distortion is hardly generated because a line image having a very small viewing angle is bonded. However, due to the disadvantage that the rotating shaft or the object should not move while rotating, it is difficult to acquire the omnidirectional image while being mounted and moved in the vehicle outdoors. Therefore, the rotary line camera is effective for acquiring the omnidirectional image of the room and can be effectively used for spatial image information service on the indoor space such as 3D modeling of the room or 'next' store view.

However, in order to obtain three-dimensional information of the object by using the indoor omnidirectional image acquired by the rotary line camera, 'Wei', 'Tang', 'Shum', 'Li', 'Huang', etc. between 1999 and 2006. Although various modeling has been performed, in the previous studies, the position and attitude of the camera at the time of acquiring the omnidirectional image are not determined in the object coordinate system, and since the reference point is difficult to secure, the georeferencing of indoor omnidirectional images is performed. There is a difficult problem.

Accordingly, the present invention is to solve the above-described problems, an object of the present invention, to define the object coordinate system as a reference in the room for the indoor omnidirectional image obtained by the rotary line camera, to secure the object reference point, the rotary line The present invention provides a georeferencing method capable of providing high quality omnidirectional images by estimating an external expression element of an indoor omnidirectional image obtained by a camera and determining a most suitable projection method.

First, to summarize the features of the present invention, according to an aspect of the present invention, the method of geo-referencing the indoor omnidirectional image obtained by the line camera, the storage means by obtaining the line unit image for the omnidirectional while rotating the line camera To generate an omnidirectional image of n × m pixels projected onto the image of each image point (i, j) on the focal length (f), and to select a plurality of object reference points in the room (eg, at least three or more). ) And the coordinates of the image points (i, j) are corrected by estimating an external expression element including posture data of the line camera using the acquired collinear equation of the image. the corner point in the room as the origin and the first stage representing the object point (P ^O) to the object coordinate system defined by three axes in the three edges meeting at the corner point; A second step of converting the object point ^O P into an object point ^C P represented by a camera coordinate system defined by a position and a posture of the moment when the image is captured; A third step of calculating a j-direction distance r _p of the image points i and j with respect to the object point ^C P on the focal length f of the line camera represented by the camera coordinate system based on a predetermined projection method; ; And a fourth operation of calculating coordinates of the image points i and j by calculating pixel positions of the object point ^C P from the angle α-β formed by the object point ^C P on the image and the r _p . step; And a fifth step of estimating the external expression element of the omnidirectional image obtained by the rotary line camera.

But using the least ^(O P) more than two object points represented in the object coordinate system as a plurality of object reference point to define a corner point in the room as the origin of the object coordinate system and measuring the length of the three edges meeting at the corner point Equation 10 is used to obtain (X _b , Y _b , Z _b ) of the measurement coordinate system and convert it to coordinates (X _a , Y _a , Z _a ) of three points represented in the object coordinate system. , (X _b , Y _b , Z _b ) are three-dimensional coordinate values of three points expressed in the measurement coordinate system, and (X _a , Y _a , Z _a ) are three-dimensional coordinate values expressed in the object coordinate system. , R is the rotation transformation matrix to convert from the object coordinate system to the measurement coordinate system, and t is the transformation matrix.

In the second step, Equations 1 to 5 below are used, wherein r is a rotation radius of the line camera, α is a rotation angle in the virtual rotation coordinate system of the line camera, and β is a line. Θ is the angle between the camera and the rotation axis, θ is the X-axis rotation angle of the point where the X axis and the object point of the rotation coordinate system are projected on the XY plane of the rotation coordinate system, ε is the difference angle between θ and α, ^R P ( ^R X, ^R Y , ^R Z) is the object point ^O P coordinate in the rotational coordinate system, and ^C P ( ^C X, ^C Y, ^C Z) is the object point ^O P coordinate in the camera coordinate system.

In the third step, using [Equation 6] to [Equation 7] below, wherein, ρ is the ray and the optical axis of the point object from entering the line camera ^(O P) forming an angle, P ^C (X ^C, Y ^C, Z ^C) is the object point (P ^O) coordinates in the camera coordinate system.

In the fourth step, Equation 8 below is used, where (i, j) is an image point on an image coordinate system expressed in pixels, and α is a rotation angle in a virtual rotation coordinate system of a line camera, β Is the rotation angle in the camera coordinate system of the line camera, n is the number of pixels in the j direction of the image, μ _α is the rotation angle per pixel, and μ _j is the size of the j direction per pixel.

In the fifth step, the origin ( ^O X _R ,) in the virtual rotational coordinate system of the line camera as the external expression element using the nonlinear observation equation [Equation 9] for the image points (i, j). ^O Y _R , ^O Z _R ) and posture data (ω, φ, κ,), are estimated, but after the projection method 1, the projection method 2, the projection method 3, and the projection method 4 are all applied, the residual in the i, j direction Determines the external expression factor estimated by the smallest projection method, where the object reference point ( ^o X, ^o Y, ^o Z), the rotation radius (r) of the line camera, and the rotation angle (β) of the line camera are used. .

According to the georeferencing method according to the present invention, a high-quality omnidirectional image can be provided with respect to an indoor omnidirectional image obtained by a rotary line camera by a projection method using an external coordinate element estimated through a defined object coordinate system and an object reference point. have.

1 is a view for explaining a system for providing a high quality omnidirectional image according to an embodiment of the present invention.
FIG. 2 is a diagram for describing a 3D geometric model of each coordinate system for image acquisition of the rotary line camera of FIG. 1.
3 is a view for explaining a two-dimensional geometric model of the coordinate system of FIG.
4 is a diagram for explaining a projection method of a fisheye lens camera.
5 is a diagram for explaining an example of the object coordinate system.
6 and 7 are examples of indoor omnidirectional internal and external images, respectively.
8 and 9 illustrate object points before and after performing coordinate transformation, and are examples of object points represented by a measurement coordinate system and an object coordinate system.
FIG. 10 is an example of object points from which an abnormal point and an abnormal point are removed in FIG. 9.
11 is an example of the i-direction residual distribution diagram of the indoor omnidirectional interior image.
12 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image.
13 is an example of the i-direction residual distribution diagram of the indoor omnidirectional interior image according to the rotation angle.
14 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image according to a rotation angle.
15 is an example of the i-direction residual distribution diagram of the indoor omnidirectional interior image.
16 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image.
17 is an example of a j-direction residual distribution diagram of an indoor omnidirectional internal image according to a Z value.
18 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image according to a Z value.
19 is an example of the i-direction residual distribution diagram of the indoor omnidirectional interior image according to distance.
20 is an example of the i-direction residual distribution diagram of the indoor omnidirectional external image according to distance.
21 is an example of a j-direction residual distribution diagram of an indoor omnidirectional interior image according to distance.
22 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image according to a distance.
23 is an example of the i-direction residual distribution diagram of the indoor omnidirectional interior image according to the radiation distance.
24 is an example of the i-direction residual distribution diagram of the indoor omnidirectional external image according to the radiation distance.
25 is an example of a j-direction residual distribution diagram of an indoor omnidirectional interior image according to a radiation distance.
FIG. 26 is an example of a j-direction residual distribution diagram of an indoor omnidirectional external image according to a radiation distance. FIG.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited to the embodiments.

The present invention describes a method of georeferencing a single omnidirectional image acquired by a rotary line camera, that is, determining an external expression element. To this end, first, a rotary line camera and coordinate systems are defined, and a collinear equation for the omnidirectional image acquired by the rotary line camera is derived. In addition, we present a method to efficiently obtain the object reference point required for the georeferencing process indoors and explain the mathematical model for estimating the external expression element using the collinear equation and the object reference point.

<Definition of Rotating Line Camera and Coordinate System>

As shown in FIG. 1, in the system 100 for providing a high quality omnidirectional image through georeferencing according to the present invention, one line camera 110 having 1 × m line pixels is spaced apart from the center of rotation by r. It is a system to shoot video by rotating 360 ° clockwise. The line camera 100 is mounted on a predetermined support plate coupled to the motor rotation shaft, and the controller 130 controls the driving so that the motor rotation shaft rotates at a constant speed in a clockwise or counterclockwise direction to rotate the line camera 110. can do. One column of images obtained by rotating and photographing the line camera 100 are stored in the storage medium 131 of the control device 130. In this case, the control device 130 is configured by the line camera 110. In response to each position data (camera coordinate system) corresponding to the acquisition, image data of each line unit for all directions can be stored in the storage medium 131. The synthesizing unit 132 of the control device 130 finally uses the image data stored in the storage medium 131 according to the image bonding through geo-referencing to finally coordinates (I, j) of n × m pixels. A two-dimensional omnidirectional image may be generated.

When acquiring the omnidirectional image with the rotary line camera 110 indoors, it consists of an object coordinate system in which object points exist, a rotation coordinate system in which the camera rotates, and a camera coordinate system at the moment the image is captured. Figure 2 is a three-dimensional geometric model of each coordinate system, Figure 3 shows this as a two-dimensional geometric model.

The object coordinate system may define any corner point of the room as the origin. The pose of the object coordinate system is defined based on three axes defined by three corners perpendicular to the corner point. As shown in FIG. 2, the ^o Z axis may be defined as an edge leading to a corner point in the indoor ceiling, and the ^o X axis and ^o Y axis may be defined as two remaining corners.

The origin of the virtual rotation coordinate system is defined as the rotation center O _R. The ^R Z axis of the rotation coordinate system is defined in the zenith direction in the same direction as the ^o Z axis of the object coordinate system, and the ^R X axis of the rotation coordinate system is defined by connecting the rotation center (O _R ) and the point where the rotation starts. ^{The R} Y axis is defined by rotating the ^R X axis counterclockwise by 90 °.

The origin of the camera coordinate system is defined as a point O _C at which an image is captured. The camera coordinate system is defined differently according to the i-direction of the image because it is defined by the position and posture of the moment when the image is captured. The ^C Z axis of the camera coordinate system is defined as the optical axis, the ^C Y axis is defined as the ceiling direction, and the ^C X axis is defined by rotating the ^C Y axis counterclockwise by 90 °.

2 and 3, P is an object point and P 'is an image point of the object point. r is the rotation radius, f is the focal length of the line camera 110, ^R O _C is the position of the line camera 110 at the moment of shooting the image, α is the angle of rotation in the rotational coordinate system at the moment the image is taken to be. β represents the distorted rotation angle (or rotation angle in the camera coordinate system of the line camera) of the camera 110 in which the optical axis ^C Z axis of the line camera 110 forms the ^C X axis, and ^C X fixed at the start of shooting. The angle relative to the axis.

Collinear Equation for All Direction Images

The collinear equation is a formula for determining the point P 'from which the object point P is projected onto the image. The projection center (O _C ), the object point (P), and the image point (P') form a straight line according to the principle of center projection. Derived from the condition that exists in. The collinear equation for the omnidirectional image can be derived by dividing the coordinate transformation of the object point P, the projection to the image point P ', and the coordinate transformation of the image point P'.

First, the coordinate transformation of the object point P is an object point ^C P representing the object point ^O P ( ^o X, ^o Y, ^o Z) expressed in the object coordinate system (obtained by measuring). Convert to ( ^C X, ^C Y, ^C Z). First, by using Equation 1, ^O P is converted into an object point ^R P ( ^R X, ^R Y, ^R Z) represented by a rotational coordinate system. Next, ^R P is converted into ^C P using Equation 2 below. ^O O _R is the origin of the rotation coordinate system expressed in the object coordinate system, and ^R R _O is the rotation transformation matrix from the object coordinate system to the rotation coordinate system. ^R O _C is the origin of the camera coordinate system expressed in the rotation coordinate system, and ^C R _R is the rotation transformation matrix from the rotation coordinate system to the camera coordinate system. Here, the origin ^R O _C of the camera coordinate system may be determined as the rotation angle α at the moment of taking the picture as shown in [Equation 3]. The rotational transformation of the camera coordinate system can be determined by α and β as shown in [Equation 4]. In this case, α can be obtained from θ and ε as shown in FIG. 3, where -2π≤α≤0 and θ can be obtained from the plane coordinates of the object point ^R P expressed in the rotational coordinate system, and ε is determined by the sin law. After finding, it can obtain | require using the sum condition of a triangular cabinet. α can be defined as shown in [Equation 5], and β can be defined by any real number, and β> 0. [theta] corresponds to the X-axis rotation angle of the point at which the X-axis and the object point of the rotational coordinate system are projected on the XY plane of the rotational coordinate system, and [epsilon] corresponds to the difference angle between [theta] and [alpha].

[Equation 1]

&Quot; (2) &quot;

&Quot; (3) &quot;

&Quot; (4) &quot;

[Equation 5]

Second, the projection to the image point P 'may be determined by the projection method of the camera on the camera coordinate system. In the present invention, since a fisheye lens line camera is used, the projection method of the fisheye lens camera as shown in FIG. 4 is used. f is the focal length of the line camera (110), ρ is the image points in the projection center of the angle between the ray and the optical axis of the line camera 110, an object point ^(O P) entering the forming, r _p is the line camera 110 The distance to (P ') is the direction coordinate value of the omnidirectional image proposed by the present invention. 'Juho' et al. (2004) proposed four kinds of fisheye lens camera projection methods, but since the present invention does not know a projection method suitable for the camera used, the residuals are best applied by applying all four projection methods shown in [Equation 6]. Determine the less projection. Equation (7) is an equation for determining an angle p between the light ray of the line camera 110 and the optical axis with respect to the image point P '.

[Equation 6]

&Quot; (7) &quot;

Third, the coordinate transformation of the image point converts the image point ^C P of the object represented by the camera coordinate system to be expressed in the image coordinate system. First, since the i-direction coordinate value of the image is related to the position and attitude of the camera as shown in [Equation 8], it is determined by dividing the angle α-β formed by the object point in the image by the rotation angle μ _α per pixel. μ _α is determined by dividing 360 ° by the total number of pixels m in the i direction. Next, the j-direction coordinate value of the image may be determined by dividing r _p by the size (μ _j ) of the j-direction per pixel. However, since the j direction coordinate value of the image has a range of -n / 2 to n / 2 (n is the number of pixels in the j direction of the image), the image point (i, j), which is the direction coordinate value of the image as shown in [Equation 8]. ) (The image point on the image coordinate system expressed in pixels).

&Quot; (8) &quot;

External estimation factor estimation

In general, the estimation of an external coordinate element for an outdoor image uses a ground reference point of absolute coordinate values obtained by measuring with a Global Position System (GPS) or a total station. Total station is a device that calculates horizontal distance, azimuth angle, height difference to target object by irradiating laser to target object and analyzing reflected signal. However, for indoor images, it is difficult to use the ground reference point of the absolute coordinate value. Therefore, in order to estimate the external pointing element of the indoor image, the object coordinate system as defined above is defined indoors and the object reference point coordinates expressed in the object coordinate system are required. As described above, the object coordinate system can be set by defining one corner point of the room as the origin and three corners meeting at the origin as the attitude of each axis. The coordinate values of the corner points in the object coordinate system defined in this way may be defined by a method illustrated in FIG. 5 using a distance from an origin point.

As described above, the collinear equation of the object reference point expressed in the object coordinate system acquired mainly as a corner point and the omnidirectional image obtained by the rotary line camera 110 derived from the above can be schematically expressed by an observation equation as shown in [Equation 9]. The synthesizing unit 132 of the device 130 estimates the external expression element by using the same. That is, the image point (i, j) is the external expression element (ω, φ, κ, ^O X _R , ^O Y _R , ^O Z _R ), that is, the origin point in the rotational coordinate system ( ^O X _R , ^O Y _R , ^O Z _R ), posture data (ω, φ, κ,), object reference point ( ^o X, ^o Y, ^o Z), radius of rotation (r), and twisted angle of rotation (β) of camera 110 are nonlinear Using the nonlinear observation equation [Equation 9] for the image points (i, j), the external expression element is estimated, and the projection method 1, the projection method 2, the projection method 3, and the After all of the projection methods 4 are applied, the external expression elements estimated by the projection method having the smallest residual in the i and j directions are determined. The external expression elements ω, φ, and κ may be camera pose data for each of the front (Roll axis direction), the front right direction (Pitch axis direction), and the gravity direction (Yaw axis direction), respectively. ^O X _R , ^O Y _R , ^O Z _R ) is the origin position data of the rotation coordinate system with respect to the object reference point expressed in the object coordinate system.

&Quot; (9) &quot;

Since one object reference point corresponds to one image point, two observation equations such as Equation 8 and Equation 9 can be established. In order to solve the above six external expression elements, at least six observation equations are needed, and thus three or more reference points are required. Using three or more reference points, the above observation equations can be established and linearized to estimate external expression factors by applying the least square method. The estimated external expression element may again be used for error correction of the image points (i, j).

<Experimental Method>

First, after obtaining the omnidirectional image and the object reference point (OCP) with the rotary line camera 110, georeferencing is performed by applying four types of projection methods, and a projection method having a minimum residual is determined. In addition, the result of applying the determined projection method is analyzed to determine the suitability of georeferencing the indoor omnidirectional image obtained with the proposed rotary line camera.

<Observation image acquisition>

In this case, a line camera 110 having a 1 × 4000 pixel was used, and the focal length of the camera was 15 mm, and the angle formed by the ^R X axis of the camera and the rotating coordinate system was fixed at 90 °. First, the rotation radius is set to 0 to obtain an indoor omnidirectional image as shown in FIG. 6, and the rotation radius r is set to 18 cm to obtain an indoor omnidirectional external image as shown in FIG. 7. The size of the omnidirectional image is 8000 x 4000 pixels and the size per pixel is 10 m x 10 m. In the images shown in FIGS. 6 and 7, arbitrary 56 points were used as the object reference point.

<Acquire Object Reference Point>

Three or more object reference points are required to estimate the external expression elements of the indoor omnidirectional image acquired by the rotary line camera 110. In this experiment, a plurality of object reference points were used to determine the best projection method for the rotary line camera 110.

For example, define a corner point of the room as the origin, measure the length of the three corners that meet at that corner point, obtain (X _b , Y _b , Z _b ) of the measurement coordinate system, and three points represented by the object coordinate system. It can be estimated using Equation 10 to convert to the coordinates (X _a , Y _a , Z _a ) of. Where (X _b , Y _b , Z _b ) is the three-dimensional coordinate value of three points expressed in the measurement coordinate system, and (X _a , Y _a , Z _a ) is the three-dimensional coordinates of three points expressed in the object coordinate system. The value R is a predetermined rotation transformation matrix for converting from the object coordinate system to the measurement coordinate system, and t is a movement transformation matrix for adding a constant movement value for error correction.

&Quot; (10) &quot;

First, in order to use the parameter estimation method between the measured coordinate system and the object coordinate system, eight corner points and 74 object points were measured by the total station. The object point consists of the intersections of the bookshelves seen in the image. The coordinate transformation coefficient between the measured coordinate system and the object coordinate system was estimated using four corner points, and 74 object points were converted to the object coordinate system using the estimated coordinate transformation coefficient. Finally, 56 object reference points were obtained by removing 52 outliers and 4 corner points. The ideal point is defined as the difference between the two points, Z, that should be at similar heights in the bookcase, and the difference between the lengths of the bookcases having a constant length.

Table 1 shows the four corner point coordinates used to estimate the coordinate transformation coefficients, which are coordinate values expressed in the measurement coordinate system and the object coordinate system. [Table 2] is the estimated coordinate transformation coefficient and the origin (X _t ,) of the measurement coordinate system expressed in the object coordinate system. Y _t , Z _t ) and attitude (ω, φ, κ). [Table 3] shows the coordinates of the corner points before and after the coordinate transformation, and the eight corner points C0 to C7 expressed in the measurement coordinate system and the object coordinate system. 8 and 9 illustrate object points before and after performing coordinate transformation, and are object points represented by a measurement coordinate system and an object coordinate system. 10 is a drawing of object points with outliers and outliers removed.

[Table 1]

[Table 2]

[Table 3]

<Determine the suitable projection method for the rotary line camera>

Finally, the external markers of internal and external images were estimated using the 56 object reference points. Since the internal and external images are obtained without first acquiring the internal image and the external image, the center of rotation of the internal and external images is the same and the starting point is different. Therefore, ω and φ were similar to the origin of the rotational coordinate system of internal and external images, and κ was very different.

[Table 4] shows the results of estimating the external expression elements of the internal image and [Table 5] shows the results of estimating the external expression elements of the external image. The most precise estimate was made when applying projection 1.

[Table 4]

[Table 5]

Table 6 shows the standard deviations of the residuals in the i and j directions of the variance component and image point estimated by each projection method. The i-coordinate value of the image point is related to the rotation angle, and the j-coordinate value is related to the projection method. When the projection method 1 was applied, the observation precision and the standard deviation of the residual were the best.

TABLE 6

As a result of applying the four projection methods, the error in the i direction of the image was relatively small, but the error in the j direction was significantly different for each projection method. In addition, as the error in the j direction increases, the error in the i direction also tends to increase. This is because the projection method affecting only the j-direction affects the external expression element and the estimated external expression element affects the i-coordinate value of the image point. The precision of the estimated externally-expressed elements, and the standard deviation of the estimated variance and residuals, were the smallest when applying projection method 1. Therefore, in this study, Projection Method 1 was determined as a suitable projection method for the proposed rotary line camera.

<Analysis of the Profitability of Proposed Georeferencing>

The suitability of georeferencing can be analyzed by the distribution of residual size in the i and j directions and the relationship between the main variables and the residuals. First, the distribution tendency of the i-direction residual size of the image point calculated by the external expression element estimated by applying the projection method 1 was examined. 11 and 12 are histograms of the i-direction image point residuals of the internal and external images. It contains some outliers but does not seem to deviate significantly from the normal distribution.

Next, the rotation angle (α), which is a key variable for determining the i value of the image point, and the i-direction residual of the image point were examined. As shown in FIGS. 13 and 14, it was found that there is no significant relationship between the rotation angle and the i-direction residual of the image point.

15 and 16 are histograms of the j-direction residual magnitudes of the inner and outer image points. Although the most suitable fisheye lens projection method is applied, the standard deviation is not small due to the characteristics of fisheye lens with severe distortion.

Next, the j value of the image point is only related to the ^R Z value of the object point which is the main variable. Therefore, the relationship between the ^R Z value of the object point and the j-direction residual of the image is examined. As shown in FIGS. 17 and 18, it can be seen that there is a negative correlation between the ^R Z value and the residual in the j direction. In particular, a negative correlation was apparent between -0.5m and 0.5m.

In addition, the relationship between the distance (D) between the camera and the object point, which is an important variable affecting the image point, and the residuals in the i and j directions of the image point was examined. 19 and 20 are residual i directions of the inner and outer images according to the distance between the camera and the object point, and FIGS. 21 and 22 are residual j directions of the inner and outer images. There was no significant relationship.

Finally, the residuals in the i and j directions according to the radiation distance were examined. 23 and 24 are residual i-directions of the inner and outer images according to the radiation distance, and FIGS. 25 and 26 are residual j-directions of the inner and outer images. The distribution of j-direction residuals has a clear correlation with the radiation distance, because the effect of lens distortion has a great influence on the j-direction.

<Conclusion>

As in the present invention, in order to estimate an external expression element for georeferencing an indoor single omnidirectional image obtained with a rotary line camera, a rotary line camera and coordinate systems are defined, a collinear equation is derived, and an object is also used. In order to secure the reference point, the object coordinate system was defined, the coordinate transformation coefficient was estimated, the coordinate transformation was performed to obtain the object reference point represented by the object coordinate system, and external projection elements were estimated by applying four projection methods. By selecting the projection method with the best estimation accuracy and analyzing the distribution of residuals calculated by the selected projection method and its relationship with the main variables, we propose that the proposed georeferencing method is suitable for the indoor omnidirectional image obtained with the rotary line camera. Confirmed. In the above experimental example, the estimated position of the external expression element was estimated to be 1.4 mm in accuracy, and the posture was estimated to be 0.05 ° in accuracy. Also, the image point residual showed a standard deviation of 2.56 pixels in the i direction and 9.85 pixels in the j direction, and the j direction residual had a negative correlation with the value of the object point. In addition, since the j-direction of the image shows a clear relationship with the radiation distance, it is necessary to correct the distortion of the fisheye lens.If the calibration of the rotary line camera is performed in the future, the distance of the center of rotation (r) and the camera are distorted. It will be possible to estimate the angle of rotation (β) and the distortion coefficient of the fisheye lens camera, which will enable more accurate georeferencing and more than two georeferenced indoor omnidirectional images. It is expected that three-dimensional coordinate estimation will be possible, and furthermore, accurate indoor three-dimensional modeling will be possible.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

Omnidirectional image providing system (100)
Line camera (100)
Controller 130
Storage medium (131)
Synthesis part 132

Claims (6)

In the georeferencing method of indoor omnidirectional images obtained with a line camera, a line-by-line image of an omnidirectional image is obtained by rotating the line camera, stored in a storage means, and each image point (i) on the focal length f is used by the same. and an external expression element that generates an omnidirectional image of n × m pixels projected onto the image of j), and includes posture data of the line camera using a preselected plurality of object reference points in the room and the acquired collinear equation of the image. Correcting the coordinate values of the image points (i, j),
Calculation process of the collinearity equations, the corner point as the origin of the room and the first stage representing the object point (P ^O) to the object coordinate system defined by three axes in the three edges meeting at the corner point;
A second step of converting the object point ^O P into an object point ^C P represented by a camera coordinate system defined by a position and a posture of the moment when the image is captured;
A third step of calculating a j-direction distance r _p of the image points i and j with respect to the object point ^C P on the focal length f of the line camera represented by the camera coordinate system based on a predetermined projection method; ; And
A fourth step of calculating coordinate values of the image points (i, j) by calculating a pixel position of the object point ( ^C P) from an angle formed by the object point ( ^C P) to the image and the r _p ; And
A fifth step of estimating the external expression element of the omnidirectional image acquired by the rotating line camera;
Georeferencing method comprising a. The method of claim 1,
At least three or more object points ^O P expressed in the object coordinate system are used as the plurality of object reference points.
Define a corner point of the room as the origin of the object coordinate system and obtain the (X _b , Y _b , Z _b ) of the measured coordinate system by measuring the lengths of the three corners that meet at the corner point, Equation to convert to coordinates (X _a , Y _a , Z _a )

(X _b , Y _b , Z _b ) are three-dimensional coordinate values of three points expressed in the measurement coordinate system, and (X _a , Y _a , Z _a ) are three points expressed in the object coordinate system. The three-dimensional coordinate value of, R is a rotation transformation matrix for converting from the object coordinate system to the measurement coordinate system, t is a geo-referencing method, characterized in that the movement transformation matrix. The method of claim 1,
In the second step,
Equations

Where ^O O _R is the origin of the rotation coordinate system expressed in the object coordinate system, ^R R _O is the rotation transformation matrix from the object coordinate system to the rotation coordinate system, r is the rotation radius of the line camera, and α is the virtual Rotation angle in the rotational coordinate system, β is the angle between the line camera and the rotation axis, θ is the X-axis rotation angle of the point projected to the XY plane of the rotation coordinate system, ε is the difference between θ and α Where ^R P ( ^R X, ^R Y, ^R Z) is the object point ( ^O P) coordinate in the rotational coordinate system, and ^C P ( ^C X, ^C Y, ^C Z) is the object point ( ^O P) in the camera coordinate system. Georeferencing method characterized in that the coordinate. The method of claim 1,
In the third step,
Equations

, Where: ρ is the angle between the optical axis and the ray of the object point ^O P coming into the line camera, f is the focal length of the line camera, and ^C P ( ^C X, ^C Y, ^C Z) is the object point ^O in the camera coordinate system. P) Georeferencing method characterized in that the coordinate. The method of claim 1,
In the fourth step,
Equation

Where (i, j) is an image point on an image coordinate system expressed in pixels, α is a rotation angle in a virtual rotation coordinate system of the line camera, β is a rotation angle in the camera coordinate system of the line camera, and n is The number of pixels in the j direction of the image, μ _α is the rotation angle per pixel, μ _j is the size of the j direction per pixel. 5. The method of claim 4,
In the fifth step,
Using the object reference point, the rotation radius of the line camera, and the angle at which the line camera and the rotation axis are distorted, the origin and posture data of the virtual rotation coordinate system of the line camera as the external expression element are estimated with respect to the image point (i, j). But
And applying the projection method 1, the projection method 2, the projection method 3, and the projection method 4 to determine the external expression element estimated by the projection method having the smallest residual in the i and j directions.

Images