JP7293931B2 - Position measurement data generation device, position measurement data generation method, and position measurement data generation program - Google Patents

Description

The present disclosure relates to a position measurement data generation device, a position measurement data generation method, and a position measurement data generation program.

Patent Document 1 describes a method of capturing an image in which the line of sight of a camera is tilted with respect to a measurement area on the ground, and measuring the position of an object recognized by image processing. and the image coordinate system corresponding to the imaging plane of the camera are assumed to be two-dimensionally connected, and the positional relationship between the world coordinate system and the image coordinate system with respect to this intermediate coordinate plane is defined. A position measurement method based on image processing is disclosed, which is characterized by creating a physical model and using this model to determine the position on the ground from the position of the object on the image.

JP-A-10-332334

However, the technique described in Patent Document 1 assumes that the ground surface, which is the imaging range of the camera, is horizontal. There is a problem that the accuracy may decrease.

The present disclosure provides a position measurement data generation device, a position measurement data generation method, and a position measurement data generation program that can suppress a decrease in position measurement accuracy on an uneven ground surface. With the goal.

A position measurement data generating device according to a first aspect of the present disclosure is provided with a measuring unit that measures a three-dimensional position. and a three-dimensional position of the measurement vehicle measured by the measurement unit when the measurement vehicle travels on the road; A detection unit (66) for detecting an image position, a correspondence information generation unit (68) for generating correspondence information in which the image position and the three-dimensional position are associated with each other, and the image detected by the detection unit. and a vertex information generation unit (70) for generating vertex information in which positions are associated with the three-dimensional positions representing vertices forming a triangle in the coordinate system of the three-dimensional positions.

A position measurement device according to a second aspect of the present disclosure, based on the correspondence information and vertex information generated by the position measurement data generation device according to any one of the first to fourth aspects, on the captured image. a conversion unit (62) for converting an arbitrary image position in the above into the three-dimensional position.

A method for generating position measurement data according to a third aspect of the present disclosure includes: a computer (30A) having a measuring unit that measures a three-dimensional position; A photographed image of the measurement vehicle and a three-dimensional position of the measurement vehicle measured by the measurement unit when the measurement vehicle travels on the road are obtained, and the position of the measurement vehicle is acquired from the photographed image. detecting an image position, generating correspondence information in which the image position and the three-dimensional position are associated with each other, and representing the detected image position as a vertex forming a triangle in the coordinate system of the three-dimensional position; Generate vertex information associated with three-dimensional positions.

A position measurement data generation program (40) according to a fourth aspect of the present disclosure includes a computer provided with a measurement unit for measuring a three-dimensional position, and a camera installed at a position capable of photographing a road on which a measurement vehicle travels. A photographed image of the measurement vehicle and a three-dimensional position of the measurement vehicle measured by the measurement unit when the measurement vehicle travels on the road are obtained, and the position of the measurement vehicle is acquired from the photographed image. detecting an image position, generating correspondence information in which the image position and the three-dimensional position are associated with each other, and representing the detected image position as a vertex forming a triangle in the coordinate system of the three-dimensional position; A position measurement data generation program for executing processing including generation of vertex information associated with a three-dimensional position.

ADVANTAGE OF THE INVENTION According to this indication, it has the effect that it can suppress that the measurement accuracy of the position in the ground surface which is not flat can fall.

1 is a configuration diagram of a position measurement system; FIG. It is a figure for demonstrating the road which a camera image|photographs, and a world coordinate system. It is a block diagram which shows the hardware constitutions of a position measuring device. 3 is a functional block diagram of a CPU of the position measuring device; FIG. 6 is a flowchart of position measurement data generation processing; It is a figure for demonstrating the image position of the measurement vehicle detected from the picked-up image. FIG. 10 is a diagram for explaining the correspondence between the image position of the measurement vehicle and the three-dimensional position of the measurement vehicle; FIG. 4 is a diagram for explaining generation of vertex information; FIG. 10 is a diagram for explaining selection of vertices of a triangle; FIG. 4 is a diagram for explaining a case of transforming an arbitrary point in an image coordinate system into an arbitrary point in a world coordinate system; FIG. 10 is a diagram for explaining an error that occurs in coordinate transformation by linear approximation; FIG. 10 is a diagram for explaining an error that occurs in coordinate transformation by linear approximation; It is a figure for demonstrating the case where the depth direction seen from the camera is along the lane width direction. FIG. 4 is a diagram for explaining coordinate conversion; FIG. 4 is a diagram for explaining coordinate conversion; 6 is a flowchart of position measurement data generation processing; FIG. 4 is a diagram for explaining correction of an image position; FIG. 10 is a diagram for explaining addition of vertices; FIG. 10 is a diagram for explaining selection of vertices of a triangle; It is a figure for demonstrating the case where a triangle is enlarged. FIG. 4 is a diagram for explaining the accuracy of coordinate transformation near vertices; FIG. 4 is a diagram for explaining the accuracy of coordinate transformation near vertices;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

FIG. 1 is a configuration diagram of a position measurement system 1 according to this embodiment. The position measurement system 1 has a configuration in which a position measurement device 10 , a photographing device 14 , and an in-vehicle device 12 are connected via a network 16 .

The in-vehicle device 12 is mounted, for example, on a measurement vehicle S as shown in FIG. As shown in FIG. 1, the in-vehicle device 12 includes a measuring section 18 that measures a three-dimensional position. The measurement unit 18 measures the three-dimensional position of the vehicle-mounted device 12 in a three-dimensional space, that is, the three-dimensional position of the measurement vehicle S, using, for example, GNSS (Global Navigation Satellite System). A three-dimensional position in a three-dimensional space, which is a real space, is expressed as coordinates in a world coordinate system in which x-, y-, and z-axes are orthogonal to each other, as shown in FIG.

The photographing device 14 includes a camera 20 and an angle-of-view adjusting device 22 . The camera 20 is a monocular camera having an imaging device such as a CCD (Charge-coupled devices) sensor or a CMOS (Complementary metal-oxide-semiconductor) sensor. The camera 20 is installed at a position capable of photographing the road RD on which the measurement vehicle S travels, as shown in FIG. 2, for example. In this embodiment, a case where the road RD is a road consisting of four lanes L1 to L4 with two lanes on each side will be described as an example. Also, the road RD refers to a road included in the photographing range of the camera 20 in this embodiment.

The camera 20 photographs the road RD on which the measurement vehicle S and other vehicles travel at predetermined photographing intervals. The captured image data of the captured image captured by the camera 20 is transmitted to the position measuring device 10 via the network 16 and stored in the position measuring device 10 . Here, the photographed image data is, for example, data in which a photographed image and a photographing time are associated with each other. Note that the shooting interval is set to an interval at which multiple shots are taken while the measurement vehicle S is traveling on the road RD. Also, the camera 20 may shoot the road RD as a moving image. In this case, the shooting interval is defined by the frame rate.

The measurement vehicle S travels in advance on the road RD photographed by the camera 20, and measures the three-dimensional position in the world coordinate system at predetermined measurement intervals. It should be noted that conversion accuracy is higher when the measurement interval is shorter than the photographing interval. Therefore, it is preferable to set the measurement interval shorter than the photographing interval.

Three-dimensional position measurement data obtained by measuring the three-dimensional position of the measurement vehicle S is transmitted to the position measuring device 10 via the network 16 and stored in the position measuring device 10 . Here, the three-dimensional position measurement data includes, for example, the measured three-dimensional position, the measurement time when the three-dimensional position was measured, the movement vector of the measurement vehicle S at the measurement time, and the movement of the measurement vehicle S at the measurement time. are data in which speed and are associated with each other.

The angle-of-view adjusting device 22 has a function of adjusting the angle of view of the camera 20 to an arbitrary angle of view based on an instruction from the position measuring device 10 . In this embodiment, a case where the photographing device 14 includes the angle-of-view adjusting device 22 will be described, but the angle of view of the camera 20 may be fixed and the angle-of-view adjusting device 22 may be omitted.

Although the details will be described later, the position measurement device 10 detects the image position of the measurement vehicle S on the captured image by performing image processing on the captured image captured by the camera 20 . Therefore, it is preferable to provide a marker, for example, on the roof of the measurement vehicle S to facilitate detection of the measurement vehicle S. Examples of markers include predetermined specific patterns such as chessboard patterns, but are not limited to these.

FIG. 3 is a configuration diagram of the position measuring device 10. As shown in FIG. The position measuring device 10 is composed of a device including a general computer.

As shown in FIG. 3 , the position measuring device 10 has a controller 30 . The controller 30 includes a CPU (Central Processing Unit) 30A as an example of a computer, a ROM (Read Only Memory) 30B, a RAM (Random Access Memory) 30C, a nonvolatile memory 30D, and an input/output interface (I/O) 30E. . A CPU 30A, a ROM 30B, a RAM 30C, a nonvolatile memory 30D, and an I/O 30E are connected via a bus 30F.

Also, the operation unit 32, the display unit 34, the communication unit 36, and the storage unit 38 are connected to the I/O 30E.

The operation unit 32 includes, for example, a mouse and a keyboard.

The display unit 34 is configured by, for example, a liquid crystal display.

The communication unit 36 is an interface for performing data communication with external devices such as the in-vehicle device 12 and the imaging device 14 .

The storage unit 38 is composed of a non-volatile storage device such as a hard disk. The storage unit 38 stores a position measurement data generation program 40, a position measurement program 42, three-dimensional position measurement data 44, photographed image data 46, association information 48, vertex information 50, and the like, which will be described later. The CPU 30A reads and executes the position measurement data generation program 40 and the position measurement program 42 stored in the storage unit 38 .

Next, the functional configuration of the CPU 30 of the position measuring device 10 will be described.

As shown in FIG. 4 , the CPU 30A functionally includes a position measurement data generation device 60 and a conversion section 62 . The position measurement data generation device 60 includes an acquisition unit 64 , a detection unit 66 , a correspondence information generation unit 68 and a vertex information generation unit 70 .

The acquisition unit 64 obtains a captured image of the measurement vehicle S captured by the camera 20, which is installed at a position capable of capturing a road RD on which the measurement vehicle S travels and which includes the measurement unit 18 for measuring the three-dimensional position, and the measurement vehicle S. and the three-dimensional position of the measurement vehicle S measured by the measurement unit 18 when S travels on the road RD.

In this embodiment, as an example, the three-dimensional position of the vehicle S to be measured is the three-dimensional position measured when traveling in each of the oncoming lanes of the road RD.

The detection unit 66 detects the image position of the measurement vehicle S from the captured image captured by the camera 20 .

The association information generation unit 68 generates association information 48 that associates the image position of the measurement vehicle S in the captured image with the three-dimensional position. The generated association information 48 is stored in the storage unit 38 .

The vertex information generation unit 70 generates vertex information 50 in which the image positions detected by the detection unit 66 are associated with three-dimensional positions representing vertices forming a triangle in a three-dimensional coordinate system, that is, a world coordinate system. . The generated vertex information 50 is stored in the storage unit 38 .

The conversion unit 62 converts an arbitrary image position on the captured image into a three-dimensional image in the world coordinate system based on the position measurement data including the correspondence information 48 and the vertex information 50 generated by the position measurement data generation device 60. Convert to position. Specifically, the conversion unit 62 calculates a conversion parameter based on the vertex information of a triangle that includes the position of the image to be converted within the region, and converts the position of the image to be converted into a three-dimensional position using the calculated conversion parameter. Convert.

Next, the operation of the position measuring device 10 according to this embodiment will be described. First, position measurement data generation processing will be described. As shown in FIG. 3, the position measurement data generation program 40 is stored in the storage unit 38 . The position measurement data generation process shown in FIG. 5 is executed by the CPU 30A reading and executing the position measurement data generation program 40. FIG. The position measurement data generation process shown in FIG. 5 is executed when the operator operates the operation unit 32 to instruct execution of the position measurement data generation process. It should be noted that the processing shown in FIG. 5 may be repeatedly and automatically executed each time a predetermined time elapses without intervention of the operator.

In addition, the image data 46 captured by the camera 20 when the measurement vehicle S travels on the road RD and the three-dimensional position measured by the measurement unit 18 of the in-vehicle device 12 when the measurement vehicle S travels on the road RD are shown. It is assumed that the three-dimensional position measurement data 44 has already been stored in the storage unit 38 . Moreover, in this embodiment, for the sake of simplicity, the case where the number of measurement vehicles S is one will be described.

In addition, the camera 20 and the measurement unit 18 each have a time synchronization function. Time is assumed to be synchronized. As a time synchronization method, various known methods can be used, for example, a method such as NTP (Network Time Protocol) can be used, but the method is not limited to this.

In step S100, the CPU 30A acquires the captured image data 46 captured by the camera 20 when the measurement vehicle S travels on the road RD by reading from the storage unit .

In step S102, the CPU 30A detects the image position of the measurement vehicle S from the captured image based on the captured image data. Specifically, first, the vehicle is detected from the photographed image using a technique such as pattern matching. Then, for example, when a marker is provided on the roof portion of the measurement vehicle S to facilitate detection of the measurement vehicle S, the vehicle provided with the marker is extracted as the measurement vehicle from among the detected vehicles. . In addition, when the measurement vehicle S is not provided with a marker, shape images of the front side shape and the rear side shape of the measurement vehicle S are stored in advance in the storage unit 38, and the shape images are extracted from the photographed images. By doing so, the measurement vehicle S may be specified.

Then, as shown in FIG. 6, the coordinates of the point at the lower left corner of the bounding box BB of the measurement vehicle S detected from the photographed image G are taken as the image position of the measurement vehicle S. FIG. In FIG. 6, the image position of the measurement vehicle S traveling on the lane L1 detected from the captured image G captured at a certain time is represented as p ₁ (u ₁ , v ₁ ). It should be noted that the image coordinate system in the photographed image G is a two-dimensional coordinate system composed of mutually orthogonal u-axis and v-axis, as shown in FIG. The origin Om of the image coordinate system is the point at the upper left corner of the captured image G. As shown in FIG.

In addition, since the measurement vehicle S traveling on the road RD is photographed a plurality of times by the camera 20, the measurement vehicle S is detected from each of the plurality of consecutively photographed images of the measurement vehicle S traveling on the road RD. do. In the captured image G of FIG. 6, the image position p ₃ (u ₃ , v ₃ ). Further, the image position detected from the captured image captured when the measurement vehicle S travels in the opposite lane L4 is represented as p ₂ (u ₂ , v ₂ ).

By detecting the image positions of the measurement vehicle S from a plurality of captured images in this manner, a plurality of image positions of the measurement vehicle S traveling in the lanes L1 and L4 are detected for each of the lanes L1 and L4.

In step S104, the CPU 30A acquires the three-dimensional position measurement data corresponding to each of the plurality of image positions of the measurement vehicle S detected in step S102 by reading from the storage unit . Here, the three-dimensional position measurement data corresponding to the image position of the measurement vehicle S is the measurement data of the three-dimensional position measured at the closest time to the photographing time of the photographed image in which the image position of the measurement vehicle S was photographed. be.

As a result, for example, as shown in FIG. 7, three-dimensional position P ₁ (x _{1 ,} y ₁ , z ₁ ) corresponding to image position p ₁ (u 1 , v ₁ ), image position p ₂ (u ₂ , v ₂ ) corresponding to the three-dimensional position P ₂ (x ₂ , y _{2 ,} z ₂ ), and the three-dimensional position P ₃ (x 3 , y ₃ , z ₃ ) corresponding to the image position p ₃ (u ₃ , v ₃ ). is obtained.

By the way, for example, if the photographing time of the photographed image at which the image position p ₁ (u ₁ , v ₁ ) was detected coincides with the measurement time at which the three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ) was measured, is not limited. Therefore, the three-dimensional position of the measuring vehicle S needs to be corrected.

Therefore, in step S106, the CPU 30A corrects the three-dimensional position using the photographing time as the reference time. That is, the three-dimensional position of the measurement vehicle S at the reference time is calculated. Specifically, for example, the three-dimensional position at the reference time is calculated based on the movement vector and the movement speed of the measurement vehicle S at the measurement time, which are included in the three-dimensional position measurement data, and the difference between the photographing time and the measurement time. do.

In step S108, CPU 30A generates association information that associates the image position of measurement vehicle S detected in step S102 with the three-dimensional position corrected in step S106, and stores the information in storage unit . For example, in the example of FIG. 7, the image position p ₁ (u ₁ , v ₁ ) is associated with the corrected three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ). Similarly, the image position p ₂ (u ₂ , v ₂ ) is associated with the corrected three-dimensional position P ₂ (x ₂ , y ₂ , z ₂ ). Also, the image position p ₃ (u ₃ , v ₃ ) is associated with the corrected three-dimensional position P ₃ (x ₃ , y ₃ , z ₃ ).

In step S110, CPU 30A generates vertex information in which the image positions detected in step S102 are associated with three-dimensional positions representing vertices forming a triangle in the world coordinate system of three-dimensional positions, and stores the vertex information in storage unit 38. do. For example, in the example of FIG. 8, the image positions p ₁ , p ₂ , and p ₃ representing the vertices of the triangle TG in the photographed image G, and the three-dimensional positions P ₁ , P 2 , P ₂ , Generate vertex information associated with _P3 .

In practice, a large number of image positions may be detected from a photographed image, and how to select vertices poses a problem. How to select vertices will be described below. Further, hereinafter, when the three-dimensional position is not distinguished, it is simply referred to as a three-dimensional position P.

For example, as shown in FIG. 9, a plurality of three-dimensional positions P are measured when the measurement vehicle S travels on the lane L1, and a plurality of three-dimensional positions P are measured when the measurement vehicle S travels on the opposite lane L4 of the lane L1. Assume that the original position P is detected.

In this case, two adjacent three-dimensional positions Pa and Pb are selected from the plurality of three-dimensional positions P measured along the lane L1. Next, assuming that the perpendicular bisector N of the line segment connecting the two selected three-dimensional positions Pa and Pb is drawn toward the lane L4, the shortest cubic line from the perpendicular bisector N is drawn. Select the original position Pc. Then, vertex information is generated in which the three-dimensional positions Pa and Pb measured on the lane L1 and the three-dimensional position Pc measured on the lane L4 are associated as vertices of triangles. This vertex information generation processing is executed for all adjacent three-dimensional positions P measured on the lane L1. In addition, the same processing is performed for the adjacent three-dimensional position P measured on the lane L4, which is the oncoming lane.

Next, processing for converting an arbitrary image position on the captured image into a three-dimensional position based on the correspondence information and vertex information generated by executing the above processing will be described.

First, an approximation principle for converting a two-dimensional image position in the image coordinate system into a three-dimensional position in the world coordinate system using linear approximation will be described.

As shown in FIG. 10, the three-dimensional position of the three-dimensional world coordinate system C3 corresponding to the arbitrary image position of the two-dimensional image coordinate system C2 is the triangle TG containing the arbitrary image position of the image coordinate system C2. It is obtained by coordinate transformation based on the image positions p1 to p3 of the vertices and the three-dimensional positions P1 to P3 of the vertices of the triangle TW in the world coordinate system C3 associated with the vertices p1 to p3 of the triangle TG.

For simplicity of explanation, a spatial coordinate system is adopted such that the XZ plane consisting of the X and Z axes in the world coordinate system is parallel to the optical axis Zcam of the camera 20, as shown in FIG. In this case, as shown in FIG. 11, when the XZ plane is viewed from the Y-axis direction, the length in the X direction in the world coordinate system per unit pixel in the v direction in the captured image, that is, the depth as seen from the camera 20 The length in the direction depends on the pixel position in the v direction. For example, when the distance from the image position p1 to the image position p3 in the image coordinate system C2 is divided into four equal parts in the v direction, the respective distances d1 to d4 are all the same distance. On the other hand, the distances D1 to D4 in the world coordinate system C3 corresponding to the distances d1 to d4 in the image coordinate system C2 are not the same. That is, the distance D1 on the short distance side from the camera 20 is the shortest, and the distance D4 on the long distance side from the camera 20 is the longest. Thus, the length in the depth direction viewed from the camera 20 in the image coordinate system C2 and the length in the depth direction viewed from the camera 20 in the world coordinate system C3 have a nonlinear relationship. That is, when the XZ plane is viewed from the Y-axis direction in the world coordinate system C3, the triangle in the image coordinate system C2 and the triangle in the world coordinate system C3 are not similar. Therefore, when the coordinates of an arbitrary point in the image coordinate system C2 are transformed into the coordinates in the world coordinate system C3 by linear approximation, an error occurs in the depth direction as seen from the camera 20 .

On the other hand, as shown in FIG. 12, when the XY plane is viewed from the Z-axis direction, the length in the Y-axis direction in the world coordinate system C3 per unit pixel in the u-direction in the image coordinate system C2 is They are the same and do not differ depending on the pixel position. For example, when the distance from the image position p1 to the image position p3 in the image coordinate system C2 is divided into two equal parts in the u direction, the respective distances d1 and d2 are all the same distance. Also, the distances D1 and D2 in the world coordinate system C3 corresponding to the distances d1 and d2 in the image coordinate system C2 are the same. That is, when the XY plane is viewed from the Z-axis direction, the triangle in the image coordinate system C2 and the triangle in the world coordinate system C3 are similar. Therefore, no error occurs when the coordinates of an arbitrary point in the image coordinate system C2 are transformed into the coordinates of the world coordinate system by linear approximation.

Therefore, if an error in the depth direction as seen from the camera 20 caused by conversion by linear approximation is acceptable, the coordinates of any point in the image coordinate system C2 can be coordinate-transformed into the coordinates in the world coordinate system. For example, as shown in FIG. 6, when the road RD extends in the depth direction when viewed from the camera 20, for example, when the optical axis direction of the camera 20 extends along the lane width direction A of the road RD as illustrated in FIG. , the error in the length in the depth direction viewed from the camera 20 is often permissible. This is because, when the road RD extends in the depth direction as viewed from the camera 20, a large number of three-dimensional positions of the measurement vehicle S are measured along the road RD, and thus adjacent three-dimensional positions along the road RD are measured. This is because the distance becomes shorter and the influence of error becomes smaller. Note that the shorter the measurement interval of the three-dimensional position and the shooting interval of the camera 20, the smaller the error. Even if the road RD runs along the depth direction when viewed from the camera 20, if the three-dimensional position measurement interval or the camera 20 photographing interval does not satisfy the accuracy required by the application, the error cannot be allowed. . Therefore, it is preferable to adjust the measurement interval and the photographing interval according to the accuracy required by the application.

In the following, as shown in FIG. 6, the case where the road RD extends in the depth direction when viewed from the camera 20 will be described. On the other hand, as shown in FIG. 13, when the optical axis direction of the camera 20 is along the lane width direction A of the road RD, as shown in FIG. In many cases, the length error in the depth direction as seen from the camera 20 cannot be allowed as compared to the case. This is because when the optical axis direction of the camera 20 is along the lane width direction A of the road RD, the three-dimensional position of the vehicle S to be measured along the lane width direction A of the road RD is small. This is because the distance between adjacent three-dimensional positions becomes longer, and the influence of errors becomes greater. Therefore, it is necessary to generate vertex information by setting the position where the lane of the road RD and the triangle intersect as a new vertex. Details of this processing will be described later.

A specific conversion method will be described below.

For example, as shown in FIG. 14, an arbitrary image position p _i (u _i , v i ) within the triangle TG in the image coordinate system C2 is converted to a three-dimensional position P _i (x _i , v _i ) in the world coordinate system C3 shown in FIG. y _i , z _i ) will be described.

Assuming that the approximation principle described above holds, an arbitrary image position p _i (u _i , v _i ) within the triangle TG in the image coordinate system C2 is a two-dimensional can be transformed into a point in the triangle TW of .

_{Therefore , the image position p _ _ _} A first calculation formula for calculating _i (u _i , v _i ) and a three-dimensional position P ₁ (x ₁ , y ₁ , z ₁ ), P ₂ (x ₂ , y ₂ , z ₂ ), P ₃ (x ₃ , y ₃ , z ₃ ) based on the three-dimensional position P _i (x _i , y _i , _zi ) is the same as the second calculation formula. That is, both calculation formulas have the same coefficients, but differ only in the dimensions of the variables.

First, the first calculation formula in the image coordinate system C2 is represented by the following formula.

p _i =p ₁ +α _i (p ₂ −p ₁ )+β _i (p ₃ −p ₁ )
={1−(α _i +β _i )}p ₁ +α _i ×p ₂ +β _i ×p ₃ (1)

However, α _i + β _i ≤ 1, 0 ≤ α _i , 0 ≤ β _i (2)

In the above equation (1), p _i , p ₁ , p ₂ and p ₃ represent vectors.

Conversion parameters α _i and β _i are calculated from the above equations (1) and (2).

Then, the second calculation formula in the world coordinate system C3 is expressed by the following formula like the first calculation formula.

P _i ={1−(α _i +β _i )}P ₁ +α _i ×P ₂ +β _i ×P ₃ (3)

In the above equation (3), P _i , P ₁ , P ₂ and P ₃ represent vectors.

The three-dimensional position P _i is calculated by substituting the conversion parameters α _i and β _i calculated from the equations (1) and (2) into the equation (3).

Position measurement processing for converting an arbitrary image position on a captured image into a three-dimensional position using the conversion method described above will now be described with reference to the flowchart shown in FIG.

The position measurement process shown in FIG. 16 is executed when the operator instructs execution of the position measurement process.

First, in step S200, the CPU 30A causes the display section 34 to display the captured image captured by the camera 20. FIG.

In step S202, the CPU 30A receives the designation of the image position _pi to be converted by the operator. The operator operates the operation unit 32 to designate an image position p _i to be converted, for example, a vehicle running on a road.

In step S204, the CPU 30A transforms the specified image position _pi into a three-dimensional position _pi in the world coordinate system. Specifically, the vertex information stored in the storage unit 38 is referred to, and the vertex information of the vertex of the triangle containing the image position _pi designated in step S202 is extracted. The image positions of the vertices represented by the extracted vertex information are p ₁ , p ₂ , and p ₃ .

Next, referring to the correspondence information stored in the storage unit 38, the three-dimensional positions P 1 and P 2 corresponding to the image positions _{p 1} , p ₂ and p ₃ of _the three vertices represented by the detected vertex information. , _P3 respectively.

Then, the image positions p ₁ , p ₂ and p ₃ are substituted into the equation (1), and the conversion parameters α _i and β _i are calculated by the equations (1) and (2). Next, by substituting the calculated transformation parameters α _i and β _i and the extracted three-dimensional positions P ₁ , P ₂ and P ₃ into the above equation (3), the image position p _i specified in step S202 Calculate the three-dimensional position P _i corresponding to .

In step S206, the CPU 30A causes the display unit 34 to display the three-dimensional position P _i calculated in step S204. Thereby, for example, the three-dimensional position of the vehicle specified by the operator can be known. In the processing of FIG. 16, a case has been described in which a photographed image is displayed and the image position specified by the operator is converted into a three-dimensional position in the world coordinate system, but the present invention is not limited to this. For example, a vehicle is detected from a photographed image using a known method such as pattern matching, and a process of converting the image position of the detected vehicle into a three-dimensional position in the world coordinate system is automatically executed at predetermined time intervals. good too.

Thus, in this embodiment, the road photographed by the camera 20 is represented by a set of triangles whose vertices are the image positions of the measurement vehicle S detected from the photographed images. It is associated with the three-dimensional position of the measurement vehicle in the coordinate system C3. Then, an arbitrary image position within the triangle in the image coordinate system C2 is transformed into a three-dimensional position in the world coordinate system C3. As a result, even if the ground surface for position detection is not flat, the accuracy of position detection is lower than when the ground surface, which is the shooting range of the camera, is assumed to be horizontal. can be suppressed. In addition, since the installation position information of the camera is unnecessary, the installation work of the camera is facilitated.

By the way, as described above, when the optical axis direction of the camera 20 is along the lane width direction A of the road RD as shown in FIG. In this case, the vertex information is generated with the position where the lane of the road RD and the triangle intersect as a new vertex. Specifically, as shown in FIG. 17, first, the image positions p1 to p3 are corrected so as to be pixels on the road and the surface on which the measurement vehicle S is installed. Also, the three-dimensional positions P1 to P3 corresponding to the image positions p1 to p3 are corrected so that the three-dimensional positions P1 to P3 match the corrected image positions p1 to p3. For example, the installation position information about the installation position of the measurement unit 18 and the shape information about the shape of the measurement vehicle S are stored in advance in the storage unit 38, and based on the installation position information and the shape information, the image positions p1 to p3 are determined. The three-dimensional positions P1 to P3 are corrected so that the three-dimensional positions P1 to P3 match the corrected image positions p1 to p3. As the shape information, it is preferable to prepare a three-dimensional model or the like according to the type of the vehicle S to be measured (sedan, wagon, truck, bus), for example.

It also adds vertices on the boundaries of adjacent lanes. Specifically, for example, as shown in FIG. 18, boundary lines K1 to K3 of adjacent lanes of lanes L1 to L4 are detected using a known method. Here, the coordinate information of the boundary lines K1 to K3 in the world coordinate system C3 is stored in the storage unit 38 in advance. Then, the points of intersection between the sides of the triangle TG whose vertices are the image positions p ₁ , p ₂ , and p ₃ and the boundary lines K1 to K3 are added as new vertices. Also, based on the coordinate information of the boundary lines K1 to K3 in the world coordinate system C3, the three-dimensional position corresponding to the image position representing the newly added vertex is calculated, and the two are associated. Note that when calculating the three-dimensional position corresponding to the image position representing the newly added vertex, it is assumed that the lane width of each lane is known and that the boundary line and the travel route of the measurement vehicle S are parallel. to calculate.

In the example of FIG. 18, three-dimensional positions P ₄ and P ₅ are added as vertices on the boundary line K1 between the lanes L1 and L2, and three-dimensional positions P 6 and P ₆ are added as vertices on the boundary line K2 between the lanes L2 and L3. _P7 is added, and three-dimensional positions _P8 and _P9 are added as vertices on the boundary line K3 between the lanes L3 and L4.

Then, vertex information is generated with the newly added vertex as the vertex of a new triangle. For example, two three-dimensional positions added on the same boundary line are selected as the vertices of the base of the triangle, and three-dimensional positions added on the boundary lines on both sides of this base, or if there is no boundary line next to the three-dimensional position of the measurement vehicle S Select the third vertex from the three-dimensional position on the travel path. At this time, the vertex closer to the perpendicular bisector of the line connecting the two three-dimensional positions added on the same boundary line is selected as the third vertex.

For example, as shown in FIG. 19, newly added three-dimensional positions P ₄ and P ₅ will be described as an example. In this case, assuming that the perpendicular bisector N of the line connecting the three-dimensional positions _P4 and _P5 is drawn in the lane width direction, the boundary line K1 where the three-dimensional positions _P4 and _P5 exist is adjacent to Of the three-dimensional positions _P6 and _P7 newly added on the matching boundary line K2, the three-dimensional position _P7 closer to the perpendicular bisector N is selected as the vertex. On the opposite side of the boundary line K2 to the boundary line _{K1 ,} there is no adjacent boundary line. is selected as the _vertex . As a result, a triangle T1 whose vertices are the three-dimensional positions P ₄ , P ₅ and P ₇ and a triangle T2 whose vertices are the three-dimensional positions P ₄ , P ₅ and P ₁ are newly defined. New triangles are defined by performing similar processing for the other three-dimensional positions P ₁ , P ₃ , P ₆ , P ₇ , P ₈ and P ₉ . As a result, since the number of vertices increases along the lane width direction A, it is possible to suppress deterioration in conversion accuracy when converting an arbitrary image position in the image coordinate system C2 to a three-dimensional position in the world coordinate system C3. can.

In this embodiment, a case has been described in which coordinate transformation is performed using the three-dimensional position measurement data obtained by measuring the three-dimensional position of the measurement vehicle S traveling in lanes L1 and L4. The defined triangle does not cover the shoulder of the road RD. Therefore, as shown in FIG. 20, vertex information of an enlarged triangle TW2 may be generated by enlarging the triangle TW1 beyond the roadway to the sidewalk or building. Specifically, a well-known technique such as edge detection is used to detect the boundary between the roadway and the sidewalk and the boundary between the roadway and the building, and the vertex information of the enlarged triangle is generated by enlarging the triangle to the detected boundary.

Also, the accuracy of coordinate transformation is better at positions closer to the vertices of the triangle. For example, as shown in FIG. 21, the neighboring region R1 of the three-dimensional position P1, the neighboring region R2 of the three-dimensional position P2, and the neighboring region R3 of the three-dimensional position P3 have higher accuracy of coordinate conversion than other regions. Become. Therefore, it is preferable to measure the three-dimensional position P by driving the measurement vehicle S in all the lanes L1 to L4 as shown in FIG. As a result, the number of regions R near the vertices where the precision of coordinate transformation is high increases, and the rate of coordinate transformation near the vertices increases.

Further, in the present embodiment, the case where the angle of view of the camera 20 is fixed has been described. Vertex information may be generated.

Moreover, it is preferable to run the measurement vehicle S to acquire the three-dimensional position measurement data even after the correspondence information and the vertex information are once generated, and to update the correspondence information and the vertex information. As a result, even when the angle of view of the camera 20 shifts due to disturbance or the like, it is possible to prevent the accuracy of the coordinate conversion from deteriorating.

Further, in the present embodiment, the case where the position measurement device 10 includes the position measurement data generation device 60 and the conversion unit 62 has been described. 10 may be configured separately.

In this embodiment, the road RD on which the measurement vehicle S travels has been described as a four-lane road with two lanes on each side. It may be a one-lane road for traffic. That is, the number of lanes is not limited.

Although the embodiments have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above embodiments without departing from the gist of the invention, and the forms with such changes or improvements are also included in the technical scope of the present invention.

In addition, the above embodiments do not limit the claimed invention, and all combinations of features described in the embodiments are essential to the solution of the invention. Not exclusively. Inventions at various stages are included in the embodiments described above, and various inventions can be extracted by combining a plurality of disclosed constituent elements. Even if some constituent elements are deleted from all the constituent elements shown in the embodiments, as long as an effect is obtained, a configuration in which these several constituent elements are deleted can be extracted as an invention.

Further, in the above embodiment, the case where the position measurement data generation program 40 and the position measurement program 42 are pre-installed in the storage unit 38 has been described, but the present invention is not limited to this. For example, the position measurement data generation program 40 and the position measurement program 42 may be stored in a storage medium such as a CD-ROM (Compact Disc Read Only Memory) and provided, or may be provided via a network. good.

Furthermore, in the above embodiment, the case where the position measurement data generation process and the position measurement process are realized by a computer using a software configuration by executing a program has been described, but the present invention is limited to this. not something. For example, the position measurement data generation process and the position measurement process may be realized by a hardware configuration or a combination of hardware and software configurations.

In addition, the configuration of the position measuring device 10 described in the above embodiment (see FIG. 3) is merely an example, and unnecessary portions may be deleted or new portions added without departing from the gist of the present invention. It goes without saying that it is possible to

In addition, the processing flow of the position measurement data generation program 40 and the position measurement program 42 described in the above embodiment (see FIGS. 5 and 16) is also an example, and unnecessary steps can be performed within the scope of the present invention. may be deleted, new steps may be added, or the processing order may be changed.

62 conversion unit, 64 acquisition unit, 66 detection unit, 68 correspondence information generation unit, 70 vertex information generation unit

Claims (9)

A photographed image of the measurement vehicle taken by a camera equipped with a measurement unit for measuring a three-dimensional position and installed at a position capable of photographing a road on which the measurement vehicle travels; an acquisition unit (64) for acquiring a three-dimensional position of the measurement vehicle measured by the measurement unit;
a detection unit (66) for detecting the image position of the measurement vehicle from the captured image;
an association information generating unit (66) for generating association information in which the image position and the three-dimensional position are associated;
From among the image positions detected by the detection unit, two image positions adjacent to each other along the road and a perpendicular bisector of a line segment connecting the two image positions are directed to the oncoming lane of the road. and the image position of the shortest distance from the perpendicular bisector on the oncoming traffic lane as vertices of a triangle, and the selected three image positions are: a vertex information generation unit (70) for generating vertex information associated with the three three-dimensional positions in the coordinate system of the three-dimensional position based on the correspondence information;
A conversion unit (62) for calculating a conversion parameter based on the vertex information of the triangle including the correspondence information and the image position to be converted, and converting the image position to be converted into the three-dimensional position using the calculated conversion parameter. )and,
Position measurement data generation device (60). 2. The position measurement data generation device according to claim 1, wherein the three-dimensional position of the measurement vehicle is a three-dimensional position measured when traveling in each of the oncoming lanes of the road. 3. The position measurement data generation device according to claim 1, wherein the vertex information generator generates vertex information of an enlarged triangle obtained by enlarging the triangle. When the optical axis direction of the camera is along the lane width direction of the road, the vertex information generation unit generates vertex information using the three-dimensional position where the lane of the road and the triangle intersect as a new vertex. The position measurement data generation device according to any one of claims 1 to 3. A computer (30A)
A photographed image of the measurement vehicle taken by a camera equipped with a measurement unit for measuring a three-dimensional position and installed at a position capable of photographing a road on which the measurement vehicle travels; obtaining a three-dimensional position of the measurement vehicle measured by the measurement unit;
detecting the image position of the measurement vehicle from the captured image;
generating association information that associates the image position with the three-dimensional position;
Two image positions adjacent to each other along the road and a perpendicular bisector of a line connecting the two image positions are drawn from among the detected image positions toward the oncoming lane of the road. Assuming that three image positions are selected as the vertices of a triangle consisting of the image position of the shortest distance from the perpendicular bisector on the oncoming lane , and the selected three image positions are stored in the correspondence information Based on, generating vertex information associated with the three three-dimensional positions in the coordinate system of the three-dimensional position ,
calculating a conversion parameter based on the vertex information of the triangle including the correspondence information and the image position to be converted, and converting the image position to be converted into the three-dimensional position using the calculated conversion parameter;
Position measurement data generation method. 6. The position measurement data generating method according to claim 5 , wherein the three-dimensional position of the measurement vehicle is a three-dimensional position measured when traveling in each of the oncoming lanes of the road. 7. The method of generating position measurement data according to claim 5 , further comprising generating vertex information of an enlarged triangle obtained by enlarging said triangle. When the optical axis direction of the camera is along the lane width direction of the road , vertex information is generated with a three-dimensional position where the lane of the road and the triangle intersect as a new vertex. 2. The method for generating position measurement data according to item 1. to the computer,
A photographed image of the measurement vehicle taken by a camera equipped with a measurement unit for measuring a three-dimensional position and installed at a position capable of photographing a road on which the measurement vehicle travels; obtaining a three-dimensional position of the measurement vehicle measured by the measurement unit;
detecting the image position of the measurement vehicle from the captured image;
generating association information that associates the image position with the three-dimensional position;
From among the detected image positions, two image positions adjacent to each other along the road and a perpendicular bisector of a line segment connecting the two image positions are drawn toward the oncoming lane of the road. Assuming that three image positions are selected as vertices of a triangle consisting of the image position of the shortest distance from the perpendicular bisector on the oncoming traffic lane , and the selected three image positions are stored in the correspondence information Based on, generating vertex information associated with the three three-dimensional positions in the coordinate system of the three-dimensional position ,
calculating a conversion parameter based on the vertex information of the triangle including the correspondence information and the image position to be converted, and converting the image position to be converted into the three-dimensional position using the calculated conversion parameter;
A position measurement data generation program (40) for executing processing including:

Images