JP7113997B1 - Information processing device, information processing method, and information processing program - Google Patents

Abstract

An information processing device (1) includes a data acquisition unit (31), a first display processing unit (40), a specifying unit (34), a second display processing unit (41), a distance calculation unit (35 ). A data acquisition unit (31) acquires captured image data and three-dimensional point group data. A first display processing section (40) causes the display section (12) to display the captured image based on the data of the captured image. A specifying unit (34) specifies a three-dimensional point corresponding to a point specified by the user in the captured image displayed on the display unit (12) from among a plurality of three-dimensional points. A second display processing unit (41) causes the display unit (12) to display the three-dimensional point specified by the specifying unit (34) in a state of being superimposed on the captured image. When the identifying unit (34) identifies three-dimensional points corresponding to the two points designated by the user, the distance calculating unit (35) determines the distance corresponding to the two points identified by the identifying unit (34). Calculate the distance between two 3D points.

Description

The present disclosure relates to an information processing device, an information processing method, and an information processing program for calculating a distance between two points specified on a captured image.

Conventionally, structures such as tunnels and bridges are regularly inspected. A technique is adopted in which images are collected and an inspection operator confirms the state of the structure based on the collected captured images.

When checking the state of a structure, there is a need to measure the distance between two points specified by an inspection operator on a captured image for the purpose of checking the size of deformation of the structure. In order to measure the distance between two points specified by the inspection operator, the coordinates of the two specified points are required. Therefore, Patent Literature 1 discloses a technique for measuring the distance between two points specified in a captured image by image analysis.

JP 2020-165729 A

In the technique described in Patent Document 1, a diagram such as a plane, a pseudo-curve, or a straight line is calculated from three or more points specified on a captured image, and the distance between the calculated diagrams is calculated. However, the accuracy of distance measurement may deteriorate depending on the position or range of the diagram.

The present disclosure has been made in view of the above, and an object thereof is to obtain an information processing apparatus capable of accurately measuring a distance.

In order to solve the above-described problems and achieve the object, the information processing apparatus of the present disclosure includes a data acquisition unit, a first display processing unit, a specifying unit, a second display processing unit, and a distance calculation unit. and The data acquisition unit acquires captured image data obtained by imaging a structure and three-dimensional point group data representing the structure with a plurality of three-dimensional points in a three-dimensional coordinate system. The first display processing unit causes the display unit to display the captured image based on the data of the captured image. The specifying unit specifies a three-dimensional point corresponding to the point specified by the user in the captured image displayed on the display unit, from among a plurality of three-dimensional points. The second display processing unit causes the display unit to display the three-dimensional point specified by the specifying unit in a state of being superimposed on the captured image. When the specifying unit specifies three-dimensional points corresponding to the two points specified by the user, the distance calculating unit calculates the distance between the two three-dimensional points corresponding to the two points specified by the specifying unit. calculate.

Advantageous Effects of Invention According to the present disclosure, there is an effect that distance measurement can be performed with high accuracy.

1 is a diagram showing an example of a configuration of a measurement processing system according to a first embodiment; FIG. FIG. 2 is a diagram showing an example of the configuration of a measurement device included in the measurement vehicle according to the first embodiment; FIG. FIG. 2 is a diagram showing an example of a three-dimensional point cloud of a structure measured by a three-dimensional measurement unit of the measurement device provided in the measurement vehicle according to the first embodiment; 1 is a diagram showing an example of a configuration of an information processing apparatus according to a first embodiment; FIG. FIG. 4 is a diagram showing an example of a captured image data table stored in a captured image data storage unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of a three-dimensional point cloud data table stored in the three-dimensional point cloud data storage unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of a specific point data table stored in a specific point data storage unit of the information processing apparatus according to the first embodiment; FIG. FIG. 4 is a diagram showing an example of a captured image displayed on the display unit by the first display processing unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining a method of moving a three-dimensional point group by a first display processing unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of a captured image displayed on a display unit by a second display processing unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of distances between specific points displayed on the display unit by the third display processing unit of the information processing apparatus according to the first embodiment; FIG. 5 is a diagram showing an example of a total distance of connection lines displayed on the display unit by the third display processing unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of a specific point determination method by the specifying unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of processing by a processing unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of captured image display processing by the processing unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of distance measurement processing between specific points by the processing unit of the information processing apparatus according to the first embodiment; 3 is a flowchart showing an example of specific point determination processing by the processing unit of the information processing apparatus according to the first embodiment; 3 is a flowchart showing an example of three-dimensional point group processing by the processing unit of the information processing apparatus according to the first embodiment; 3 is a flowchart showing an example of connection line total distance measurement processing by the processing unit of the information processing apparatus according to the first embodiment; 1 is a diagram showing an example of a hardware configuration of an information processing apparatus according to a first embodiment; FIG.

An information processing apparatus, an information processing method, and an information processing program according to embodiments will be described in detail below with reference to the drawings.

Embodiment 1.
1 is a diagram illustrating an example of a configuration of a measurement processing system according to a first embodiment; FIG. As shown in FIG. 1 , the measurement processing system 100 according to the first embodiment includes an information processing device 1 and a measurement vehicle 2 .

The measurement vehicle 2 includes a vehicle body 60 and a measurement device 70 mounted on the vehicle body 60 . The measuring device 70 captures images of structures existing around the vehicle body 60 while the vehicle body 60 is running, and measures the three-dimensional shape of the structure. In the example shown in FIG. 1, the structures are roads 4, tunnels 5, guardrails 6, and the like, but are not limited to such examples, and may be, for example, signboards, traffic lights, bridges, and the like. The vehicle body 60 of the measurement vehicle 2 is a vehicle that travels on the road. In the example shown in FIG. 1, it is an automobile that travels on the road 4 as the travel path. There may be.

The measurement device 70 includes captured image data including captured image data obtained by capturing an image of a structure, and three-dimensional point cloud data representing a structure with a plurality of three-dimensional points in a three-dimensional coordinate system. Generate dimensional point cloud data. Hereinafter, a three-dimensional point group representing a structure with a plurality of three-dimensional points in a three-dimensional coordinate system may be referred to as a three-dimensional point group of the structure or a three-dimensional point group.

The measuring device 70 can transmit and receive data to and from the information processing device 1 via the network 3 , and transmits captured image data, three-dimensional point cloud data, and the like to the information processing device 1 via the network 3 . The network 3 is, for example, a WAN (Wide Area Network) such as the Internet, but may be a LAN (Local Area Network) or other networks.

FIG. 2 is a diagram illustrating an example of a configuration of a measurement device included in the measurement vehicle according to the first embodiment; As shown in FIG. 2 , the measurement device 70 included in the measurement vehicle 2 includes an imaging section 71 , a three-dimensional measurement section 72 , a position detection section 73 , a processing section 74 and a communication section 75 .

The imaging unit 71 captures images of structures existing around the vehicle body 60 while the vehicle body 60 is running, and outputs data of captured images of the structures obtained by the imaging to the processing unit 74 . The imaging unit 71 is, for example, an omnidirectional camera, a semi-spherical camera, or a panorama camera, but is not limited to such examples.

An omnidirectional camera is also called a 360-degree camera, and outputs omnidirectional image data as captured image data. output as A panoramic camera outputs, for example, a horizontal 360-degree panoramic image or a horizontal 180-degree panoramic image as captured image data.

The three-dimensional measurement unit 72 measures structures existing around the vehicle body 60 while the vehicle body 60 is running, and sends three-dimensional point cloud data including three-dimensional point cloud data of the structures obtained by the measurement to the processing unit 74 . Output. The three-dimensional measurement unit 72 includes, for example, a laser scanner device, and generates three-dimensional point cloud data of a structure from measurement data obtained by such a laser scanner device.

FIG. 3 is a diagram showing an example of a three-dimensional point group of a structure measured by a three-dimensional measurement unit of the measurement device provided in the measurement vehicle according to the first embodiment. A 3D point group 80 shown in FIG. 3 includes a 3D point group 81 of the road 4 , a 3D point group 82 of the tunnel 5 , and a 3D point group 83 of the guardrail 6 . The three-dimensional point group data output from the three-dimensional measurement unit 72 includes data indicating the coordinates of each of the plurality of three-dimensional points forming the three-dimensional point group 80 .

The position detection unit 73 detects the position of the measuring device 70 when the imaging unit 71 captures an image of the structure while the vehicle body 60 is running, and outputs position data, which is data of the detected position, to the processing unit 74 . The position detection unit 73 receives a plurality of positioning signals transmitted from a plurality of positioning satellites in GNSS (Global Navigation Satellite System), and detects the position of the measuring device 70 based on the received plurality of positioning signals.

The position data output from position detection unit 73 includes latitude, longitude, and altitude data. The position of the measuring device 70 that is arranged near the imaging unit 71 and detected by the position detection unit 73 is used as the viewpoint position of the captured image that is the image captured by the imaging unit 71 .

The processing unit 74 acquires captured image data output from the imaging unit 71 , three-dimensional point cloud data output from the three-dimensional measurement unit 72 , and position data output from the position detection unit 73 . The processing unit 74 generates captured image data based on the captured image data output from the imaging unit 71 and the position of the vehicle body 60 output from the position detection unit 73 . The captured image data includes captured image data and viewpoint position data.

The processing unit 74 outputs the generated captured image data and the three-dimensional point cloud data output from the three-dimensional measurement unit 72 to the communication unit 75 . The communication unit 75 is wirelessly connected to the network 3 , and transmits captured image data and three-dimensional point cloud data acquired from the processing unit 74 to the information processing device 1 via the network 3 . Data transfer between the measuring device 70 and the information processing device 1 can also be performed using a recording medium. Also, the three-dimensional point cloud data may be generated by the processing section 74 based on the measurement data output from the three-dimensional measurement section 72 .

4 is a diagram illustrating an example of a configuration of an information processing apparatus according to the first embodiment; FIG. As shown in FIG. 4 , the information processing device 1 includes a communication section 10 , an input section 11 , a display section 12 , a storage section 13 and a processing section 14 .

The communication unit 10 is communicably connected to the network 3 by wire or wirelessly, and transmits and receives information to and from an external device such as a measuring device 70 , a printer (not shown), or a terminal device via the network 3 .

The input unit 11 includes, for example, a mouse and a keyboard, but may also be a touch pad. The display unit 12 is, for example, a liquid crystal display, an organic EL (ElectroLuminescence) display, or a projector. In the example shown in FIG. 4, the input unit 11 and the display unit 12 are included in the information processing device 1. The input unit 11 and the display unit 12 are connected to the information processing device 1 as external devices of the information processing device 1. may be

The storage unit 13 includes a captured image data storage unit 20 , a three-dimensional point cloud data storage unit 21 and a specific point data storage unit 22 . The captured image data storage unit 20 stores a captured image data table including a plurality of captured image data transmitted from the measuring device 70 .

5 is a diagram illustrating an example of a captured image data table stored in a captured image data storage unit of the information processing apparatus according to the first embodiment; FIG. The captured image data table shown in FIG. 5 includes “target ID (IDentifier)”, “image ID”, “viewpoint position”, and “data file” for each captured image data.

“Target ID” is identification information unique to each target to be measured by the measuring device 70 . The objects to be measured are, for example, a plurality of structures including the road 4, the tunnel 5, and the guardrail 6 as shown in FIG. , and may be structures other than roads 4, tunnels 5, and guardrails 6.

“Image ID” is unique identification information for each captured image. "Viewpoint position" is the viewpoint position of the captured image, and is indicated by latitude, longitude, and altitude. The viewpoint position of the captured image is the position of the imaging unit 71 when the captured image is obtained by the imaging unit 71 of the measuring device 70 or a position corresponding to the position of the imaging unit 71 .

A “data file” is data of a captured image, and includes pixel data indicating the color of each pixel included in the captured image and distance data indicating the distance from the viewpoint position to the captured image. Color data is, for example, RGB data, and R, G, and B gradations are, for example, 256 gradations. The data file is, for example, data in JPEG (Joint Photographic Experts Group) format or MP (Multi Picture) 4 format, but is not limited to these formats.

In the captured image data table shown in FIG. 5, the captured image with the target ID of "T001" and the image ID of "IM1001" has a viewpoint position of P (x1, y1, z1) and a data file of "IMGF1001". be. The captured image with the target ID of "T001" and the image ID of "IM1002" has a viewpoint position of P(x2, y2, z2) and a data file of "IMGF1002".

The 3D point cloud data storage unit 21 shown in FIG. 4 stores a 3D point cloud data table including a plurality of 3D point cloud data transmitted from the measuring device 70 . 6 is a diagram illustrating an example of a three-dimensional point cloud data table stored in a three-dimensional point cloud data storage unit of the information processing apparatus according to the first embodiment; FIG.

The three-dimensional point cloud data table shown in FIG. 6 includes “object ID” and “data file” for each three-dimensional point cloud data. “Object ID” is identification information unique to each measurement object. A “data file” is data of a three-dimensional point group, and includes brightness data and position data of each three-dimensional point included in the three-dimensional point group. For example, when the three-dimensional measurement unit 72 includes a laser scanner device, the brightness data is the reflection intensity data of the three-dimensional point. Position data is data indicating the position of a three-dimensional point, and includes, for example, latitude, longitude, and altitude data.

In the 3D point cloud data table shown in FIG. 6, the 3D point cloud with the target ID "T001" has the data file "DGF1001", and the 3D point cloud with the target ID "T002" has the data file It is "DGF1002".

The specific point data storage unit 22 shown in FIG. 4 stores data of specific points, which are three-dimensional points corresponding to points in the captured image designated by the user of the information processing apparatus 1 . Hereinafter, the user of the information processing device 1 may be simply referred to as the user.

7 is a diagram of an example of a specific point data table stored in a specific point data storage unit of the information processing apparatus according to the first embodiment; FIG. The specific point data table shown in FIG. 7 includes a "specific point ID" and a "specific point position" for each specific point. "Specific point ID" is identification information unique to each measurement target. "Specific point position" is data indicating the position of a specific point, and includes latitude, longitude, and altitude data of the specific point.

In the specific point data table shown in FIG. 7, the specific point with the specific point ID "S001" is located at P (x11, y11, z11), and the specific point with the specific point ID "S002" is located at P (x12, y12, z12).

Returning to FIG. 4, the description of the information processing apparatus 1 is continued. The processing unit 14 shown in FIG. 4 includes an input reception unit 30, a data acquisition unit 31, a thinning unit 32, a display processing unit 33, a specification unit 34, a distance calculation unit 35, and a data output unit 36. .

The input reception unit 30 receives an input operation to the input unit 11 by the user. For example, the input reception unit 30 receives a display request for a captured image from the user via the input unit 11 . The captured image display request includes, for example, information such as the target ID of the measurement target to be displayed, the viewpoint position, the line-of-sight direction, and the angle of view.

The input reception unit 30 also receives a designation operation from the user via the input unit 11 . A designation operation is an operation by which a user designates an arbitrary position within a captured image. In the following description, a position within a captured image specified by a user may be referred to as a specified point. Also, the input reception unit 30 receives an output request from the user via the input unit 11 .

The data acquisition unit 31 acquires captured image data and three-dimensional point cloud data transmitted from the measurement device 70 of the measurement vehicle 2 and received by the communication unit 10 via the network 3 from the communication unit 10 . The data acquisition unit 31 adds the captured image data acquired from the communication unit 10 to the captured image data table stored in the captured image data storage unit 20, and converts the three-dimensional point cloud data acquired from the communication unit 10 into three-dimensional point data. Add to the three-dimensional point cloud data table stored in the cloud data storage unit 21 .

Further, when the input reception unit 30 receives a display request for a captured image from the user, the data acquisition unit 31 acquires the captured image data and the three-dimensional point cloud data corresponding to the display request received by the input reception unit 30 . is obtained from the storage unit 13 . The data acquiring unit 31 also adds specific point data, which is data of the specific points specified by the specifying unit 34, to the specific point data table stored in the specific point data storage unit 22, or adds specific point data to the specific point data table. Acquire the included specific point data.

The thinning unit 32 thins out some of the three-dimensional points included in the three-dimensional point cloud data acquired by the data acquiring unit 31 in response to the display request. The viewpoint position is specified by viewpoint position data included in the captured image data acquired by the data acquisition unit 31 in response to the display request.

The thinning unit 32, for example, deletes some or all of the three-dimensional points that satisfy the thinning conditions that do not or have little effect on the determination process of the specific point, which is the three-dimensional point corresponding to the designated point. Decimate dimension points. Thinning-out conditions that do not or have little effect on the determination processing of specific points are, for example, positions that are at a distance greater than or equal to a preset distance from the viewpoint position, or where the density of three-dimensional points is greater than or equal to a preset density. It is a condition that

For example, the thinning unit 32 selects, among a plurality of 3D points included in the 3D point cloud data acquired by the data acquisition unit 31 in response to a display request, 3D points whose distance from the viewpoint position is a preset distance or more. 3D points are thinned out by deleting the 3D points at their positions.

In addition, the thinning unit 32 selects a plurality of three-dimensional points included in the three-dimensional point cloud data acquired by the data acquisition unit 31 in response to a display request, and divides a plurality of three-dimensional points within a predetermined distance from each other. If points exist, the 3D points are thinned out by deleting some of the plurality of 3D points. Note that the thinning condition for the three-dimensional points by the thinning unit 32 is not limited to the above example, as long as the thinning condition does not or has little effect on the specific point determination process.

In this way, the thinning unit 32 thins out the three-dimensional points by deleting the three-dimensional points that have little or no influence on the determination processing of the specific points. , and the processing load on the display processing unit 33 can be reduced.

Note that the thinning unit 32 performs thinning processing when, for example, the number of three-dimensional points included in the three-dimensional point cloud data acquired by the data acquiring unit 31 in response to a display request is less than a preset number. You can do nothing. Further, the thinning unit 32 may not perform the thinning process when the processing load of the information processing apparatus 1 is less than a preset load.

Further, when the storage unit 13 stores setting data as to whether or not to perform the thinning process according to the object to be measured, the thinning part 32 determines whether to perform the thinning process based on the setting data. can also The thinning unit 32 can also determine whether or not to perform the thinning process according to user settings.

The display processing unit 33 generates a two-dimensional image based on the captured image data and the three-dimensional point cloud data obtained by the data obtaining unit 31 in response to a display request, and displays the generated two-dimensional image on the display unit 12. Let When the thinning processing is performed by the thinning unit 32, the display processing unit 33 replaces the 3D point cloud data acquired by the data acquiring unit 31 in response to the display request with the thinning unit 32. A two-dimensional image is generated using the three-dimensional point cloud data that has undergone the thinning process.

The display processing section 33 includes a first display processing section 40 , a second display processing section 41 and a third display processing section 42 . The display processing unit 33 has a first mode and a second mode as operation modes.

The operation mode can be changed by the user, and the user can change the operation mode by performing a specific operation on the input unit 11 . In this case, the user can also set the operation mode for each measurement target.

Moreover, the display processing unit 33 can change the operation mode according to the number of three-dimensional points included in the three-dimensional point cloud data. For example, the display processing unit 33 operates in the first operation mode when the number of 3D points included in the 3D point cloud data is equal to or less than the threshold, and operates in the second operation mode otherwise. .

First, the processing of the first display processing unit 40 when the operation mode is the first mode will be described. When the operation mode is the first mode, the first display processing unit 40 converts the captured image represented by the captured image data and the 3D point group represented by the 3D point cloud data into the same 3D coordinate system. Deploy. The captured image is arranged in a spherical shape in a three-dimensional coordinate system, with the viewpoint position specified by the captured image data as the center of the sphere and the distance specified by the distance data of the captured image data as the radius. Also, the first display processing unit 40 renders the three-dimensional point group colorless and transparent.

Then, the first display processing unit 40 generates a two-dimensional image of the captured image and the three-dimensional point group arranged in the same three-dimensional coordinate system, viewed from the viewpoint position and having the angle of view specified by the display request. and outputs the generated two-dimensional image data to the display unit 12 to display the two-dimensional image including the captured image on the display unit 12 . Since the color of the 3D point group is made transparent, the user cannot visually recognize the 3D point group in the 2D image generated by the first display processing unit 40 . In the following description, a two-dimensional image viewed from a viewpoint position and having an angle of view specified by a display request may be referred to as a two-dimensional image viewed from the viewpoint position.

8 is a diagram illustrating an example of a captured image displayed on the display unit by the first display processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 8, the first display processing section 40 can cause the display section 12 to display a two-dimensional image 90 including a captured image. Since the color of the 3D point group is made transparent, the user cannot visually recognize the 3D point group in the 2D image 90 .

Next, processing of the first display processing unit 40 when the operation mode is the second mode will be described. When the operation mode is the second mode, the first display processing unit 40 converts the captured image represented by the captured image data and the 3D point group represented by the 3D point cloud data to the same 3D coordinate system. The three-dimensional point group is moved to a position farther from the viewpoint position than the captured image.

9 is a diagram for explaining a method of moving a three-dimensional point cloud by the first display processing unit of the information processing apparatus according to the first embodiment; FIG. In the example shown in FIG. 9, for convenience of explanation, a plan view of a captured image and a three-dimensional point group arranged in the same three-dimensional coordinate system is shown. In FIG. 9, the Z-axis direction is the vertical direction, and the Y-axis positive direction is the viewing direction.

The first display processing unit 40 acquires information on the distance from the viewpoint position to the captured image based on the distance data included in the captured image data. Note that the first display processing unit 40 can, for example, calculate the radius of the spherical surface from the spherical shape of the captured image that is spherically arranged in the three-dimensional coordinate system as the distance from the viewpoint position to the captured image. The distance from the viewpoint position to the captured image acquired or calculated by the first display processing unit 40 is the Euclidean distance in the three-dimensional coordinate system, but is not limited to this example.

The first display processing unit 40 arranges the 3D points so that the distance from the viewpoint position to each 3D point of the 3D point group is longer than the distance from the viewpoint position to the captured image in the 3D coordinate system. A moving distance, which is the distance to move the group, is calculated. The first display processing unit 40 moves the three-dimensional point group by the calculated moving distance so that the distance from the viewpoint position becomes farther than the captured image in the three-dimensional coordinate system, as shown in FIG. As a result, the three-dimensional point group seen from the viewpoint position is on the back side of the captured image and cannot be visually recognized from the viewpoint position.

The first display processing unit 40 generates data of a two-dimensional image in which the captured image and the three-dimensional point group arranged in the same three-dimensional coordinate system are viewed from the viewpoint position, and displays the data of the generated two-dimensional image. By outputting to the unit 12, the display unit 12 is caused to display a two-dimensional image including the captured image.

Since the 3D point cloud seen from the viewpoint position is on the back side of the captured image, the user cannot visually recognize the 3D point cloud in the 2D image generated by the first display processing unit 40 . Thereby, the first display processing unit 40 can allow the user to visually recognize only the captured image without making the three-dimensional point group colorless and transparent. Therefore, the first display processing unit 40 does not need to change the color information of the three-dimensional point group, and can reduce the processing load.

When the operation mode is the first mode, the second display processing unit 41 displays a plurality of points included in the three-dimensional point group arranged in the same three-dimensional coordinate system as the captured image by the first display processing unit 40. Among the three-dimensional points, the specific points and peripheral points are highlighted so as to be distinguishable from the captured image. The highlighting is performed, for example, by conspicuously changing the color, size, pattern, and shape of the specific point and peripheral points.

A specific point is a three-dimensional point specified by the specifying unit 34, and a peripheral point is a three-dimensional point around the specific point. The three-dimensional points around the specific point are, for example, three-dimensional points within a preset distance range from the specific point. The preset distance range increases as the distance from the viewpoint position increases, but may be the same regardless of the distance from the viewpoint position.

The second display processing unit 41 highlights the specific point and the surrounding points so as to be distinguishable from the captured image, and then displays the captured image and the three-dimensional point group arranged in the same three-dimensional coordinate system from the viewpoint position. 2D image data is generated. Then, the second display processing unit 41 outputs the generated two-dimensional image data to the display unit 12 so that the two-dimensional image including the captured image and the highlighted specific points and peripheral points is displayed on the display unit 12. display.

10 is a diagram illustrating an example of a captured image displayed on the display unit by the second display processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 10 , the second display processing unit 41 causes the display unit 12 to display a two-dimensional image 90 including a captured image 91 and highlighted specific points 93 and peripheral points 94 . 10 by moving the cursor 92 and specifying a point on the two-dimensional image 90 shown in FIG. The image 90 can be displayed on the display unit 12 .

When the operation mode is the second mode, the second display processing unit 41 selects specific points among the three-dimensional point group arranged in the same three-dimensional coordinate system as the captured image by the first display processing unit 40 . The peripheral points are returned to the positions before the movement by the first display processing unit 40 . Then, the second display processing unit 41 generates data of a two-dimensional image in which the picked-up image and the three-dimensional point group arranged in the same three-dimensional coordinate system are viewed from the viewpoint position. The 3D point group at this time is a 3D point group in which specific points and peripheral points among a plurality of 3D points are returned to their original positions. not returned to

The second display processing unit 41 outputs the generated two-dimensional image data to the display unit 12, thereby causing the display unit 12 to display the two-dimensional image including the captured image and the highlighted specific points and peripheral points. . The second display processing unit 41 can increase the size of the three-dimensional points according to the distance from the viewpoint position so that each three-dimensional point is displayed on the display unit 12 with the same size. .

The third display processing unit 42 causes the display unit 12 to display the distance calculated by the distance calculation unit 35 . For example, the third display processing unit 42 causes the display unit 12 to display the distance between specific points, which is the distance between two specific points calculated by the distance calculation unit 35 .

The display position of the distance between specific points is, for example, a position near the line connecting two specific points, or a position near either the start point or the end point of distance measurement, but it is easy for the user to visually recognize it. Any position is acceptable as long as it is possible, and it is not limited to such an example. The starting point of distance measurement is a first specific point described later, and the end point of distance measurement is a second specific point described later.

11 is a diagram illustrating an example of distances between specific points displayed on the display unit by the third display processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 11, the third display processing unit 42 superimposes a straight line 95 connecting two specific points 93 and a distance 96 between specific points arranged near the straight line 95 on a captured image 91. The two-dimensional image 90 thus obtained can be displayed on the display unit 12 .

In addition, the third display processing unit 42 displays the total connection line distance, which is the distance of the line formed by connecting the three or more specific points calculated by the distance calculation unit 35 in the order designated by the user. 12 is displayed. The display position of the connection line total distance is, for example, a position near a line connecting three or more specific points with a straight line in the order specified by the user, or a position near either the start point or the end point of distance measurement.

12 is a diagram illustrating an example of a total connection line distance displayed on the display unit by the third display processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 12, the third display processing unit 42 generates a line 97 formed by connecting three or more specific points with a straight line in the order specified by the user, and a connection line arranged at a position near the line 97. A two-dimensional image 90 in which the total distance 98 is superimposed on the captured image 91 can be displayed on the display unit 12 .

Returning to FIG. 4, the description of the information processing apparatus 1 is continued. When the input receiving unit 30 receives a specified point, which is a point specified by the user in the captured image 91, the specifying unit 34 of the processing unit 14 selects a point among a plurality of three-dimensional points forming the three-dimensional point cloud. A three-dimensional point corresponding to the designated point is specified from the . The specifying unit 34 causes the data obtaining unit 31 to perform a process of storing the specified three-dimensional point data as the specified point data in the specified point data storage unit 22 .

For example, in a three-dimensional coordinate system in which the captured image and the three-dimensional group are arranged by the display processing unit 33, the specifying unit 34 assumes that the viewpoint position is a viewpoint position A and the specified point is a specified point B, and the viewpoint position A is the starting point. Then, the straight line L is extended toward the specified point B, and the first three-dimensional point within the distance d from the straight line L is determined as the specified point. If a three-dimensional point within the distance d from the straight line L is not found even if the straight line L is extended to the specified point B, the specifying unit 34 passes through the specified point B and further extends the straight line L. The first three-dimensional point whose distance from is within the distance d is determined as the specific point.

The distance d is set as a parameter and can be changed by an input operation to the input section 11 . By setting the distance d to a small value, the 3D points closer to the straight line L can be narrowed down as specific points. can be specified as a specific point even if it is somewhat away from the straight line L.

Note that the identification unit 34 shortens the distance d as the number of three-dimensional points arranged in the same three-dimensional coordinate system as that of the captured image by the display processing unit 33 increases, or adjusts the distance d to the distance d within a preset range from the designated point. The distance d can be shortened as the number of dimensional points increases.

13 is a diagram illustrating an example of a specific point determination method by the specifying unit of the information processing apparatus according to the first embodiment; FIG. In the example shown in FIG. 13, for convenience of explanation, a plan view of a captured image and a three-dimensional point group arranged in the same three-dimensional coordinate system is shown. In FIG. 13, the Z-axis direction is the vertical direction, and the Y-axis positive direction is the viewing direction.

As shown in FIG. 13, the specifying unit 34 determines a three-dimensional point that is within the distance d from a straight line L starting at the viewpoint position A and passing through the designated point B and that is closest to the viewpoint position A as a specific point. do.

Further, when the specifying unit 34 newly specifies a three-dimensional point corresponding to the designated point from among the plurality of three-dimensional points forming the three-dimensional point group, the newly specified three-dimensional point has already been specified. It is determined whether or not it is within a preset range from the specific point. The preset range is, for example, a preset range from an already specified specific point, or a range within a cube of a preset size centering on the already specified specific point.

When determining that the newly specified three-dimensional point is within a preset range from the already specified specific point, the specifying unit 34 determines that the already specified specific point is the newly specified three-dimensional point. A specific point within a preset range is defined as a target specific point.

Then, the specifying unit 34 causes the data acquisition unit 31 to execute a process of deleting the data of the target specified point from the specified point data storage unit 22, and further specifies the data of the newly specified three-dimensional point as the data of the specified point. Correction processing is performed to cause the data acquisition unit 31 to execute the processing to be stored in the point data storage unit 22 . Such correction processing is processing by overwriting the specific points stored in the specific point data storage unit 22 .

In this way, when the specifying unit 34 determines that there is the above-described target specific point when specifying the three-dimensional point corresponding to the specified point, the target specific point is deleted from the specified point and replaced with a new specified point. Take corrective action. As a result, the identifying unit 34 can cancel the identification of the already identified three-dimensional point, and the user can fine-tune the position of the specific point to be distance-measured or correct the error of the specific point. and correction of specific points can be easily performed. Therefore, the information processing device 1 can improve the accuracy of distance measurement.

For example, when a click operation or a tap operation is performed, the specifying unit 34 provisionally specifies a position on the captured image 91 as a specified point, and then, when a double-click operation or a double-tap operation is performed on the provisionally specified specified point. In this case, the tentatively identified designated point can also be actually identified.

When the specified point is temporarily specified, the second display processing unit 41 determines whether the specified point temporarily specified among the specified points already specified by the specifying unit 34 is within a preset range. A particular point can be highlighted by changing at least one of the color, size, and pattern of the particular point. As a result, when the specified point is temporarily specified, the second display processing unit 41 sets in advance the specified point temporarily specified among the specified points already specified by the specifying unit 34. The user can easily grasp the specific point to be corrected within the range. Note that the method of highlighting is not limited to this example.

When the identifying unit 34 determines that there is no target specific point when a three-dimensional point corresponding to the newly specified point is specified, the specifying unit 34 stores the data of the three-dimensional point corresponding to the newly specified specified point as the specified point. A registration process for causing the data acquisition unit 31 to store the data in the specific point data storage unit 22 as the data of .

The distance calculation unit 35 shown in FIG. 4 has, as measurement modes, a specific point distance measurement mode for calculating the distance between specific points and a connection line total distance measurement mode for calculating the connection line total distance. The distance is calculated in the operation mode selected according to the input operation to the input unit 11 .

When the operation mode is the distance measurement mode between specific points, the distance calculation unit 35 calculates two points corresponding to two points specified by the user based on the data stored in the specific point data storage unit 22. It is determined whether or not the specific point has been specified by the specifying unit 34 . The two specific points are a first specific point and a second specific point. The first specific point is the specific point corresponding to the point designated by the user first, and the second specific point is the specific point corresponding to the point designated by the user next.

When determining that the first specific point and the second specific point are specified by the specifying unit 34, the distance calculating unit 35 calculates the coordinates of the first specific point and the second specific point specified by the specifying unit 34. Based on this, the distance between the first specific point and the second specific point is calculated.

When the operation mode is the connection line total distance measurement mode, the distance calculation unit 35 corresponds to three or more points specified by the user based on the data stored in the specific point data storage unit 22. It is determined whether or not three or more specific points have been specified by the specifying unit 34 .

When determining that three or more specific points have been identified by the identifying unit 34, the distance calculating unit 35 determines whether or not the distance calculation condition is satisfied. The distance calculation condition is that a distance calculation button (not shown), which is a GUI (Graphical User Interface) button displayed on the display unit 12 by the display processing unit 33, is operated by the user's input operation to the input unit 11. be.

Note that the distance calculation conditions are not limited to the examples described above. For example, the distance calculation condition may be a condition that the number of specific points specified by the specifying unit 34 reaches a preset number, or a condition that the specified points last specified among three or more specific points by the specifying unit 34. The condition may be such that there is no next specific point after the point is specified until a preset time elapses.

When the operation mode is the connection line total distance measurement mode, the distance calculation unit 35 connects the three or more specific points with straight lines in the order in which the corresponding designated points are designated by the user, and the specific points connected by the straight lines are connected. The distance between the points is calculated, and the sum of the distances between the specific points is calculated as the total connecting line distance.

For example, the specified points are the first specified point, the second specified point, and the third specified point, and the order in which the corresponding specified points are specified is the first specified point and the second specified point. , and the third particular point. In this case, the distance calculator 35 calculates the distance between the first specific point and the second specific point and the distance between the second specific point and the third specific point, The total distance of the connection line is calculated by summing the two distances of .

In this way, the distance calculation unit 35 can connect three or more specific points with a straight line in the specified order, and calculate the total distance between the specific points connected with the straight line as the total connection line distance. Distance measurements can be made.

When the input reception unit 30 receives an output request from the user, the data output unit 36 shown in FIG. A data file containing data indicating the distance is output as calculation result data. The distance calculated by the distance calculator 35 is the above-described distance between specific points or total distance of connection lines.

The calculation result data output from the data output unit 36 is, for example, a csv format data file, but is not limited to such an example. For example, the calculation result data may be text data in a format other than the csv format, such as captured image data obtained by capturing an image displayed on the display unit 12 as shown in FIG. 11 or FIG. There may be.

The data output unit 36 acquires the data stored in the storage unit 13 via the data acquisition unit 31, and performs data shaping to format the acquired data into data such as csv format. For example, by connecting the coordinates of two specific points and the distance between the specific points separated by commas, or by connecting the coordinates of three or more specific points and the total distance of connecting lines separated by commas, data shaping. Note that data shaping is not limited to csv format data shaping as long as the coordinate information and distance information can be shaped into a format that can be used by the user in a useful manner.

For example, the data output unit 36 outputs calculation result data by transmitting the calculation result data to an external device via the communication unit 10 and the network 3, or outputs the calculation result data to an external memory via a memory interface (not shown). data can be output. The external device is, for example, a terminal device or a printer, and the external memory is a USB (Universal Serial Bus) memory or memory card.

Next, processing by the processing unit 14 of the information processing device 1 will be described using a flowchart. 14 is a flowchart illustrating an example of processing by a processing unit of the information processing apparatus according to the first embodiment; FIG.

As shown in FIG. 14, the processing unit 14 of the information processing device 1 determines whether or not there is a display request (step S10). If the processing unit 14 determines that there is a display request (step S10: Yes), it performs captured image display processing (step S11). The picked-up image display processing of step S11 is the processing of steps S20 to S26 shown in FIG. 15, and will be described in detail later.

When the processing of step S11 is completed, or when it is determined that there is no display request (step S10: No), the processing unit 14 determines whether the measurement mode is the distance measurement mode between specific points (step S12). ).

When the processing unit 14 determines that the measurement mode is the distance measurement mode between specific points (step S12: Yes), it performs distance measurement processing between specific points (step S13). The distance measurement process between specific points in step S13 is the process of steps S30 to S40 shown in FIG. 16, and will be described in detail later.

When the processing of step S13 is completed, or when it is determined that the measurement mode is not the distance measurement mode between specific points (step S12: No), the processing unit 14 determines whether the measurement mode is the connection line total distance measurement mode. (step S14).

When the processing unit 14 determines that the measurement mode is the connection line total distance measurement mode (step S14: Yes), the processing unit 14 performs connection line total distance measurement processing (step S15). The connection line total distance measurement process of step S15 is the process of steps S70 to S77 shown in FIG. 19, and will be described in detail later.

When the processing of step S15 is completed, or when it is determined that the measurement mode is not the connection line total distance measurement mode (step S14: No), the processing unit 14 determines whether or not there is an output request (step S16). ).

When determining that there is an output request (step S16: Yes), the processing unit 14 performs output processing (step S17). In the output process of step S17, the processing unit 14 uses, for example, a data file containing data indicating the coordinates of the specific point determined by the identifying unit 34 and data indicating the distance calculated by the distance calculating unit 35 as calculation result data. Output.

When the process of step S17 is finished, or when it is determined that there is no output request (step S16: No), the processing unit 14 determines whether or not it is time to end the operation (step S18). For example, when the processing unit 14 determines that the power supply (not shown) of the information processing device 1 is turned off or determines that an operation end operation has been performed on the input unit 11, the processing unit 14 determines that it is time to end the operation. do.

If the processing unit 14 determines that it is not the time to end the operation (step S18: No), the process proceeds to step S10. 14 ends.

15 is a flowchart illustrating an example of captured image display processing by the processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 15, the processing unit 14 acquires captured image data corresponding to the display request from the captured image data storage unit 20 of the storage unit 13 (step S20), and stores the three-dimensional point cloud data storage unit of the storage unit 13. 21 acquires the three-dimensional point group data corresponding to the display request (step S21).

Then, the processing unit 14 arranges the captured image represented by the captured image data acquired in step S20 and the three-dimensional point group represented by the three-dimensional point cloud data acquired in step S21 in the same three-dimensional coordinate system ( step S22).

Next, the processing unit 14 determines whether or not the operation mode is the first mode (step S23). When the processing unit 14 determines that the operation mode is the first mode (step S23: Yes), the 3D point group arranged in the same 3D coordinate system as the captured image is colorless and made transparent. (Step S24).

In addition, when the processing unit 14 determines that the operation mode is not the first mode but the second mode (step S23: No), the processing unit 14 stores the three-dimensional point group arranged in the same three-dimensional coordinate system as the captured image. The position from the viewpoint position is moved farther than the captured image (step S25).

When the process of step S24 or the process of step S25 ends, the processing unit 14 generates data of a two-dimensional image, and outputs the data of the generated two-dimensional image to the display unit 12 (step S26 ), the process shown in FIG. 15 is terminated. The two-dimensional image data generated in step S26 is two-dimensional image data obtained by viewing the picked-up image and the three-dimensional point group arranged in the same three-dimensional coordinate system from the viewpoint position.

16 is a flowchart illustrating an example of distance measurement processing between specific points by the processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 16, the processing unit 14 determines whether or not there is a user's designation operation (step S30).

When the processing unit 14 determines that there is a user's designation operation (step S30: Yes), the processing unit 14 performs specific point determination processing (step S31). The process of step S31 is the process of steps S50 to S52 shown in FIG. 17, and will be described in detail later.

Next, the processing unit 14 determines whether or not there is a first specific point (step S32). If the processing unit 14 determines that there is a first specific point (step S32: Yes), it determines whether the specific point determined in step S31 is within a preset range from the first specific point. (step S33).

When the processing unit 14 determines that there is no first specific point (step S32: No), or determines that the specific point determined in step S31 is within a preset range from the first specific point (Step S33: Yes), the new specific point determined in step S31 is used as the first specific point, and the first specific point is corrected (step S34).

Further, when the processing unit 14 determines that the specific point determined in step S31 is not within the preset range from the first specific point (step S33: No), the new specific point determined in step S31 is A second specific point is determined (step S35).

The processing unit 14 performs three-dimensional point group processing when the process of step S34 is finished or when the process of step S35 is finished (step S36). The three-dimensional point group processing of step S36 is the processing of steps S60 to S65 shown in FIG. 18, and will be described in detail later.

Next, the processing unit 14 generates a two-dimensional image in which the captured image and the three-dimensional point group arranged in the same three-dimensional coordinate system are viewed from the viewpoint position, and in which the specific point and the peripheral points are highlighted. Data is generated, and the generated two-dimensional image data is output to the display unit 12 (step S37).

Next, the processing unit 14 determines whether or not there is a second specific point (step S38). If the processing unit 14 determines that there is a second specific point (step S38: Yes), it calculates the distance between the first specific point and the second specific point (step S39). Then, the processing unit 14 causes the display unit 12 to display the distance calculated in step S39 (step S40).

When the process of step S40 ends, the processing unit 14 determines that there is no designation operation by the user (step S30: No), or determines that there is no second specific point (step S38: No). , the processing shown in FIG. 16 ends.

17 is a flowchart illustrating an example of specific point determination processing by the processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 17, the processing unit 14 determines whether or not the operation mode is the first mode (step S50).

When the processing unit 14 determines that the operation mode is not the first mode but the second mode (step S50: No), the position of the three-dimensional point group is temporarily changed to the position before being moved in step S25. return (step S51).

When the process of step S51 is completed, or when the operation mode is determined to be the first mode (step S50: Yes), the processing unit 14 specifies a three-dimensional point corresponding to the specified point (step S52). , the process shown in FIG. 17 ends. Although not shown, when the processing unit 14 temporarily returns the position of the three-dimensional point cloud to the position before the movement in step S25 after step S52 is completed, the position of the three-dimensional point cloud is changed to It is returned to the position after it was moved in step S25.

18 is a flowchart illustrating an example of three-dimensional point cloud processing by the processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 18, the processing unit 14 determines whether or not the operation mode is the first mode (step S60).

When the processing unit 14 determines that the operation mode is the first mode (step S60: Yes), the processing unit 14 identifies three-dimensional points within a range in which the distance from the specific point is set in advance as peripheral points (step S61).

Further, when the processing unit 14 determines that the operation mode is not the first mode but the second mode (step S60: No), the position of the three-dimensional point group is returned to the position before being moved in step S25. (Step S62). Then, the processing unit 14 identifies, as peripheral points, three-dimensional points whose distance from the specific point is within a preset range (step S63). The point is returned to the post-movement position to which it was moved in step S25 (step S64).

When the process of step S61 or the process of step S64 ends, the processing unit 14 colors the specific point and the surrounding points (step S65), and ends the process shown in FIG.

19 is a flowchart illustrating an example of connection line total distance measurement processing by the processing unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 19, the processing unit 14 determines whether or not there is a user's designation operation (step S70).

When the processing unit 14 determines that there is a user's designation operation (step S70: Yes), the processing unit 14 performs specific point determination processing (step S71). The processing of step S71 is the same as the processing of steps S50 to S52 described above. Then, the processing unit 14 performs three-dimensional point group processing (step S72). The processing of step S72 is the same as the processing of steps S60 to S65 described above.

Next, the processing unit 14 generates a two-dimensional image in which the captured image and the three-dimensional point group arranged in the same three-dimensional coordinate system are viewed from the viewpoint position, and in which the specific point and the peripheral points are highlighted. Data is generated, and the generated two-dimensional image data is output to the display unit 12 (step S73).

Next, the processing unit 14 determines whether or not there are three or more specific points (step S74). When determining that there are three or more specific points (step S74: Yes), the processing unit 14 determines whether or not the distance calculation condition is satisfied (step S75).

If the processing unit 14 determines that the distance calculation condition is satisfied (step S75: Yes), the processing unit 14 calculates the distance of a line formed by connecting the specific points with straight lines in the designated order (step S76). Then, the processing unit 14 causes the display unit 12 to display the distance calculated in step S76 (step S77).

When the process of step S77 is completed, the processing unit 14 determines that there is no designation operation by the user (step S70: No), or determines that there are no three or more specific points (step S74: No). , or when it is determined that the distance calculation condition is not satisfied (step S75: No), the process shown in FIG. 19 is terminated.

20 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to the first embodiment; FIG. As shown in FIG. 20, the information processing apparatus 1 includes a computer having a processor 101, a memory 102, a communication device 103, an input device 104, a display device 105, and a bus .

Processor 101 , memory 102 , communication device 103 , input device 104 , and display device 105 can send and receive information from each other, eg, via bus 106 . Storage unit 13 is implemented by memory 102 . The communication unit 10 is implemented by the communication device 103 . The input unit 11 is implemented by the input device 104 . The display unit 12 is implemented by the display device 105 .

The processor 101 reads an information processing program from a recording medium set in a recording medium drive and installs the read information processing program in the memory 102 . The recording medium drive is, for example, a CD (Compact Disc)-ROM drive, a DVD (Digital Versatile Disc)-ROM drive, or a USB drive, and the recording medium is, for example, a CD-ROM, a DVD-ROM, or a non-volatile Such as semiconductor memory.

The processor 101 executes the functions of the processing unit 14 by reading and executing programs stored in the memory 102 . The processor 101 is an example of a processing circuit, for example, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration).

The memory 102 includes one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Registered Trademark) (Electrically Erasable Programmable Read Only Memory). include. The information processing apparatus 1 may include integrated circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array).

The information processing apparatus 1 may be configured by a client device, may be configured by a server device, or may be configured by a client device and a server device. When the information processing apparatus 1 is composed of two or more devices, each of the two or more devices has the hardware configuration shown in FIG. 20, for example. Note that communication between two or more devices is performed via the communication device 103 . Further, the information processing device 1 may be composed of two or more server devices. For example, the information processing device 1 may be configured with a processing server and a data server.

As described above, the information processing apparatus 1 according to the first embodiment includes the data acquisition unit 31, the first display processing unit 40, the specifying unit 34, the second display processing unit 41, the distance calculation unit 35, and the and The data acquisition unit 31 acquires captured image data obtained by imaging a structure and three-dimensional point group data representing the structure with a plurality of three-dimensional points in a three-dimensional coordinate system. The first display processing unit 40 causes the display unit 12 to display the captured image based on the data of the captured image. The specifying unit 34 specifies a three-dimensional point corresponding to the point specified by the user in the captured image displayed on the display unit 12 from among a plurality of three-dimensional points. The second display processing unit 41 causes the display unit 12 to display the three-dimensional point specified by the specifying unit 34 in a state in which it is superimposed on the captured image. When the specifying unit 34 specifies the three-dimensional points corresponding to the two points specified by the user, the distance calculating unit 35 calculates the distance between the two three-dimensional points corresponding to the two points specified by the specifying unit 34. Calculate the distance of As a result, the information processing apparatus 1 calculates the distance between the points specified by the user using the three-dimensional points corresponding to the points specified by the user in the captured image, so that the distance can be accurately measured. can be done.

The second display processing unit 41 causes the display unit 12 to display the three-dimensional point specified by the specifying unit 34 in a state of being superimposed on the captured image while emphasizing it so as to be distinguishable from the captured image. As a result, the information processing apparatus 1 can allow the user to easily grasp the three-dimensional point corresponding to the point specified by the user, so that distance measurement can be performed with high accuracy.

Also, the captured image is an omnidirectional image, a hemispherical image, or a panoramic image. The specifying unit 34 selects the specified point from among the plurality of three-dimensional points based on the distance of each of the plurality of three-dimensional points from a straight line passing through the viewpoint position of the captured image and the specified point in the three-dimensional coordinate system. Identify the 3D point that corresponds to the point. As a result, the information processing device 1 can accurately measure the distance.

Further, the second display processing unit 41 makes the color of the three-dimensional point group transparent, and then colors the three-dimensional points specified by the specifying unit 34 and causes the display unit 12 to display them. As a result, the information processing apparatus 1 can prevent the captured image from becoming difficult to visually recognize due to the three-dimensional point cloud, so that distance measurement can be performed with high accuracy.

Further, the second display processing unit 41 moves the position of the captured image in the three-dimensional coordinate system to a position farther from the viewpoint position than the three-dimensional point cloud, and then moves the position of the three-dimensional point specified by the specifying unit 34. is returned to the state before the movement, the three-dimensional point specified by the specifying unit 34 is displayed on the display unit 12 . Thereby, the information processing apparatus 1 can reduce the processing load without performing the process of changing the color information of the three-dimensional point group.

The information processing apparatus 1 also includes a thinning unit 32 that thins out some of the three-dimensional points from among the plurality of three-dimensional points based on the distance from the viewpoint position or the distance between the three-dimensional points. The specifying unit 34 specifies a 3D point corresponding to the point specified by the user from among a plurality of 3D points from which some of the 3D points are thinned out by the thinning unit 32 . Thereby, the information processing device 1 can perform complicated distance measurement.

Further, when the 3D point newly specified as the 3D point corresponding to the point specified by the user is within a preset range from the already specified 3D point, the specifying unit 34 determines the already specified 3D point. Unidentify dimension points. As a result, the information processing device 1 can improve the accuracy of distance measurement.

Further, when the specifying unit 34 specifies the coordinates of the three-dimensional points corresponding to the three or more points specified by the user, the distance calculating unit 35 calculates the coordinates of the three or more points specified by the specifying unit 34. The distance of a line formed by connecting the corresponding three-dimensional points with straight lines in the order specified by the user is calculated. Thereby, the user can finely adjust the position of the specific point to be distance-measured or correct the error of the specific point, and can easily correct the specific point. Therefore, the information processing device 1 can improve the accuracy of distance measurement.

The information processing apparatus 1 also includes a data output unit 36 that outputs a data file including data representing the coordinates of the three-dimensional point specified by the specifying unit 34 and data representing the distance calculated by the distance calculation unit 35 . Thereby, the user can use the data file output from the information processing apparatus 1 to save or analyze the distance calculation result.

The configuration shown in the above embodiment is an example, and can be combined with another known technique, and part of the configuration can be omitted or changed without departing from the scope of the invention. It is possible.

1 information processing device, 2 measurement vehicle, 3 network, 4 road, 5 tunnel, 6 guardrail, 10, 75 communication unit, 11 input unit, 12 display unit, 13 storage unit, 14, 74 processing unit, 20 captured image data storage 21 three-dimensional point cloud data storage unit 22 specific point data storage unit 30 input reception unit 31 data acquisition unit 32 thinning unit 33 display processing unit 34 identification unit 35 distance calculation unit 36 data output unit , 40 first display processing unit, 41 second display processing unit, 42 third display processing unit, 60 vehicle body, 70 measurement device, 71 imaging unit, 72 three-dimensional measurement unit, 73 position detection unit, 80, 81, 82, 83 three-dimensional point group, 90 two-dimensional image, 91 captured image, 92 cursor, 93 specific point, 94 peripheral point, 95, L straight line, 96 distance between specific points, 97 line, 98 connection line total distance, 100 measurement processing system, 101 processor, 102 memory, 103 communication device, 104 input device, 105 display device, 106 bus.

Claims (11)

a data acquisition unit that acquires captured image data obtained by imaging a structure and three-dimensional point group data representing the structure with a plurality of three-dimensional points in a three-dimensional coordinate system;
a first display processing unit for displaying the captured image on a display unit based on the data of the captured image;
a specifying unit that specifies, from among the plurality of three-dimensional points, a three-dimensional point corresponding to a point specified by a user in the captured image displayed on the display unit;
a second display processing unit that causes the display unit to display the three-dimensional point specified by the specifying unit in a state of being superimposed on the captured image;
When the specifying unit specifies three-dimensional points corresponding to the two points specified by the user, calculating the distance between the two three-dimensional points corresponding to the two points specified by the specifying unit. an information processing apparatus comprising: a distance calculation unit that The second display processing unit
2. The information according to claim 1, wherein the three-dimensional point specified by the specifying unit is displayed on the display unit in a state of being emphasized so as to be distinguishable from the captured image and superimposed on the captured image. processing equipment. The captured image is
Spherical image, hemispherical image, or panoramic image,
The identification unit
the designated point out of the plurality of three-dimensional points based on the distance of each of the plurality of three-dimensional points from a straight line passing through the viewpoint position of the captured image and the designated point in a three-dimensional coordinate system; 3. The information processing apparatus according to claim 1, wherein a three-dimensional point corresponding to the point is specified. The second display processing unit
4. The information processing apparatus according to claim 3, wherein after making the color of the three-dimensional point group transparent, the three-dimensional points specified by the specifying unit are colored and displayed on the display unit. The second display processing unit
After moving the position of the three-dimensional point group in the three-dimensional coordinate system to a position farther from the viewpoint position than the captured image , the position of the three-dimensional point specified by the specifying unit is returned to the state before the movement. 4. The information processing apparatus according to claim 3, wherein the three-dimensional point specified by the specifying unit is displayed on the display unit by returning. a thinning unit that thins out some of the three-dimensional points from among the plurality of three-dimensional points based on the distance from the viewpoint position or the distance between the three-dimensional points;
The identification unit
from claim 3, wherein a three-dimensional point corresponding to the point specified by the user is specified from among a plurality of three-dimensional points from which the partial three-dimensional points are thinned out by the thinning unit. 6. The information processing apparatus according to any one of 5. The identification unit
If the newly specified 3D point as the 3D point corresponding to the point specified by the user is within a preset range from the already specified 3D point, the already specified 3D point is canceled. The information processing apparatus according to any one of claims 1 to 5, characterized in that: The distance calculation unit
When coordinates of three-dimensional points corresponding to three or more points specified by the user are specified by the specifying unit, three-dimensional points corresponding to the three or more points specified by the specifying unit are specified. 8. The information processing apparatus according to any one of claims 1 to 7, wherein a distance of a line formed by connecting straight lines in the order designated by the user is calculated. 2. A data output unit for outputting a data file including data indicating the coordinates of the three-dimensional point specified by the specifying unit and data indicating the distance calculated by the distance calculating unit. 9. The information processing device according to any one of 8 to 8. a data acquisition step of acquiring captured image data obtained by imaging a structure and three-dimensional point group data representing the structure with a plurality of three-dimensional points in a three-dimensional coordinate system;
a first display processing step of displaying the captured image on a display unit based on the data of the captured image;
a specifying step of specifying, from among the plurality of three-dimensional points, a three-dimensional point corresponding to a point specified by a user in the captured image displayed on the display unit;
a second display processing step of causing the display unit to display the three-dimensional point specified by the specifying step in a state of being superimposed on the captured image;
When three-dimensional points corresponding to the two points specified by the user are specified by the specifying step, calculating the distance between the two three-dimensional points corresponding to the two points specified by the specifying step. and a distance calculation step for calculating the distance. a data acquisition step of acquiring captured image data obtained by imaging a structure and three-dimensional point group data representing the structure with a plurality of three-dimensional points in a three-dimensional coordinate system;
a first display processing step of displaying the captured image on a display unit based on the data of the captured image;
a specifying step of specifying, from among the plurality of three-dimensional points, a three-dimensional point corresponding to a point specified by a user in the captured image displayed on the display unit;
a second display processing step of causing the display unit to display the three-dimensional point specified by the specifying step in a state of being superimposed on the captured image;
When three-dimensional points corresponding to the two points specified by the user are specified by the specifying step, calculating the distance between the two three-dimensional points corresponding to the two points specified by the specifying step. and a step of calculating a distance to be calculated.

Images