JP2019095876A - Three-dimensional point group display device, three-dimensional point group display system, method for displaying three-dimensional point group, three-dimensional point group display program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a 3D point cloud display device for surely performing matching between 3D point cloud data and image data in matching 3D point cloud data and image data. SOLUTION: A three-dimensional point cloud display device 1 has a data acquisition unit 2 for acquiring image data 22 obtained by photographing a monitored object with a camera 21, and a range of the monitored objects captured by the camera 21. User input to the display processing unit 3 that performs processing to display the 3D point cloud data 31 obtained by measuring including at least a part on the screen 4a and the 3D point cloud data 31 displayed on the screen 4b. A calculation range setting unit 6 for setting a calculation range is provided. [Selection diagram] Fig. 1

Description

  The present invention relates to a three-dimensional point cloud display device, a three-dimensional point cloud display system, a three-dimensional point cloud display, and a method of semi-automatically matching and displaying an image captured by a camera on a display screen of the three-dimensional point cloud display device The present invention relates to a method, a three-dimensional point cloud display program, and a recording medium.

  When wide area infrastructure monitoring such as rivers is performed, three-dimensional point cloud data of terrain or structure obtained by laser scanning may be displayed on a three-dimensional point cloud display device. Then, by superimposing and displaying image data such as a moving image or a still image captured by a camera on the screen of the three-dimensional point cloud display device, it is possible to check the real time state by the image and the surrounding environment. It becomes possible to grasp exactly. In order to realize this function, it is necessary to match three-dimensional point cloud data with image data.

  Conventionally, in order to obtain matching between 3D point group data and image data, 3D coordinate data of the peripheral position of the mobile terminal acquired from a camera or a camera for capturing a real image of the peripheral is obtained from the server, Feature information is extracted for each of the acquired three-dimensional coordinate data and an image obtained by ortho-correcting the current real image captured by the camera, and the extracted feature information is matched (Patent Document 1).

JP, 2016-170060, A (paragraph [0007])

  Patent Document 1 assumes that facility information is superimposed and displayed on a real space image using augmented reality in a mobile terminal, and the user of the mobile terminal is often close to the facility. Therefore, after acquiring the current approximate position from the GPS, it is configured to transmit a request for three-dimensional map information including a region with a radius of 20 meters centering on the approximate position to the three-dimensional map information transmission unit of the server .

  However, when performing wide-area infrastructure monitoring, it is not limited to the fact that there is an object of an image captured by a camera around the camera, and three-dimensional surroundings based on GPS data of the current approximate position as in Patent Document 1. Even if it tried to match map information and a real image, there was a possibility that it could not match.

  The present invention has been made to solve the above-described problems, and in performing matching between three-dimensional point cloud data and image data, matching between three-dimensional point cloud data and image data is reliably performed. The object is to obtain a three-dimensional point cloud display device.

  A three-dimensional point cloud display device according to the present invention includes a data acquisition unit that acquires image data obtained by photographing a monitoring target with a camera, and at least a part of a range photographed by the camera among monitoring targets. The display processing unit performs processing to display the 3D point cloud data obtained by measurement on the screen, and the operation range setting to set the operation range by user input to the 3D point cloud data displayed on the screen A feature information extraction unit for extracting feature information of selected data selected from 3D point cloud data based on the settings of the calculation range setting unit and the image data obtained by the data obtaining unit; 3 based on the matching result of the matching unit that matches the feature information of the three-dimensional point cloud data and the feature information of the image data based on the result of the extraction by And a, a superimposed display processing unit that superimposes the display image data on the Mototengun data.

  A three-dimensional point cloud display device according to the present invention includes a data acquisition unit that acquires image data obtained by photographing a monitoring target with a camera, and at least a part of a range photographed by the camera among monitoring targets. For the display processing unit that displays the 3D point cloud data obtained by measurement on the screen and the 3D point cloud data displayed on the screen, setting of the operation range based on the attribute information included in the image data Operation range setting unit for performing the process, and a feature information extraction unit for extracting each feature information of the selected data selected from the three-dimensional point cloud data based on the settings of the operation range setting unit and the image data acquired by the data acquisition unit Based on the result of extraction by the feature information extraction unit, the matching unit matches the feature information of the three-dimensional point cloud data and the feature information of the image data, and the matching result of the matching unit Hazuki, and a, a superimposed display processing unit that superimposes the display image data on the 3D point group data in the screen.

  Further, in the 3D point cloud display system according to the present invention, a camera for shooting a monitoring target, a data acquisition unit for obtaining image data obtained by shooting with the camera, and a camera among the monitoring targets. A display processing unit that performs processing for displaying on a screen three-dimensional point cloud data obtained by measurement including at least a part of the range, and calculation range by user input for three-dimensional point cloud data displayed on the screen Feature information extraction for extracting each feature information of the selected data selected from the 3D point cloud data based on the setting of the calculation range setting unit for setting the data and the calculation range setting unit and the data acquisition unit A matching unit that matches feature information of the three-dimensional point cloud data with feature information of the image data based on the result of the extraction by the unit and the feature information extraction unit; The basis of the results, and a, a superimposed display processing unit that superimposes the display image data on the 3D point group data in the screen.

  In the three-dimensional point cloud display method according to the present invention, a data acquisition step of acquiring image data obtained by photographing a monitoring target with a camera, and at least a part of a range photographed by the camera among monitoring targets Display processing step to display 3D point group data obtained by including and obtained on the screen, and calculation range to set calculation range by user input to 3D point group data displayed on the screen A setting step, a feature information extraction processing step of extracting feature information of each of selected data selected from 3D point cloud data based on the setting in the calculation range setting step and the image data acquired by the data acquisition step; Matching that matches feature information of 3D point cloud data and feature information of image data based on the result of extraction by the information extraction processing step And management step, on the basis of the matching result of the matching processing step, and a superimposed display process scan to superimpose image data on the 3D point group data in the screen.

  Further, a three-dimensional point cloud display program according to the present invention includes a data acquisition step of acquiring image data obtained by photographing a monitoring target with a camera, and at least a part of a range photographed by the camera among monitoring targets. And a display processing step for displaying the 3D point group data obtained by measurement on the screen, and an operation for setting an operation range by user input to the 3D point group data displayed on the screen A range setting step, a feature information extraction processing step of extracting feature information of each of selected data selected from 3D point cloud data based on the settings in the calculation range setting step and image data acquired by the data acquisition step; Based on the result of extraction by the feature information extraction processing step, the feature information of the three-dimensional point cloud data and the feature information of the image data are matched. And quenching process step, on the basis of the result of the matching matching processing steps to execute the superimposing display processing step of displayed superimposed image data on the 3D point group data in the screen, to the computer.

  The recording medium according to the present invention has the above-described three-dimensional point cloud display program recorded therein.

  Further, according to the three-dimensional point cloud display device of the present invention, it becomes possible to reliably perform matching between three-dimensional point cloud data and image data.

  Further, according to the three-dimensional point cloud display system according to the present invention, it becomes possible to reliably perform matching between three-dimensional point group data and image data.

  Further, according to the three-dimensional point group display method according to the present invention, it becomes possible to reliably perform matching between three-dimensional point group data and image data.

  Further, according to the three-dimensional point group display program according to the present invention, it becomes possible to reliably perform matching between three-dimensional point group data and image data.

  Furthermore, according to the recording medium of the present invention, the above program is recorded, and it becomes possible to reliably perform matching between 3D point cloud data and image data.

FIG. 1 is a block diagram showing a configuration of a three-dimensional point cloud display system according to Embodiment 1. FIG. 1 is a diagram illustrating an example of a hardware configuration of a three-dimensional point cloud display device according to a first embodiment. FIG. 6 is a diagram showing an example of the arrangement of cameras and laser scanners in the case of assuming a river as a monitoring target according to the first embodiment. FIG. 5 is a flowchart showing the operation of the three-dimensional point cloud display device according to the first embodiment. FIG. 6 is a view showing the contents of a screen of a display for displaying image data and three-dimensional point group data according to the first embodiment. FIG. 7 is a diagram showing a configuration of a message input screen that functions as an operation range setting unit according to Embodiment 1. It is a figure for demonstrating the content of the characteristic information extraction operation | work which concerns on Embodiment 1, and a matching operation | work. FIG. 7 is a diagram showing that the superposition display processing unit according to Embodiment 1 causes the display device to superimpose and display image data on three-dimensional point group data. FIG. 17 is a block diagram in the case of setting an operation range using a pointing device as an input unit according to a second embodiment. 15 is a setting screen for setting a calculation range using the pointing device according to the second embodiment. FIG. 16 is a block diagram showing a configuration of an arithmetic unit according to Embodiment 3. FIG. 14 is a flowchart showing the operation of the three-dimensional point cloud display device according to the third embodiment. FIG. 18 is a view showing a modification according to the third embodiment. FIG. 18 is a block diagram showing a configuration of a three-dimensional point cloud display system according to a fourth embodiment. FIG. 18 is a flowchart showing the operation of the three-dimensional point cloud display device according to the fourth embodiment. FIG. 18 is a diagram showing the contents of a screen of a display for displaying three-dimensional point group data according to a fourth embodiment. FIG. 14 is a flowchart showing the operation of a modification according to the fourth embodiment.

  Hereinafter, a three-dimensional point cloud display device, a three-dimensional point cloud display system device, and a three-dimensional point cloud display method according to an embodiment of the present invention will be described in detail based on the drawings. The present invention is not limited by the embodiments, and the embodiments can be combined or appropriately changed. The same reference numerals are given to the same configuration in each embodiment, and the description thereof will be omitted.

Embodiment 1
Embodiment 1 of the present invention will be described with reference to FIGS. 1 to 8. The outline of the first embodiment is that a calculation range setting unit is provided which receives setting of a calculation range for performing a matching operation on three-dimensional point group data displayed on a screen.

  FIG. 1 is a block diagram showing the configuration of a three-dimensional point cloud display system 40 according to the first embodiment. The three-dimensional point cloud display system 40 includes a three-dimensional point cloud display device 1, a camera unit 20, and a storage unit 30. Although the storage unit is disclosed as a configuration different from that of the three-dimensional point cloud display device 1, if the three-dimensional point cloud display device 1 itself has a sufficient storage area, the three-dimensional point cloud display device 1 may A storage unit 30 may be provided.

  The three-dimensional point cloud display device 1 includes a program P (to be described in detail later) as a three-dimensional point cloud display program for realizing various functions of the three-dimensional point cloud display device 1, and a network (not shown) from the camera unit 20. The data acquisition unit 2 acquires the image data 22 transmitted via the image and the three-dimensional point cloud data 31 called from the storage unit 30, the image data 22 acquired by the data acquisition unit 2, the three-dimensional point cloud data 31, and A display processing unit 3 performing processing for fetching data relating to setting of the calculation range by the calculation range setting unit 6 from the data acquisition unit 2 and displaying it on a display 4 such as a display, and external connected devices such as a mouse, track ball And an input receiving unit 5 for receiving an input from the input unit 7.

  The three-dimensional point cloud display device 1 also includes an operation unit 8 that performs various operations. The arithmetic unit 8 corresponds to the viewpoint corresponding unit 9 that associates the viewpoint from the camera 21 of the image data 22 acquired by the data acquisition unit 2 with the viewpoint of the three-dimensional point cloud data 31, the image data 22, and the three-dimensional point cloud data 31. Including a feature information extraction unit 10 that takes in and extracts each feature information related to edge information and the like, and a matching unit 11 that performs matching on the image data 22 and the three-dimensional point cloud data 31 based on each extracted feature information There is.

  The data acquisition unit 2 communicates with the storage unit 30 when receiving an input related to the setting by the input unit 7 on the data range acquisition screen (not shown) displayed on the display 4, the display 4 The data acquisition is performed while narrowing the data range of the three-dimensional point cloud data 31 to be displayed. For example, it is assumed that the monitoring target described in FIG. 3 is selected, the acquisition time of the three-dimensional point cloud data 31 is selected, and the like. Further, the data acquisition unit 2 includes an operation range setting unit 6 that sets an operation range in the operation unit 8.

  In addition, the display processing unit 3 has a superimposed display processing unit 12 that aligns the image data 22 on the three-dimensional point cloud data 31 and displays the image data 22 on the basis of the matching result by the matching unit 11.

  The timing of processing of each of the feature extraction unit 10, the matching unit 11, and the superimposed display processing unit 12 will be described. The feature extraction unit 10 performs the feature extraction operation using the image data 22 at a certain point in time and the matching operation by the matching unit 11 using the extracted feature information, at least the three-dimensional point cloud display system 40 When constructing, it is sufficient to carry out as initial setting. At the stage of operating the three-dimensional point cloud display system 40, the superimposed display processing unit 12 uses the matching result of the matching unit 11 performed as the initial setting to generate the image data 22 acquired each time from the data acquisition unit 2. It may be superimposed on the position corresponding to each point of the dimensional point cloud display data 31.

  The camera unit 20 transmits data to a data acquisition unit 2 of the three-dimensional point cloud display device 1 with a camera 21 for capturing a monitoring target such as an environment such as a river or equipment such as a factory and image data 22 captured by the camera 21. And a transmitting unit 23.

  The image data 22 captured by the camera 21 may be a still image or a moving image. However, in the feature extraction unit 10 and the matching operation by the matching unit 11 using the extracted feature information, when the image data 22 is a moving image, an image captured during a predetermined period It is preferable to use the average value of the moving image data as the data 22, or to use one image at any time in a predetermined period. That is, when matching is performed based on a moving image that changes in real time, the amount of calculation is required when performing the task of feature extraction to the feature extraction unit 10 and the matching operation by the matching unit 11 using the extracted feature information. There is a possibility of

  The storage unit 30 stores, as the three-dimensional point cloud data 31, a plurality of points acquired by scanning the monitoring target and its surroundings in advance by the laser scanner 24 (see FIG. 3) or the like. The three-dimensional point cloud data 31 is, for example, a point in the space of the world coordinate system configured by the X axis, the Y axis, and the Z axis. The three-dimensional point cloud data 31 may also have absolute coordinates by using GPS data (not shown) or the like.

  FIG. 2 is a diagram showing an example of the hardware configuration of the three-dimensional point cloud display device 1. As shown in FIG. 2, the three-dimensional point cloud display device 1 includes a processor 100, a memory 101, and an interface circuit 102. The processor 100, the memory 101, and the interface circuit 102 are communicably connected by a system bus 103.

  The processor 100 implements the function of the display processing unit 3 in the three-dimensional point cloud display device 1a as a computer by calling a program P stored in the memory 101 as a recording medium, or the function of the superimposed display processing unit 12 , The function of the feature information extraction unit 10 in the calculation unit 8, the function of the matching unit 11, or the other data processing function.

  Furthermore, when the user performs an input in the input unit 7 connected via the interface circuit 102, the processor 100 receives the input by the input reception unit 5 in the memory 101, thereby the three-dimensional point cloud data 31 of the storage unit 30. It is possible to process (extract, process, convert, etc.) data of

  The processor 100 is an example of a processing circuit, and includes one or more of a central processing unit (CPU), a digital signal processor (DSP), and a large scale integration (LSI). The memory 101 includes a random access memory (RAM), a read only memory (ROM), a flash memory, a non-volatile or volatile semiconductor memory such as an EPROM (enable program read only memory), a magnetic disk, a flexible memory, an optical disk, It includes one or more of a compact disc and a DVD (Digital Versatile Disc). Further, although the configuration of this hardware has been described for the first embodiment, each embodiment of the present invention can adopt the same hardware configuration.

  FIG. 3 is a view showing an example of the arrangement of the camera 21 and the laser scanner 24 when the river 25 is assumed to be a monitoring target. The camera 21 and the laser scanner 24 are installed on a platform 26 overlooking the river 25. The plateau 26 is located farther from the river shore 125 than the river 25.

  By photographing the river 25 with the camera 21, it becomes possible to monitor, in real time, changes in water level, possibility of river flooding and the like. In addition to the river 25 to be monitored, the camera 21 may capture images of surrounding environments and structures together. For example, by setting the shooting direction of the camera 21 as the shooting direction 121a and using the shooting frame 29 containing the warehouse 27 and the house 28 as structures present in the vicinity of the river 25 as the shooting target, the situation of the river 25 can be understood easily it can.

  On the other hand, the scanning unit of the laser scanner 24 is movable, and normally, an area wider than the camera 21 is scanned and stored in the storage unit 30 as three-dimensional point cloud data 31. However, the present invention is not limited to such a configuration, and the laser scanner 24 may be provided so as to obtain three-dimensional point group data 31 by performing measurement including at least a part of the range photographed by the camera 21 among monitoring targets. . That is, it is only necessary to have a portion where the range of data of the image data 22 and the three-dimensional point cloud data 31 overlaps the monitoring target.

  The shooting direction 121 a indicates, for example, a direction connecting the center of the shooting frame 29 and the camera 21. The shooting direction 121a of the camera 21 is obtained by an azimuth sensor or the like (not shown), and the shooting position 121b of the camera 21 is obtained by GPS data (not shown) or the like. Information on the photographing direction 121a and the photographing position 121b is embedded and recorded in the image data 22, for example.

  In addition, when performing matching by the matching unit 11, the difference between the image data 22 and the three-dimensional point cloud data is smaller if the imaging timing by the camera 21 and the scanning timing by the laser scanner 24 are the same timing or closer. Is suitable for various calculations by the calculation unit 8. However, it is possible to perform various calculations by the calculation unit 8 even if the timings of the two data are different.

  FIG. 4 is a flowchart showing the operation of the three-dimensional point cloud display device 1. Description will be given with reference to FIGS. 1 and 5 to 8 as appropriate. First, as shown in FIG. 1, the data acquisition unit 2 acquires the image data 22 and the three-dimensional point cloud data 31 (data acquisition step ST1 in FIG. 4). It should be noted that in the following description of each step, in order to simplify the description, the names of the steps such as the steps ST1 and ST2 may be simply omitted.

  As shown in FIG. 5, the display processing unit 3 processes the image data 22 and the three-dimensional point group data 31 received from the data acquisition unit 2 to display the image data 22 and the three-dimensional point group data 31. (Display processing step ST2). The three-dimensional point group data 31 which is the world coordinate system is displayed on the display 4 so as to be switchable as a screen coordinate system. Next, the user performs input using the input unit 7 and sets the coordinates (position) of the camera 21 as shown on the screen 4 a of the display 4 (FIG. 5) by the viewpoint handling unit 9 via the input reception unit 5. (Viewpoint correspondence processing step ST3).

  Thereafter, the screen 4b (FIG. 6) in which the viewpoint of the camera 21 and the viewpoint of the three-dimensional point cloud data 31 coincide with each other is displayed (viewpoint corresponding screen display processing step ST4). The contents of the viewpoint correspondence processing step ST3 and the viewpoint correspondence screen display processing step ST4 will be described in detail with reference to FIGS. 5 and 6.

  FIG. 5 is a view showing the contents of the screen 4 a of the display 4 for displaying the image data 22 and the three-dimensional point cloud data 31. The relationship between the river 25 to be monitored and the structure around it and the three-dimensional point cloud data 31 will be described. In the screen 4a, the river 25 is a point cloud river 25a, the river shore 125 is a point cloud river shore 125a, the boundary between the river 25 and the river shore 125 is a first point cloud boundary 25a1, a second point cloud boundary 25a2, a warehouse 27 is a point cloud warehouse 27a, a house 28 is displayed by a three-dimensional point cloud data 31 as a point cloud house 28a, and a camera 21 as a point cloud camera 21a.

  The relationship between the image data 22 and the river 25 to be monitored and the structures around it will be described. The display 4 creates the image data display area 41 (image frame) with the contents of the image data 22 in the display area of the corner on the screen 4a, and checks both the three-dimensional point cloud data 31 and the image data 22 together it can. The operation of setting the calculation range by the calculation range setting unit 6 described in detail later can be performed efficiently. The image data 22 displayed in the image data display area 41 may be displayed in real time or may be captured at a certain time.

  In the image data display area 41, the river 25 has the image river 25b, the color shades of the water of the river 25, etc., the image river water 25b1, the warehouse 27 has the image warehouse 27b, and the house 28 has been displayed by the image data 22 as the image house 28b. .

  The image data display area 41 is disposed at the corner of the screen 4a (for example, the end area of the diagonal of the screen 4a), thereby reducing the visibility of the three-dimensional point group data 31 in the display area of the screen 4a. It is possible to efficiently grasp the state of both the image data 22 and the three-dimensional point cloud data 31. However, without providing the image data display area 41, only the three-dimensional point cloud data 31 may be displayed on the screen 4a, and the image data 22 may be displayed on another screen (not shown).

  By the way, in order to perform a matching operation by the matching unit 11 between the three-dimensional point cloud data 31 and the image data 22, it is preferable to extract feature information by the feature information extraction unit 10 in a state in which the viewpoints match each other. That is, if the viewpoints are shifted from each other, the feature information of the three-dimensional point cloud data 31 extracted by the feature information extraction unit 10 and the feature information of the image data 22 are extracted in a state of different aspect ratios. This is because there is a risk that the matching operation by the matching unit 11 can not be performed with high accuracy. However, it is possible to perform matching work to some extent without necessarily matching the viewpoints.

  Therefore, for example, when the cursor 29 by the mouse as the input unit 7 is aligned with the position of the point cloud camera 21a on the screen 4a in order to align the viewpoints of the three-dimensional point group data 31 and the image data 22, the point cloud camera 21a The coordinates of are displayed (viewpoint correspondence processing step ST3). Then, a message 42 indicating whether the viewpoint of the viewer is to be adjusted to the viewpoint of the camera is displayed. Then, “Yes” is selected in the check box “Yes” (viewpoint corresponding process step ST3). Thereafter, as shown in FIG. 6, the display 4 displays the screen 4b such that the viewpoints of the three-dimensional point cloud data 31 and the image data 22 coincide with each other (viewpoint corresponding screen display processing step ST4 in FIG. 4).

  The point cloud camera 21a may not exist on the screen 4a. In such a case, for example, when the camera 21 is installed, a facility management database (not shown) that records and manages absolute coordinates of the installation position of the camera is referred to. Then, the data of the absolute coordinates referred to may be input by the input unit 7, and the viewpoint correspondence processing step ST3 may be executed by correlating with the absolute coordinates of the point cloud data 31 or the like.

  FIG. 6 is a diagram showing the configuration of the message input screen 44 functioning as the calculation range setting unit 6. The calculation range setting unit 6 sets a calculation range to be calculated in the three-dimensional point group data 31 (calculation range setting step ST5 in FIG. 4). That is, the operation unit 8 performs an operation on the selected data 31 a selected from the three-dimensional point cloud data 31 based on the setting of the operation range by the operation range setting unit 6.

  In addition, only the range of the matching operation by the matching unit 11 is set by the setting of the calculation range setting unit 6, and extraction of the feature information by the feature information extraction unit 10 may be performed in the entire region of the three-dimensional point cloud data 31 The range of both of the extraction of feature information by the information extraction unit 10 and the matching operation by the matching unit 11 may be set. That is, at least the range of the matching operation of the matching unit 11 in the three-dimensional point cloud data 31 may be set.

  A specific input method for the message input screen 44 will be described. In order to match the image data 22 with the three-dimensional point cloud data 31, it is preferable to set an area of the three-dimensional point cloud data 31 in which the entire image data 22 is included. However, if a wide area in the three-dimensional point cloud data 31 is set as the calculation range, the amount of calculation by the calculation unit 8 will be large. Therefore, it is preferable to set a structure or the like displayed in common on the screen frame 41 and the screen 4b as a mark and a range around the mark as a calculation range for calculation.

  For example, when the warehouse 27 (point cloud warehouse 27a, image warehouse 27b) is a mark existing in common, the cursor 29 is placed on the point cloud warehouse 27a to display the coordinates (X1, Y1, Z1) of the point cloud warehouse 27a. The calculation range (X1-.alpha.1.ltoreq.X1.ltoreq.X1 + .alpha., Y1-.beta.1.ltoreq.X1.ltoreq.X1 + .beta., Z1-.gamma.1.ltoreq.X1.ltoreq.X1 + .gamma.2) is set on the message input screen 44 based on the information of the coordinates. As described above, by setting the calculation range using the coordinates of the structure displayed in common to the three-dimensional point group data 31 and the image data 22 as a mark, the matching operation can be reliably performed while suppressing the amount of calculation. It becomes possible.

  In the calculation range setting step ST5, the calculation range area 44a to be the calculation range may be displayed on the screen 4b based on the calculation range input on the message input screen 44. In this way, it is possible for the user to know which area is set as the calculation range, and when the set calculation range is unsuitable, it is possible to easily correct.

  In addition, although the setting method of the calculation range by the user input in the message input screen 44 was demonstrated as an example of the calculation range setting part 6, it is not restricted to this. For example, the calculation range may be set using attribute information included in the image data 22. As the attribute information, for example, information on the photographing direction 121a and the photographing position 121b regarding the camera 21 and information on a lens and the like regarding the camera 21 are assumed as needed.

  Specifically, since the approximate range to be captured by the image data 22 is known from the imaging direction 121a, the imaging position 121b, the focal length of the lens of the camera 21, etc., a range in which a predetermined margin is given to the approximate range. May be set as the calculation range. Alternatively, temporary setting of the calculation range may be automatically performed, and the user may adjust based on the temporary setting to set the formal calculation range.

  FIG. 7 shows a feature information extraction operation for extracting each piece of feature information from the image data 22 and the three-dimensional point cloud data 31 obtained by the feature information extraction unit 10 via the data acquisition unit 2 (feature information extraction processing in FIG. Step ST6) and the contents of the matching work (matching process step ST7 in FIG. 4) for extracting the position and size of the image data 22 that the respective feature information in the set calculation range most closely matches by the matching unit 11 It is a figure for demonstrating. For example, edge information is assumed as feature information.

  Edge information is information on boundaries obtained by extracting feature information of the image data 22 and the three-dimensional point cloud data 31. For example, in the case of the image data 22, when the monitoring target is a river, the boundary line can be extracted because the color tone is largely different between the water surface and the land portion. In addition, when the monitoring target is a road, the border line can be extracted because the color of the road and the shoulder of the road are different.

  On the other hand, in the case of three-dimensional point group data 31, the point group is a set of data of x coordinate, y coordinate and z coordinate. For example, when the monitoring target is a river, the water surface is almost parallel to the xy plane. Because the land area in the vicinity is inclined, the boundary line between the water surface and the land area can be taken.

  FIGS. 7A and 7B are diagrams for explaining extraction of edge information related to the three-dimensional point cloud data 31. FIG. FIG. 7A shows the initial state, and FIG. 7B shows the extraction result of the edge information. In order to make the explanation easy to understand, the display of the image data display area 41 is omitted on the screen 4b.

  On the other hand, FIGS. 7C and 7D are diagrams for explaining extraction of edge information related to the image data 22. FIG. FIG. 7C shows an initial state, and FIG. 7D shows the extraction result of edge information. The image data display area 41 of (C) and (D) of FIG. 7 is enlarged and described so as to be easily compared with (A) and (B) of FIG. 7. FIG. 7E is a diagram showing the result of the matching operation based on the extraction results of FIG. 7B and FIG. 7D.

  Hereinafter, the feature information extraction work and the matching work will be described in detail. In FIG. 7B, edge information is extracted as the first point group boundary 25a1 as the first point group boundary edge 45a1, and edge information is extracted as the second point group boundary 25a2 as the second point group boundary edge 45a2. It is displayed that the point cloud warehouse 27a has edge information extracted as a point cloud warehouse edge 46a, and that the point cloud house 28a has edge information extracted as a point cloud house edge 47a.

  On the other hand, in FIG. 7D, the image river 25b extracts edge information as the image river edge 45b, the image warehouse 27b extracts edge information as the image warehouse edge 46b, and the image house 28b as the image house edge 47b. Is extracted. Although the image river water 25b1 is displayed in FIG. 7C, since it is not extracted as edge information, there is no corresponding one in FIG. 7D.

  In (B) or (D) of FIG. 7, in order to make the edge processing more appropriate while grasping the state of the edge processing, for example, it is possible to adjust parameters related to edge extraction by moving the bar. It is also good. This can be expected to improve the accuracy of the matching operation in FIG. 7 (E).

  FIG. 7E shows the display 4 displaying the screen 4c. In the screen 4c, the edge information of the three-dimensional point group data 31 and the edge information of the image data 22 are superimposed in the matching work in the matching processing step ST7 using the edge information acquired in the feature information extraction processing step ST6. The result is displayed.

  Specifically, the screen 4 c extracts the position on the three-dimensional point group data and the size of the image data 22 where the edge information of the image data 22 most overlaps the edge information of the three-dimensional point group data 31 and extracts the extraction result It becomes a display by adjusting the position and size based on. As a result of the matching work, the image warehouse edge 46b approximately overlaps the point cloud warehouse edge 46a, the image housing 47b approximately overlaps the point cloud housing 47a, and the image river edge 45b at the first point cloud boundary edge 45a1. It can be read that the

  FIG. 8 is a diagram showing that the superimposed display processing unit 12 causes the display unit 4 to superimpose the image data 22 on the three-dimensional point cloud data 31 based on the matching result (superimposed display processing step ST8 in FIG. 4). For example, after the screen 4c in FIG. 7E is displayed, when a predetermined time such as a few seconds has elapsed or when the screen 4c is clicked with the cursor 29, for example, the screen is displayed as shown in FIG.

  Specifically, the image data 22 is output in three-dimensional coordinates in the space where the point cloud data 31 exists, using the matching result and the information on the photographing direction 121a and the photographing position 121b. Then, the image data 22 is enlarged or reduced based on the matching result without changing the aspect ratio (aspect ratio), and the display processing unit 3 displays the image data 22 superimposed on the three-dimensional point cloud data 31. I do. That is, the image river 25b is superimposed and displayed on the point cloud river 25a, the image warehouse 27b is superimposed and displayed on the point cloud warehouse 27a, and the image housing 28b is superimposed and displayed on the point cloud house 28a.

  If the image data 22 used by the matching unit 11 is image data at a certain point, the superimposed display processing unit 12 uses the matching result to make the image data 22 acquired by the camera 21 three-dimensional from now on. It may be superimposed on the point cloud data 31 at an appropriate position.

  Thus, on the screen 4c, the user perceives the real-time information to be monitored with the contents displayed in the image data display area 41a where the image data 22 is displayed, and the situation around the image data display area 41a is 3 It can be grasped by the contents of the dimensional point cloud data 31, and it becomes possible to monitor a comprehensive monitoring target.

  Although it has been described in FIG. 1 that the three-dimensional point cloud display device 1 includes the computing unit 8, for example, in a server or a workstation (not shown) having higher computing power than the three-dimensional point cloud display device 1. The arithmetic unit 8 may be stored. By doing so, it becomes possible to perform feature information extraction work and matching work in a shorter time. In addition, the storage unit 30 may also be stored in a server (not shown).

  Although the feature information extraction work and the matching work are displayed on the screen 4b and the screen 4c in FIG. 7, the process may be performed as an internal process without displaying the feature information extraction work and the matching work on the screen 4b or the screen 4c. The reduction in the load on the display processing unit 3 allows for an increase in the processing capacity of the three-dimensional point cloud display device 1, and the processing speed of the feature information extraction work and the matching work can be improved compared to the case where the internal processing is not performed.

  According to the three-dimensional point cloud display device 1 according to the first embodiment of the present invention, the data acquisition unit 2 for acquiring the image data 22 obtained by photographing the monitoring target with the camera 21, and the camera among the monitoring targets 21. Display processing unit 3 performing processing for displaying on screen 4 a the three-dimensional point cloud data 31 obtained by measurement including at least a part of the range photographed by 21 and three-dimensional point cloud data displayed on screen 4 b The data acquisition unit 2 acquires selection data 31 a selected from the three-dimensional point cloud data 31 based on the setting of the calculation range setting unit 6 that sets the calculation range by user input with respect to 31 and the calculation range setting unit 6. The feature information of the three-dimensional point cloud data 31 and the feature information of the image data 22 based on the feature information extraction unit 10 for extracting each feature information of the image data 22 and the extraction result by the feature information extraction unit 10 Because the matching unit 11 for matching each other and the superimposed display processing unit 12 for superimposing the image data on the three-dimensional point cloud data 31 in the screen 4 c based on the matching result of the matching unit 11 are provided. As compared with the configuration in which the real image is matched with the surrounding three-dimensional map information based on the GPS data, it becomes possible to reliably perform the matching between the three-dimensional point cloud data 31 and the image data 22.

  Further, in the three-dimensional point cloud display device 1 according to the first embodiment of the present invention, the display processing unit 3 causes the image data 22 to be displayed inside the screen 4a when the setting by the calculation range setting unit 6 is performed. Since the image data display area 41 is created, the operation of setting the calculation range by the calculation range setting unit 6 can be performed efficiently.

  Further, according to the three-dimensional point cloud display device 1 according to the first embodiment of the present invention, the image data display area 41 is arranged at the corner of the screen 4a when setting by the calculation range setting unit 6. Therefore, it is possible to suppress a decrease in visibility in the display area of the three-dimensional point group data 31 on the screen 4a, and it is possible to efficiently grasp the state of both the image data 22 and the three-dimensional point group data 31. .

  Further, according to the three-dimensional point cloud display device 1 according to the first embodiment of the present invention, the calculation range setting unit 6 displays the calculation range area 44a corresponding to the calculation range set by user input on the screen 4b. Therefore, it is possible for the user to know which area is set as the calculation range, and when the set calculation range is unsuitable, it is possible to easily correct.

  According to the three-dimensional point cloud display device 1 according to the first embodiment of the present invention, the data acquisition unit 2 for acquiring the image data 22 obtained by photographing the monitoring target with the camera 21, and the camera among the monitoring targets 21. Display processing unit 3 performing processing for displaying on screen 4 a the three-dimensional point cloud data 31 obtained by measurement including at least a part of the range photographed by 21 and three-dimensional point cloud data displayed on screen 4 b 31 based on the setting of the calculation range setting unit 6 for setting the calculation range based on the attribute information (for example, information on the photographing direction 121a and the photographing position 121b) included in the image data 22 Feature information extraction unit 10 for extracting feature information of each of selection data 31a selected from point cloud data 31 and image data 22 obtained by data acquisition unit 2, and feature information extraction unit Based on the result of the extraction by 0, based on the matching result of the matching unit 11 for matching the feature information of the three-dimensional point group data 31 and the feature information of the image data 22, the three-dimensional points in the screen 4c Since the superposition display processing unit 12 for superimposing and displaying the image data on the group data 31 is provided, compared with the configuration in which the real image is matched with the surrounding three-dimensional map information based on the GPS data, It becomes possible to reliably perform matching between the group data 31 and the image data 22.

  Further, according to the three-dimensional point cloud display system 40 according to the first embodiment of the present invention, the camera 21 that captures the monitoring target and the data acquisition unit 2 that acquires the image data 22 obtained by capturing with the camera 21 And the display processing unit 3 performing processing for displaying on the screen 4 a the three-dimensional point cloud data 31 obtained by measuring at least a part of the range photographed by the camera 21 among the monitoring targets, and the screen 4 b Selection data selected from the three-dimensional point group data 31 based on the setting of the calculation range setting unit 6 for setting the calculation range by user input to the displayed three-dimensional point group data 31 and the calculation range setting unit 6 31a and a feature information extraction unit 10 for extracting feature information of each of the image data 22 acquired by the data acquisition unit 2, and three-dimensional point cloud data 31 based on the extraction result by the feature information extraction unit 10. Superimposing display processing in which image data is superimposed and displayed on the three-dimensional point cloud data 31 in the screen 4 c based on the matching unit 11 that matches the feature information and the feature information of the image data 22 and the matching result of the matching unit 11 Since the unit 12 is provided, the matching between the three-dimensional point cloud data 31 and the image data 22 can be reliably performed compared to the configuration in which the actual three-dimensional map information and the surrounding three-dimensional map information are matched based on GPS data. It becomes possible.

  Further, according to the three-dimensional point cloud display system 40 according to the first embodiment of the present invention, when the display processing unit 3 performs setting by the calculation range setting unit 6, the image data 22 is displayed in the screen 4a. Since the image data display area 41 is created, the operation of setting the calculation range by the calculation range setting unit 6 can be performed efficiently.

  Further, according to the three-dimensional point cloud display method according to the first embodiment of the present invention, data acquisition step ST1 of acquiring the image data 22 obtained by photographing the monitoring target with the camera 21; Display processing step ST2 for displaying on the screen 4a the three-dimensional point cloud data 31 obtained by measurement including at least a part of the range photographed by the camera 21; 3D point cloud displayed on the screen 4b Selection data 31a selected from 3D point cloud data based on the setting in the superposition display processing step of setting the calculation range by user input to the data and the setting in the calculation range setting step ST5 and the image acquired by the data acquisition unit 3D point cloud data 31 based on feature information extraction processing step ST6 for extracting feature information of each piece of data and the result of extraction by the feature information extraction unit. Superimposed display in which the image data 22 is superimposed and displayed on the three-dimensional point cloud data 31 in the screen 4c based on the result of the superimposed display processing step ST7 for matching feature information of feature information and image data and the matching processing step ST7. Since processing step ST8 is provided, the matching between the three-dimensional point group data and the image data can be reliably performed as compared with the configuration in which the actual three-dimensional map information and the surrounding three-dimensional map information are matched based on GPS data. It will be possible.

  Further, according to the three-dimensional point cloud display method according to the first embodiment of the present invention, the display processing step ST2 displays the image data 22 inside the screen 4b when performing the setting in the calculation range setting step ST5. Since the image data display area 41 to be generated is created, the operation of setting the calculation range in the calculation range setting step ST5 can be performed efficiently.

  Further, according to the three-dimensional point cloud display program according to the first embodiment of the present invention, a data acquisition step ST1 for acquiring image data 22 obtained by photographing the monitoring target with the camera 21; Display processing step ST2 for displaying on the screen 4a the three-dimensional point cloud data 31 obtained by measurement including at least a part of the range photographed by the camera 21; 3D point cloud displayed on the screen 4b Selection data 31a selected from 3D point cloud data based on the setting in the superposition display processing step of setting the calculation range by user input to the data and the setting in the calculation range setting step ST5 and the image acquired by the data acquisition unit Three-dimensional point group data based on feature information extraction processing step ST6 for extracting feature information of each data and the result of extraction by the feature information extraction unit The image data 22 is superimposedly displayed on the three-dimensional point cloud data 31 in the screen 4c based on the result of the superposition display processing step ST7 for matching the feature information of the image data and the feature information of 1 and the matching processing step ST7. Since the superimposing display processing step ST8 is executed by the computer, the matching between the three-dimensional point cloud data and the image data is assured compared to the configuration in which the actual three-dimensional map information and the surrounding three-dimensional map information are matched. It will be possible to

  Further, according to the three-dimensional point cloud display program according to the first embodiment of the present invention, the display processing step ST2 displays the image data 22 inside the screen 4b when setting in the calculation range setting step ST5. Since the image data display area 41 to be generated is created, the operation of setting the calculation range in the calculation range setting step ST5 can be performed efficiently.

  Further, according to the recording medium in the first embodiment of the present invention, since the program P is recorded, various functions by the program P can be executed.

Second Embodiment
Second Embodiment A second embodiment of the present invention will be described with reference to FIG. 9 and FIG. In the first embodiment, setting of the calculation range by the calculation range setting unit 6 is performed by user input of coordinates of each axis of X axis, Y axis and Z axis, but in the present embodiment, the mouse as the input unit 7 is used. Setting the calculation range is a different part by using the cursor movement by the like.

  FIG. 9 is a block diagram in the case of setting an operation range using a pointing device as the input unit 7. As a pointing device, for example, a mouse 7a, a joystick 7b, a track ball (not shown), etc. are assumed. The input receiving unit 5 receives the input from these pointing devices, and the setting of the calculation range by the movement of the cursor 29 is executed in the calculation range setting unit 6a in the data acquisition unit 2a.

  FIG. 10 is a setting screen for setting a calculation range using a pointing device. With respect to the display in the screen coordinate system displayed in the screen 4b, an area which the user wants to set as a calculation range is considered. For example, the cursor 29 is moved by the operation of the mouse 7a to determine the calculation range in the X-axis direction and the Z-axis direction. Then, when determining the calculation range in the Y-axis direction which is the depth, while shifting the viewpoint in the Y-axis direction using the viewpoint moving tool 51, the same operation as the moving operation of the cursor 29 in the X-axis or Z-axis is performed. It is good.

  In the three-dimensional point cloud display device 1 according to the second embodiment of the present invention, the calculation range setting unit 6 sets the calculation range using the pointing devices 7a and 7b, so the user can calculate visually It will be possible to do range. That is, which range is to be set as the calculation range can be easily performed as compared with the case of inputting a numeral serving as coordinates.

Third Embodiment
Third Embodiment A third embodiment of the present invention will be described with reference to FIGS. 11 and 12. In the first embodiment, the matching operation is an operation of extracting the position and size of the image data in which the feature information of each of the image data and the three-dimensional point group data most matches within the set operation range. The difference is that it has a function to prompt resetting of the calculation range when the degree of is not good.

  FIG. 11 is a block diagram showing the configuration of the arithmetic unit 8a. In addition to the viewpoint handling unit 9, the feature information extraction unit 10, and the matching unit 11, the calculation unit 8a includes a matching degree determination unit 11a that determines the degree of matching. The matching degree determination unit 11a functions to grasp how much the result of the matching operation in the matching unit 11 is with respect to the predetermined matching degree. As a method of obtaining the degree of matching, for example, the number of pixels in which the image data 22 does not match the number of pixels matching the three-dimensional point group data 31 is counted in the calculation range set in the three-dimensional point group data 31 Alternatively, it may be obtained from the ratio of the number of matching pixels to the total number of pixels.

  FIG. 12 is a flowchart showing the operation of the three-dimensional point cloud display device 1. Steps ST1 to ST8 are the same as in the first embodiment, and steps ST2 to ST4 are omitted. The matching degree confirmation step ST17 and the warning display step ST18 for prompting resetting of the calculation range are steps different from those in the first embodiment.

  For example, when the result of the matching operation in the matching unit 11 does not reach the predetermined matching degree in the matching degree confirmation step ST17, the flow branches to the warning display step ST18, and the display processing unit 3 A warning 144 is notified that the calculation range should be set and the calculation should be performed again. Thereafter, based on the display of the warning 144, the user shifts again to the calculation range setting step ST5 and executes setting of the calculation range by the cursor 29 or the like. The warning 144 may be anything other than characters, as long as it can be transmitted to the user, such as voice.

  The three-dimensional point cloud display device 1 according to Embodiment 3 of the present invention includes a matching degree determination unit 11a that determines the result of matching by the matching unit 11, and resets the calculation range according to the determination result. Since the warning 144 for prompting is notified, it is possible to prevent the superimposed display of the image data 22 on the three-dimensional point cloud data 31 from being displayed with poor accuracy.

Modification 3a
The modification 3a will be described with reference to FIG. When the user again sets the calculation range in the calculation range setting step ST5 after receiving the warning display step ST18, even if the calculation range is set anew, even if the calculation range is newly set, the matching work is performed again. The degree of matching does not improve, and there is a possibility that the degree of matching determined in advance may not be reached.

  Therefore, the area of the previous calculation range is displayed as the previous calculation range area 51, and based on that, it becomes possible to set the calculation range as the latest calculation range area 52. That is, in the calculation range set previously, it is possible to grasp the area which is insufficient for the matching operation in the matching unit 11, and it becomes possible to set the latest calculation range area 52 more efficiently.

  In the three-dimensional point cloud display device 1 according to the modification 3a of the third embodiment of the present invention, when the warning 144 is notified and the setting of the calculation range is performed again, the previous calculation range region 51 which became the warning target is displayed on the screen. Since it is displayed, there is an effect that efficient resetting can be performed when resetting the calculation range.

Fourth Embodiment
Fourth Embodiment A fourth embodiment of the present invention will be described using FIGS. 14 to 16. In the first embodiment, it is mainly assumed that the camera is a fixed camera with no change in the field of view, but in the fourth embodiment, it is a movable camera that can be photographed by changing the angle between left and right or up and down. Assuming that the matching operation is performed using image data obtained by photographing an area larger than that of the first embodiment.

  FIG. 14 is a block diagram showing the configuration of the three-dimensional point cloud display system 40a. A configuration different from the three-dimensional point cloud display system 40 in FIG. 1 of the first embodiment will be described. The camera unit 20a has a movable camera 21a and a data receiving unit 123 for receiving a movable camera control signal 13a for controlling the movable camera 21a sent from the three-dimensional point cloud display device 1a.

  The three-dimensional point cloud display device 1a includes a camera control unit 13 that sends a movable camera control signal 13a, a movable camera 21a, a program Pa as a three-dimensional point cloud display program that generates the movable camera control signal 13a, and a movable camera 21a. And an image combining unit that combines the plurality of pieces of image data 22 captured by the image processing unit to create combined image data 22a. Further, the storage unit 30 has a function of receiving from the data acquisition unit 2 a plurality of image data 22 synthesized by the plurality of image synthesis units 14 and storing the image data 22.

  When the input reception unit 5 receives an instruction to control the movable camera 21 a by the input from the input unit 7, the instruction is transmitted to the camera control unit 13. The camera control unit 13 receives the instruction, and transmits the movable camera control signal 13a to the data receiving unit 123 in the camera unit 20a. The movable camera 21a is configured to perform various operations in response to the movable camera control signal 13a.

  FIG. 15 is a flowchart showing the operation of the three-dimensional point cloud display system 40a when the movable camera control signal 13a is transmitted from the camera control unit 13 and the movable camera 21a receives the movable camera control signal 13a. The operation of the three-dimensional point cloud display system 40a will be described with reference to FIGS. 14 and 16 as appropriate.

  First, the movable camera 21a shoots in a direction such as an initial position. Then, for example, imaging is performed continuously in the horizontal direction such that the boundary portions of the image data 22 overlap one another at a plurality of predetermined angles (a plurality of imaging steps STA). The captured image data 22 is sequentially stored in the storage unit 30 via the data transmission unit 23 and the data acquisition unit 2 (image storage step STB). That is, the movable camera 21a performs shooting in a plurality of shooting directions.

  Thereafter, when photographing by the movable camera 21a is finished at all of a plurality of predetermined angles, the data acquisition unit 2 communicates with the storage unit 30, and sends the plurality of stored image data 22 to the image combining unit 14. The image combining unit 14 combines the plurality of received image data 22 to create combined image data 22a (image data combining step STC). After the composite image data 22a is created, steps ST1 to ST8 are executed as in the first embodiment.

  FIG. 16 is a diagram showing the contents of the screen 4 c in the display 4 for displaying the three-dimensional point cloud data 31. The composite image data 22a is displayed in the image data display area 41a. Of the image data display area 41a, the image data 22 displayed in the first image data display area 41a1 is photographed by the imaging range as the initial position of the movable camera 21a, and is displayed in the second image data display area 41a2. The image data 22 to be captured is the one captured in the imaging range when the imaging direction of the movable camera 21a is changed.

  A boundary between a shooting range in a certain shooting direction (for example, an initial position) and a shooting range in another shooting direction when the shooting direction is changed is displayed as a boundary line 42. If an abnormality occurs in an image such as noise different from usual appearing in the composite image data 22a, the boundary line 42 is displayed to display the first image data display area 41a1 or the second image data display area. Since it can be easily grasped which of 41a2 is the cause, it is efficient in regenerating the composite image data 22a.

  In addition, after making synthetic image data 22a from the image data 22 which varied the imaging | photography direction and acquired the continuous imaging range, extraction of the feature information by the feature extraction part 10 and matching by the matching part 11 are demonstrated. However, the image data 22 does not necessarily have to be a continuous shooting range. That is, the first image data (not shown) photographed in a certain photographing direction and the second image data (not shown) photographed while varying the photographing direction may have their respective image data separated, in this case, As the relationship between the first image data and the second image data, the relationship between the respective image data such as the photographed angle information may be grasped.

  In this manner, by using the plurality of image data 22 captured according to the plurality of imaging directions, extraction of feature information by the feature extraction unit 10 can be increased more than in the case of the first embodiment, The matching result by the matching unit 11 can be matched in a wider area. As a result, the image data 22 can be superimposed and displayed on the three-dimensional point cloud data 31 on the screen 4 more accurately than in the first embodiment.

  In the three-dimensional point cloud display device 1 according to the fourth embodiment of the present invention, when the camera 21 is the movable camera 21a whose imaging direction is variable, the data acquiring unit 2 is plural in plural directions photographed by the movable camera 21a. The image data 22 is acquired, and the feature information extraction unit 10 extracts feature information using the plurality of image data 22. Therefore, on the screen 4c, more accurately, on the three-dimensional point cloud data 31. It is possible to display the image data 22 in a superimposed manner.

Modification 4a
In this modification, the specification of the movable camera whose unit moving amount in the photographing direction for each command is unknown is obtained by utilizing the image taken by the movable camera and the three-dimensional point cloud data, and it is useful for monitoring work. This will be described with reference to FIG.

  FIG. 17 is a flowchart showing the operation of this modification. Steps STA to ST7 are the same as in the fourth embodiment. Step ST9 is the content added in this modification.

  In step STA, the user transmits an instruction to change the shooting direction to the movable camera 21 via the camera control unit 13 described in FIG. The movable camera 21a in this modification does not know the unit movement amount, and, for example, the user continues the movement instruction step (not shown) for moving the movable camera 21a leftward for 1 second in a sense. It was.

  Therefore, when the imaging direction is changed in step STA, the information on the movement instruction step is recorded, and the unit movement amount calculation step ST9 is performed using the information on the movement instruction step and the result in the matching processing step ST7. The unit moving amount (the unit moving amount in the photographing direction) of the movable camera 21a for each unit is obtained.

  By having the unit movement amount calculation step ST9, the unit movement amount of the movable camera 21a is determined for each unit movement instruction step, so that more accurate operation can be performed when performing monitoring work.

Reference Signs List 1 3D point cloud display device 2 data acquisition unit 3 display processing unit 4 display 4 a, 4 b, 4 c screen 5 input reception unit 6 operation range setting unit 7 input unit 7 a mouse (pointing device) , 7b joystick (pointing device), 8 operation unit, 9 viewpoint corresponding unit, 10 feature information extraction unit, 11 matching unit, 11a matching degree determination unit, 12 superposition display processing unit, 20 camera units, 21 cameras, 22 image data, 23 data transmission unit, 30 storage unit, 31 three-dimensional point group data, 31 a selection data, 40 three-dimensional point group display system, 41 image data display area, 44 a calculation range area (calculation range), 45 a 1 first point group Boundary edge (edge information, feature information), 45a2 Second point group boundary edge (edge information, feature information), 45b Image river Edge (edge information, feature information), 46a Point cloud warehouse edge (edge information, feature information), 46b Image warehouse edge (edge information, feature information), 47a Point cloud house edge (edge information, feature information), 47b image Residential edge (edge information, feature information), 51 previous calculation range area, 101 memory (recording medium), 121a shooting direction (attribute information), 121b shooting position (attribute information), 144 warning P, Pa program (3D point group Display program), ST1 data acquisition step, ST2 display processing step, ST5 operation range setting step, ST6 feature information extraction processing step, ST7 matching processing step, ST8 superposition display processing step, ST9 unit movement amount calculation step, STA multiple imaging step, STB image storage step, STC image data synthesis step

Claims (16)

A data acquisition unit that acquires image data obtained by photographing a monitoring target with a camera;
A display processing unit that performs processing for displaying on a screen three-dimensional point cloud data obtained by measuring at least a part of a range captured by the camera among the monitoring targets;
A calculation range setting unit configured to set a calculation range by user input to the three-dimensional point group data displayed on the screen;
A feature information extraction unit that extracts feature information of each of the selected data selected from the three-dimensional point cloud data based on the setting of the calculation range setting unit and the image data acquired by the data acquisition unit;
A matching unit that matches feature information of the three-dimensional point cloud data with feature information of the image data based on a result of the extraction by the feature information extraction unit;
A superimposed display processing unit configured to display the image data superimposed on the three-dimensional point group data in the screen based on the matching result of the matching unit;
3D point cloud display equipped with The three-dimensional point cloud display device according to claim 1, wherein the display processing unit creates an image data display area for displaying the image data inside the screen when the setting by the calculation range setting unit is performed. . The three-dimensional point cloud display device according to claim 2, wherein the image data display area is disposed at a corner of the screen. The three-dimensional point cloud display device according to any one of claims 1 to 3, wherein the calculation range setting unit displays a calculation range region corresponding to the calculation range on the screen. The three-dimensional point cloud display device according to any one of claims 1 to 4, wherein the calculation range setting unit sets the calculation range using a pointing device. A matching degree determination unit that determines a result of the matching performed by the matching unit;
The three-dimensional point cloud display device according to any one of claims 1 to 5, wherein a warning to prompt resetting of the calculation range is notified according to the result of the determination. The three-dimensional point according to claim 6, wherein, when the warning is notified and the setting of the calculation range is performed again, the previous calculation range area, which is the area of the previous calculation range that became the warning target, is displayed on the screen. Group display device. The data acquisition unit acquires a plurality of the image data in a plurality of directions captured by the movable camera, when the camera is a movable camera whose imaging direction is variable.
The three-dimensional point cloud display device according to any one of claims 1 to 7, wherein the feature information extraction unit extracts feature information using a plurality of the image data. A data acquisition unit that acquires image data obtained by photographing a monitoring target with a camera;
A display processing unit that performs processing for displaying on a screen three-dimensional point cloud data obtained by measuring at least a part of a range captured by the camera among the monitoring targets;
A calculation range setting unit configured to set a calculation range based on attribute information included in the image data with respect to the three-dimensional point cloud data displayed on the screen;
A feature information extraction unit that extracts feature information of each of the selected data selected from the three-dimensional point cloud data based on the setting of the calculation range setting unit and the image data acquired by the data acquisition unit;
A matching unit that matches feature information of the three-dimensional point cloud data with feature information of the image data based on a result of the extraction by the feature information extraction unit;
A superimposed display processing unit configured to display the image data superimposed on the three-dimensional point group data in the screen based on the matching result of the matching unit;
3D point cloud display equipped with A camera that shoots the monitoring target,
A data acquisition unit for acquiring image data obtained by photographing with the camera;
A display processing unit that performs processing for displaying on a screen three-dimensional point cloud data obtained by measuring at least a part of a range captured by the camera among the monitoring targets;
A calculation range setting unit configured to set a calculation range by user input to the three-dimensional point group data displayed on the screen;
A feature information extraction unit that extracts feature information of each of the selected data selected from the three-dimensional point cloud data based on the setting of the calculation range setting unit and the image data acquired by the data acquisition unit;
A matching unit that matches feature information of the three-dimensional point cloud data with feature information of the image data based on a result of the extraction by the feature information extraction unit;
A superimposed display processing unit configured to display the image data superimposed on the three-dimensional point group data in the screen based on the matching result of the matching unit;
3D point cloud display system equipped with. The three-dimensional point cloud display system according to claim 10, wherein the display processing unit creates an image data display area for displaying the image data inside the screen when the setting by the calculation range setting unit is performed. . A data acquisition step of acquiring image data obtained by photographing the monitoring target with a camera;
A display processing step of performing processing of displaying on a screen three-dimensional point cloud data obtained by measuring at least a part of a range photographed by the camera among the monitoring targets;
A calculation range setting step of setting a calculation range by user input to the three-dimensional point cloud data displayed on the screen;
Feature information extraction processing step of extracting feature information of each of the selected data selected from the three-dimensional point cloud data and the image data acquired by the data acquisition step based on the setting in the calculation range setting step;
A matching processing step of matching feature information of the three-dimensional point group data with feature information of the image data based on a result of the extraction by the feature information extraction processing step;
A superimposed display processing step of causing the image data to be superimposed and displayed on the three-dimensional point group data in the screen based on the matching result in the matching processing step;
A three-dimensional point cloud display method having The three-dimensional point cloud display method according to claim 12, wherein the display processing step creates an image data display area for displaying the image data inside the screen when the setting in the calculation range setting step is performed. . A data acquisition step of acquiring image data obtained by photographing the monitoring target with a camera;
A display processing step of performing processing of displaying on a screen three-dimensional point cloud data obtained by measuring at least a part of a range photographed by the camera among the monitoring targets;
A calculation range setting step of setting a calculation range by user input to the three-dimensional point cloud data displayed on the screen;
Feature information extraction processing step of extracting feature information of each of the selected data selected from the three-dimensional point cloud data and the image data acquired by the data acquisition step based on the setting in the calculation range setting step;
A matching processing step of matching feature information of the three-dimensional point group data with feature information of the image data based on a result of the extraction by the feature information extraction processing step;
A superimposed display processing step of causing the image data to be superimposed and displayed on the three-dimensional point cloud data in the screen based on the matching result of the matching processing step;
Three-dimensional point cloud display program for making a computer execute. The three-dimensional point cloud display program according to claim 14, wherein the display processing step creates an image data display area for displaying the image data in the screen when the setting in the calculation range setting step is performed. .   A recording medium recording the three-dimensional point cloud display program according to claim 14 or 15.

Images