JP7044331B2 - Image processing systems, image processing methods and programs for efficiently inspecting structures such as bridges - Google Patents

Description

The present invention relates to an image processing system, an image processing method and a program for efficiently inspecting a structure such as a bridge, and in particular, an inspection work for the presence or absence of damage such as a crack in a bridge can be easily and quickly performed. It relates to an image processing system, an image processing method and a program for efficiently inspecting a structure such as a bridge suitable for accurate and low cost implementation.

Conventionally, when inspecting concrete walls of small-scale bridges in particular, the dimensions of cracks are directly measured on-site with a crack scale, etc., marked with chalk, and sketched and photographed. The method of recording the position and size of the concrete is generally performed, but such work requires a lot of time and labor and is very inefficient.
In addition, the majority of bridges that exist in Japan and require regular inspection for cracks and damage are about 2 m to 15 m in length and several m to 10 m below the girder. In many cases, in order to acquire a plurality of images taken from the front, it is necessary to form a scaffold and shoot while moving the shooting position, and such work is difficult.
If a crack having a width of 0.2 mm occurs on the concrete wall surface of the bridge, the depth is predicted to be about 10 cm, and there is a possibility that corrosion may occur in the internal reinforcing bar.
Therefore, in order to detect the existence and position of cracks having a width of about 0.1 mm from the front image, it is necessary to acquire a detailed image of the structure.

Using a drone for a small bridge with a length of about 2 m to 15 m requires advanced maneuvering techniques such as avoiding obstacles such as tree branches, and requires expensive equipment. be.
Conventionally, in order to inspect a structure, a method of acquiring a plurality of image data using a moving body and integrating them has been proposed (Patent Documents 1 to 4), but these methods include the following methods. There is such a problem.
・ There is usually no passage under a small bridge, and it is difficult to move the moving body properly.
-Multiple image data to be integrated must be taken from the front of the structure. -There is a need for overlapping parts in adjacent image data.

Further, the above-mentioned conventional technique has a problem that it is not possible to provide a simple image processing system capable of identifying the existence and position of fine cracks in a structure existing in an inaccessible site such as a small bridge.
Therefore, the inventors of the present invention have already taken a plurality of detailed images from a fixed position in Japanese Patent Application No. 2018-30606, calculated the distance and inclination of each detailed image, and integrated the converted into a front image for details. The invention for obtaining a wide-area front image is disclosed (see FIG. 14).
The present invention discloses a method of converting image data obtained by shooting from an oblique direction into front image data by using three or more laser pointers and a camera whose positional relationship is specified in advance. ..
The invention of the present application is further developed while partially utilizing the system and method of the invention disclosed in Japanese Patent Application No. 2018-30606.

Japanese Unexamined Patent Publication No. 2012-18073 Japanese Unexamined Patent Publication No. 2009-85785 Japanese Unexamined Patent Publication No. 2004-37440 Japanese Unexamined Patent Publication No. 2012-177569

In the invention of Japanese Patent Application No. 2018-30606, even when the structure to be inspected is only dotted with damaged parts such as cracks, a detailed image of a wide range (wide area) of the structure without omission is obtained. There is a problem that the number of shots is unnecessarily large and it takes time to work.
Therefore, the problem to be solved by the present invention is
(A) An image processing system for inspecting structures such as bridges, and an image processing system that can easily, quickly, accurately, and inexpensively identify the existence and position of fine cracks in an image processing method. Provides an image processing method and
(B) An image processing system for inspecting structures such as bridges, which makes it easy to perform photography work even for structures existing in sites that are difficult to access, and which can accurately and quickly process the obtained image data, and While providing an image processing method,
(C) An image processing system for efficiently inspecting a structure such as a bridge, a method for efficiently inspecting a structure such as a bridge using an image processing system, and a program that can be used for the method. Is to provide.

Another object of the present invention is
In addition to providing an image processing system, image processing method and program that can acquire front images from images that are not taken from the front of the structure and integrate them, as well as
Provided are an image processing system, an image processing method and a program capable of integrating a plurality of images even if there is no overlapping part in adjacent image data, and further,
By using an image processing system that can further improve the efficiency of inspection work and improve the reliability of the data obtained by inspection work by appropriately referring to and using the work of visually checking the damaged part on site. The purpose is to provide a method for efficiently inspecting structures such as bridges and a program that can be used for that method.
Furthermore, another object of the present invention is
An image processing system that can add information such as the length and width to the damaged part of the structure detected in the image data and record it, and
It is an object of the present invention to provide an image processing system capable of comparing and contrasting image data obtained in the past with respect to the same structure and newly obtained image data, and confirming a change with time of a damaged portion in the structure.

(1) The present invention is "an image processing system for efficiently inspecting structures such as bridges.
An imaging device consisting of a camera, at least three laser pointers, and a fixing means capable of holding them integrally.
A data processing device for processing image data obtained by the photographing device is provided.
Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and direction by the photographing apparatus, and the laser pointer. Detailed image data obtained by photographing the central part of the partial image data so that at least three irradiation points can be captured, and two wide and narrow image data sets obtained from the image data set.
Other wide and narrow 2 obtained from other partial image data obtained by photographing a part different from a part of the structure from the same position and other detailed image data corresponding to the other partial image data. With one image dataset,
A plurality of specific part detailed image data in which a specific part of a structure is separately photographed from the same position by the photographing apparatus, and
An image processing system that operates based on
The data processing device is
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. As well as being able to get
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the specific partial detailed image data are calculated, and the distance data, the tilt data, and the like. And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. The most important feature is "an image processing system for efficiently inspecting structures such as bridges, which is characterized by being able to do so."

(2) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, there are four laser pointers in the photographing apparatus, and each of these laser pointers is the optical axis of the camera. It may be arranged in parallel and may emit laser light so that an irradiation point can be obtained at a position corresponding to the four corners on the diagonal line of the detailed image data.
(3) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device automatically detects and displays a candidate for a damaged portion of the structure in the wide area front image data. Further, it may be possible to delete or correct the candidate of the damaged portion.
(4) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device learns from the history of deletion or correction of a candidate for a damaged portion, and then damages the structure. It may be one that can be reflected when the candidate of the part is automatically detected.

(5) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device is composed of a host computer and a local terminal device connected to the host computer. , The host computer learns from the coarse wide area front image data of the structure, the generation of the partially detailed wide area front image data of the structure, and the history of deletion or correction of the candidate of the damaged part. It is possible to perform a process of reflecting the candidate for the damaged portion of the structure when automatically detecting the damaged portion, and the local terminal device may be able to delete or correct the candidate for the damaged portion.
(6) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing apparatus includes the length of the damaged portion of the structure detected in the wide area front image data. It may be one that can be recorded by adding information such as width.
(7) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing apparatus obtains image data obtained in the past and newly obtained image data of the same structure. It may be possible to compare and contrast, and it may be possible to confirm the change with time of the damaged portion in the structure.

(8) Further, the present invention is "a method for efficiently inspecting a structure such as a bridge using an image processing system.
An imaging device consisting of a camera, at least three laser pointers, and a fixing means capable of holding them integrally.
Using an image processing system equipped with a data processing device that processes the image data obtained by the photographing device,
1) Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and the same direction by the photographing apparatus. Detailed image data obtained by photographing the central part of the partial image data so that at least three irradiation points by the laser pointer are captured, and two wide and narrow image data sets obtained from the partial image data.
Other wide and narrow 2 obtained from other partial image data obtained by photographing a part different from a part of the structure from the same position and other detailed image data corresponding to the other partial image data. With one image dataset,
By inputting to the image processing system
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. Steps to get and
2) The damaged part of the structure found visually is determined to be a specific part, and a plurality of specific part detailed image data in which the specific part of the structure is separately photographed from the same position by the photographing apparatus is applied to the image processing system. By inputting
The distance data from at least three irradiation points by the laser pointer to the structure and the tilt data in the three-dimensional direction of the camera in the specific part detailed image data are calculated, and the distance data, the tilt data, And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. Steps and
This is a method for efficiently inspecting structures such as bridges using an image processing system characterized by being composed of. "
(9) In a method for efficiently inspecting a structure such as a bridge using the image processing system of the present invention, there are four laser pointers in the photographing apparatus, and each of these laser pointers is the light of the camera. It may be arranged in parallel with the axis and may be capable of emitting a laser beam so that an irradiation point can be obtained at a position corresponding to the four diagonal corners of the detailed image data.

(10) Further, the present invention can be used when "a method for efficiently inspecting a structure such as a bridge using an image processing system is executed by a data processing device such as a computer, a tablet, or a mobile phone. It ’s a program that you can do
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. As well as being able to get
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the specific partial detailed image data are calculated, and the distance data, the tilt data, and the like. And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. It is a program that can be done.
(11) The program of the present invention automatically detects and displays a candidate for a damaged portion of a structure in the wide area front image data, a process for accepting deletion or correction of the candidate for the damaged portion, and the damaged portion. It may further include a process of learning from the history of deletion or modification of the candidate and then reflecting it when automatically detecting the candidate of the damaged part of the structure.

(1) Since the present invention configured as described above uses an imaging device in which the laser light from each laser pointer and the optical axis of the camera are fixed, the coordinate positions of each irradiation point by the laser pointer and the imaging device. From the relative relationship with the coordinate position of each irradiation point by the laser pointer in the image data taken in the state where the positional relationship with the target is specified in advance using, the distance data from the camera to the structure and the camera 3 Tilt data in the dimensional direction can be calculated. In this case, the distance data and the tilt data can be calculated by using the relative relationship between the three irradiation points. Therefore, when four or more laser pointers are adopted, at least three laser pointers are used. Image processing can be performed if the irradiation point is clear.
Such image processing can be performed by using the invention disclosed in Japanese Patent Application No. 2018-30606.
Furthermore, since multiple image data in different states obtained by shooting from a fixed position are rearranged and integrated at the correct position corresponding to the subject, even if there is no overlapping part in the adjacent image data. , Multiple images can be integrated.
Since the present invention can construct a wide-range front image of a structure from a plurality of image data obtained by shooting from a fixed position, it is not essential to shoot while moving the shooting device. It is possible to easily, quickly, accurately and at low cost to inspect structures existing in sites that are difficult to access.
Further, in the present invention, by a method using two wide and narrow image data sets, a wide area front image data of a structure can be quickly obtained with a small number of images, and a wide area front image data of the structure can be quickly obtained. In addition, a plurality of specific parts of a structure (specifically, a damaged part visually confirmed and a part requiring particularly detailed image data) are combined with a plurality of specific parts. Can be integrated to obtain partially detailed wide-area frontal image data of structures with a unified scale scale.
Therefore, in the present invention, it is not necessary to photograph the structure from the front, it is not necessary to change the photographing location, and it is not necessary to photograph a wide range of the structure in detail without omission, and the quality of the inspection data is not required. It is possible to process the inspection work extremely efficiently by reducing the number of shots without reducing the number of images.

(2) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, there are four laser pointers in the photographing apparatus, and each of these laser pointers is parallel to the optical axis of the camera. When it is arranged and the laser beam can be emitted so that the irradiation point can be obtained at the position corresponding to the four corners on the diagonal line of the detailed image data, one irradiation by the laser pointer is performed. Subsequent image processing can be performed even if the points are unclear, and it is possible to easily confirm that the shooting range in the image data is appropriate, and individual shooting work can be performed quickly. Not only can it be done, but also the image processing work after shooting can be carried out quickly and appropriately.
(3) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device automatically detects and displays a candidate for a damaged portion of the structure in the wide area front image data. If the damaged part candidate can be deleted or corrected, the damaged part in the structure can be quickly picked up, and the damaged part in the structure can be accurately data by visual confirmation at the site. It is possible to improve the reliability of the inspection.
(4) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device learns from the history of deletion or correction of a candidate for a damaged portion, and then damages the structure. If it can be reflected in the automatic detection of the candidate part, it is possible to improve the accuracy of picking up the damaged part in the structure and customize the property of the detection target, and the candidate of the damaged part. It is possible to reduce or improve the efficiency of the work of deleting or modifying the system, and to improve the reliability, convenience and versatility of the system.

(5) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing device is composed of a host computer and a local terminal device connected to the host computer. , The host computer learns from the coarse wide area front image data of the structure, the generation of the partially detailed wide area front image data of the structure, and the history of deletion or correction of the candidate of the damaged part. If a process that can be reflected when automatically detecting a candidate for a damaged part of a structure can be performed and the candidate for the damaged part can be deleted or corrected by the local terminal device, it is visually observed at the site. By checking, the damaged part in the structure can be accurately converted into data, the burden on the local terminal device and the load on data communication can be reduced, and the efficiency and convenience of data processing work can be improved. ..
(6) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing apparatus has a length of the damaged portion of the structure detected in the wide area front image data. When information such as width can be added and recorded, information on the state of the damaged part can be easily confirmed at a glance, and an extremely convenient system can be provided.
(7) In the image processing system for efficiently inspecting a structure such as a bridge of the present invention, the data processing apparatus obtains image data obtained in the past for the same structure and newly obtained image data. If it is possible to compare and contrast and confirm the change over time in the damaged part of the structure, it is possible to provide an extremely useful database that can contribute to the planning and execution of repair work. ..

(8) In the method for efficiently inspecting a structure such as a bridge using the image processing system of the present invention, the damaged part of the structure found visually is determined to be a specific part, and the image is taken from the same position. By inputting a plurality of specific part detailed image data separately captured by the apparatus into the image processing system, a partially detailed wide area front image of the structure having a unified scale scale is obtained. Since the data can be obtained, it is possible to efficiently perform the photographing work for the damaged part that requires a detailed image of the structure, and it is easy to confirm the position where the damaged part exists in the structure.
(9) In a method for efficiently inspecting a structure such as a bridge using the image processing system of the present invention, there are four laser pointers in the photographing apparatus, and each of these laser pointers is the optical axis of the camera. If they are arranged in parallel and can emit laser light so that irradiation points can be obtained at positions corresponding to the four diagonal corners of the detailed image data, one laser pointer is used. Even if the irradiation point of the laser is unclear, the subsequent image processing can be performed, and further, it is possible to easily confirm that the shooting range in the image data is appropriate, and the individual shooting work can be performed quickly. Not only can it be done quickly, but also the image processing work after shooting can be done quickly and appropriately.
(10) In a program that can be used when a method for efficiently inspecting a structure such as a bridge using the image processing system of the present invention is executed by a data processing device such as a computer, a tablet, or a mobile phone. A wide-area front image data of a structure can be obtained from two wide and narrow image data sets by a simple method, and a wide-area front image of the structure is partially detailed by combining a plurality of specific partial detailed image data. Since image data can be obtained, it is possible to maintain the quality of inspection and make the inspection work of the structure quick and easy even with a small number of photographs taken from a fixed position.
(11) In a program that can be used when a method for efficiently inspecting a structure such as a bridge using the image processing system of the present invention is executed by a data processing device such as a computer, a tablet, or a mobile phone. From the process of automatically detecting and displaying the candidate of the damaged part of the structure in the wide area front image data, the process of accepting the deletion or correction of the candidate of the damaged part, and the history of the deletion or correction of the candidate of the damaged part. If it further includes a process to learn and then reflect when automatically detecting a candidate for a damaged part of the structure, improve the accuracy of picking up the damaged part in the structure or customize the properties of the detection target. It is possible to reduce or improve the efficiency of the work of deleting or correcting the candidate of the damaged part, and to improve the reliability, convenience and versatility of the system.

For the purpose of inspecting structures such as bridges, it is not necessary to strictly specify the positions of cracks and damages in the image processing system, image processing method and program of the present invention. The “fixed position” for photographing a structure is not limited to the case where the photographing device is supported by a tripod, but also includes the case where the image is taken by hand.
Therefore, the image processing system, the image processing method, and the program of the present invention are extremely easy to handle and easy to use when inspecting a structure such as a bridge.

FIG. 1 is a diagram showing an image processing system and various image data for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 2 is a perspective view showing an outline of an embodiment of an image processing system for efficiently inspecting a structure such as a bridge embodying the present invention, and a plan view showing a state in which a bridge is photographed by an imaging device. .. FIG. 3 is a front view showing an imaging device of an image processing system for efficiently inspecting a structure such as a bridge embodying the present invention and an irradiation point by a laser pointer. FIG. 4 is a side view showing a state in which a bridge is photographed by an imaging device of an image processing system for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 5 is a perspective view showing another embodiment of an image processing system for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 6 is a diagram for explaining calculation processing when performing position / orientation estimation processing of a camera by an image processing system for efficiently inspecting a structure such as a bridge embodying the present invention. FIG. 7 is a flowchart showing a procedure of an image processing method for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 8 is a flowchart showing a procedure for performing imaging after selecting an appropriate imaging position in an image processing method for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 9 is a flowchart showing a procedure of position / orientation estimation processing with improved reliability in an image processing method for efficiently inspecting a structure such as a bridge embodying the present invention. FIG. 10 is a flowchart showing a procedure for processing a specific partial detailed image in an image processing method for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 11 is a flowchart showing a procedure when there are three laser pointers in an image processing method for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 12 is a diagram for explaining the resolution of an image obtained in an image processing method for efficiently inspecting a structure such as a bridge that embodies the present invention. FIG. 13 is a front view showing a board used for adjusting and calibrating an imaging device that embodies the present invention. FIG. 14 is a diagram showing a wide range of detailed frontal images of a structure obtained by an image processing method for inspecting a structure such as a bridge different from the present invention.

The present invention is "an image processing system for efficiently inspecting structures such as bridges, which includes a camera, at least three laser pointers, and a fixing means capable of integrally holding them. It is provided with a photographing device comprising a photographing device and a data processing device for processing image data obtained by the photographing device.
Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and direction by the photographing apparatus, and the laser pointer. The detailed image data obtained by photographing the central portion of the partial image data so that at least three irradiation points are captured, and the two wide and narrow image data sets obtained from the above, and a part of the structure from the same position are different. The other wide and narrow image data sets obtained from the other partial image data obtained by photographing the portion and the other detailed image data corresponding to the other partial image data, and the photographing device from the same position. It is an image processing system that operates based on a plurality of specific part detailed image data obtained by separately photographing a specific part of a structure.
The data processing device calculates distance data from at least three irradiation points by the laser pointer to the structure from the camera and tilt data in the three-dimensional direction of the camera in the detailed image data, and the distance data, A structure in which a plurality of partial image data are combined and integrated based on the inclination data and the magnification of the partial image data and the detailed image data in each of the two wide and narrow image data sets, and the scale of the scale is unified. Coarse wide area front image data can be obtained, and distance data from at least three irradiation points by the laser pointer to the structure and tilt data of the camera in the three-dimensional direction in the specific part detailed image data can be obtained. Is calculated, and based on the distance data, the tilt data, and the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated, and the scale of the structure is unified. It is an image processing system for efficiently inspecting structures such as bridges, which is characterized by being able to obtain detailed wide-area frontal image data.

Further, the present invention is "a method for efficiently inspecting a structure such as a bridge using an image processing system, and can integrally hold a camera, at least three laser pointers, and the like. An image processing system including a photographing device including a fixing means, a photographing device including the photographing device, and a data processing device for processing image data obtained by the photographing device is used.
1) Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and the same direction by the photographing apparatus. Detailed image data obtained by photographing the central part of the partial image data so that at least three irradiation points by the laser pointer are captured, two wide and narrow image data sets obtained from the same, and a part of the structure from the same position. The image processing system comprises another partial image data obtained by photographing different parts and two other wide and narrow image data sets obtained from the other partial image data and other detailed image data corresponding to the other partial image data. By typing in
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. Steps to get and
2) The damaged part of the structure found visually is determined to be a specific part, and a plurality of specific part detailed image data in which the specific part of the structure is separately photographed from the same position by the photographing apparatus is applied to the image processing system. By inputting
The distance data from at least three irradiation points by the laser pointer to the structure and the tilt data in the three-dimensional direction of the camera in the specific part detailed image data are calculated, and the distance data, the tilt data, And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. Steps and
This is a method for efficiently inspecting structures such as bridges using an image processing system characterized by being composed of. "
Further, the present invention is "a program for causing a data processing device such as a computer, a tablet, or a mobile phone to execute an image processing method for efficiently inspecting a structure as described above".
The present invention can be suitably embodied by the embodiments described below.

In the present invention, the "front image" is an image seen from the horizontal direction of the vertical wall surface of the structure, and an image seen from directly below the horizontal bottom surface and the ceiling surface of the structure. The horizontal floor surface and upper surface of the structure are images viewed from directly above.

Hereinafter, an embodiment of an image processing system 1 for efficiently inspecting a structure such as a bridge that embodies the present invention will be described.
As shown in FIGS. 1 and 2, the image processing system 1 for inspecting a structure 2 such as a bridge according to an embodiment of the present invention includes a photographing device 10 and a data processing device 20.
The photographing device 10 is composed of one camera 11, four laser pointers Lp1 to Lp4, and a fixing means 12 capable of integrally holding them.

As the camera 11, a digital camera having about 10 to 40 million pixels can be preferably used, but the image resolution required for inspection of the structure 2 may be low, or the structure 2 may be used. When a plurality of image data can be obtained from a small and close place, a digital camera or a mobile phone 22 that can obtain an image resolution of about several million pixels may be used.
Further, the camera 11 preferably has a zoom function capable of shooting at different magnifying powers, but the present invention is embodied even if image data having different magnifying powers can be obtained by exchanging lenses. Can be carried out. However, the work efficiency will decrease.
If the focal distance information, that is, the data for calculating the magnification of each image data, is not automatically recorded in the data of each captured image, the focus of the data of each captured image at the time of shooting is separately recorded. It is necessary to record the distance and input it to the data processing device 20.
When the magnification is constant, such as when changing lenses or when shooting at both ends of the zoomable range, the process of inputting the focal length of each image into the data processing device 20 should be performed easily. Can be done.
The camera 11 has the number of effective pixels and optics of the camera 11 according to the range of pixel resolution (actual size per pixel) of the image captured by the camera 11 and the distance to the structure 2 at the time of imaging. The specifications of the camera 11 such as the zoom magnification can be selected.
Further, the camera 11 can confirm the shooting location and focusing from a remote location by a monitor connected wirelessly or by wire, and is relative to the structure 2 obtained by the position / orientation estimation process described later. The zoom mechanism and shutter of the camera 11 may be remotely controlled so that an image having a unified scale scale can be obtained from the positional relationship.
Further, the camera 11 may be a digital camera having a function of shooting a moving image, a mobile phone, or a video camera 11a as shown in FIG.
When the moving image data of the structure 2 is obtained by the video camera 11a, appropriate image data in the moving image data may be extracted and selected, and the subsequent image processing may be performed.

Further, in the present invention, the camera 11 preferably uses a zoom function to magnify and photograph a part of the structure at the same position and in the same direction as the partial image data obtained by photographing a part of the structure 2. Two wide and narrow image data sets can be obtained from the detailed image data obtained by photographing the central portion of the partial image data so that at least three irradiation points by the laser pointers Lp1 to Lp4 are captured. It needs to be a thing.
The practically preferable range of the magnification of the partial image data and the detailed image data constituting the two wide and narrow image data sets in the present invention is about 2 to 5 times, which is used when carrying out the present invention. The zoom magnification of the camera may be about 5 to 10 times.
However, the magnification of the partial image data and the detailed image data constituting the two wide and narrow image data sets in the present invention may be about 10 times in principle, although the accuracy may decrease.

The fixing means 12 uses the four laser pointers Lp1 to Lp4 so that the laser light from the four laser pointers Lp1 to Lp4 is irradiated to the vicinity of the four corners of the detailed image captured by the camera 11. It can be arranged and fixed in parallel with the optical axis of the camera 11 and on a diagonal line centered on the optical axis.
The fixing means 12 can be made of a rigid material such as synthetic resin, light metal, or wood, so that the positional relationship between the optical axis of the camera 11 and the optical axes of the laser pointers Lp1 to Lp4 does not always get out of order. It suffices if they can be held in a square, and the shape may be a rectangle, a square, another quadrangle, an X-shape, a circle, a polygon, a triangle, a V-shape, an A-shape or the like, or a plate-like shape as appropriate. It is selectable.
Further, in order to obtain a plurality of image data from a fixed position by the photographing device 10, a tripod 13 may be attached to the fixing means 12 and an image may be taken by the camera 11.

In the present invention, each laser pointer Lp1 to Lp4 can preferably be used with a laser pointer of about 1 mW capable of irradiating an object with a distance of several meters to 20 meters with a laser beam, and a green or red pointer having excellent visibility is preferable. ..
Further, the shape of the bright spots of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 is preferably a perfect circle and a luminance distribution that follows a normal distribution from the center.
As an example, the diameter of the bright spot can be about 4 mm at a distance of 4 m.
In order to confirm the fixed state of the laser pointers Lp1 to Lp4 by the photographing apparatus 10 used in the present invention and adjust the laser pointers Lp1 to Lp4 so as to be fixed at the correct position, the laser pointers Lp1 to Lp1 in the captured image are shown on the left side of FIG. It is preferable to use the adjustment board 30 marked with a mark indicating the ideal irradiation position of Lp4.
Further, as the photographing apparatus 10 used in the present invention, as shown in the lower part of FIG. 3, eight laser pointers Lp1 to Lp8 may be arranged diagonally. By increasing the number of laser pointers in this way, the choice of irradiation points by the laser pointers in the image processing described later increases, and the accuracy of the image processing can be improved.

As the data processing device 20, a notebook personal computer 21 capable of reading image data obtained by photographing a structure 2 with a camera 11 via a recording medium or wirelessly or by wire is preferably used. can do.
Further, the data processing device 20 may be configured such that the host computer 23 and the mobile phone 22 or the tablet terminal 24 connected to the host computer 23 are local terminal devices.
In this case, as will be described later, the host computer 23 generates coarse wide-area front image data of the structure and partially detailed wide-area front image data of the structure, and deletes or deletes candidates for damaged parts. It is possible to learn from the repair history and reflect it when the candidate for the damaged part of the structure is automatically detected next, and the candidate for the damaged part can be deleted or corrected by the local terminal device. Can be done. In that case, the damaged part in the structure can be accurately converted into data by visual confirmation at the site, and the burden on the local terminal device and the load on data communication can be reduced to improve the efficiency and convenience of data processing work. It is possible to improve the sex.

Next, the data processing device 20 will be described.
The data processing device 20 enlarges and photographs a part of the structure 2 at the same position and in the same direction as the partial image data obtained by photographing a part of the structure 2 by the photographing apparatus 10. The detailed image data obtained by photographing the central portion of the partial image data so that at least three irradiation points by the laser pointer are captured, and the two wide and narrow image data sets obtained from the same . Two other wide and narrow images obtained from other partial image data obtained by photographing a part different from a part of the structure 2 from a position and other detailed image data corresponding to the other partial image data. It operates based on an image data set and a plurality of specific part detailed image data in which a specific part of the structure 2 is separately photographed by the photographing apparatus 10 from the same position.
Specifically, the data processing device 20 includes distance data from at least three irradiation points by the laser pointers Lp1 to Lp4 in the detailed image data to the structure 2 from the camera 11 and a three-dimensional direction of the camera 11. The tilt data in the above should be calculated, and a plurality of partial image data should be combined based on the distance data, the tilt data, and the magnification of the partial image data and the detailed image data in each of the two wide and narrow image data sets. Coarse wide area front image data of the structure with unified scale can be obtained by rearranging and integrating in the correct position, and at least three by the laser pointers Lp1 to Lp4 in the specific partial detailed image data. The distance data from the irradiation point of the camera 11 to the structure 2 and the tilt data of the camera 11 in the three-dimensional direction are calculated, and based on the magnification of the distance data, the tilt data, and the detailed image data of each specific portion. Therefore, a plurality of specific partial detailed image data can be combined, rearranged and integrated to obtain partially detailed wide area front image data of a structure having a unified scale scale.

More specifically, the data processing device 20 is
(A) In the detailed image data of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4, for each of the detailed image data obtained by photographing a part of the structure 2 from a fixed position by the photographing apparatus 10. Irradiation point extraction process to extract the coordinate position in
(A) For each of the detailed image data, the positional relationship between the coordinate positions of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 obtained by the irradiation point extraction process and the target in advance using the photographing apparatus. From the relative relationship with the coordinate position of each irradiation point by the laser pointers Lp1 to Lp4 in the image data taken in the specified state, the distance data from the camera 11 to the structure 2 and the three dimensions of the camera 11 Position / orientation estimation processing that calculates tilt data in the direction,
(C) For each of the partial image data and the detailed image data in the two wide and narrow image data sets, the distance data, the inclination data, and the partial in the two wide and narrow image data sets obtained by the position and orientation estimation process. Front image acquisition processing that converts to front image data of structure 2 with a unified scale scale based on the magnification of image data and detailed image data, and
(D) The plurality of front image data obtained by the front image acquisition process are combined with the distance data, the inclination data, and the partial image data in each of the two wide and narrow image data sets obtained by the position and orientation estimation process. Image integration processing to obtain coarse wide-area front image data of a structure with a unified scale scale by combining multiple partial image data, rearranging and integrating them based on the magnification of detailed image data.
Can be done.
Here, the "magnification magnification" of the partial image data and the detailed image data is preferably calculated from the focal length of the camera 11 when each image is taken.

Further, in the present invention, the data processing device 20 is
(E) The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the specific part detailed image data are calculated, and each specific part detailed image. It is possible to combine a plurality of specific partial detailed image data based on the magnification of the data, rearrange them, and integrate them to obtain a partially detailed wide area front image data of a structure having a unified scale scale. It is a thing.

Further, in the present invention, the data processing device 20 can automatically detect and display a candidate for a damaged portion.
For example, damaged parts such as cracks and holes are displayed blacker than the surroundings in the image data, so they are automatically picked up as damaged parts and displayed in vivid colors to indicate the existing location. Can be clarified.
Further, in the present invention, the data processing device 20 can accept a process of deleting or correcting an automatically detected candidate for a damaged portion by visually confirming an actual damaged portion of the structure at the site. Such modifications include the process of entering and recording data such as length, width and type of damaged portion (see wide area frontal image data at the bottom of FIG. 1).
Further, in the present invention, the data processing device 20 learns from the history of the process of deleting or correcting the candidate of the damaged portion, and then reflects it when automatically detecting and displaying the candidate of the damaged portion. Can be done.

Further, in the present invention, the data processing device 20 can display and record the damaged portion to be added in the front image data by a sketch operation with a finger, a touch pen, a mouse, or the like by visually checking at the site. It may be possible (see the wide area front image data described at the bottom of FIG. 1).
Further, in the present invention, the data processing device 20 can compare and contrast the image data obtained in the past with respect to the same structure 2 and the newly obtained image data, and the damaged portion in the structure 2 can be compared and contrasted with time. It may be something that can confirm the change.

In the position / orientation estimation process, the data processing device 20 sets the coordinate positions of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 obtained by the irradiation point extraction process for each of the plurality of image data. Position / orientation estimation processing for calculating the distance data from the camera 11 to the structure 2 and the tilt data of the camera 11 in the three-dimensional direction from the arrangement position data of the camera 11 and the laser pointers Lp1 to Lp4 in the photographing device 10. It may be something that does.
Here, in the position / orientation estimation process by the data processing device 20, if the coordinate positions of three irradiation points among the irradiation points P1 to P4 of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 can be specified, the structure 2 can be specified from the camera 11. It is possible to calculate the distance data up to and the tilt data of the camera in the three-dimensional direction.
Therefore, even if any one of the irradiation points P1 to P4 is unclear and difficult to discriminate, the subsequent image processing can be performed.

Further, in the position / orientation estimation process, the data processing device 20 obtains the coordinate positions of the four irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 obtained by the irradiation point extraction process, and the photographing device 10. It is selected from the correspondence of four points with the coordinate positions of the four irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 in the image data taken in the state where the positional relationship with the target is specified in advance. By calculating the distance data and the inclination data from two or more combinations of correspondences of three different points, the accuracy of the data may be improved.

Further, in a preferred embodiment of the present invention, the four irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 should be arranged on the planes facing each other, to be exact, on the vertices of a rectangle or a square. In the position / orientation estimation process, the data processing device 20 uses the photographing device 10 to form a plane facing from different distances in advance, preferably a 10 cm square as shown on the right side of FIG. Distance data and tilt data are calculated based on a plurality of image data obtained by photographing the calibration boards 31 arranged in a pattern, and the deviation of the optical axes of the camera 11 and the four laser pointers Lp1 to Lp4 is detected. Therefore, it is preferable that the coordinate positions of the irradiation points P1 to P4 in the image data of the structure 2 can be corrected.

Further, in a preferred embodiment of the present invention, the data processing apparatus 20 has damage such as cracks 5 or peeling caused in the structure 2 from a wide range front image of the structure 2 obtained by the image integration process. It is possible to perform the measurement processing of.
In such measurement processing, the coordinate position of damage (distance between irradiation points P1 to P4) in each image data is determined from the distance data between the camera 11 and the object, the tilt data of the camera 11, and the resolution of the image data. It is possible because it can be specified.
As described above, the damaged portion is automatically detected and displayed as a candidate for the damaged portion, and the candidates are visually confirmed at the site, and the data processing device 20 at the site, preferably a mobile phone, is used. Alternatively, more accurate inspection result data can be obtained by adding, deleting or modifying with a local terminal device such as a tablet.

Next, an image processing method for inspecting a structure 2 such as a bridge will be described using the "image processing system 1 for efficiently inspecting a structure such as a bridge" of the present invention described above. ..
The image processing method for inspecting a structure 2 such as a bridge of the present invention preferably has a pixel resolution (actual size per pixel) of an image taken by the camera 11 of 0.1 mm / pixel or more. A structure capable of confirming the existence and position of fine cracks 5 generated in the structure 2 from a wide range front image of the structure 2 using an image processing system 1 within a range of 0.3 mm / pixel. 2 is an image processing method for inspecting, and as shown in FIG. 7, it is composed of the following steps S1 to S7.

Partial image data and a central portion thereof are obtained by photographing a part of the structure 2 in the same direction from a fixed position away from the structure 2 by the photographing apparatus 10 having the camera 11 and four laser pointers Lp1 to Lp4. Step S1 (image data set shooting) for acquiring two wide and narrow image data sets of detailed image data taken.
The wide and narrow of the partial image data and the detailed image data obtained by photographing the central portion thereof by photographing the other part of the structure 2 in the same direction from the same fixed position away from the structure 2 by the photographing apparatus 10. Step S2 (taking another image data set) to acquire two image data sets.
Step S3 (step S3) of extracting the coordinate positions in the image data of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 for each of the plurality of detailed image data by the irradiation point extraction process by the data processing device 20. Irradiation point extraction process).
For each of the plurality of detailed image data by the position / orientation estimation process by the data processing device 20, the coordinate positions of the irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 obtained by the irradiation point extraction process and the coordinate positions. From the camera 11 based on the relative relationship with the coordinate position of each irradiation point by the laser pointers Lp1 to Lp4 in the image data taken in a state where the positional relationship with the target is specified in advance using the photographing device 10. Step S4 (position / orientation estimation process) for calculating the distance data to the structure 2 and the tilt data of the camera 11 in the three-dimensional direction.

By the front image acquisition process by the data processing device 20, each of the plurality of partial image data is rearranged and integrated based on the distance data, the tilt data and the enlargement magnification obtained by the position / orientation estimation process. Step S5 (coarse wide area front image acquisition process) for converting the structure 2 into coarse wide area front image data having a unified scale.
As shown in another routine shown in FIG. 10, the data processing apparatus is used for detailed image data of a specific part obtained by taking a damaged part in a structure visually confirmed at the site as a specific part and taking the specific part from the same fixed position as described above. The structure 2 is partially detailed by rearranging it based on the distance data, the inclination data, and the magnification obtained by the position / orientation estimation process performed by 20 and integrating it with the coarse wide area front image data of the structure 2. Step S6 (partially detailed wide area front image acquisition process) for constructing the wide area front image.
From the partially refined wide area front image of the structure 2 obtained by the partially refined wide area front image acquisition process, the damaged part such as the crack 5 or the peeling generated in the structure 2 is automatically removed. Step S7 (information addition processing of the damaged part) in which information on the damaged part such as the width and length of cracks and the like occurring in the structure 2 is added and recorded while being detected and displayed.

In the image processing method for efficiently inspecting the structure of the present invention, the position / orientation estimation process by the data processing device is performed on a plane facing from different distances in advance using the photographing device 10, preferably. As shown on the right side of FIG. 13, distance data and tilt data are calculated based on a plurality of image data obtained by photographing a calibration board 31 in which 10 cm square squares are arranged in a checkered pattern, and the cameras 11 and 4 are used. It may further include step S0 (calibration) for detecting the deviation of the optical axes of the laser pointers Lp1 to Lp4. Thereby, the coordinate positions of the irradiation points P1 to P4 in the image data of the structure 2 can be corrected.
Further, in the image processing method for efficiently inspecting the structure of the present invention, the range of the structure 2 photographed from the photographing position is set up, down, left and right of the optical axis of the camera 11 before the photographing step S1. It is preferable that the step S1-0 for selecting a fixed position away from the structure 2 so as to be within the range of 60 degrees at the maximum is further included (see FIG. 8).
In the image processing method for efficiently inspecting the structure of the present invention, the shooting position by the camera 11 is not limited to one place, and when the structure 2 is long in the horizontal direction or the vertical direction, it is not limited to one position. It is also possible to construct a wider frontal image of the structure 2 based on the moving distance at that time or by collating the feature points in the image data by taking a plurality of pictures from a plurality of places.

Further, in the image processing method for efficiently inspecting the structure of the present invention, four irradiations by the laser pointers Lp1 to Lp4 obtained by the irradiation point extraction processing in step S4 of the position / orientation estimation processing are performed. The four irradiation points P1 to P4 by the laser pointers Lp1 to Lp4 in the image data taken in a state where the positional relationship between the coordinate positions of the points P1 to P4 and the target is specified in advance using the photographing device 10. Step S3 that can improve the accuracy of the data by calculating the distance data and the inclination data from two or more combinations of the correspondences of three different points selected from the correspondences of the four points with the coordinate positions. -1 may be adopted (see FIG. 9).

ｉｉ）回転と移動による距離変化により、

Next, a specific procedure (calculation processing theory) of the position / orientation estimation process by the data processing device 20 will be described below.
As shown in FIG. 6, the image data (captured image) of the structure 2 captured by the camera 11 is rotated in the X-axis (horizontal direction) and the Y-axis (vertical direction) from the image captured in the reference state. , Z-axis (front-back direction) is considered to be moving.
Therefore,
i) From the relationship between the bright spot coordinates in the reference state and the bright spot coordinates after rotation / movement

ii) Due to the change in distance due to rotation and movement

が成り立つので、以下のパラメータが既知であれば、方程式を解くことができる。

の「輝点座標のペア（組み合わせ）が３点以上」
「各レーザーポインタの傾き」：

「基準状態での撮影装置と壁面との距離」：Ｚ
「撮影時の焦点距離」：

From i) and ii) above, the equation for distance

Therefore, if the following parameters are known, the equation can be solved.

"There are 3 or more pairs (combinations) of bright spot coordinates"
"Tilt of each laser pointer":

"Distance between the shooting device and the wall surface in the standard state": Z
"Focal length at the time of shooting":

Here, an embodiment of the image processing system 1 for efficiently inspecting the structure 2 such as a bridge of the present invention will be described (see FIG. 2).
<Structure to be inspected and degree of damage>
When performing image processing for inspection to detect cracks having a width of 0.1 mm or more on the concrete wall surface 3 of a bridge That is, an important element in this system is image resolution, and resolution (front facing and flat surface). When shooting),
-Total number of pixels of the camera [pix]
・ Camera sensor size [mm]
・ Distance to the wall surface at the time of shooting [m]
・ Focal length (zoom value) [mm]
It is determined by the four parameters of.
Then, while considering these values, it is necessary to appropriately select the specifications of the camera to be used and the enlargement magnification of the partial image data and the detailed image data constituting the two wide and narrow image data sets.
<Camera to use>
Sony digital camera: Cyber-shot DSC-RX10M4
Resolution: 5472 x 3648
Optical zoom magnification: Up to 25x The focal length at the time of shooting is recorded in the image data (in Exif).
<Detailed image specifications>
Image resolution: 0.2 mm / pix
Maximum shooting range: 1.1m x 0.7m
Maximum shooting distance: 0.73m (18.25m when using 25x zoom)
<Laser pointer>
Placed at the apex of a rectangle 48 cm in the horizontal direction and 32 cm in the vertical direction The camera is fixed so that the optical axis is placed at the center position of the rectangle (this placement is the placement position data used in the position / orientation estimation process. )
<About the magnification of partial image data and detailed image data that make up two wide and narrow image data sets>
As already described, in actual operation, from the viewpoint of work efficiency and processing accuracy, the enlargement magnification is preferably about 2 to 5 times.
<Number of partial images taken>
When shooting with 30% overlap to facilitate image integration by setting the magnification of the partial image data and detailed image data that make up the two wide and narrow image data sets to 3 (wall width / 2.4) x (Height of wall surface / 1.5) However, in order to take a picture of the entire wall surface without omission, the number after the decimal point is rounded up.

In the present invention, the "data processing device 20" that performs irradiation point extraction processing, position / orientation estimation processing, frontal image acquisition processing, image integration processing, and crack measurement processing is a PC (computer), a tablet, and a mobile phone (smartphone). The program of the present invention operates in those data processing devices 20, and the image processing method of the present invention (data processing in each step described in the present specification) is applied to those data processing devices. It is something that can be done.
Here, the individual processes of the irradiation point extraction process, the front image acquisition process, and the crack measurement process can be easily understood and processed by those skilled in the art by using conventionally known image processing methods and programs. A detailed description will be omitted.

INDUSTRIAL APPLICABILITY According to the present invention, in addition to bridges, image processing for inspecting columnar objects such as embankments, dams, roads, tunnels, buildings, and piers, wall surfaces 3, floor surfaces, bottom surfaces 4, and ceiling surface surfaces of other structures 2. It can be used as a system, and is not limited to inspection of vertical or horizontal planes, and image data for inclined planes can also be rearranged and integrated as front images, such as bridges. An image processing system, an image processing method, and a program for efficiently inspecting the structure of the above.

The present invention may be carried out by appropriately changing the shape, dimensions, angles, installation positions, sizes and the like of each part of the photographing apparatus 10 used in the image processing system 1 without departing from the gist of the present invention.
As an example, the image processing system 1 for inspecting the structure of the present invention includes a camera 11, four laser pointers Lp1 to Lp4, and a fixing means capable of integrally holding them. In the apparatus, the fixing means constituting the photographing apparatus is four so that the laser light from the four laser pointers Lp1 to Lp4 is irradiated to the vicinity of the four corners of the image captured by the camera 11. The laser pointers Lp1 to Lp4 may be arranged and fixed so as to "expand" on the diagonal line about the optical axis of the camera 11.
Here, "expanding" means that each laser beam from the laser pointers Lp1 to Lp4 spreads at a certain angle with respect to the optical axis of the camera 11.
In this embodiment, the photographing apparatus can be miniaturized.

Further, in the embodiment shown in FIG. 5 of the present invention, a video camera 11a as a camera and a photographing device 10 provided with fixing means (not shown) for holding three laser pointers Lp1 to Lp3 are used.
In this embodiment, three fixing means constituting the photographing device 10 are provided so that the laser light from the three laser pointers Lp1 to Lp3 is irradiated within the range of the image photographed by the camera 11a. Laser pointers Lp1 to Lp3 can be arranged and fixed on the apex of a triangle arranged parallel to the optical axis of the camera 11a and around the optical axis.
The arrangement of the three laser pointers Lp1 to Lp3 is preferably on the apex of an equilateral triangle having a base along the horizontal direction as a preferred embodiment.
In the case of this embodiment, one embodiment of the image processing method for inspecting the structure of the present invention comprises the following steps S10 to S17 as shown in FIG.

Distance data and tilt data are calculated based on a plurality of detailed image data obtained by photographing planes facing each other from different distances in advance using the photographing device 10, and the optical axes of the camera 11a and the three laser pointers Lp1 to Lp3. Step S10 (calibration) for detecting the deviation of.
Thereby, the coordinate positions of the irradiation points P1 to P3 in the detailed image data of the structure 2 can be corrected.
For such calibration, as an example, as shown on the right side of FIG. 13, a calibration board 31 in which 10 cm square squares are arranged in a checkered pattern can be preferably used.

Partial image data and a central portion thereof are obtained by photographing a part of the structure 2 in the same direction from a fixed position away from the structure 2 by the photographing apparatus 10 having the camera 11a and three laser pointers Lp1 to Lp3. Step S11 (image data set shooting) for acquiring two wide and narrow image data sets of detailed image data taken.
Partial image data by photographing the other part of the structure 2 in the same direction from the same fixed position as described above away from the structure 2 by the photographing apparatus 10 having the camera 11a and the three laser pointers Lp1 to Lp3. Step S12 (taking another image data set) to acquire two wide and narrow image data sets of detailed image data obtained by photographing the central portion thereof.
Step S13 (step S13) of extracting the coordinate positions in the image data of the irradiation points P1 to P3 by the laser pointers Lp1 to Lp3 for each of the plurality of detailed image data by the irradiation point extraction process by the data processing device 20. Irradiation point extraction process).

For each of the plurality of detailed image data by the position / orientation estimation process by the data processing device 20, the coordinate positions of the irradiation points P1 to P3 by the laser pointers Lp1 to Lp3 obtained by the irradiation point extraction process and the coordinate positions. The relative relationship with the coordinate position of each irradiation point by the laser pointers Lp1 to Lp3 in the image data taken in a state where the positional relationship with the target is specified in advance using the photographing device 10, or the camera in the photographing device. Step S14 (position / orientation estimation process) for calculating the distance data from the camera 11a to the structure 2 and the tilt data of the camera 11a in the three-dimensional direction from the arrangement position data of the laser pointers Lp1 to Lp3.

By the front image acquisition process by the data processing device 20, each of the plurality of partial image data is rearranged and integrated based on the distance data, the tilt data and the enlargement magnification obtained by the position / orientation estimation process. Step S15 (coarse wide area front image acquisition process) for converting the structure 2 into coarse wide area front image data having a unified scale.
As shown in another routine shown in FIG. 10, the data processing apparatus is used for detailed image data of a specific part obtained by taking a damaged part in a structure visually confirmed at the site as a specific part and taking the specific part from the same fixed position as described above. The structure 2 is partially detailed by rearranging it based on the distance data, the inclination data, and the magnification obtained by the position / orientation estimation process performed by 20 and integrating it with the coarse wide area front image data of the structure 2. Step S16 (partially detailed wide area front image acquisition process) for constructing the wide area front image.
From the partially refined wide area front image of the structure 2 obtained by the partially refined wide area front image acquisition process, the damaged part such as the crack 5 or the peeling generated in the structure 2 is automatically removed. Step S17 (information addition processing of the damaged part) of step S17 (information addition processing of the damaged part) in which information on the damaged part such as the width and length of cracks and the like occurring in the structure is added and recorded while being detected and displayed.

In the image processing method for efficiently inspecting the structure of the above embodiment of the present invention, since the photographing apparatus 10 having only three laser pointers Lp1 to Lp3 is used, it is possible to reduce the size and weight. It is possible to reduce the system construction cost.
Further, when the video camera 11a is used as the camera, the shooting work can be performed quickly and smoothly.
When the moving image data of the structure 2 is obtained by the video camera 11a, appropriate image data in the moving image data may be extracted and selected, and the subsequent image processing may be performed.

Further, in the image processing method of the present invention, the resolution of the obtained planar image of the structure 2 can be easily changed by using the zoom function of the camera 11.
Further, in order to partially increase the resolution, the zoom magnification may be increased and the positions of the irradiation points P1 to P4 may be as close as possible to the four corners of the photographing range.
On the contrary, if it is acceptable to increase the shooting efficiency and lower the resolution, the zoom magnification may be lowered to shoot a wide range of the structure 2. In this case, the irradiation points P1 to P4 are within the shooting range. It will be arranged near the center of the four corners (see FIG. 12).

Further, the present invention uses a mobile phone or tablet (PC) as the cameras 11 and 11a of the image processing system 1, and also uses the mobile phone or tablet (PC) as the data processing device 20 to obtain a photographing device 10. It is also possible to use the data processing device 20 in combination.
In that case, the image processing system for efficiently inspecting a structure such as a bridge of the present invention can be made smaller and lighter, and the image processing work can be smoothly performed.
Further, since the image processing system of the present invention can be provided with a communication function, data obtained at a remote site can be quickly transferred to a large monitor such as a business office or a place where a PC is located by wireless communication. You can also do it.

In the image processing system of the present invention, by tracing a crack on a PC screen with a mouse pointer, the length and position of the crack are specified, and the data of the length and position is partially detailed in a wide area front. It can also be recorded in association with image data.
In the image processing system of the present invention, the data processing device 20 automatically detects a candidate for a damaged portion of a structure in the coarse wide area front image data of the structure 2 or the partially detailed wide area front image data. In addition, the candidate for the damaged part can be deleted or corrected.
Further, in the image processing system of the present invention, the data processing device 20 may learn from the history of deletion or modification of the candidate for the damaged portion, and then reflect it when automatically detecting the candidate for the damaged portion of the structure. can.
In the image processing system of the present invention, the data processing device 20 is derived from a host computer 23 and a local terminal device (notebook personal computer 21, mobile phone 22, tablet terminal 24, etc.) connected to the host computer 23. Assuming that it is configured, the host computer 23 generates coarse wide-area front image data of the structure and partially detailed wide-area front image data of the structure, and deletes or corrects candidates for damaged parts. It is possible to perform a process of learning from the history of the above and reflecting it when the candidate of the damaged part of the structure is automatically detected next, and it is also possible to delete or correct the candidate of the damaged part by the local terminal device. can.
In the image processing system of the present invention, the data processing device 20 enlarges / reduces and displays an arbitrary part of the coarse wide area front image data of the structure and the partially detailed wide area front image data of the structure. By selecting or clicking a specific part, that part can be displayed as a separate screen.
Further, in the image processing system of the present invention, the data processing device 20 can display the image data obtained in the past for the same structure 2 and the newly obtained image data in comparison and contrast with each other. It is also possible to confirm the change over time in the damaged portion of the object 2.

The present invention is not limited to the embodiments and examples described above, and may be appropriately modified and embodied without departing from the spirit of the present invention.
For example, the detailed image data in each image data set may be included in the partially refined wide area front image data together with the specific partial detailed image data and rearranged at the correct position for integration. In that case, the detailed image data can be used without waste.
Further, the process of automatically detecting and displaying the candidate of the damaged part of the structure by the data processing apparatus of the present invention includes a coarse wide area front image data of the structure or a partially detailed wide area front image data of the structure. , Any image data may be used.

The present invention is excellent in reliability, operability, convenience, and versatility, and simplifies, speeds up, and improves efficiency of work and processing required for inspection of damaged parts of structures such as shooting work and image analysis work at low cost. It can be industrially and suitably used as an image processing system, an image processing method and a program for efficiently inspecting a structure such as a bridge.

1 Image processing system 2 Structure 3 Wall surface 4 Bottom surface 5 Crack 10 Imaging device 11 Camera 11a Video camera (camera)
12 Fixing means 13 Tripod 20 Data processing device 21 Notebook personal computer (data processing device)
22 Mobile phone (data processing device)
23 Host computer 24 Tablet terminal 30 Adjustment board 31 Calibration board Lp1 to Lp8 Laser pointer P1 to P4 Irradiation point

Claims (11)

An image processing system for efficiently inspecting structures such as bridges.
An imaging device consisting of a camera, at least three laser pointers, and a fixing means capable of holding them integrally.
A data processing device for processing image data obtained by the photographing device is provided.
Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and direction by the photographing apparatus, and the laser pointer. Detailed image data obtained by photographing the central part of the partial image data so that at least three irradiation points can be captured, and two wide and narrow image data sets obtained from the image data set.
Other wide and narrow 2 obtained from other partial image data obtained by photographing a part different from a part of the structure from the same position and other detailed image data corresponding to the other partial image data. With one image dataset,
A plurality of specific part detailed image data in which a specific part of a structure is separately photographed from the same position by the photographing apparatus, and
An image processing system that operates based on
The data processing device is
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. As well as being able to get
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the specific partial detailed image data are calculated, and the distance data, the tilt data, and the like. And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. An image processing system for efficiently inspecting structures such as bridges, which is characterized by being able to. There are four laser pointers in the photographing apparatus, and each of these laser pointers is arranged parallel to the optical axis of the camera and at positions corresponding to the four corners on the diagonal line of the detailed image data. The image processing system for efficiently inspecting a structure such as a bridge according to claim 1, wherein a laser beam can be emitted so that an irradiation point can be obtained. The data processing apparatus is characterized in that it can automatically detect and display a candidate for a damaged portion of a structure in the wide area front image data, and can delete or correct the candidate for the damaged portion. Or an image processing system for efficiently inspecting a structure such as a bridge according to 2. The data processing device is characterized in that it can learn from the history of deletion or modification of a candidate for a damaged portion and then reflect it when automatically detecting a candidate for a damaged portion of a structure. An image processing system for efficiently inspecting structures such as bridges according to 3. The data processing device is composed of a host computer and a local terminal device connected to the host computer.
The host computer learns from the coarse wide area front image data of the structure, the generation of the partially detailed wide area front image data of the structure, and the history of deletion or correction of the candidate of the damaged part, and then follows. It is possible to perform processing that reflects when automatically detecting a candidate for a damaged part of a structure.
The image processing system for efficiently inspecting a structure such as a bridge according to claim 4, wherein the candidate for the damaged portion can be deleted or corrected by the local terminal device. The data processing apparatus according to claim 1 to 5, wherein the data processing apparatus can add information such as the length and width to the damaged portion of the structure detected in the wide area front image data and record the data. An image processing system for efficiently inspecting structures such as bridges according to any one of the items. The data processing device can compare and contrast the image data obtained in the past for the same structure with the newly obtained image data, and can confirm the change with time of the damaged part in the structure. An image processing system for efficiently inspecting a structure such as a bridge according to any one of claims 1 to 6, which is a feature. It is a method to efficiently inspect structures such as bridges using an image processing system.
An imaging device consisting of a camera, at least three laser pointers, and a fixing means capable of holding them integrally.
Using an image processing system equipped with a data processing device that processes the image data obtained by the photographing device,
1) Partial image data obtained by photographing a part of the structure by the photographing apparatus and detailed image data obtained by magnifying a part of the structure in the same position and the same direction by the photographing apparatus. Detailed image data obtained by photographing the central part of the partial image data so that at least three irradiation points by the laser pointer are captured, and two wide and narrow image data sets obtained from the partial image data.
Other wide and narrow 2 obtained from other partial image data obtained by photographing a part different from a part of the structure from the same position and other detailed image data corresponding to the other partial image data. With one image dataset,
By inputting to the image processing system
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data are calculated, and the distance data, the tilt data, and individual data are calculated. Based on the magnification of the partial image data and the detailed image data in the two wide and narrow image data sets, a plurality of partial image data are combined and integrated to obtain a coarse wide area front image data of a structure having a unified scale. Steps to get and
2) The damaged part of the structure found visually is determined to be a specific part, and a plurality of specific part detailed image data in which the specific part of the structure is separately photographed from the same position by the photographing apparatus is applied to the image processing system. By inputting
The distance data from at least three irradiation points by the laser pointer to the structure and the tilt data in the three-dimensional direction of the camera in the specific part detailed image data are calculated, and the distance data, the tilt data, And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. Steps and
A method for efficiently inspecting structures such as bridges using an image processing system characterized by consisting of. There are four laser pointers in the photographing apparatus, and each of these laser pointers is arranged parallel to the optical axis of the camera and at positions corresponding to the four corners on the diagonal line of the detailed image data. A method for efficiently inspecting a structure such as a bridge by using the image processing system according to claim 8, wherein a laser beam can be emitted so that an irradiation point can be obtained. It can be used when the method for efficiently inspecting a structure such as a bridge by using the image processing system according to claim 8 is executed by a data processing device such as a computer, a tablet, or a mobile phone. It ’s a program,
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the detailed image data according to claim 8 are calculated, and the data is calculated. Based on the distance data, the tilt data, and the magnification of the partial image data and the detailed image data in each of the two wide and narrow image data sets, a plurality of partial image data were combined and integrated, and the scale of the scale was unified. It is possible to obtain coarse wide area front image data of the structure and at the same time.
The distance data from the camera to the structure and the tilt data in the three-dimensional direction of the camera from at least three irradiation points by the laser pointer in the specific partial detailed image data are calculated, and the distance data, the tilt data, and the like. And , based on the magnification of each specific part detailed image data, a plurality of specific part detailed image data are combined and integrated to obtain a partially detailed wide area front image data of a structure having a unified scale scale. A program characterized by being able to. A process of automatically detecting and displaying a candidate for a damaged part of a structure in the wide area front image data,
Processing to accept deletion or correction of the damaged part candidate,
A process of learning from the history of deletion or correction of the damaged part candidate, and then reflecting it when automatically detecting the damaged part candidate of the structure.
10. The program of claim 10, further comprising:

Images