KR102423561B1 - Intelligent dam management system based on digital twin - Google Patents

Abstract

The present invention relates to a dam safety management system, and more specifically, to a digital twin-based intelligent dam safety management system that can significantly improve speed, accuracy, and efficiency through technological advancement of dam infrastructure safety inspection and facility management. will be.
To this end, the present invention establishes a 3D dam modeling based on an image captured by at least one of a drone and a photographing means for a dam in which a measuring means is installed in a facility, and subdivides the 3D dam modeling for each management area unit, 3D a digital twin generation module for displaying at least one of facility information and measurement information of a measurement means on the dam modeling; The flight path information is stored for each management area unit and the image taken by a drone equipped with a thermal imaging camera is mapped to 3D dam modeling, and the 3D dam modeling before and after mapping is compared and expressed while leaking or leaking the dam through the thermal image orthographic image. a 3D modeling mapping module that enables confirmation of white changes; and a concrete crack identification module for determining concrete cracks by analyzing the dam concrete structure image taken by the drone in a deep learning method.

Description

Digital twin-based intelligent dam safety management system {INTELLIGENT DAM MANAGEMENT SYSTEM BASED ON DIGITAL TWIN}

The present invention relates to a dam safety management system, and more specifically, to a digital twin-based intelligent dam safety management system that can significantly improve speed, accuracy, and efficiency through technological advancement of dam infrastructure safety inspection and facility management. will be.

Korea's water infrastructure is world-class, starting with the Soyanggang Dam in 1967, building and managing water resource water infrastructure facilities (56 places) such as multi-purpose dams and weirs, and is the core infrastructure for national industrial development.

In particular, Korea Water Resources Corporation is responsible for water supply of 12.5 billion ㎥ (60% of the total) and flood control of 5.3 billion ㎥ (95% of the country), but the aging rate of dams and reservoirs among the entire SOC infrastructure is significantly higher.

Recently, public interest in water infrastructure safety management is rising due to a series of accidents, and large-scale dam safety accidents due to deterioration of water infrastructure and climate change are frequent around the world.

However, although the safety management of dams, which can lead to disasters, is a very important issue as described above, the problems of current dam management include the following.

- It is necessary to prepare structural and non-structural measures to increase the ability of existing facilities to respond to climate change and increased earthquake occurrence.

- New demands are emerging in the field of facility safety, such as aging of facilities, reduction of SOC investment, and increasing demand for development and application of advanced technologies such as 4th industrial revolution technology

- Insufficient introduction of the latest technologies such as the 4th industrial revolution technology in the field of water resource facility maintenance compared to domestic SOC management companies and institutions

- Insufficient computerization of essential information such as drawings, construction, physical properties, and measurement information of dams completed before 2000

- Inspection and diagnosis data are individually preserved in the form of paper reports → Reliance on the know-how of experienced users as it entails considerable time and experience in user accessibility restrictions and information retrieval

- Systematic facility and maintenance and reinforcement history management is required due to the distribution of existing maintenance history and restrictions on accessibility

- Currently, dam safety inspection materials are preserved in the form of the inspection subject's report and scanned electronic document, which limits the user's accessibility, entails considerable time and experience in determining the exact location, and relies on the know-how of the experienced in providing and sharing knowledge.

- As most of the legal safety inspection and diagnosis work of dam facilities is conducted mainly on visual inspections in areas with restricted access, they are classified as large and small dangerous tasks, the accuracy of data is low, and it is heavily dependent on labor and time.

- It takes a considerable amount of time to master the work of new and transferred employees due to the know-how-oriented safety management of existing engineers

- Measurement data analysis takes a considerable amount of time because it is difficult to intuitively understand information and lacks the engineering meaning of measurement information because it provides simple temporal changes.

- Three-dimensional deformation monitoring is required as it is limited to identify deformation and deterioration phenomena other than the installation location of major instruments such as external displacement gauges

Therefore, it is necessary to prepare a plan to promote technological advancement of dam infrastructure safety inspection and facility management by applying the 4th industrial revolution technology.

Republic of Korea Patent Publication No. 10-0773393 Republic of Korea Patent Publication No. 10-2017-0074411

The present invention has been devised to solve the problems of the prior art, and a digital twin-based intelligent dam safety management system that can significantly improve speed, accuracy, efficiency, etc. through technological advancement of dam infrastructure safety inspection and facility management Its purpose is to provide

In order to achieve the above object, the present invention builds a 3D dam modeling based on an image captured by at least any one of a drone and a photographing means for a dam in which a measuring means is installed in the facility, and 3D dam modeling is performed in a management area unit a digital twin generation module for subdividing into each and displaying at least any one or more of facility information and measurement information of measurement means on 3D dam modeling; The flight path information is stored for each management area unit and the image taken by a drone equipped with a thermal imaging camera is mapped to 3D dam modeling, and the 3D dam modeling before and after mapping is compared and expressed while leaking or leaking the dam through the thermal image orthographic image. a 3D modeling mapping module that enables confirmation of white changes; and a concrete crack identification module for determining concrete cracks by analyzing the dam concrete structure image taken by the drone in a deep learning method.

Here, the drone has a vertical overlap of 80% or more and a lateral overlap of 70% or more so that the photographed image is in the form of a grid, or the flight path information that overlaps the images photographed while flying in the east-west and north-south directions To store flight path information characterized in that

In addition, the measurement means is equipped with an IoT function, and the digital twin generation module is characterized in that the measurement information received from the measurement means is linked to the 3D dam modeling so that it can be inquired in real time.

Additionally, the digital twin generation module converts a 2D drawing scanned for a dam into 3D, can be linked to 3D dam modeling, and can periodically pop-up by setting a checklist for daily inspection for 3D dam modeling. do it with

In addition, it is characterized in that it further includes AR (Augmented Reality) glasses capable of displaying at least any one of 3D dam modeling, facility information, and measurement information of a measurement means in conjunction with the digital twin generation module.

In addition, the 3D modeling mapping module synchronizes 3D dam modeling before and after mapping to compare and express changes over time.

In addition, the concrete crack discrimination module maintains the same scale for the concrete crack image and deforms according to at least one method of rotation, brightness control, sharpness control, left/right reversal, and vertical reversal while maintaining the same scale for a plurality of deformed concrete crack images It is characterized by reinforcing the learning data for discriminating concrete cracks by learning , and then extracting at least one of the crack length and width after removing noise on the concrete crack image.

In addition, it characterized in that it further comprises a water leakage prediction module for predicting the amount of water leakage by machine learning by extracting data related to the amount of water leakage from the measurement information of the measuring means.

The digital twin-based intelligent dam safety management system according to the present invention as described above may exhibit the following effects.

1. Promotion of technological innovation in safety work field by realizing an intelligent information platform for dam facility management based on digital twin

2. Three-dimensional and intuitive dam safety inspection by drone digital mapping and image analysis, overcoming restricted access areas, and significant improvement in inspection efficiency and accuracy

3. Concrete structure reality modeling and AI deep learning automatic crack detection to reduce inspection/diagnosis time and labor

4. Quick decision support by visualizing key measurement information and intuitive data analysis

5. 3D digital facility management and dam management status briefing using AR (augmented reality) glass

6. Education and training available using the digital twin model of dam safety management for new employees

1 is a diagram schematically showing a digital twin-based intelligent dam safety management system according to the present invention
2A and 2B are diagrams showing the flight direction of a drone used in a digital twin-based intelligent dam safety management system according to the present invention;
3 is a diagram showing an example of a digital twin-based intelligent dam safety management system structure according to the present invention in connection with AR function
4 is a photograph showing an example of a method of checking the whiteness or leakage of a dam through a thermal image orthographic image in a digital twin-based intelligent dam safety management system according to the present invention
5 is a flowchart schematically illustrating a learning/analysis process of a concrete crack identification module in a digital twin-based intelligent dam safety management system according to the present invention;
6 is a photograph showing a case in which a concrete crack image is transformed by a concrete crack identification module for learning reinforcement in a digital twin-based intelligent dam safety management system according to the present invention;
7 is a view showing the VGG Net structure applied to the concrete crack identification module among the digital twin-based intelligent dam safety management system according to the present invention

The terms or words used in the present specification and claims should not be construed as being limited to their ordinary or dictionary meanings, and the inventor may properly define the concept of the term in order to best describe his invention. Based on the principle that there is, it should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Accordingly, the configuration shown in the embodiments and drawings described in the present specification is only one of the most preferred embodiments of the present invention, and does not represent all of the technical spirit of the present invention. It should be understood that various equivalents and modifications may be made.

Hereinafter, a digital twin-based intelligent dam safety management system according to the present invention will be described with reference to the drawings.

1 is a diagram schematically illustrating a digital twin-based intelligent dam safety management system according to the present invention.

The present invention is basically configured to include a digital twin generation module, a 3D modeling mapping module, and a concrete crack identification module.

More specifically, the present invention builds 3D dam modeling based on images taken by drones, photographing means, etc. for dams in which measuring means are installed in facilities, and manages 3D dam modeling in the management area (eg, concrete dam body, Yeosu waterway, fill dam body) , binocular) subdivided by unit, and a digital twin generation module for displaying at least any one of facility information and measurement information of measurement means on 3D dam modeling, and a thermal imaging camera A 3D modeling mapping module that maps images taken by a drone equipped with a dam to 3D dam modeling, and compares and expresses 3D dam modeling before and after mapping, and enables checking of dam leaks or changes in white matter through thermal image orthographic images, and drone shooting A dam concrete structure image is analyzed by a deep learning method and a concrete crack identification module is included to identify concrete cracks.

The digital twin generation module is preferentially installed in a dam facility such as a concrete dam body, a waterway, a fill dam body, and both eyes (left eye, right eye), a temperature meter, a rainfall meter, a water level meter, a water leak meter, a pore water pressure gauge, a positive pressure gauge, etc. Measurement information can be received from the same measurement means, and 3D dam modeling is performed after excluding overlapping parts in the process of linking images taken by drones or images taken by shooting means such as a DSLR (Digital Single Lens Reflex) camera. function to convert to .

2A and 2B are diagrams illustrating the flight direction of a drone used in the digital twin-based intelligent dam safety management system according to the present invention.

Referring to FIGS. 2A and 2B, the drone is equipped with a DSLR mirrorless camera of 40 million pixels or more, for example, and the images taken with 80% or more of vertical overlap and 70% or more of lateral overlap can be overlapped in a grid form. It is possible to create precise 3D dam modeling through the process of merging only the disparity part between the captured images by flying so that In situations where bonding may be difficult, flight path information is stored so that images taken while flying in the east-west and north-south directions are more densely overlapped than in the grid form.

For reference, as another example of generating the 3D dam modeling, a laser scanner is installed in a drone to scan a laser beam to the dam along the flight path, and the dam appearance is displayed at regular intervals on three-dimensional coordinates through the direction and distance of the reflected beam. It can be expressed as a set of points distributed as It is possible to create an overall 3D dam modeling by converting the point cloud data of this 3D scan information into 3D reference data.

Here, as the point cloud is densely emitted from the laser scanner and reflected from the object and returned to the receiver, a number of point groups each have x, y, and z position coordinates. It means information provided in the form of a number of points, that is, a point cloud, including the corresponding three-dimensional position through dense sampling of the target surface. measurement information can be provided.

The digital twin generation module matches the actual dam management area to visually distinguish the corresponding area on the 3D dam modeling, and provides facility information such as basic specifications, maintenance/reinforcement history, location, etc. on the 3D dam modeling. Measurement information such as the amount of water leakage, pore water pressure, positive pressure, etc. measured through the measuring means can be displayed visually along with the location of the measuring means. It is also possible to link the received measurement information to 3D dam modeling so that it can be viewed in real time in the modeling.

In addition, the digital twin generation module can convert the 2D drawing scanned for the dam into 3D and link it to 3D dam modeling or link the 2D drawing itself to 3D dam modeling, which is a blank 3D image because there is no photographed image. It is used to fill the dam modeling part or to enable 3D dam modeling to check 2D drawings for a specific part, and 3D dam modeling according to the hot spot method It may also be possible to expose only relevant information without confusion and simply by clicking on the spot by linking to it.

Additionally, the digital twin generation module may set a checklist for daily inspection for 3D dam modeling and pop-up periodically, and an example of such a checklist is shown in Table 1 below.

division check list

daily inspection * Automatic measurement system and CCTV check
- Are there any unmeasured sensors for important measurement items?
- Is on-site inspection or crisis response necessary due to abnormal measurement? (Interest → Caution → Alert)
- Is there any abnormality in the middle structure (dam body, Yeosu waterway, water intake tower) where CCTV is installed?








Weekly inspection * Automatic measurement system and CCTV check
- Are there any unmeasured sensors for important measurement items?
- Is on-site inspection or crisis response necessary due to abnormal measurement? (Interest → Caution → Alert)
- Is there any abnormality in the middle structure (dam body, Yeosu waterway, water intake tower) where CCTV is installed? * dam floor
- Are there longitudinal or transverse cracks?
- Are there settlements, holes and horizontal displacements? * Downstream slope
- Is there any leakage or wetting?
- Is there any change in existing leaks or wet areas?
- Does muddy water occur in leaks or wet areas?
- Are there any unusual deformations such as holes and sliding (collapse)? * Upstream slope
- Are there areas of erosion due to water level fluctuations?
- Are there any unusual deformations such as holes and sliding (collapse)?
- Is there a whirlpool phenomenon in the reservoir near the upstream slope? * Both eyes and downstream areas
- Is there any leakage or wetting?
- Is there any change in existing leaks or wet areas?
- Are there any unusual deformations such as holes and sliding (collapse)?
- Is there any leakage or wetting in the downstream river bank?

As described above, through the checklist that the digital twin generation module pops up, the inspection task, which is the first half of the dam, can be smoothly performed. It may be helpful in the inspection work of new employees and transferees by marking them together.

3 is a diagram showing an example of a digital twin-based intelligent dam safety management system structure according to the present invention in connection with the AR function.

The present invention further includes AR (Augmented Reality) glasses for displaying 3D dam modeling, facility information, measurement information of measurement means, etc. in conjunction with a digital twin generation module, so that AR glasses can be used for various purposes, such as support for visual/intuitive field management tasks. can be utilized, and referring to FIG. 2, for example, during on-site inspection/diagnosis, basic specifications, maintenance, Maintenance and reinforcement information or physical meaning/safety range/location information of major measurement means such as water leak gauge, pore water pressure gauge, and positive pressure gauge can be displayed on AR glasses. .

ㅇ dam facility management task AR glass briefing

- Visualize and deliver the overall operation method

- Possible to deliver images according to drone digital mapping

- It is possible to deliver key measurement information as a 3D virtual image through MR (Mixed Reality)

- AR implementation for each major event: Produces dam safety management events such as drone matching and measurement with AR, and can visualize the maintenance system of major machinery and equipment in 3D

ㅇ Implementation of digital model to promote public dam in Water Culture Center

- Implementation of realistic public dam function and convenience promotion system using digital twin model

- Make the dam safety management method easy to understand as an interactive event

- Maximizes the publicity effect to visitors by replacing the existing model exhibits

- Avatar production and briefing matching: Maximizes the delivery power through the avatar in parallel with the voice to explain each major issue

4 is a photograph showing an example of a method of checking the dam whiteness or leakage through a thermal image orthographic image in the digital twin-based intelligent dam safety management system according to the present invention.

The 3D modeling mapping module receives images taken by drones or artificially taken after the first creation of 3D dam modeling in the digital twin generation module, maps them to the existing 3D dam modeling, and then synchronizes the 3D dam modeling before and after mapping. By comparing and expressing changes over time, it is possible to analyze the deformation/change of the facility, and as the image taken by the drone with the thermal imaging camera is reflected, it is possible to check the thermal orthographic image on the 3D dam modeling. Make sure to check the whiteness.

5 is a flowchart schematically illustrating a learning/analysis process of a concrete crack identification module in a digital twin-based intelligent dam safety management system according to the present invention, and FIG. 6 is a digital twin-based intelligent dam safety management system according to the present invention. It is a photograph showing a case in which the concrete crack image is transformed by the concrete crack identification module for reinforcement, and FIG. 7 shows the VGG Net structure applied to the concrete crack identification module among the digital twin-based intelligent dam safety management system according to the present invention. It is a drawing.

The concrete crack discrimination module functions to discriminate concrete cracks according to the deep learning method. ), brightness control, sharpness control, left-right reversal, vertical reversal, etc., while learning the deformed plurality of concrete crack images to strengthen the learning data for discriminating concrete cracks, thereby reducing the number of learning Since a large amount of training data can be acquired even with data, that is, concrete crack images, the accuracy of the training model can be improved.

For example, it is possible to check whether concrete cracks in a dam image with VGG (Visual Geometry Group) Net, which is a kind of artificial intelligence neural network. As it has n convolution filters and 1 pooling structure compared to the structure, it has higher accuracy than CNN.

In addition, the concrete crack identification module can extract the crack length and width after removing the noise on the concrete crack image. As an example of the process for removing the noise on the crack image, the image is first processed smoothly through blurring. One example is to process the image so that only the cracks are clearly identified in the black-and-white state through gray-scaling and binarization.

The concrete crack identification module extracts and augments the crack image based on the photographed image (image rotation, transformation, etc.) In more detail, referring to FIG. 5 , as a learning process, dam shooting image labeling, dam shooting image augmentation, crack discrimination artificial intelligence learning and crack discrimination model generation are in progress, and dam shooting as an analysis process Image loading, crack identification, detailed crack diagnosis, and storage of crack diagnosis results can be performed.

On the other hand, the present invention may further include a water leakage prediction module for estimating the amount of water leakage by machine learning by extracting data related to the amount of water leakage from the measurement information of the measurement means, which can be received by the measurement means, the amount of rainfall, water level, It predicts the amount of water leakage by continuously learning the data related to the amount of water leakage, such as the amount of water leakage, pore water pressure, positive pressure, and so on. .

The leak amount prediction module selects data related to water leakage from the measurement information collected from the measurement means by data mining, then performs a pre-processing process to be suitable for learning, and selects a learning model through performance comparison to improve the accuracy of the leak amount prediction model It optimizes the learning model by adjusting the following parameters, and sets the final leak amount prediction model by comparing the accuracy of the measured value and the predicted value for performance verification, thereby enabling prediction of the amount of water leakage by artificial intelligence.

For reference, machine learning algorithms applicable to the leak prediction module include the following.

1. Support Vector Machine

- Basically designed to solve binary classification problems

- Can be used for non-linear classification as well as linear classification

- It is possible to extract the effective natural frequency through binary classification for each learned mode

2. Naive Bayes Classification

- A type of probability classifier that applies Bayes' theorem that assumes independence between features

Like the support vector machine, it uses the natural frequencies learned for each mode.

- Extract natural frequencies through binary classification

3. Decision tree

- Aims to create a model that predicts the value of the target variable for the input variable

- Regression models with continuous output variables have a disadvantage in that predictive power is lowered.

4. Artificial Neural Network

- Function inference, regression analysis, time series prediction, approximate modeling

- Classification algorithms such as pattern recognition, sequence recognition, and sequential decision

In the above description of the present invention with reference to the accompanying drawings, a specific shape and direction have been mainly described, but the present invention is capable of various modifications and changes by those skilled in the art, and such modifications and changes are included in the scope of the present invention. should be interpreted as

Claims (10)

3D dam modeling is built based on images captured by at least one of drones and shooting means for dams with measuring means installed in facilities, 3D dam modeling is subdivided by management area unit, and facility information on 3D dam modeling and a digital twin generating module for displaying at least one of the measurement information of the measurement means;
The flight path information is stored for each management area unit and the image taken by a drone equipped with a thermal imaging camera is mapped to 3D dam modeling, and the 3D dam modeling before and after mapping is compared and expressed while leaking or leaking the dam through the thermal image orthographic image. a 3D modeling mapping module that enables confirmation of white changes;
Analyze the dam concrete structure image taken by the drone with VGG (Visual Geometry Group) Net, which is a kind of artificial intelligence neural network, at least one of the following: Loading the dam image, discriminating cracks, diagnosing detailed cracks, and saving crack diagnosis results. Concrete crack identification module for;
The drone flies so that the images taken with more than 80% of the north-central degree and 70% of the north-central degree or more are overlapped in a grid form, and only the disparity part between the photographed images is merged to create a 3D dam modeling, east-west It stores flight path information so that images taken while flying in the direction and north-south direction are overlapped,
The concrete crack identification module processes the image through blurring the noise on the concrete crack image, then processes and removes the image so that only the crack part is identified in the black and white state through gray-scaling and binarization, and then extracts the crack length and width. ,
It extracts data related to the amount of water leakage from the measurement information of the measurement means and predicts the amount of water leakage by machine learning of at least any one of a support vector machine, a naive Bayes classification, a decision tree, and an artificial neural network. ,
The leak amount prediction module selects the data related to the leak from the measurement information collected from the measurement means by data mining and performs a preprocessing process, selects a learning model through performance comparison of the leak amount prediction model, and then selects the learning model through parameter adjustment A digital twin-based intelligent dam safety management system, characterized in that it optimizes and sets the leak amount prediction model by comparing the accuracy of the measured value and the predicted value.
delete The method according to claim 1,
The measuring means has an IoT function,
The digital twin generation module is a digital twin-based intelligent dam safety management system, characterized in that the measurement information received from the measurement means is linked to 3D dam modeling so that it can be inquired in real time.
The method according to claim 1,
The digital twin generation module converts a 2D drawing scanned for a dam into 3D and can be linked to 3D dam modeling. A digital twin-based intelligent dam safety management system.
The method according to claim 1,
The digital twin generation module is a digital twin-based intelligent dam safety management system, characterized in that it can pop-up periodically by setting a checklist for daily inspection for 3D dam modeling.
The method according to claim 1,
Digital twin-based intelligent dam safety management system, characterized in that it further comprises AR (Augmented Reality) glasses capable of displaying at least one of 3D dam modeling, facility information, and measurement information of measurement means in conjunction with the digital twin generation module .
The method according to claim 1,
The 3D modeling mapping module is a digital twin-based intelligent dam safety management system, characterized in that it is possible to compare and express changes over time by synchronizing 3D dam modeling before and after mapping.
The method according to claim 1,
The concrete crack discrimination module learns a plurality of deformed concrete crack images while maintaining the same scale for the concrete crack image and transforming it according to at least one method of rotation, brightness control, sharpness control, left/right reversal, and up/down reversal Digital twin-based intelligent dam safety management system, characterized in that it reinforces the learning data for identifying cracks in concrete.
delete delete

Images