KR102453446B1 - System for checking defect and method thereof - Google Patents

Abstract

The present invention relates to a system and a method for inspecting a defect. The system comprises: a control device transmitting a light command signal to a drone and receiving a defect detection result of a subject from the drone; the drone receiving the flight command signal to fly; a gymball attached to a front side of the drone; a camera fixated to the gymball and enabling a viewing angle according to driving of the gymball; and a lidar sensor fixated to the gymball and detecting a distance from the subject.

Description

Defect checking system and method thereof

The present invention relates to a defect inspection system and method therefor. Specifically, the present invention relates to a defect inspection system and method in which a drone performs a top mission flight at a nose position of a subject after taking off first, and performing a bottom mission at a nose position of a subject after taking off a second time.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

Recently, a technology for photographing using a drone has been commercialized. In addition, the technology of photographing using a drone is being grafted into various technical fields, and the need for a defect detection system using AI is increasing in order to check the blades of a wind power generator.

In the case of manually inspecting the existing wind turbine blades, a large number of personnel and equipment are required, and a large amount of time and money may be consumed.

In addition, when inspecting the blade using a drone, there was a need to accurately photograph the blade surface by adjusting the camera installed in the drone at various angles in order to photograph the wind turbine blade surface perpendicular to the blade surface.

Accordingly, there is a high demand for a system and method for accurately detecting a defective part while minimizing time and cost for inspecting a wind turbine blade.

An object of the present invention is to transmit a flight command signal to a drone and receive a defect detection result of a subject from the drone, a control device that receives a flight command signal to fly, a gimbal attached to the front of the drone, and a gimbal fixed to , to provide a defect inspection system including a camera and a gimbal whose viewing angle is adjusted according to the driving of the gimbal, and a lidar sensor that detects a distance from a subject.

Another object of the present invention is to provide a defect checking method in which a drone performs a top mission flight at a nose position of a subject after taking off first, and performing a bottom mission at a nose position of a subject after taking off a second time.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the appended claims.

A defect inspection system according to some embodiments of the present invention for solving the above problems transmits a flight command signal to a drone, a control device for receiving a defect detection result of a subject from the drone, and receives the flight command signal to fly A drone, a gimbal attached to the front of the drone, a camera fixed to the gimbal and having a viewing angle adjusted according to the driving of the gimbal, and a lidar sensor fixed to the gimbal and detecting a distance to the subject have.

In addition, the gimbal includes a supporting arm connected to the front of the drone and fixing the camera to remain horizontal even in the movement of the drone, and a driving motor connected to the supporting arm to change the angle of the camera. can

In addition, the lidar sensor may be located under the camera.

In addition, the control device may include a control unit for transmitting the flight command signal to the drone, and a display unit for displaying the defect detection result.

In addition, the defect detection result includes an image captured by the camera and a defect part of the subject included in the image, and a defect inspection server that generates report data, chart data, and history data using the defect detection result may further include.

In addition, further comprising a mission processor attached to the drone, wherein the mission processor receives the flight command signal from the control device, and generates a gimbal control signal, a camera control signal, a lidar sensor control signal and a flight controller control signal to the gimbal, the camera, the lidar sensor and the flight controller, respectively, and the flight controller may receive the flight controller control signal and control the flight of the drone.

Defect checking method according to some embodiments of the present invention for solving the above problems, the drone receiving a flight command signal from a control device;

Based on the flight command signal, the drone first takes off and moves to the position of the subject's nose, the drone performing a top mission flight at the position of the nose, after the top mission flight is finished, The first landing of the drone, the second taking off of the drone and moving to the position of the nose of the subject, the steps of the drone performing a bottom mission flight from the position of the nose, and the bottom mission flight is finished After that, the drone may include a second landing.

In addition, the step of moving to the position of the nose of the subject includes recognizing the subject with a lidar sensor included in the drone, and using distance data received from the lidar sensor to set a constant distance with the tower of the subject. It may include the steps of flying upward by a distance, generating real-time image data from a camera included in the drone, and recognizing, by the drone, a nose portion of the subject using the real-time image data.

In addition, performing the top mission flight includes flying along the lower surface of the first blade of the subject and the leading edge of the first blade, the right side of the second blade of the subject and the second blade It may include flying along the leading edge, and flying along the lower surface of the third blade of the subject and the trailing edge of the third blade.

In addition, the performing of the bottom mission flight includes flying along the upper surface of the third blade of the subject and the leading edge of the third blade, the left side of the second blade of the subject and the second blade It may include flying along the trailing edge of the subject, and flying along the upper surface of the first blade of the subject and the trailing edge of the first blade.

The specific effects of the present invention in addition to the above will be described together while explaining the specific details for carrying out the invention below.

The defect inspection system and method of the present invention may include a gimbal attached to the front of the drone, a camera whose viewing angle is adjusted according to the driving of the gimbal, and a lidar sensor fixed to the gimbal and detecting a distance from the subject. . Therefore, a gimbal is attached to the front of the drone to adjust the camera's viewing angle up and down, and by measuring the distance from the subject with the lidar sensor, the drone can fly stably while maintaining the distance from the subject.

In addition, the defect inspection system and method of the present invention may perform a top mission flight at the position of the subject's nose after the first takeoff of the drone, and the second takeoff to perform a bottom mission at the nose position of the subject. Therefore, the drone performs a top mission flight, takes off a second time after landing, and recognizes the nose part of the subject, and the flight algorithm can be simplified by performing the bottom mission at the position of the nose.

The specific effects of the present invention in addition to the above will be described together while explaining the specific details for carrying out the invention below.

1 is a conceptual diagram for explaining a defect inspection system according to some embodiments of the present invention.
FIG. 2 is a block diagram for explaining in detail the drone of FIG. 1 and the server of the defect inspection system.
FIG. 3 is a block diagram illustrating in detail the control device and the server of the defect inspection system of FIG. 1 .
FIG. 4 is a block diagram illustrating in detail the defect inspection system of FIG. 1 .
5 is a view for explaining a drone according to some embodiments of the present invention.
FIG. 6 is a view for explaining a mission processor of the drone of FIG. 2 .
7 is a diagram for explaining a connection relationship between the camera, gimbal, and lidar sensor of FIG. 2 .
8 is a diagram for explaining a connection relationship between the camera and the gimbal of FIG. 7 .
9 is an exemplary diagram of a user interface for describing report data according to some embodiments of the present invention.
10 is an exemplary diagram of a user interface for describing historical data according to some embodiments of the present invention.
11 is an exemplary diagram of a user interface for explaining chart data according to some embodiments of the present invention.
12 is a block diagram schematically illustrating a first deep learning module and a second deep learning module according to some embodiments of the present invention.
13 is a diagram illustrating the configuration of the first deep learning module and the second deep learning module of FIG. 12 .
14 is a flowchart illustrating a defect checking method according to some embodiments of the present invention.
FIG. 15 is a view for explaining the defect checking method of FIG. 14 .
16 is a flowchart for explaining in detail the step of moving to the position of the nose of FIG. 14 .
FIG. 17 is a view for explaining the step of ascending flight away from the tower by a certain distance by using the distance data received from the lidar sensor of FIG. 16 .
FIG. 18 is a diagram for explaining a step of recognizing a nose portion of the subject of FIG. 16 .
19 is a flowchart for describing in detail the step of performing the top mission flight of FIG. 14 .
FIG. 20 is a view for explaining a step of performing the top mission flight of FIG. 14 .
21 is a flowchart for explaining in detail the step of performing the bottom mission flight of FIG. 14 .
22 is a view for explaining a step of performing the bottom mission flight of FIG. 14 .

Terms or words used in this specification and claims should not be construed as being limited to a general or dictionary meaning. In accordance with the principle that the inventor can define a term or concept of a word to best describe his/her invention, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. In addition, the embodiments described in the present specification and the configurations shown in the drawings are only one embodiment in which the present invention is realized, and do not represent all the technical spirit of the present invention, so they can be replaced at the time of the present application. It should be understood that there may be various equivalents and modifications and applicable examples.

Terms such as first, second, A, B, etc. used in this specification and claims may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term 'and/or' includes a combination of a plurality of related listed items or any of a plurality of related listed items.

Terms used in this specification and claims are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. It should be understood that terms such as “comprise” or “have” in the present application do not preclude the possibility of addition or existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification in advance. .

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, each configuration, process, process or method included in each embodiment of the present invention may be shared within a range that does not technically contradict each other.

Hereinafter, with reference to FIGS. 1 to 22, a defect inspection system light method according to some embodiments of the present invention will be described in detail.

1 is a conceptual diagram for explaining a defect inspection system according to some embodiments of the present invention, and FIG. 2 is a block diagram for explaining in detail the drone of FIG. 1 and a server of the defect inspection system. FIG. 3 is a block diagram for explaining in detail the control device and the server of the defect inspection system of FIG. 1 , and FIG. 4 is a block diagram for describing the defect inspection system of FIG. 1 in detail. 5 is a diagram for explaining a drone according to some embodiments of the present invention, and FIG. 6 is a diagram for explaining a mission processor of the drone of FIG. 2 . FIG. 7 is a diagram for explaining a connection relationship between the camera, gimbal, and lidar sensor of FIG. 2 , and FIG. 8 is a diagram for explaining a connection relationship between the camera and the gimbal of FIG. 7 .

1 to 8 , a defect inspection system 1000 according to some embodiments of the present invention may include a drone 100 , a control device 200 , and a server 300 . Hereinafter, each component included in the defect inspection system 1000 will be described in detail.

First, referring to FIGS. 1 to 4 , the drone 100 , the control device 200 , and the server 300 may transmit data through a network. Also, the drone 100 , the control device 200 , and the server 300 may receive data through a network.

In this case, the network may include a network based on a wired Internet technology, a wireless Internet technology, and a short-range communication technology. Wired Internet technology may include, for example, at least one of a local area network (LAN) and a wide area network (WAN).

Wireless Internet technology is, for example, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet) Access), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS) and 5G New Radio (NR) technology. However, the present embodiment is not limited thereto.

Short-range communication technologies include, for example, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, and Near Field Communication: At least one of NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and 5G NR (New Radio) may include. However, the present embodiment is not limited thereto.

The drone 100 , the control device 200 , and the server 300 communicating through the network may comply with technical standards and standard communication methods for mobile communication. For example, standard communication methods include Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO). , at least one of Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTEA), and 5G New Radio (NR). may include. However, the present embodiment is not limited thereto.

For example, the drone 100 may be connected to the control device 200 through a short-range communication technology, and the control device 200 and the server 300 may be connected to the control device 200 through a wired/wireless Internet technology. However, the present invention is not limited thereto.

The drone 100 may be equipped with a camera 110 , a gimbal 120 , a lidar sensor 130 , a memory 140 , and a mission processor 150 .

The camera 110 may capture an image and a moving picture of a person, a background, and an object in response to a camera control signal. The captured image and video may be stored in the memory 140 .

In this case, the image data or real-time image data captured by the camera 110 may be an image or an image including the appearance of an object. Hereinafter, for convenience of description, the appearance of an object photographed by the camera 110 will be defined as a 'subject'. In this case, the subject may be, for example, a wind power generator. However, the present embodiment is not limited thereto.

The gimbal 120 is connected to the front of the drone 100 , and the camera 110 can be fixed even when the drone 100 moves. Also, the gimbal 120 may adjust the viewing angle of the camera 110 . A detailed description thereof will be described in detail with reference to FIGS. 5 to 8 .

The lidar sensor 130 is fixed to the gimbal 120 and may detect a surrounding subject. In this case, the lidar sensor 130 may be located under the camera 110 . The lidar sensor 130 detects a surrounding subject so that the drone 100 and the subject do not collide with each other, and through this, the drone 100 can maintain a distance from the subject when the drone 100 flies.

The memory 140 may store image data or real-time image data acquired from the camera 110 . Specifically, the memory 140 may store data related to the previously analyzed defect state of the subject. In this case, the defect may exist in the form of, for example, a defect (crack). For example, the memory 140 may store the size and location of a defect in the subject. Also, the memory 140 may store a random number for image data or real-time image data.

The mission processor 150 is electrically connected to the camera 110 , the gimbal 120 , the lidar sensor 130 , and the memory 140 , and the mission processor 150 may control the components. For example, the mission processor 150 may transmit a gimbal control signal, a lidar sensor control signal, and a memory control signal to the lidar sensor 130 , the gimbal 120 , and the memory 140 , respectively. Through this, the mission processor 150 may control the driving of the lidar sensor 130 , the gimbal 120 , and the memory 140 , respectively.

The mission processor 150 may include a flight control unit 151 and a mission control unit 153 . The mission controller 153 may receive a flight command signal from the control device 200 and generate a flight controller control signal. The flight controller 151 may receive a flight controller control signal from the mission controller 153 and control the flight of the drone 100 . For example, when receiving a flight command signal requesting autonomous flight from the control device 200 , the flight controller 151 may control the drone 100 to fly autonomously. However, the present embodiment is not limited thereto, and may receive a flight command signal requesting manual flight.

The mission control unit 153 receives a flight command signal from the control device 200 and generates a camera control signal, a gimbal control signal, and a lidar sensor signal, respectively, to the camera 110 , the gimbal 120 and the lidar sensor ( ). 130) can be transmitted.

Specifically, the mission controller 153 may transmit a camera control signal, a gimbal control signal, and a lidar sensor signal to the camera driver 118 , the gimbal driver 128 , and the lidar driver 138 .

In this case, the camera control signal may include a photographing start or photographing stop signal, the gimbal control signal may include a signal for operating a driving motor of the gimbal, and the lidar control signal may include a sensor operation or operation stop signal. For example, when the mission control unit 153 transmits a photographing start signal among the camera control signals to the camera driver 118 , the camera driver 118 may operate the camera 110 to photograph. However, the present embodiment is not limited thereto, and the camera control signal, the gimbal control signal, and the lidar control signal may be variously modified and implemented.

Specifically, referring to FIGS. 5 to 8 , the gimbal 120 may be installed in front of the drone 100 . A mission processor 150 may be installed in region A of the rear of the drone 100 . However, the present embodiment is not limited thereto, and the position of the mission processor 150 may be variously modified.

The gimbal 120 is connected to the front of the drone 100 , and the camera 110 and the lidar sensor 130 may be installed in the region B of the gimbal 120 . The camera 110 may be fixed to the gimbal 120 attached to the front of the drone 100 . That is, the camera 110 may also be disposed in front of the drone 100 . Also, the lidar sensor 130 may be disposed in front of the drone 100 .

The gimbal 120 is connected to the first and second supporting arms 121 and 123 and the first and second supporting arms 121 and 123 that are fixed so as to remain horizontal even in the movement of the drone 100 and are connected to the camera 110 . It may include first and second driving motors 125 and 127 for changing the angle of . The first and second driving motors 125 and 127 may rotate to change the viewing angle of the camera 110 . For example, when the camera 110 is facing forward and the first and second driving motors 125 and 127 rotate 45˚ clockwise, the viewing angle of the camera 110 may be inclined by 45˚. have. However, the present embodiment is not limited thereto, and the first and second driving motors 125 and 127 may rotate at various angles.

Meanwhile, a damper DP may be positioned between the drone 100 and the gimbal 120 . The damper DP may be a device that absorbs vibration energy of the drone 100 .

Through this, the drone 100 according to some embodiments of the present invention has a gimbal 120 attached to the front so that the viewing angle of the camera 110 can be adjusted, so that a subject can be photographed from various angles.

Again, referring to FIGS. 1 to 4 , the control device 200 may include a control unit 210 and a display unit 220 .

The control unit 210 may transmit a flight command signal to the drone 100 . In this case, the flight command signal may include an autonomous flight or a manual flight signal. The autonomous flight signal may be a signal for autonomously flying the drone 100 using the mission processor 150 . The manual flight signal may be a signal for manually flying by continuously transmitting a signal to the drone 100 using the control unit 210 .

The display unit 220 is, for example, a personal computer, a workstation, a personal digital assistant (PDA), a portable computer, a web tablet, a wireless It can be applied to a wireless phone, a mobile phone, a digital music player, a memory card, or any electronic product that can transmit and/or receive information in a wireless environment. .

The server 300 may include a communication unit 310 , a flight path determination unit 320 , a metadata management unit 330 , a result data generation unit 340 , and a memory unit 350 .

The communication unit 310 may transmit a flight command signal to the control device 200 . The communication unit 310 may receive a flight signal from the drone 100 . Specifically, the control device 200 may receive the flight command signal transmitted from the communication unit 310 , and the control device 200 may transmit the received flight command signal to the drone 100 . Accordingly, the communication unit 310 may receive the flight signal transmitted by the drone 100 . In this case, the flight signal may be a signal transmitted when the drone 100 takes off from the ground.

Also, the communication unit 310 may receive image data captured by the camera 110 and real-time image data. Also, the communication unit 310 may receive distance data from the lidar sensor 130 . In this case, the distance data may be data representing a distance between the lidar sensor 130 and the subject.

The flight path determiner 320 may generate an autonomous flight path using real-time image data and distance data. For example, real-time image data and distance data may be provided to the flight path determining unit 320 through the communication unit 310 . A detailed description thereof will be described in detail with reference to FIG. 12 .

The metadata management unit 330 may generate metadata using image data. For example, the image data may be provided to the metadata management unit 330 through the communication unit 310 . The metadata may include at least one of a photographed location of image data, a distance to a subject, and an angle of the subject.

In addition, the metadata may generate and store a random number. In this case, the random number of the meta data may be stored as a random number identical to the random number generated when the image data corresponding to the image data is stored in the memory 140 .

The metadata management unit 330 may compare random numbers to match metadata and image data. For example, if the random number of the image data stored in the memory 140 is 123 and the random number of the metadata generated by the metadata management unit 330 is 456, the image data and the metadata do not correspond to the metadata. The management unit 330 cannot match the image data and the metadata.

As another example, when the random number of the image data stored in the memory 140 is 123 and the random number of the metadata generated by the metadata management unit 330 is 123, since the image data and the metadata are corresponding data, the metadata management unit ( 330) may match image data and metadata.

The result data generator 340 may generate report data, chart data, and history data using the image data. For example, the image data may be provided to the result data generation unit 340 through the communication unit 310 . A detailed description thereof will be described in detail with reference to FIGS. 9 to 11 .

In the present specification, it has been described that the flight path determining unit 320 , the meta data management unit 330 , and the result data generating unit 340 are all included in the server 300 for convenience of description, but embodiments are not limited thereto. According to some embodiments, at least some of the flight path determining unit 320 , the metadata management unit 330 , and the result data generating unit 340 may be included in the mission processor 150 of the drone 100 . For example, the metadata management unit 330 may be included in the server 300 , and the flight path determining unit 320 and the result data generation unit 340 may be included in the mission processor 150 . As another example, the metadata management unit 330 and the result data generation unit 340 may be included in the server 300 , and the flight path determining unit 320 may be included in the mission processor 150 . A person having ordinary knowledge in the technical field of the present invention according to the usage environment or the specification of the mission processor 150, among the flight path determining unit 320, the meta data management unit 330, and the result data generation unit 340 At least some functions may be implemented in the drone 100 on-device.

The memory unit 350 may store data received from the communication unit 310 and data generated by the flight path determination unit 320 , the metadata management unit 330 , and the result data generation unit 340 . Specifically, distance data, image data, and real-time image data received from the communication unit 310 and the autonomous flight path generated by the flight path determining unit 320 and metadata generated by the metadata management unit 330 and result data generation unit The report data, chart data, and history data generated in 340 may be stored.

As shown in FIG. 4 , the communication device TM is a device that performs communication between the drone 100 and the control device 200 , and may be a device that performs short-range networking such as Wi-Fi or RF communication. However, the present embodiment is not limited thereto.

Hereinafter, result data according to some embodiments of the present invention will be described with reference to FIGS. 9 to 11 .

9 is an exemplary diagram of a user interface for explaining report data according to some embodiments of the present invention, FIG. 10 is an exemplary diagram of a user interface for describing history data according to some embodiments of the present invention, and FIG. 11 is an exemplary diagram of a user interface for describing chart data according to some embodiments of the present invention.

Referring to FIG. 9 , the display 220 may display report data RE. The report data RE may include comment data SM, table data AN, and image data IM.

The comment data SM may include a detailed comment on which defect exists in which position of the subject. The comment data SM may describe the location, form, and type of the defect CP, and may describe to which category it belongs.

The table data AN may provide at least one of a type, category, state, action request deadline, description, and location of the defect in a tabular form. However, the present embodiment is not limited thereto.

The image data IM may include an image of the defect CP so that the position and shape of the defect CP may be known. In this case, the image data IM may include an image from which a location can be identified and an enlarged image from which a shape can be identified in detail.

Referring to FIG. 10 , the display 220 may display history data HS1 and HS2. The history data HS1 and HS2 may include first history data HS1 and second history data HS2 . The first history data HS1 and the second history data HS2 may be data representing different viewpoints of the same part. Through this, it is possible to clearly and intuitively express which part of the subject changes over time.

Referring to FIG. 11 , the display 220 may display chart data CA. The chart data CA may be displayed in the form of a map and various graphs. Through this, the defect inspection system of the present invention can graphically and intuitively provide the position of the subject, the position of the defect of the subject, the characteristics of the defect, and the like.

Hereinafter, in order to describe in detail the deep learning module for generating autonomous flight routes and result data, further reference is made to FIGS. 13 and 14 .

12 is a block diagram schematically illustrating a first deep learning module and a second deep learning module according to some embodiments of the present invention, and FIG. 13 is a first deep learning module and a second deep learning module of FIG. It is a drawing showing the configuration.

1 and 12 , the server 300 may include a deep learning module. The deep learning module may include a first deep learning module DM1 and a second deep learning module DM2. For example, the flight path determining unit 320 may include a first deep learning module DM1, and the result data generating unit 340 may include a second deep learning module DM2. As described above, in the defect inspection system according to some other embodiments, at least a portion of the flight path determining unit 320 and the result data generating unit 340 may be included in the mission processor 150 of the drone 100 . For example, the flight path determining unit 320 may be included in the mission processor 150 , and the result data generating unit 340 may be included in the server 300 . In this case, the first deep learning module DM1 may be included in the mission processor 150 of the drone 100 , and the second deep learning module DM2 may be included in the server 300 . A person having ordinary knowledge in the technical field of the present invention according to the user environment or the specification of the mission processor 150, the first deep learning module (DM1) and the second deep learning module (DM2) in the server 300 It may be possible to choose whether to implement it or whether to implement it on-device in the drone 100 .

The first deep learning module DM1 may derive the autonomous flight path FR by performing an algorithm by inputting the distance data DA and the real-time image data VA. The second deep learning module DM2 may derive the report data RE, the history data HI, and the chart data CA, which are result data, by performing an algorithm with the image data IM as input.

The first and second deep learning modules DM1 and DM2 may use various well-known neural network structures. For example, the first and second deep learning modules (DM1, DM2) include at least one of a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Deep Belief Network (DBN), and a Graph Neural Network (GNN). can do.

On the other hand, learning of the first and second deep learning modules DM1 and DM2 is performed by adjusting the weight of the connection line between nodes (and adjusting the bias value if necessary) so that a desired output is obtained for a given input. can In addition, the first and second deep learning modules DM1 and DM2 may continuously update a weight value by learning.

Hereinafter, an example of the above-described first and second deep learning modules DM1 and DM2 will be described.

13 and 14 , the deep learning module DM may include first and second deep learning modules DM1 and DM2.

The first deep learning module (DM1) includes an input layer (Input) including an input node that receives distance data (DA) and real-time image data (VA), and an output node that outputs an autonomous flight path (FR) It includes an output layer (Output) and M hidden layers disposed between the input layer and the output layer.

The second deep learning module DM2 has an input layer (Input) including an input node that receives image data (IM), and an output that outputs report data (RE), history data (HI) and chart data (CA). It includes an output layer (Output) including nodes, and M hidden layers disposed between the input layer and the output layer.

Here, a weight may be set on an edge connecting the nodes of each layer. The presence or absence of such weights or edges may be added, removed, or updated during the learning process. Accordingly, through the learning process, weights of nodes and edges disposed between k input nodes and i output nodes may be updated.

Before the first deep learning module DM1 performs learning, all nodes and edges may be set to initial values. However, when accumulated information is input, the weights of nodes and edges are changed, and in this process, parameters input as learning factors (ie, distance data and real-time image data) and values assigned to output nodes (ie, autonomy) flight paths) can be matched.

Before the second deep learning module DM2 performs learning, all nodes and edges may be set to initial values. However, when cumulative information is input, the weights of nodes and edges are changed, and in this process, parameters input as learning factors (ie, image data) and values assigned to output nodes (ie, report data, history data) , chart data) can be matched.

Additionally, when using a cloud server, the deep learning module (DM) may receive and process a large number of parameters. Therefore, the deep learning module (DM) can perform learning based on massive data.

In addition, the weights of nodes and edges between the input node and the output node constituting the deep learning module (DM) may be updated by the learning process of the deep learning module (DM). In addition, the parameters output from the first deep learning module (DM1) are autonomous flight paths and the parameters output from the second deep learning module (DM2) can be further expanded to various data in addition to report data, history data, and chart data. Of course.

Hereinafter, a defect checking method according to some embodiments of the present invention will be described with reference to FIGS. 4 and 14 to 22 . Descriptions overlapping with the above-described embodiments will be simplified or omitted.

14 is a flowchart for explaining a defect checking method according to some embodiments of the present invention, and FIG. 15 is a diagram for explaining the defect checking method of FIG. 14 . 16 is a flowchart for explaining in detail the step of moving to the position of the nose of FIG. 14, and FIG. 17 is a step of ascending flying away from the tower by a certain distance using the distance data received from the lidar sensor of FIG. 16 It is a drawing for explaining. FIG. 18 is a view for explaining the step of recognizing the nose portion of the subject of FIG. 16 , and FIG. 19 is a flowchart for explaining in detail the step of performing the top mission flight of FIG. 14 . FIG. 20 is a view for explaining the step of performing the top mission flight of FIG. 14 , and FIG. 21 is a flowchart for explaining in detail the step of performing the bottom mission flight of FIG. 14 . 22 is a view for explaining a step of performing the bottom mission flight of FIG. 14 .

14 and 15 , a flight signal is first received (S100). It may refer to the preparation step of FIG. 15 .

Specifically, referring to FIG. 4 , the mission processor 150 may include a flight control unit 151 and a mission control unit 153 . The mission controller 153 may receive a flight command signal from the control device 200 and generate a flight controller control signal. The flight controller 151 may receive a flight controller control signal from the mission controller 153 and control the flight of the drone 100 .

Again, referring to FIGS. 14 and 15 , the first takes off and moves to the position of the subject's nose ( S200 ). In this case, the subject may be the wind power generator WT, and the nose may mean a portion where the blades of the wind power generator WT are combined.

In detail, referring to FIG. 16 , the subject is recognized by the lidar sensor (S210), and it flies upwardly away from the tower by a certain distance using the distance data received from the lidar sensor (S220).

Specifically, referring to FIG. 17 , the lidar sensor 130 of the drone 100 may detect the tower TO of the wind power generator WT. Since the lidar sensor 130 may measure the distance DI to the tower TO, the drone 100 may perform an ascending flight along the tower TO to the nose.

Again, referring to FIG. 16 , the camera generates real-time image data (S230), and recognizes the nose portion of the subject (S240).

Specifically, referring to FIGS. 2 and 18 , as in b1 , the nose NO may be a portion in which the first blade BL1 , the second blade BL2 , and the third blade BL3 are merged. The camera 110 of the drone 100 may detect the nose NO while capturing an image of the nose NO.

Again, referring to FIGS. 14 and 15 , a top mission flight is performed ( S300 ).

In detail, referring to FIG. 19 , it flies along the lower surface and the leading edge of the first blade ( S310 ).

Specifically, referring to FIGS. 2 and 20 , the three blades of the wind power generator WT may be stopped in a lying Y-shape. In this case, the number of blades of the wind power generator WT is exemplary and may be varied. Also, the resting shape of the blades may vary.

The first blade BL1 refers to a blade facing the lower right when viewed from the front of the wind power generator WT, and the second blade BL2 and the third blade BL3 are sequentially from the first blade BL1. Each of the blades located in the counterclockwise direction may refer to. Each blade may have two faces and a leading edge located in the front and a trailing edge located in the rear.

The top mission flight first detects defects while flying along the lower surface and leading edge of the first blade BL1 (①, ②). At this time, the camera 110 attached to the gimbal 120 located in the front of the drone 100 can freely secure the view in the upward direction, so that compared to the camera attached to the gimbal located below the drone, It can be much more liberating to shoot. Since the detection of the defect proceeds more precisely as the field of view perpendicular to the surface is secured, the camera 110 fixed to the front gimbal 120 can find the defect much more precisely.

Again, referring to FIG. 19, it flies along the right side and the leading edge of the second blade (S320).

The top mission flight then detects defects while flying along the right side and leading edge of the second blade BL2 (③, ④). By the camera fixed to the gimbal 120 installed in front of the drone 100, the camera 110 can take a picture perpendicular to the right side, so that more precise measurement is possible.

Then, it flies along the lower side of the third blade and the trailing edge (S330).

The top mission flight then detects defects while flying along the lower surface and trailing edge of the third blade BL3 (⑤, ⑥). Also, the camera 110 attached to the gimbal 120 located in the front of the drone 100 can freely secure the view in the upward direction, so that compared to the camera attached to the gimbal located below the drone, It can be much more liberating to shoot.

In this way, flying along one surface and one edge for each blade is to optimize the flow of flight. Accordingly, shooting of the two faces of each blade and half of the two edges can be completed with a top mission flight.

Again, referring to FIGS. 14 and 15 , a first landing is made ( S400 ). Then, it takes off a second time and moves to the position of the subject's nose (S500).

The reason for landing and taking off again may be to make precise measurements by clarifying the flight path of the drone. That is, effective measurement is performed through an optimal movement route, but landing and take-off can be performed to reduce errors that may occur due to a long flight time.

Then, the bottom mission flight is performed (S600).

In detail, referring to FIG. 21 , it flies along the upper surface and leading edge of the third blade ( S610 ).

Specifically, referring to FIGS. 2 and 22 , the bottom mission flight first detects defects while flying along the upper surface and leading edge of the third blade BL3 (⑦, ⑧).

Again, referring to FIG. 21 , it flies along the left side of the second blade and the trailing edge ( S620 ).

Specifically, referring to FIG. 22 , the bottom mission flight detects defects while flying along the left surface and the trailing edge of the second blade BL2 ( ⑨ , ⑩ ). By the camera fixed to the gimbal 120 installed in front of the drone 100, the camera 110 can take a picture perpendicular to the left side, so that more precise measurement is possible.

Then, it flies along the upper surface and the trailing edge of the first blade (S630).

The bottom mission flight then detects defects while flying along the upper surface and trailing edge of the first blade BL1 (⑪, ⑫). Also, the camera 110 attached to the gimbal 120 located in the front of the drone 100 can freely secure the view in the upward direction, so that compared to the camera attached to the gimbal located below the drone, It can be much more liberating to shoot.

Accordingly, measurements not completed with the top mission flight may be completed with the bottom mission flight. That is, all sides and edges of all blades can be measured.

Again, referring to FIG. 14 , a second landing is made ( S700 ).

The defect inspection method according to the present embodiment can accurately and closely detect the defect through the optimal movement line for detecting the defect of the Y-shaped blade lying down. In addition, error-free defect detection may be possible through adjustment of flight time.

The above description is merely illustrative of the technical idea of this embodiment, and various modifications and variations will be possible without departing from the essential characteristics of the present embodiment by those of ordinary skill in the art to which this embodiment belongs. Accordingly, the present embodiments are intended to explain rather than limit the technical spirit of the present embodiment, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

Claims (10)

a control device for transmitting a flight command signal to the drone and receiving a defect detection result of a subject from the drone;
a drone that receives the flight command signal and flies;
a gimbal attached to the front of the drone;
a camera fixed to the gimbal and having a viewing angle adjusted according to the driving of the gimbal; and
It is fixed to the gimbal, including a lidar sensor for detecting the distance to the subject,
The drone is
First take off based on the flight command signal and move to the position of the nose of the subject,
Perform a top mission flight at the position of the nose,
After the top mission flight is over, the first landing,
After the second takeoff, it moves to the position of the subject's nose,
Perform a bottom mission flight from the position of the nose,
After the bottom mission flight is over, the second landing
Fault checking system.
The method of claim 1,
The gimbal is
a supporting arm connected to the front of the drone and fixing the camera to remain horizontal even in the movement of the drone;
and a drive motor connected to the supporting arm to change the angle of the camera
Fault checking system.
The method of claim 1,
The lidar sensor is located under the camera
Fault checking system.
The method of claim 1,
The control device is
a control unit for transmitting the flight command signal to the drone;
and a display unit for displaying the defect detection result
Fault checking system.
5. The method of claim 4,
The defect detection result includes an image taken with the camera and a defective part of the subject included in the image,
Using the defect detection result, further comprising a defect inspection server for generating report data, chart data, and history data
Fault checking system.
The method of claim 1,
Further comprising a mission processor attached to the drone,
The mission processor,
Receives the flight command signal from the control device, generates a gimbal control signal, a camera control signal, a lidar sensor control signal, and a flight controller control signal and transmits them to the gimbal, the camera, the lidar sensor and the flight controller, respectively,
The flight controller receives the flight controller control signal, and controls the flight of the drone.
Fault checking system.
receiving, by the drone, a flight command signal from a control device;
based on the flight command signal, the drone first takes off and moves to the position of the subject's nose;
performing, by the drone, a top mission flight at the position of the nose;
After the top mission flight is finished, the first landing of the drone;
moving the drone to the position of the nose of the subject by taking off a second time;
performing, by the drone, a bottom mission flight from the position of the nose; and
After the bottom mission flight is finished, the drone comprising the step of a second landing
How to check for defects.
8. The method of claim 7,
The step of moving to the position of the nose of the subject is,
Recognizing the subject with a lidar sensor included in the drone;
Using the distance data received from the lidar sensor, the step of flying upward by a certain distance from the tower of the subject;
generating real-time image data from a camera included in the drone;
Recognizing, by the drone, a nose portion of the subject using the real-time image data
How to check for defects.
8. The method of claim 7,
The step of performing the top mission flight,
Flying along the lower surface of the first blade of the subject and the leading edge of the first blade;
Flying along the right side of the second blade of the subject and the leading edge of the second blade;
Comprising the step of flying along the lower surface of the third blade of the subject and the trailing edge of the third blade
How to check for defects.
8. The method of claim 7,
The step of performing the bottom mission flight,
Flying along the upper surface of the third blade of the subject and the leading edge of the third blade;
Flying along the left side of the second blade of the subject and the trailing edge of the second blade;
Comprising the step of flying along the upper surface of the first blade of the subject and the trailing edge of the first blade
How to check for defects.

Images