LU502731B1 - Method for monitoring abnormality of power production, apparatus, computer device, and storage medium therefor - Google Patents

Abstract

A method and apparatus for monitoring abnormality of power production, including: obtaining a working scene image acquired by a camera arranged at a power production site; inputting the working scene image to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; when the detection result that the power production site is abnormal is obtained according to the detection model, calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal, giving an abnormal alarm.

Description

METHOD FOR MONITORING ABNORMALITY OF POWER PRODUCTION, APPARATUS, COMPUTER DEVICE, AND STORAGE MEDIUM THEREFOR

TECHNICAL FIELD The present invention relates to the technical field of power production, in particular to a method and apparatus for monitoring abnormality of power production, an apparatus, a computer device, and a storage medium therefor.

BACKGROUND There are more than 200 cameras in a production area of a pumped storage power plant with an installed capacity of 1,200 MW. There are more than 100 cameras in a production area of a 500 kV substation. In the past, it was only possible to perform manual on-site video review, massive video data could not be used for data analysis, and an industrial television system was only a system for post-event analysis or view confirmation. Meanwhile, automated sensors cannot fully sense abnormal conditions such as oil leakage, water leakage, and part falloff of devices and so on. Thus, the ideal inspection effect of full-range and full-time coverage still cannot be achieved by relying on strict management and conscientious and responsible inspectors. There are many types of power plant/substation devices, including oil, water, gas, and electrical devices. Due to mechanical defects such as sealing performance degradation and part defect and falloff, "three leakages" which are water leakage, oil leakage, and steam leakage of devices occur occasionally during production. If abnormal conditions such as oil leakage, water leakage and steam leakage of production devices are not detected in time, defects are often caused to affect upgrades, resulting in outage of main devices, and even other safety accidents are brought. High-temperature steam leakage in gas-fired power plants/thermal power plants often will reduce the power generation efficiency of units due to heat medium leakage, thereby affecting the economic indicators of the entire units. In addition, high-temperature steam leakage may cause personal injury to operation and maintenance personnel. Therefore, optical/infrared cameras are urgently needed in the production of power plants to actively detect oil leakage, water leakage and steam leakage so as to find abnormalities in time.

SUMMARY Based on this, it is necessary to provide a method with simple management and high detection accuracy for monitoring abnormality of power production, an apparatus, a computer device, and a storage medium therefor for the above technical problems.

Provided is a method for monitoring abnormality of power production, the method including: obtaining a working scene image acquired by a camera arranged at a power production site; inputting the working scene image to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; when the detection result that the power production site is abnormal 1s obtained according to the detection model, calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal,

giving an abnormal alarm.

In one of embodiments, inputting the working scene image to the pre-trained detection model according to the detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result, includes: obtaining the detection frequency of the ledger object corresponding to the working scene image; and when detection time corresponding to the detection frequency is reached, inputting the working scene image to the detection model, detecting whether the power production site is abnormal by the detection model, and outputting a classification result and a segmentation result of an abnormal area segmented from the working scene image when the classification result is abnormal, where the classification result includes normal, oil leakage, steam leakage, and water leakage.

In one of embodiments, calling the corresponding secondary abnormality detection model according to the type of abnormality to perform secondary detection on the work site, includes: if the type of abnormality is oil leakage and/or water leakage, inputting the working scene image to an oil and water leakage secondary recognition model that uses a residual network structure, and outputting a secondary abnormality detection result by the oil and water leakage secondary recognition model.

In one of embodiments, calling the corresponding secondary abnormality detection model according to the type of abnormality to perform secondary detection on the work site, includes: if the type of abnormality is steam leakage, obtaining an infrared image acquired by an infrared camera of the ledger object corresponding to the working scene image; and inputting the infrared image into an infrared image recognition model, extracting features of the infrared image by a feature extraction network of the infrared image recognition model, and inputting the features of the infrared image to a classifier so as to obtain a secondary abnormality detection result about whether the steam leakage occurs.

In one of embodiments, obtaining the detection frequency of the ledger object corresponding to the working scene image, includes: obtaining a camera name and a preset position of the working scene image; matching the ledger object according to the camera name and the preset position; if a single ledger object is matched, obtaining the detection frequency corresponding to the ledger object; and if a plurality of ledger objects are matched, obtaining a minimum detection frequency in detection frequencies of the plurality of ledger objects.

In one of embodiments, a process for training a detection model includes: obtaining an image sample set including samples, and segmentation labels for labeling pixel points of the samples; training a feature extraction network and a segmentation network of the detection model according to the segmentation labels of the samples labeled in the image sample set, obtaining classification labels of the samples according to the trained feature extraction network and segmentation network; and training the detection model according to the classification labels.

In one of embodiments, obtaining the classification labels of the samples according to the trained feature extraction network and segmentation network, includes: inputting the samples to the detection model, and extracting image features by the trained feature extraction network in the detection model; inputting the image features to the trained segmentation network, and outputting segmentation results of abnormal areas; and generating the classification labels of the samples according to the segmentation 5 results.

Provided is an apparatus for monitoring abnormality of power production, the apparatus including: an image obtaining module for obtaining a working scene image acquired by a camera arranged at a power production site; a detection module for inputting the working scene image to a detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; a secondary detection module for calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site when the detection result that the power production site is abnormal is obtained according to the detection model; and an alarm module for giving an abnormal alarm 1f the secondary detection still shows that the power production site 1s abnormal.

Provided is a computer device including a memory and a processor, the memory storing computer programs, when executing the computer programs, the processor implements the following steps: obtaining a working scene image acquired by a camera arranged at a power production site; inputting the working scene image to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result;

when the detection result that the power production site is abnormal 1s obtained according to the detection model, calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal, giving an abnormal alarm.

Provided is a computer readable storage medium storing computer programs thereon, when the computer programs 1s executed by a processor, cause the processor to implement the following steps: obtaining a working scene image acquired by a camera arranged at a power production site; inputting the working scene image to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; when the detection result that the power production site is abnormal 1s obtained according to the detection model, calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal, giving an abnormal alarm.

According to the above-mentioned method for monitoring abnormality of power production, an apparatus, computer device and storage medium therefor, the working scene image acquired by the camera at the production site is acquired; after the abnormality is detected by means of the pre-trained detection model, the secondary abnormality detection model corresponding to the type of abnormality is called to perform the secondary detection on the work site; higher-level features are extracted by means of a neural network model; and secondary detection and recognition are further performed, such that the detection accuracy is improved and the false alarm rate is reduced. In addition, when the abnormality is detected by means of the method, only the camera needs to be arranged at the power production site. The camera has a wide acquisition range, and does not need to be arranged complexly, such that the management cost can be reduced.

BRIEF DESCRIPTION OF DRAWINGS FIG. 1 is an application environment diagram of a method for monitoring abnormality of power production in an embodiment; FIG. 2 is a schematic flowchart of a method for monitoring abnormality of power production in an embodiment; FIG. 3A is a working scene image of water leakage in an embodiment; FIG. 3B is a segmentation result of a water leakage area segmented from a working scene image in an embodiment; FIG. 4A is a working scene image of steam leakage in an embodiment; FIG. 4B is a segmentation result of a steam leakage area segmented from a working scene image in an embodiment; FIG. 5A is a working scene image of oil leakage in an embodiment; FIG. 5B is a segmentation result of an oil leakage area segmented from a working scene image in an embodiment;

FIG. 6 is a schematic flowchart of steps of training a detection model in an embodiment; FIG. 7 is a schematic structural diagram of a detection model in an embodiment; FIG. 8 is a schematic structural diagram of a segmentation network in an embodiment; FIG. 9 is a schematic structural diagram of a classification network in an embodiment; FIG. 10 is a structural block diagram of an apparatus for monitoring abnormality of power production in an embodiment; and FIG. 11 is an internal structural diagram of a computer device in an embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS The specific implementation of the present invention is further described below with reference to the accompanying drawings and examples, but the implementation and protection of the present invention are not limited thereto. It should be pointed out that, if there are any processes not described in detail below, those skilled in the art can realize or understand them with reference to the prior art. A method for monitoring abnormality of power production provided by the present invention can be applied to an application environment as shown in FIG. 1. A camera 102 and an infrared camera 104 for a ledger object are arranged at a power production site. The camera 102 and the infrared camera 104 are connected to a monitoring terminal 106 in a control room. The monitoring terminal receives a working scene image acquired by the camera and an infrared image acquired by the infrared camera. The monitoring terminal obtains the working scene image acquired by the camera arranged at the power production site; according to a detection frequency of the ledger object corresponding to the working scene image, the working scene image 1s input to a detection model so as to obtain an abnormality detection result; when the detection result that the power production site is abnormal is obtained according to the detection model, a corresponding secondary abnormality detection model is called according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal, an abnormal alarm is given.

The monitoring terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones or tablet computers.

In an embodiment, as shown in FIG. 2, a method for monitoring abnormality of power production is provided. By taking the method applied to the monitoring terminal in FIG. 1 as an example for description, the method includes the following steps: Step 202: a working scene image acquired by a camera arranged at a power production site is obtained.

Specifically, the camera is arranged at the power production site to acquire the working scene image. The image acquired by the camera is stored in a database, a file name is set to be a camera name + a preset position + time, and the name is stored in the database. If the camera name is "middle aisle of unit #1 on 6.5-meter floor of main factory (dome camera)", the preset position is "1", and the time is "2020-09-24 16:13:14", then the corresponding working scene image is "middle aisle of unit #1 on 6.5-meter floor of main factory (dome camera) 1 2020-09-24 16:13:14". A video acquired by the camera is switched and inspected according to a set time interval in a monitoring frame.

Step 204: the working scene image is input to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result.

The ledger object refers to a power production device. Different power production devices have different detection frequencies. For example, the set algorithmic frequencies of ledger objects, which are a unit #1, a unit #2, a unit #3, a unit #4, etc. are 30 min/times.

The set algorithmic frequencies of ledger objects, which are a 400V power disc of the unit #1, the 400V power disc of the unit #1, the 400V power disc of the unit #1, the 400V power disc of the unit #1, etc. are 60 min/times. The set algorithmic frequencies of ledger objects, which are an oil sump tank of a governor of the #2 unit, an oil pressure tank of the governor of the unit #2, etc. are 10 min/times.

When detection time is reached, the corresponding working scene image is input to the detection model according to the detection frequency of the ledger device so as to obtain the abnormality detection result.

That the working scene image is input to a detection model according to the detection frequency of the ledger object corresponding to the working scene image so as to obtain an abnormality detection result, includes: the detection frequency of the ledger object corresponding to the working scene image is obtained; when detection time corresponding to the detection frequency is reached, the working scene image is input to the detection model, it is detected whether the power production site is abnormal by the detection model, and a classification result and a segmentation result of an abnormal area segmented from the working scene image when the classification result is abnormal are output, where the classification result includes normal, oil leakage, steam leakage, and water leakage.

The oil leakage, the steam leakage, and the water leakage are three types of abnormal conditions that exist.

Specifically, the camera name and the preset position of the working scene image are obtained; the ledger object is matched according to the camera name and the preset position; if a single ledger object is matched, the detection frequency corresponding to the ledger object is obtained; and if a plurality of ledger objects are matched, a minimum detection frequency in detection frequencies the plurality of ledger objects is obtained.

The camera name is usually related to an installation position and a type of the camera. For example, the camera name "middle aisle of unit #1 on 6.5-meter floor of main factory (dome camera)" represents that the installation position of the camera is the middle aisle of the unit #1 on the 6.5-meter floor of the main factory, and the type of camera is the dome camera. The preset position is a way of associating a monitored key area with an operational state of the dome camera. When a tripod operates to a place that needs to be emphatically monitored, a command of setting a preset point is sent to the dome camera, and the dome camera will record an orientation of the tripod and a status of the camera at this time, and associate them with a number of the preset point. When a user operates the monitoring tripod of the terminal by a control device to monitor a target, an operator can set a preset position for the current monitored target, for example, a moving point tripod can be rotated in all directions at 365° or 360° for monitoring; the operator can set a location of a power production device as the preset position; and the preset position that is set can be saved on a decoder of the monitoring tripod of the terminal by software operation of the control device. When the user needs to quickly monitor a monitored target, a position that needs to be monitored can be called out by a call command of the control device. It can be understood that different preset positions correspond to different production areas.

The ledger object is matched according to the camera name and the preset position of the camera. If the single ledger object is matched with the camera, the set algorithmic frequency of the ledger object is set to be a frequency of sending data of the camera to an algorithm for detection. If the plurality of ledger objects are matched with the camera, the minimum frequency in frequencies of the ledger objects matched with the camera is set to be a frequency of sending the data of the camera to the algorithm for detection.

In addition, a method for server resource allocation and estimation can be calculated out according to the frequency of sending the data of the camera to the algorithm, and a calculation method is as the following formula (1): S=(NXxA)/TXBxC/DXE (1) where S represents an estimated value of resources required by a server; N represents the total number of times of executing the algorithm during a peak period in a cycle (a product of a duration of the peak period and the algorithmic frequency), A represents a ratio of time of the peak period in the cycle to total time of the cycle; T represents the duration of the peak period in the cycle; B represents a ratio of actual operation of the algorithm to the complexity of a test environment; C represents a redundancy reserve for future development of a system, which needs to be estimated according to the application system; D represents an optimal utilization rate of the system, the optimal utilization rate is 75% because the excessively high utilization rate of the system results in system bottleneck, and this value is generally set to be C=75%; and E is the amount of resources required by a single algorithm in the test environment.

The detection model is obtained by training according to label data in advance, and detects whether a working scene is abnormal, and the classification result and the segmentation result of the abnormal area segmented from the working scene image when the classification result is abnormal are output, where the classification result includes normal, oil leakage, steam leakage, and water leakage. The water leakage refers to an abnormal condition such as leakage, ejection or surface water immersion of a water- related component such as a water storage tank or a water tube. The oil leakage 1s similar to the water leakage, and refers to an abnormal condition such as leakage, ejection or surface oil immersion of an oil-containing component such as an oil storage tank or an oil tube. The steam leakage includes high-temperature steam leakage and white steam leakage.

Specifically, if the classification result is any one of the oil leakage, the steam leakage, and the water leakage, the segmentation result of the abnormal area segmented from the working scene image is an abnormal part displayed in the image.

As shown in FIG. 3A that is a working scene image of water leakage, a water leakage area is in a box, and FIG. 3B 1s a segmentation result of the water leakage area segmented from the working scene image. As shown in FIG. 4A that is a working scene image of steam leakage, a steam leakage area is in a box, and FIG. 4B is a segmentation result of the steam leakage area segmented from the working scene image. As shown in FIG. SA that is a working scene image of oil leakage, an oil leakage area is in a box, and FIG. 5B is a segmentation result of the oil leakage area segmented from the working scene image.

Step 206: when the detection result that a work site is abnormal is obtained according to the detection model, a corresponding secondary abnormality detection model is called according to a type of abnormality to perform secondary detection on the work site.

Specifically, if the classification result obtained by the detection model is any one of the oil leakage, the steam leakage, and the water leakage, the secondary detection is performed on the work site according to the secondary abnormality detection model corresponding to the type of abnormality. That is to say, different types of abnormalities correspond to different secondary abnormality detection models, and for the types of abnormalities, different secondary abnormality detection models can be set separately according to the types of abnormalities, such that the detection accuracy can be further improved.

Step 208: if the secondary detection still shows that the power production site is abnormal, an abnormal alarm is given.

Specifically, if the secondary detection still shows that the power production site is abnormal, a result is output and an alarm is given, where the output alarm result includes a camera name, a monitored object, a spatial position path, etc.; otherwise, the flow returns to start to continue monitoring.

After three conditions of the oil leakage, the water leakage and the steam leakage are recognized, the name of the camera giving the alarm and the content of the monitored object will be output in addition to an alarm prompt. For example, if the classification result is the oil or water leakage, the result is output and the alarm is given, where the output alarm result includes a camera name, a monitored object, a spatial position path, etc.; otherwise, the flow returns to start to continue monitoring. For example, if the classification result is the steam leakage, the result is output and the alarm is given, where the output alarm result includes a camera name, a monitored object, a spatial position path, etc.; otherwise, the flow returns to start to continue monitoring.

According to the above-mentioned method for monitoring abnormality of power production, the working scene image acquired by the camera at the production site is acquired; after the abnormality is detected by means of the pre-trained detection model, the secondary abnormality detection model corresponding to the type of abnormality is called to perform the secondary detection on the work site; higher-level features are extracted by means of a neural network model; and secondary detection and recognition are further performed, such that the detection accuracy 1s improved and the false alarm rate is reduced. In addition, when the abnormality is detected by means of the method, only the camera needs to be arranged at the power production site. The camera has a wide acquisition range, and does not need to be arranged complexly, such that the management cost can be reduced.

In an embodiment, a process for training a detection model, as shown in FIG. 6, includes: S602: an image sample set including samples, and segmentation labels for labeling pixel points of the samples are obtained.

Specifically, a large number of work site images including image data of oil leakage, water leakage, steam leakage, normal and other scenes are collected, then pixel-level labeling is performed on all the images, segmentation labels are made, and all pixels are divided into 4 categories: normal, water leakage, oil leakage, and steam leakage. The images are randomly divided into two parts, one part is used as a training image sample set, and the other part is used as a test data set.

S604: a feature extraction network and a segmentation network of the detection model are trained according to the segmentation labels of the samples labeled in the image sample set.

In one embodiment, the detection model 70 has a structure as shown in FIG. 7, and includes a feature extraction network 701, a segmentation network 702, a spatial attention network 703, and a classification network 704. The feature extraction network 701 is respectively connected with the segmentation network 702 and the spatial attention network 703, and the spatial attention network 703 is connected with the classification network 704.

First, parameters of a spatial attention module and a classification module are fixed, and the samples of the image sample set are input to the feature extraction network 701 to extract image features. The feature extraction network is a multi-layer convolutional neural network for extracting advanced features F € R“W" of an input image. Generally, convolutional neural networks such as VGG19, MobileNet, etc. can be selected and used.

Second, the image features are input to the segmentation network, and segmentation results of abnormal areas are output.

The abnormal area refers to the oil, water or steam leakage area determined by the features. A segmentation module is the multi-layer convolutional neural network, a final layer of the network 1s connected to a sigmoid activation function, and a mask code w € RY of the oil, water or steam leakage area is output. On the basis of a general classification neural network, a water, oil or steam leakage segmentation module branch is added to output an area mask code of the water, oil or steam leakage area. A spatial attention mask code output by the branch is combined with features of a classification branch to improve the classification accuracy.

Specifically, the segmentation network may use a network structure as shown in FIG. 8, where conv3x3 represents a convolution block with a convolution kernel size of 3x3 and a step length of 1, including convolution operation and a nonlinear unit rule activation function.

Finally, according to the differences between predicted segmentation results and labeled segmentation results, the segmentation network and the feature extraction network of the detection model are adjusted. A loss function may be a loss function L1 or L2. When a loss of the model converges and does not decrease, parameters of the detection model are trained. S606: classification labels of the samples are obtained according to the trained feature extraction network and segmentation network. The samples in the sample set do not have classification labels, and the classification labels are obtained by means of the trained feature extraction network and segmentation network. Specifically, sample images are input to the detection model; image features are extracted by the feature extraction network of the detection model; the image features are input to the segmentation network; segmentation results of abnormal areas are output, and the classification labels of the samples are generated according to the segmentation results.

Specifically, the samples of the image sample set are input to the detection model, and the image features are extracted by the feature extraction network 701. The feature extraction network is the multi-layer convolutional neural network for extracting advanced features F € RW of the input image. Generally, the convolutional neural networks such as VGG19, MobileNet, etc. can be selected and used.

The abnormal area refers to the oil, water or steam leakage area determined by the features. The segmentation module is the multi-layer convolutional neural network, the final layer of the network is connected to the sigmoid activation function, and the mask code w ER“ of the oil, water or steam leakage area is output.

Specifically, the segmentation network may use the network structure as shown in FIG. 8, where conv3x3 represents the convolution block with the convolution kernel size of 3x3 and the step length of 1, including the convolution operation and the nonlinear unit rule activation function.

The classification labels are not made in the sample set. Classification label data needs to be dynamically generated during training. A method for generating the classification label data is as follows: an output result w of the segmentation module is binarized to obtain w”. A binarization threshold can be set as required, has a range of (0, 1) , and may be 0.5. Then the classification labels of water leakage, oil leakage, and steam leakage are calculated according to the following formulas: s=f/ if Ew’©v” >T, @) 0 others.

=? if Ew” © v* >T, 3) 0 others.

s2={1 if wl © vw”? >T, (4) 0 others.

where 8%. 81. 5? are respectively labels for water leakage, oil leakage, and steam leakage, and T is a threshold for determining water leakage, oil leakage, and steam leakage, and can be set as required. A classification model can be trained by using the dynamically generated labels.

In actual scenes, the frequency of oil leakage, water leakage, and steam leakage is relatively low, and it is difficult to collect positive sample data of water leakage, oil leakage, and steam leakage. It is difficult to design and train a detection model for data with only a few positive samples. Based on this, the classification labels are obtained according to the trained feature extraction network and segmentation network. Specifically, a two-stage classification method is proposed, in which candidate areas are generated by using the segmentation network and then are classified to balance the number of positive and negative samples so as to improve the accuracy of a classifier.

S608: the detection model is trained according to the classification labels and the segmentation labels of the samples labeled in the image sample set.

Specifically, the segmentation results and the image features are input to the spatial attention network to obtain attention features, the attention features are input to the classification network to obtain predicted abnormality classification results, and the detection model is adjusted according to the differences between the predicted classification results and the labeled classification results, and is iteratively trained, until a training completion condition is met, such that the trained detection model is obtained.

The changes of oil leakage and water leakage on the image are often relatively subtle and slow. The actual application scene environment is complex, and background and surface materials are different, such that the imaging differences among the oil leakage, the water leakage and the steam leakage are relatively large. For different complex scenes, it is a major difficulty in ensuring the accuracy and robustness of a detection system. In view of this difficulty, the network is made to focuses on a specific area in combination with a spatial attention mechanism, thereby reducing the interference of a complex background to a detection algorithm, and improving the robustness and accuracy of the algorithm.

Specifically, the spatial attention network is pixel-level operation for the features.

A formula F'=F OQ W © F may be used, where W = [w,w, ..,w], and WE RW © is pixel point multiplication, and @ is pixel addition.

The classification network includes a plurality of convolution layers, pooling layers, fully connected layers, and output layers, and 1s used to output a score result s of water leakage detection. s = (s!, s?, s3) ER? is a vector with 3 dimensions. The values of the dimensions represent scores of water leakage, oil leakage, and steam leakage, respectively. The higher the value 1s, the greater the confidence ratio of the corresponding state 1s.

Specifically, the classification network may use a network structure as shown in FIG. 9, where conv3x3 represents the convolution block with the convolution kernel size of 3x3 and the step length of 1, including the convolution operation and the nonlinear unit rule activation function. Global pooling represents that the resolution of input features is pooled to a specific size such as 7x7 by using global pooling operation. MLP is a multi- layer perceptron network model that expands pooled features into a 1-dimensional vector and then inputs the vector to a two-fully connected layer neural network. Finally, the classification results are obtained by a softmax function.

Specifically, the differences between the predicted classification results and the labeled classification results are back-propagated, and the detection model is adjusted, and iteratively trained, until the training completion condition is met, such that the trained detection model is obtained. The training completion condition may be that the number of iterations reaches the maximum number of iterations, or the model accuracy meets the requirements. The iteration refers to re-performing of the above training process for the samples. During training, a test may be performed by using the test data set to observe a training effect.

Compared with a conventional method for extracting features manually, the above- mentioned method for monitoring abnormality of power production has the advantages that the method using the convolutional neural network has the better feature extraction capability, the classification network combining the segmentation module and the spatial attention module can focus on possible water, oil and steam leakage areas, and the feature extraction capability of the network and the robustness of the network model can be improved.

In another embodiment, that a corresponding secondary abnormality detection model is called according to a type of abnormality to perform secondary detection on a work site includes: if the type of abnormality is oil leakage and/or water leakage, the working scene image is input to an oil and water leakage secondary recognition model which uses a residual network structure, and a secondary abnormality detection result is output by the oil and water leakage secondary recognition model.

If the type of abnormality detected by the detection model is oil leakage and/or water leakage, the image with detected oil and water leakage is input to the oil and water leakage secondary recognition model for secondary recognition.

ResNet50 (a 50-layer residual network) is used as the network structure of the oil and water leakage secondary recognition model, and the number of hidden units in the final fully connected layer is set to be 3, which respectively correspond to classification scores of three conditions including oil leakage, water leakage, and normal. ResNet50 is trained with data of three categories including oil leakage, water leakage, and normal by means of stochastic gradient descent. In order to prevent overfitting of the network, network training is performed by means of dropout (randomly discarding neurons).

In another embodiment, if the type of abnormality is steam leakage, an infrared image acquired by an infrared camera of the ledger object corresponding to the working scene image is obtained; and the infrared image is input into an infrared image recognition model, features of the infrared image are extracted by a feature extraction network of the infrared image recognition model, and the features of the infrared image are input to a classifier so as to obtain a secondary abnormality detection result about whether the steam leakage occurs.

Specifically, according to the ledger object corresponding to a file name of the image with detected steam leakage, the infrared camera for monitoring the ledger object is matched, and infrared image data at corresponding time is obtained; and the obtained corresponding infrared image is recognized by means of an infrared image recognition module.

The infrared image module includes a feature extraction module and a classification module, where the feature extraction module uses a conventional histogram of oriented gradient (HOG) feature extraction method, and the classification module is a support vector machine (SVM) classifier. According to the present invention, positive samples (infrared image samples, in various attitudes and forms, that contain steam leakage) and negative samples (any infrared image samples that do not contain steam leakage) of the steam leakage image are used, and HOG features are extracted and then sent to the SVM classifier to finally form a training model for classification. The classification result output by the classifier is a result of secondary recognition of steam leakage.

The method for monitoring abnormality of power production in the present invention has the following effects:

1. A multi-task learning network combined with the spatial attention mechanism is proposed, and can simultaneously detect three abnormal states of oil leakage, water leakage and steam leakage.

2. In combination with the actual scene (power plant scene), a matching method for the shooting content of the polling camera and the monitored object is proposed, and by setting the frequency that needs to be recognized for the monitored object, the camera can automatically match with the processing frequency.

3. An estimation formula for computing resources is proposed, can calculate the computing resources required by algorithm configuration of the entire scene, and has strong practicability.

4. On the basis of the general classification neural network, the water, oil or steam leakage segmentation module branch is added to output the area mask code of the water,

oil or steam leakage area. The spatial attention mask code output by the branch is combined with the features of the classification branch to improve the classification accuracy.

5. After the detection of the network for oil, water and steam leakage detection is passed for the first time, a secondary recognition module is arranged for secondary verification to reduce the false alarm rate of detection, and secondary recognition of oil and water leakage is separated from secondary recognition of steam leakage, where the secondary recognition of water and oil leakage adopts a different monitoring network from first recognition, adopts other deep network, and has higher reliability than recognition using a same network structure, and the secondary recognition of steam leakage is performed by matching the monitored object by an image name and then matching the corresponding infrared image data; and a method for recognizing the infrared image is designed for recognition, thereby improving the detection accuracy of steam.

6. After three conditions including the oil leakage, the water leakage and the steam leakage are recognized, the name of the camera giving the alarm and the content of the monitored object will be output in addition to the alarm prompt.

It should be understood that although the steps in the above flowcharts are displayed in sequence according to the directions of arrows, these steps are not necessarily performed in sequence indicated by the arrows. Unless explicitly stated herein, the performing of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, and may be performed at different time. These steps or stages are also not necessarily performed in sequence, and may be performed in turn or alternately with other steps or at least a part of other steps or stages.

In an embodiment, as shown in FIG. 10, provided is an apparatus for monitoring abnormality of power production, including: an image obtaining module 1001 for obtaining a working scene image acquired by a camera arranged at a power production site; a detection module 1002 for inputting the working scene image to a detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; a secondary detection module 1003 for calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site when the detection result that the power production site is abnormal is obtained according to the detection model; and an alarm module 1004 for giving an abnormal alarm if the secondary detection still shows that the power production site is abnormal.

According to the above-mentioned apparatus for monitoring abnormality of power production, the working scene image acquired by the camera at the production site is acquired; after the abnormality is detected by means of the pre-trained detection model, the secondary abnormality detection model corresponding to the type of abnormality is called to perform the secondary detection on the work site; higher-level features are extracted by means of a neural network model; and secondary detection and recognition are further performed, such that the detection accuracy 1s improved and the false alarm rate is reduced. In addition, when the abnormality is detected by means of the method, only the camera needs to be arranged at the power production site. The camera has a wide acquisition range, and does not need to be arranged complexly, such that the management cost can be reduced.

In another embodiment, the detection module includes: a detection frequency determination module for obtaining the detection frequency of the ledger object corresponding to the working scene image; and a prediction module for, when detection time corresponding to the detection frequency is reached, inputting the working scene image to the detection model, detecting whether the power production site is abnormal by the detection model, and outputting a classification result and a segmentation result of an abnormal area segmented from the working scene image when the classification result is abnormal, where the classification result includes normal, oil leakage, steam leakage, and water leakage.

In another embodiment, the secondary detection module 1s configured to, 1f the type of abnormality is oil leakage and/or water leakage, input the working scene image to an oil and water leakage secondary recognition model which uses a residual network structure, and output a secondary abnormality detection result by the oil and water leakage secondary recognition model.

In another embodiment, the secondary detection module is configured to, 1f the type of abnormality 1s steam leakage, obtain an infrared image acquired by an infrared camera of the ledger object corresponding to the working scene image, input the infrared image to an infrared image recognition model, extract features of the infrared image by a feature extraction network of the infrared image recognition model, and input the features of the infrared image to a classifier so as to obtain a secondary abnormality detection result about whether the steam leakage occurs.

In another embodiment, the detection frequency determination module 1s configured to obtain a camera name and a preset position of the working scene image, match the ledger object according to the camera name and the preset position, obtain the detection frequency corresponding to the ledger object if a single ledger object is matched, and obtain a minimum detection frequency in detection frequencies of a plurality of ledger objects if the plurality of ledger objects are matched.

In another embodiment, the apparatus for monitoring abnormality of power production further includes: a sample set processing module for obtaining an image sample set including samples, and segmentation labels for labeling pixel points of the samples; a primary training module for training a feature extraction network and a segmentation network of the detection model according to the segmentation labels of the samples labeled in the image sample set; a classification module for obtaining classification labels of the samples according to the trained feature extraction network and segmentation network; and a secondary training module for training the detection model according to the classification labels.

In another embodiment, the classification module is configured to input the samples to the detection model, extract image features by the feature extraction network trained in the detection model, input the image features to the trained segmentation network, output segmentation results of abnormal areas, and generate the classification labels of the samples according to the segmentation results.

The specific limitation of the apparatus for monitoring abnormality of power production may refer to the above limitation to the method for monitoring abnormality of power production, and will not be repeated here. A part or all of the modules in the above- mentioned apparatus for monitoring abnormality of power production may be implemented by software, hardware, and a combination thereof. The above modules may be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory of the computer device in the form of software, such that the processor can call and execute operations corresponding to the above modules.

In an embodiment, a computer device is provided. The computer device may be a monitoring terminal, and its internal structure diagram may be as shown in FIG. 11. The computer device includes a processor, a memory, a communication interface, a display screen, and an input apparatus connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an operating environment for the operating system and the computer programs in the non-volatile storage medium. The communication interface of the computer device is configured to be in wired or wireless communication with an external terminal, and the wireless mode may be realized by WIFI, operator networks, near field communication (NFC) or other technologies. When the computer programs is executed by the processor, a method for monitoring abnormality of power production is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input apparatus of the computer device may be a touch layer covered on the display screen, or buttons, a trackball or a touchpad arranged on a shell of the computer device, or an external keyboard, touchpad or mouse.

Those skilled in the art can understand that the structure shown in FIG. 11 is only a block diagram of a partial structure related to a solution of the present invention, and does not constitute a limitation to the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than components shown in the figures, or a combination of certain components, or has different component arrangements.

In an embodiment, provided is a computer device including a memory storing computer programs and a processor, the memory storing computer programs, when executing the computer programs, the processor implements the steps in the embodiments of the above methods.

In an embodiment, provided is a computer readable storage on which computer programs are storied, where the computer programs, when executed by a processor, cause the processor to implement the steps in the embodiments of the above methods.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through the computer programs. The computer programs can be stored in a non- volatile computer-readable storage medium. The computer programs, when executed, may include the processes of the embodiments of the above methods. Any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention may include at least one of non-volatile and volatile memories. The non- volatile memory may include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, or an optical memory. The volatile memory may include a random access memory (RAM) or an external cache memory. By way of illustration and not limitation, the RAM may be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM).

The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combinations of these technical features, all the combinations are considered to be within the scope of this specification.

The above-mentioned embodiments only represent several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation to the scope of the invention patent. It should be noted that those of ordinary skill in the art also can make several modifications and improvements without departing from the concept of the present invention, which all belong to the scope of protection of the present invention. Therefore, the scope of protection of the patent of the present invention should be subject to the appended claims.

Claims (10)

1. A method for monitoring abnormality of power production, the method comprising: obtaining a working scene image acquired by a camera arranged at a power production site; inputting the working scene image to a pre-trained detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; when the detection result that the power production site is abnormal is obtained according to the detection model, calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site; and if the secondary detection still shows that the power production site is abnormal, giving an abnormal alarm.

2. The method according to claim 1, wherein inputting the working scene image to the pre- trained detection model according to the detection frequency of the ledger object corresponding to the working scene image so as to obtain the abnormality detection result, comprises: obtaining the detection frequency of the ledger object corresponding to the working scene image; and when detection time corresponding to the detection frequency is reached, inputting the working scene image to the detection model, detecting whether the power production site is abnormal by the detection model, and outputting a classification result and a segmentation result of an abnormal area segmented from the working scene image when the classification result is abnormal, where the classification result includes normal, oil leakage, steam leakage, and water leakage.

3. The method according to claim 1, wherein calling the corresponding secondary abnormality detection model according to the type of abnormality to perform secondary detection on the work site, comprises: if the type of abnormality is oil leakage and/or water leakage, inputting the working scene image to an oil and water leakage secondary recognition model that uses a residual network structure, and outputting a secondary abnormality detection result by the oil and water leakage secondary recognition model.

4. The method according to claim 1, wherein calling the corresponding secondary abnormality detection model according to the type of abnormality to perform secondary detection on the work site, comprises: if the type of abnormality is steam leakage, obtaining an infrared image acquired by an infrared camera of the ledger object corresponding to the working scene image; and inputting the infrared image into an infrared image recognition model, extracting features of the infrared image by a feature extraction network of the infrared image recognition model, and inputting the features of the infrared image to a classifier so as to obtain a secondary abnormality detection result about whether the steam leakage occurs.

5. The method according to claim 2, wherein obtaining the detection frequency of the ledger object corresponding to the working scene image, comprises: obtaining a camera name and a preset position of the working scene image; matching the ledger object according to the camera name and the preset position;

if a single ledger object is matched, obtaining the detection frequency corresponding to the ledger object; and if a plurality of ledger objects are matched, obtaining a minimum detection frequency in detection frequencies of the plurality of ledger objects.

6. The method according to claim 1, wherein a process for training a detection model comprises: obtaining an image sample set including samples, and segmentation labels for labeling pixel points of the samples; training a feature extraction network and a segmentation network of the detection model according to the segmentation labels of the samples labeled in the image sample set; obtaining classification labels of the samples according to the trained feature extraction network and segmentation network; and training the detection model according to the classification labels.

7. The method according to claim 6, wherein obtaining the classification labels of the samples according to the trained feature extraction network and segmentation network, comprises: inputting the samples to the detection model, and extracting image features by the trained feature extraction network in the detection model; inputting the image features to the trained segmentation network, and outputting segmentation results of abnormal areas; and generating the classification labels of the samples according to the segmentation results.

8. An apparatus for monitoring abnormality of power production, the apparatus comprising: an image obtaining module for obtaining a working scene image acquired by a camera arranged at a power production site; a detection module for inputting the working scene image to a detection model according to a detection frequency of a ledger object corresponding to the working scene image so as to obtain an abnormality detection result; a secondary detection module for calling a corresponding secondary abnormality detection model according to a type of abnormality to perform secondary detection on a work site when the detection result that the power production site is abnormal is obtained according to the detection model; and an alarm module for giving an abnormal alarm if the secondary detection still shows that the power production site is abnormal.

9. A computer device comprising a memory and a processor, the memory storing computer programs, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer programs.

10. A computer readable storage medium on which computer programs are storied, wherein the computer programs, when executed by a processor, cause the processor to implement the steps of the method according to any one of claims 1 to 7.