CN112966003A

CN112966003A - Data quality problem troubleshooting method based on neural network algorithm

Info

Publication number: CN112966003A
Application number: CN202110229700.8A
Authority: CN
Inventors: 李保平; 谢超; 王辉; 陈�峰; 李启航
Original assignee: Guangzhou Huitong Guoxin Technology Co Ltd
Current assignee: Guangzhou Huitong Guoxin Technology Co Ltd
Priority date: 2021-03-02
Filing date: 2021-03-02
Publication date: 2021-06-15

Abstract

The invention relates to a method for checking data quality problems based on a neural network algorithm, comprising the following steps: S1, compiling an experience set of on-site scheduling; S2, collecting system data, and auditing the system data; S3, establishing an event checking tree structure diagram ; S4, the reliability of each event in the event investigation tree structure diagram in the reasoning step S3; S5, calculate the weight distribution of each event in the event investigation tree structure diagram; S6, establish the investigation rule of the event investigation tree structure diagram; S7 , Check according to the checking rules. The beneficial effect is: relying on the system data, combined with the dispatching experience of the on-site dispatcher, using the big data algorithm combined with the neural network algorithm to establish an event inspection tree structure diagram, and then formulating inspection rules, when an alarm occurs in the system, the abnormal data is analyzed. The inspection rules are used for inspection to realize the preliminary location of the alarm event, and at the same time, the location result is fed back to the on-site personnel to improve the degree of automation.

Description

Data quality problem troubleshooting method based on neural network algorithm

Technical Field

The invention relates to the technical field of data processing, in particular to a data quality problem troubleshooting method based on a neural network algorithm.

Background

After the power plant or the transformer substation receives the requirement of the investigation data, the investigation process only depends on the experience of field personnel. Different dispatchers have different experiences, and the investigation sequence is different from the steps, so that only suspected problem points can be provided, and then substation operators or distribution network operators are contacted to investigate the problem of abnormal data in a coordinated manner, which requires huge investment in labor and is tedious and time-consuming in work.

Disclosure of Invention

The invention aims to overcome the problems in the prior art and provides a data quality problem troubleshooting method based on a neural network algorithm.

In order to achieve the technical purpose and achieve the technical effect, the invention is realized by the following technical scheme:

a data quality problem checking method based on a neural network algorithm comprises the following steps:

s1, enabling each dispatcher to tidy personal field dispatching experiences, and summarizing and compiling the dispatching experiences to form a field dispatching experience set;

s2, collecting system data and auditing the system data;

s3, performing data characteristic analysis on the system data after auditing is completed, and establishing an event investigation tree structure diagram;

s4, the credibility of each event in the tree structure diagram is checked by using the site scheduling experience set inference step S3 made in the step S1;

s5, calculating the weight distribution of each event in the event troubleshooting tree graph;

s6, establishing a checking rule of the event checking tree structure chart when abnormal data occur in the system;

and S7, when the system gives an alarm, analyzing the abnormal data and then checking according to the checking rule of the event checking tree-shaped structure chart in the step S6.

Wherein the scheduling experience in step S1 includes exception data of the scheduled event, an event name of the scheduled event, and scheduling content of the scheduled event.

In step S1, the field scheduling experience set summarizes according to the event names of the scheduling events, sorts according to the sequence of the system service flow, performs data characteristic analysis and qualification on the abnormal data belonging to the same scheduling event, and merges and summarizes the scheduling contents belonging to the same scheduling event.

The system data in step S2 includes ledger and measurement data, system operation and maintenance data, system alarm data, and system business process.

The auditing of the system data in step S2 includes rechecking integrity of ledgers and metering data issued by the provincial power grid data platform, re-extracting system operation and maintenance data, checking quantity and authenticity of alarm information, and re-combing system business processes.

In step S3, the event-troubleshooting tree structure diagram constructs a frame according to the system service flow, and a single event is used as an independent node, where each node includes an event name, an event data storage location link, and a qualitative word of an event data feature.

In step S4, the inference step S3 of the reliability of each event in the event-finding tree structure diagram by using the field scheduling experience set created in step S1 specifically includes: and calling scheduling experience information which is in the same field scheduling experience set as the event names of the nodes in the event-troubleshooting tree-shaped structure chart, comparing whether the data, the data characteristics and the qualitative words are consistent, if so, informing a dispatcher of carrying out manual reasoning and carrying out manual modification on the label with standard credibility of the corresponding node of the event-troubleshooting tree-shaped structure chart, and if not, informing the dispatcher of carrying out manual reasoning and carrying out manual modification.

In step S5, the calculating the weight distribution of each event in the event-based tree view specifically includes: and carrying out weight distribution from large to small according to the negative influence degree of the event on the normal operation of the power grid.

The invention has the beneficial effects that: by depending on system data, combining with scheduling experience of a field dispatcher, using a big data algorithm and a neural network algorithm to establish an event troubleshooting tree structure chart, further formulating a troubleshooting rule, analyzing abnormal data and then troubleshooting according to the troubleshooting rule when the system gives an alarm, realizing primary positioning of the alarm event, simultaneously feeding back a positioning result to the field personnel, and improving the automation degree.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

fig. 1 is a schematic structural diagram of a data quality problem troubleshooting method in an embodiment of the present invention.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In the description of the present invention, it is to be understood that the terms "opening," "upper," "lower," "thickness," "top," "middle," "length," "inner," "peripheral," and the like are used in an orientation or positional relationship that is merely for convenience in describing and simplifying the description, and do not indicate or imply that the referenced component or element must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be considered as limiting the present invention.

As shown in fig. 1, a method for troubleshooting data quality problems based on a neural network algorithm includes the following steps:

s2, collecting system data and auditing the system data;

By depending on system data, combining with scheduling experience of a field dispatcher, using a big data algorithm and a neural network algorithm to establish an event troubleshooting tree structure chart, further formulating a troubleshooting rule, analyzing abnormal data and then troubleshooting according to the troubleshooting rule when the system gives an alarm, realizing primary positioning of the alarm event, simultaneously feeding back a positioning result to the field personnel, and improving the automation degree.

The scheduling experience in step S1 includes the exception data of the scheduled event, the event name of the scheduled event, and the scheduling content of the scheduled event.

In step S1, the field scheduling experience set is summarized according to the event names of the scheduling events, sorted according to the sequence of the system service flow, subjected to data characteristic analysis and qualification on the abnormal data belonging to the same scheduling event, and merged and summarized in the scheduling contents belonging to the same scheduling event.

The auditing of the system data in step S2 includes rechecking the integrity of ledgers and metering data issued by the provincial power grid data platform, re-extracting system operation and maintenance data, checking the number and authenticity of alarm information, and rechecking the system business process.

In step S3, the event-troubleshooting tree structure diagram constructs a framework according to the system service flow, and a single event is used as an independent node, where each node includes an event name, an event data storage location link, and a qualitative word of an event data feature.

In step S5, the calculating the weight distribution of each event in the event-based tree includes: and carrying out weight distribution from large to small according to the negative influence degree of the event on the normal operation of the power grid. When one or more data characteristics are close to a plurality of nodes at the same time, the data characteristics are compared in sequence from large to small according to the weight, so that the delay time of an event with large negative influence degree on the normal operation of the power grid is avoided.

The first example of the investigation: IF 2 substation transformation ratio error = is

AND imbalance = mild

AND persistent present = is

AND has not previously occurred = is

THEN determines that the ratio data is not updated

Investigation example two: IF 2 substation transformation ratio error = is

AND imbalance = moderate

AND persistent present = is

AND not before present = no

THEN judges that the transformation ratio data is not standard and the value range is wrong

In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed.

Claims

1. a method for checking data quality problems based on neural network algorithm, is characterized in that, comprises the following steps:

S1. Let each dispatcher sort out his personal on-site scheduling experience, and summarize and compile these scheduling experiences into an on-site scheduling experience set;

S2. Collect system data and audit the system data;

S3. Perform data feature analysis on the system data after the audit is completed, and establish an event investigation tree structure diagram;

S4, using the on-site scheduling experience set made in step S1 to reason about the reliability of each event in the event investigation tree structure diagram in step S3;

S5. Calculate the weight distribution of each event in the event investigation tree diagram;

S6. Establish an inspection rule for the event inspection tree structure diagram when abnormal data occurs in the system;

S7. When an alarm occurs in the system, the abnormal data is analyzed and then checked according to the checking rules of the event checking tree structure diagram in step S6.

2 . The data quality problem troubleshooting method according to claim 1 , wherein the scheduling experience in the step S1 includes the abnormal data of the scheduling event, the event name of the scheduling event, and the scheduling content of the scheduling event. 3 .

3. The method for checking data quality problems according to claim 2, wherein the on-site scheduling experience set in the step S1 is summarized according to the event name of the scheduling event, sorted according to the order of the system business process, and will belong to The abnormal data of the same scheduling event is analyzed and characterized, and the scheduling content belonging to the same scheduling event is merged and summarized.

4 . The data quality problem troubleshooting method according to claim 1 , wherein the system data in the step S2 includes ledger and metering data, system operation and maintenance data, system alarm data, and system business process. 5 .

5. The method for checking data quality problems according to claim 4, characterized in that: in the step S2, auditing the system data includes re-checking the integrity of the ledger and metering data issued by the provincial power grid data platform, re-extracting System operation and maintenance data, check the quantity and authenticity of alarm information, and reorganize system business processes.

6. The method for checking data quality problems according to claim 1, characterized in that: in the step S3, the event checking tree structure diagram constructs a framework according to the system business process, and takes a single event as an independent node, and each node contains Event name, event data storage location link, qualitative word for event data characteristics.

7. The data quality problem investigation method according to claim 6, characterized in that, in the step S4, the on-site scheduling experience set made in the step S1 is used to infer the possibility of each event in the event investigation tree structure diagram in the step S3. Reliability specifically includes: retrieving the dispatching experience information in the on-site dispatching experience set with the same event name as the event name of the node in the event inspection tree structure diagram, comparing whether the data, data features, and qualitative words are consistent, and if they are consistent, the event inspection tree structure The corresponding nodes of the graph are standard and credible labels. If they are inconsistent, the dispatcher will be informed to perform manual reasoning and manual modification.

8 . The data quality problem troubleshooting method according to claim 1 , wherein in the step S5, calculating the weight distribution of each event in the event inspection tree diagram specifically includes: according to the degree of negative impact of the event on the normal operation of the power grid, ranging from: Weights are assigned from large to small.