CN112966003A - Data quality problem troubleshooting method based on neural network algorithm - Google Patents
Data quality problem troubleshooting method based on neural network algorithm Download PDFInfo
- Publication number
- CN112966003A CN112966003A CN202110229700.8A CN202110229700A CN112966003A CN 112966003 A CN112966003 A CN 112966003A CN 202110229700 A CN202110229700 A CN 202110229700A CN 112966003 A CN112966003 A CN 112966003A
- Authority
- CN
- China
- Prior art keywords
- event
- data
- troubleshooting
- checking
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013024 troubleshooting Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 11
- 238000010586 diagram Methods 0.000 claims abstract description 14
- 238000011835 investigation Methods 0.000 claims abstract description 14
- 230000002159 abnormal effect Effects 0.000 claims abstract description 13
- 238000012986 modification Methods 0.000 claims description 7
- 230000004048 modification Effects 0.000 claims description 7
- 238000012423 maintenance Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000012797 qualification Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000009466 transformation Effects 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Human Resources & Organizations (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Primary Health Care (AREA)
- Marketing (AREA)
- Water Supply & Treatment (AREA)
- Databases & Information Systems (AREA)
- Public Health (AREA)
- Tourism & Hospitality (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a data quality problem checking method based on a neural network algorithm, which comprises the following steps: s1, compiling a field scheduling experience set; s2, collecting system data and auditing the system data; s3, establishing an event investigation tree structure diagram; s4, reasoning step S3 event troubleshooting tree structure chart each event credibility; s5, calculating the weight distribution of each event in the event troubleshooting tree graph; s6, establishing a checking rule of the event checking tree structure chart; and S7, checking according to the checking rule. The beneficial effects are that: by depending on system data, combining with scheduling experience of a field dispatcher, using a big data algorithm and a neural network algorithm to establish an event troubleshooting tree structure chart, further formulating a troubleshooting rule, analyzing abnormal data and then troubleshooting according to the troubleshooting rule when the system gives an alarm, realizing primary positioning of the alarm event, simultaneously feeding back a positioning result to the field personnel, and improving the automation degree.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data quality problem troubleshooting method based on a neural network algorithm.
Background
After the power plant or the transformer substation receives the requirement of the investigation data, the investigation process only depends on the experience of field personnel. Different dispatchers have different experiences, and the investigation sequence is different from the steps, so that only suspected problem points can be provided, and then substation operators or distribution network operators are contacted to investigate the problem of abnormal data in a coordinated manner, which requires huge investment in labor and is tedious and time-consuming in work.
Disclosure of Invention
The invention aims to overcome the problems in the prior art and provides a data quality problem troubleshooting method based on a neural network algorithm.
In order to achieve the technical purpose and achieve the technical effect, the invention is realized by the following technical scheme:
a data quality problem checking method based on a neural network algorithm comprises the following steps:
s1, enabling each dispatcher to tidy personal field dispatching experiences, and summarizing and compiling the dispatching experiences to form a field dispatching experience set;
s2, collecting system data and auditing the system data;
s3, performing data characteristic analysis on the system data after auditing is completed, and establishing an event investigation tree structure diagram;
s4, the credibility of each event in the tree structure diagram is checked by using the site scheduling experience set inference step S3 made in the step S1;
s5, calculating the weight distribution of each event in the event troubleshooting tree graph;
s6, establishing a checking rule of the event checking tree structure chart when abnormal data occur in the system;
and S7, when the system gives an alarm, analyzing the abnormal data and then checking according to the checking rule of the event checking tree-shaped structure chart in the step S6.
Wherein the scheduling experience in step S1 includes exception data of the scheduled event, an event name of the scheduled event, and scheduling content of the scheduled event.
In step S1, the field scheduling experience set summarizes according to the event names of the scheduling events, sorts according to the sequence of the system service flow, performs data characteristic analysis and qualification on the abnormal data belonging to the same scheduling event, and merges and summarizes the scheduling contents belonging to the same scheduling event.
The system data in step S2 includes ledger and measurement data, system operation and maintenance data, system alarm data, and system business process.
The auditing of the system data in step S2 includes rechecking integrity of ledgers and metering data issued by the provincial power grid data platform, re-extracting system operation and maintenance data, checking quantity and authenticity of alarm information, and re-combing system business processes.
In step S3, the event-troubleshooting tree structure diagram constructs a frame according to the system service flow, and a single event is used as an independent node, where each node includes an event name, an event data storage location link, and a qualitative word of an event data feature.
In step S4, the inference step S3 of the reliability of each event in the event-finding tree structure diagram by using the field scheduling experience set created in step S1 specifically includes: and calling scheduling experience information which is in the same field scheduling experience set as the event names of the nodes in the event-troubleshooting tree-shaped structure chart, comparing whether the data, the data characteristics and the qualitative words are consistent, if so, informing a dispatcher of carrying out manual reasoning and carrying out manual modification on the label with standard credibility of the corresponding node of the event-troubleshooting tree-shaped structure chart, and if not, informing the dispatcher of carrying out manual reasoning and carrying out manual modification.
In step S5, the calculating the weight distribution of each event in the event-based tree view specifically includes: and carrying out weight distribution from large to small according to the negative influence degree of the event on the normal operation of the power grid.
The invention has the beneficial effects that: by depending on system data, combining with scheduling experience of a field dispatcher, using a big data algorithm and a neural network algorithm to establish an event troubleshooting tree structure chart, further formulating a troubleshooting rule, analyzing abnormal data and then troubleshooting according to the troubleshooting rule when the system gives an alarm, realizing primary positioning of the alarm event, simultaneously feeding back a positioning result to the field personnel, and improving the automation degree.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a schematic structural diagram of a data quality problem troubleshooting method in an embodiment of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
In the description of the present invention, it is to be understood that the terms "opening," "upper," "lower," "thickness," "top," "middle," "length," "inner," "peripheral," and the like are used in an orientation or positional relationship that is merely for convenience in describing and simplifying the description, and do not indicate or imply that the referenced component or element must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be considered as limiting the present invention.
As shown in fig. 1, a method for troubleshooting data quality problems based on a neural network algorithm includes the following steps:
s1, enabling each dispatcher to tidy personal field dispatching experiences, and summarizing and compiling the dispatching experiences to form a field dispatching experience set;
s2, collecting system data and auditing the system data;
s3, performing data characteristic analysis on the system data after auditing is completed, and establishing an event investigation tree structure diagram;
s4, the credibility of each event in the tree structure diagram is checked by using the site scheduling experience set inference step S3 made in the step S1;
s5, calculating the weight distribution of each event in the event troubleshooting tree graph;
s6, establishing a checking rule of the event checking tree structure chart when abnormal data occur in the system;
and S7, when the system gives an alarm, analyzing the abnormal data and then checking according to the checking rule of the event checking tree-shaped structure chart in the step S6.
By depending on system data, combining with scheduling experience of a field dispatcher, using a big data algorithm and a neural network algorithm to establish an event troubleshooting tree structure chart, further formulating a troubleshooting rule, analyzing abnormal data and then troubleshooting according to the troubleshooting rule when the system gives an alarm, realizing primary positioning of the alarm event, simultaneously feeding back a positioning result to the field personnel, and improving the automation degree.
The scheduling experience in step S1 includes the exception data of the scheduled event, the event name of the scheduled event, and the scheduling content of the scheduled event.
In step S1, the field scheduling experience set is summarized according to the event names of the scheduling events, sorted according to the sequence of the system service flow, subjected to data characteristic analysis and qualification on the abnormal data belonging to the same scheduling event, and merged and summarized in the scheduling contents belonging to the same scheduling event.
The system data in step S2 includes ledger and measurement data, system operation and maintenance data, system alarm data, and system business process.
The auditing of the system data in step S2 includes rechecking the integrity of ledgers and metering data issued by the provincial power grid data platform, re-extracting system operation and maintenance data, checking the number and authenticity of alarm information, and rechecking the system business process.
In step S3, the event-troubleshooting tree structure diagram constructs a framework according to the system service flow, and a single event is used as an independent node, where each node includes an event name, an event data storage location link, and a qualitative word of an event data feature.
In step S4, the inference step S3 of the reliability of each event in the event-finding tree structure diagram by using the field scheduling experience set created in step S1 specifically includes: and calling scheduling experience information which is in the same field scheduling experience set as the event names of the nodes in the event-troubleshooting tree-shaped structure chart, comparing whether the data, the data characteristics and the qualitative words are consistent, if so, informing a dispatcher of carrying out manual reasoning and carrying out manual modification on the label with standard credibility of the corresponding node of the event-troubleshooting tree-shaped structure chart, and if not, informing the dispatcher of carrying out manual reasoning and carrying out manual modification.
In step S5, the calculating the weight distribution of each event in the event-based tree includes: and carrying out weight distribution from large to small according to the negative influence degree of the event on the normal operation of the power grid. When one or more data characteristics are close to a plurality of nodes at the same time, the data characteristics are compared in sequence from large to small according to the weight, so that the delay time of an event with large negative influence degree on the normal operation of the power grid is avoided.
The first example of the investigation: IF 2 substation transformation ratio error = is
AND imbalance = mild
AND persistent present = is
AND has not previously occurred = is
THEN determines that the ratio data is not updated
Investigation example two: IF 2 substation transformation ratio error = is
AND imbalance = moderate
AND persistent present = is
AND not before present = no
THEN judges that the transformation ratio data is not standard and the value range is wrong
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed.
Claims (8)
1. A data quality problem checking method based on a neural network algorithm is characterized by comprising the following steps:
s1, enabling each dispatcher to tidy personal field dispatching experiences, and summarizing and compiling the dispatching experiences to form a field dispatching experience set;
s2, collecting system data and auditing the system data;
s3, performing data characteristic analysis on the system data after auditing is completed, and establishing an event investigation tree structure diagram;
s4, the credibility of each event in the tree structure diagram is checked by using the site scheduling experience set inference step S3 made in the step S1;
s5, calculating the weight distribution of each event in the event troubleshooting tree graph;
s6, establishing a checking rule of the event checking tree structure chart when abnormal data occur in the system;
and S7, when the system gives an alarm, analyzing the abnormal data and then checking according to the checking rule of the event checking tree-shaped structure chart in the step S6.
2. The data quality problem investigation method of claim 1, wherein: the scheduling experience in step S1 includes the exception data of the scheduled event, the event name of the scheduled event, and the scheduling content of the scheduled event.
3. The data quality problem investigation method of claim 2, wherein: in the step S1, the field scheduling experience set is summarized according to the event names of the scheduling events, sorted according to the sequence of the system service flow, subjected to data characteristic analysis and qualification on the abnormal data belonging to the same scheduling event, and merged and summarized in the scheduling contents belonging to the same scheduling event.
4. The data quality problem investigation method of claim 1, wherein: the system data in the step S2 includes ledger and measurement data, system operation and maintenance data, system alarm data, and system business process.
5. The data quality problem investigation method of claim 4, wherein: the auditing of the system data in step S2 includes rechecking the integrity of the ledger and metering data issued by the provincial power grid data platform, re-extracting the system operation and maintenance data, checking the number and authenticity of alarm information, and re-combing the system business process.
6. The data quality problem investigation method of claim 1, wherein: in the step S3, the event-finding tree structure diagram constructs a framework according to the system service flow, and a single event is used as an independent node, and each node includes an event name, an event data storage location link, and a qualitative word of an event data feature.
7. The method for troubleshooting data quality as recited in claim 6, wherein the step S4 of inferring the credibility of each event in the event troubleshooting tree structure diagram in the step S3 using the field scheduling experience set created in the step S1 specifically comprises: and calling scheduling experience information which is in the same field scheduling experience set as the event names of the nodes in the event-troubleshooting tree-shaped structure chart, comparing whether the data, the data characteristics and the qualitative words are consistent, if so, informing a dispatcher of carrying out manual reasoning and carrying out manual modification on the label with standard credibility of the corresponding node of the event-troubleshooting tree-shaped structure chart, and if not, informing the dispatcher of carrying out manual reasoning and carrying out manual modification.
8. The method for troubleshooting of data quality as claimed in claim 1, wherein the calculating the weight assignment of each event in the event troubleshooting tree in the step S5 specifically includes: and carrying out weight distribution from large to small according to the negative influence degree of the event on the normal operation of the power grid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110229700.8A CN112966003A (en) | 2021-03-02 | 2021-03-02 | Data quality problem troubleshooting method based on neural network algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110229700.8A CN112966003A (en) | 2021-03-02 | 2021-03-02 | Data quality problem troubleshooting method based on neural network algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112966003A true CN112966003A (en) | 2021-06-15 |
Family
ID=76276255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110229700.8A Pending CN112966003A (en) | 2021-03-02 | 2021-03-02 | Data quality problem troubleshooting method based on neural network algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112966003A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115222371A (en) * | 2022-09-08 | 2022-10-21 | 联信弘方(北京)科技股份有限公司 | Problem troubleshooting method and device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040015906A1 (en) * | 2001-04-30 | 2004-01-22 | Goraya Tanvir Y. | Adaptive dynamic personal modeling system and method |
| US20080086363A1 (en) * | 2006-10-06 | 2008-04-10 | Accenture Global Services Gmbh | Technology event detection, analysis, and reporting system |
| CN102819239A (en) * | 2011-06-08 | 2012-12-12 | 同济大学 | Intelligent fault diagnosis method of numerical control machine tool |
| US20140172919A1 (en) * | 2012-12-18 | 2014-06-19 | Cisco Technology, Inc. | Automatic correlation of dynamic system events within computing devices |
| CN106022583A (en) * | 2016-05-12 | 2016-10-12 | 中国电力科学研究院 | Electric power communication service risk calculation method and system based on fuzzy decision tree |
| CN106154209A (en) * | 2016-07-29 | 2016-11-23 | 国电南瑞科技股份有限公司 | Electrical energy meter fault Forecasting Methodology based on decision Tree algorithms |
| CN106961249A (en) * | 2017-03-17 | 2017-07-18 | 广西大学 | A kind of diagnosing failure of photovoltaic array and method for early warning |
| US20190286660A1 (en) * | 2018-03-13 | 2019-09-19 | HCA Holdings, Inc. | Techniques for generating investigatory-event mappings using graph-structure trajectories |
| US20200177608A1 (en) * | 2018-12-04 | 2020-06-04 | International Business Machines Corporation | Ontology Based Persistent Attack Campaign Detection |
-
2021
- 2021-03-02 CN CN202110229700.8A patent/CN112966003A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040015906A1 (en) * | 2001-04-30 | 2004-01-22 | Goraya Tanvir Y. | Adaptive dynamic personal modeling system and method |
| US20080086363A1 (en) * | 2006-10-06 | 2008-04-10 | Accenture Global Services Gmbh | Technology event detection, analysis, and reporting system |
| CN102819239A (en) * | 2011-06-08 | 2012-12-12 | 同济大学 | Intelligent fault diagnosis method of numerical control machine tool |
| US20140172919A1 (en) * | 2012-12-18 | 2014-06-19 | Cisco Technology, Inc. | Automatic correlation of dynamic system events within computing devices |
| CN106022583A (en) * | 2016-05-12 | 2016-10-12 | 中国电力科学研究院 | Electric power communication service risk calculation method and system based on fuzzy decision tree |
| CN106154209A (en) * | 2016-07-29 | 2016-11-23 | 国电南瑞科技股份有限公司 | Electrical energy meter fault Forecasting Methodology based on decision Tree algorithms |
| CN106961249A (en) * | 2017-03-17 | 2017-07-18 | 广西大学 | A kind of diagnosing failure of photovoltaic array and method for early warning |
| US20190286660A1 (en) * | 2018-03-13 | 2019-09-19 | HCA Holdings, Inc. | Techniques for generating investigatory-event mappings using graph-structure trajectories |
| US20200177608A1 (en) * | 2018-12-04 | 2020-06-04 | International Business Machines Corporation | Ontology Based Persistent Attack Campaign Detection |
Non-Patent Citations (1)
| Title |
|---|
| 杨剑 等: "基于告警加权的智能光网络故障诊断算法", 《湘潭大学自然科学学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115222371A (en) * | 2022-09-08 | 2022-10-21 | 联信弘方(北京)科技股份有限公司 | Problem troubleshooting method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110674189B (en) | Method for monitoring secondary state and positioning fault of intelligent substation | |
| CN106557991B (en) | Voltage monitoring data platform | |
| CN110707692A (en) | Online load analysis and modeling system and method for power system | |
| CN103501503B (en) | A kind of network problem analysis method and apparatus | |
| CN104933631A (en) | Power distribution network operation online analysis and evaluation system | |
| CN110647567B (en) | Power failure plan auxiliary decision system based on end data fusion | |
| CN112464995A (en) | Power grid distribution transformer fault diagnosis method and system based on decision tree algorithm | |
| CN101883015A (en) | Method and system for filtering construction alarms | |
| CN106897779A (en) | A kind of processing method of data center's operational system event | |
| CN112865316A (en) | Power supply service analysis command system and method based on big data | |
| CN112465167A (en) | Intelligent decision management system and method for regional power grid equipment maintenance | |
| CN118279086B (en) | Method and system for automatically generating interval protection configuration model data by SCD (stream control device) file | |
| CN112966003A (en) | Data quality problem troubleshooting method based on neural network algorithm | |
| CN108710566A (en) | A kind of distribution scheduling station integration testing framework and method | |
| CN116743079A (en) | Photovoltaic string fault processing method and device, photovoltaic management system and medium | |
| CN115409264A (en) | Power distribution network emergency repair stagnation point position optimization method based on feeder line fault prediction | |
| CN109995856A (en) | A kind of grid operation data wide area collects method and system | |
| CN113256074A (en) | Automatic process operation system and method for electric robot | |
| CN112256922A (en) | Fault power failure rapid identification method and system | |
| CN111639839A (en) | Micro-service-based power grid fault analysis method and system | |
| CN113904440B (en) | Topology analysis-based power system morning operation test scheme generation method | |
| CN110738427A (en) | electric power part work quality scoring system | |
| CN112446619B (en) | Power distribution network rush-repair processing method and device | |
| CN107748701A (en) | A kind of analysis method for reliability of electric energy measurement automation system | |
| CN115542070A (en) | Distribution network line fault positioning method and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |