CN107066500A - A kind of electrical network mass data quality indicator method based on PMS models - Google Patents
A kind of electrical network mass data quality indicator method based on PMS models Download PDFInfo
- Publication number
- CN107066500A CN107066500A CN201611265616.7A CN201611265616A CN107066500A CN 107066500 A CN107066500 A CN 107066500A CN 201611265616 A CN201611265616 A CN 201611265616A CN 107066500 A CN107066500 A CN 107066500A
- Authority
- CN
- China
- Prior art keywords
- data
- quality
- engine
- service
- pms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000009825 accumulation Methods 0.000 claims abstract description 29
- 238000012550 audit Methods 0.000 claims abstract description 24
- 230000005540 biological transmission Effects 0.000 claims abstract description 21
- 230000008569 process Effects 0.000 claims abstract description 20
- 238000012546 transfer Methods 0.000 claims abstract description 19
- 238000013524 data verification Methods 0.000 claims description 15
- 241001269238 Data Species 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000012216 screening Methods 0.000 claims description 4
- 238000003780 insertion Methods 0.000 claims description 2
- 230000037431 insertion Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 claims 1
- 238000012795 verification Methods 0.000 abstract description 8
- 238000012937 correction Methods 0.000 abstract description 3
- 230000002452 interceptive effect Effects 0.000 abstract description 3
- 238000012800 visualization Methods 0.000 abstract description 3
- 230000003862 health status Effects 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 13
- 238000004458 analytical method Methods 0.000 description 8
- 238000004519 manufacturing process Methods 0.000 description 8
- 238000013499 data model Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000009412 basement excavation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 241000894007 species Species 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- General Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Public Health (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of electrical network mass data quality indicator method based on PMS models, a kind of quality of data check system is provided, the system includes accumulation layer, engine layers, process layer and the presentation layer interconnected successively, the accumulation layer includes relational database and real-time data base, the engine layers include data transfer detecting and alarm, data quality checking engine and quality of data auditing engine, and the process layer includes data transfer detection service, data quality checking service and quality of data auditing service；The present invention can be rapidly completed the access function of real time data by real time data interactive tool；Data transmission state, data health status etc., which are directed to, according to power network observation point model tree and a variety of methods of calibration of combination carries out overall merit, pass through quality of data correction verification module, improve data quality checking accuracy in each operation system, by quality of data Audit Module, there is provided visualization, the quality of data Audit Report of integrality.
Description
Technical field
The invention belongs to power grid operation monitoring technical field, a kind of power network magnanimity number based on PMS models is specifically related to
According to quality indicator method.
Background technology
In the case where State Grid Corporation of China's " 12 planning " instructs, the construction of magnanimity near-realtime data service platform has been completed,
In part provincial company, the platform imports substantial amounts of real time data in management information great Qu, participates in the convergence analysis of information data,
Realize to the collections of the real-time class data of the systems such as production management system, SCADA, dispatching automation, measurement meter, on-line monitoring,
Access and analysis, realize the analysis and senior application (such as line loss analyzing, load Analysis) of business datum.But with power network certainly
The continuous increase of dynamicization system acquisition amount, coverage is more and more wider, and data storage will be doubled and redoubled, integrality to data,
Uniqueness, promptness, legitimacy, uniformity and accuracy etc. propose more strict requirements in order to big data mining analysis
Promote the development of power network big data business.
Engineering production management system (PMS, power production management system) is answered based on unified
Built with platform PI3000, using multilayer (being specifically divided into data Layer, service layer and presentation layer) frame of B/S and C/S mixed modes
Construction system.PI3000 platforms are the guidelines according to SG186 engineering Unified Application Platforms, and fusion current business basic software is put down
Platform theory, a set of service basic software platform towards power industry researched and developed based on model-driven and component concept.It sets
Meter target be can be directed to electric power enterprise demand complicated and changeable as one man create with maintenance service model, and opened for personalized application
Hair provides complete infrastructure, and the generation of automatic or assistance application system at utmost improves exploitation and the reality of application system
Apply efficiency.PMS thereon is built by high unity and continues extended capability with good, and standard criterion can met.
PMS set up be one using asset management as core, service coverage corporate HQ, net provincial company, prefecture-level company three
Individual aspect, through power network it is defeated, become, with production overall process integrated production management information platform, to realize power network production it is intensive
Change, become more meticulous, standardized management, improve corporate assets managerial skills tool and be of great significance.PMS as production management and
The workbench of one line teams and groups, by means such as standard criterion, flow monitoring, security monitorings, is finally realized to production management
Risk Pre-control and aid decision.
The content of the invention
Goal of the invention：In order to overcome the deficiencies in the prior art, the present invention provides a kind of power network based on PMS models
Mass data quality indicator method, can be rapidly completed the access function of real time data；By quality of data correction verification module, improve
Data quality checking accuracy in each operation system；By quality of data Audit Module, there is provided visualization, integrality.
Technical scheme：To achieve the above object, a kind of electrical network mass data quality indicator based on PMS models of the invention
Method there is provided a kind of quality of data check system, accumulation layer that the system includes interconnecting successively, engine layers, process layer and
Presentation layer, the accumulation layer includes relational database and real-time data base, and the engine layers include data transfer detecting and alarm, number
According to quality testing engine and quality of data auditing engine, the process layer includes data transfer detection service, data quality checking
Service and quality of data auditing service；
It the described method comprises the following steps：
S1 data acquisitions：The gathered data by way of FTP transmits E files, specifically data initially enter ftp server
E file datas are formed, then the E file datas enter interface server；
S2 enters accumulation layer：Data subsequently enter accumulation layer and are stored into real-time data base in accumulation layer in interface server
In；
S3 enters engine layers：Data transfer detecting and alarm is selected by engine layers；
S4 enters process layer：Data transfer detection service is entered according to data transfer detecting and alarm, FTP clothes are specifically detected
It is engaged in the data transmission state between device and interface server and data transmission state being stored in accumulation layer in relational database；
S5 enters engine layers：Data quality checking engine is selected by engine layers；
S6 enters process layer：Data quality checking engine service is entered according to data quality checking engine, specifically detected
Quality of data situation and quality of data situation is stored in accumulation layer in real-time data base in accumulation layer in relational database；
S7 enters engine layers：Quality of data auditing engine is selected by engine layers；
S8 enters process layer：Quality of data auditing service is entered according to quality of data auditing engine, according to the data of selection
Source and quality of data Audit Report in time parameter call relation database simultaneously check related data quality condition；
S9 enters presentation layer：Presentation layer is used to show related data quality condition.
Further, the step S4 comprises the following steps：
When data transmission state between ftp server and interface server interrupts for FTP, then alarm, Field Force are needed
The warning message is handled to keep FTP transmission smooth.
Further, the step S6 comprises the following steps：
S61 builds power network observation point model tree；
S62 chooses the measuring point for needing to detect in batches in observation point model tree；
S63 increases one group of quality of data method of calibration newly, and the quality of data method of calibration includes data leak source detection method, data
It is 0 detection method, data jump detection method, data burr detection method and Data duplication screening method for empty detection method, data；
S64 loads the regular library text of different types of quality of data method of calibration generation quality of data verification according to choosing measuring point
Part is simultaneously inserted quality of data verification rule base in accumulation layer in relational database；
S65 calls data quality checking service, and regular library file is verified simultaneously by reading the quality of data in relational database
The related daily data of measuring point proceed by data quality checking from real-time data base, finally by related measuring point data quality record
It is stored in relational database.
Further, the step S8 comprises the following steps：
S81 calls quality of data auditing service, using JXL technologies by the related data quality feelings stored in relational database
Condition and data transmission state generation Excel formatted files or using iText technologies by the related data stored in relational database
Quality condition and data transmission state generation PDF format file, using the Excel formatted files or PDF format file of generation as
Quality of data Audit Report；
S82 selects the quality of data Audit Report in data source and time parameter, call relation database and checks the number
According to the quality of data situation of the measuring point that power network observation point model seeds have been chosen in source.
Further, in the step S9, the presentation layer shows related data quality condition, related data quality condition
Including FTP interruptions, data be empty, data are 0, data leak source, Data duplication, data jump and data burr.
Beneficial effect：The present invention compared with the prior art, this have the advantage that：
The present invention can be rapidly completed the access function of real time data by real time data interactive tool；Surveyed according to power network
Point model tree simultaneously combines a variety of methods of calibration for the progress overall merit such as data transmission state, data health status, passes through number
According to quality indicator module, improve data quality checking accuracy in each operation system, by quality of data Audit Module there is provided
Visualization, the quality of data Audit Report of integrality；
The present invention provides a kind of electrical network mass data quality indicator method based on PMS device model structures, by integrating
Scheduling is main, distribution net equipment model, and completes and the mounting of magnanimity real time data, and a complete electricity is formed on mass data platform
Net real time data device model, configures the regular method of calibration of the different qualities of data with reference to real-time data of power grid device model and carries out
The quality indicator of mass data carries out quality of data audit to different operation systems, and defect, alarm to the quality of data etc. is tied
Fruit feeds back to management and control department, the basis of decision-making about being provided point for related service department, is mass data excavation, senior applied analysis
Strong support is provided；The quality of data verification based on mass data platform, applied customization are realized, is striven for each business department
Application and development based on real time data is attracted to be generated using the instrument easy configuration of platform, it is to avoid a large amount of unnecessary data matter
Cleaning is measured, mitigates the pressure of operation maintenance personnel, the operating efficiency of O＆M is improved.
Brief description of the drawings
Fig. 1 is the real-time data of power grid quality indicator flow chart based on PMS.
Fig. 2 is the quality of data check system Organization Chart based on PMS models.
Fig. 3 is data transfer detection module structure chart.
Fig. 4 is generation quality of data verification rule base document flowchart.
Fig. 5 is the real time data checking system functional block diagram based on PMS models.
Fig. 6 is component instance flow chart.
Embodiment
The present invention is further described below in conjunction with the accompanying drawings.
The present invention proposes a kind of electrical network mass data quality indicator method based on PMS models, the equipment mould based on PMS
Type structure, mainly integrates scheduling master, distribution net equipment model, then completes the mounting with magnanimity real time data, on magnanimity platform
A complete real-time data of power grid device model is formed, the different qualities of data are configured with reference to real-time data of power grid device model
The quality indicator that regular method of calibration carries out mass data carries out quality of data audit, reference picture 1, figure to different operation systems
1 is the real-time data of power grid quality indicator flow chart based on PMS, is to build the real-time data of power grid model based on PMS first, point
Not Jie Ru districts and cities EMS data and the PMS models that save of DMS data and access network, the PMS models saved for access network are, it is necessary to root
Real-time Data Model of the generation based on PMS models is defined according to various kinds of equipment；For accessing EMS the and DMS data of districts and cities and determining
Justice generates the Real-time Data Model based on PMS models, it is necessary to be matched according to various regions PMS, EMS matched rule of research department,
Prefecture-level company is transferred to coordinate project team to solve for the measuring point formation web inventories that it fails to match, by manually finding problem, modification
Matched rule is named or improved to specification, the E file datas for the match is successful parsing districts and cities and by the E file datas by connecing
Mouth is accessed, and then generation can correspond to the real-time data memory of correlation model, and rule base is verified by loading the quality of data,
Finally carry out the quality of data verification based on model tree.
The present invention proposes a kind of electrical network mass data quality indicator method based on PMS models, and there is provided a kind of data matter
Check system, reference picture 2 are measured, Fig. 2 is the quality of data check system Organization Chart based on PMS models, quality of data check system
Including the accumulation layer interconnected successively, engine layers, process layer and presentation layer, wherein accumulation layer is used for data storage, accumulation layer
It is interior provided with relational database, real-time data base and data quality rule storehouse (ctl files), engine layers include data transfer detection and drawn
Hold up includes data transfer detection service, data quality checking service and the quality of data and examines with data quality checking engine, process layer
Meter service, presentation layer is used for display data quality condition；Quality of data school is realized according to the quality of data check system Organization Chart
Proved recipe method, specifically includes following steps：Magnanimity real time data is acquired first, the present invention is used for access interface
The mode of FTP transmission E files gathers magnanimity real time data, and concrete methods of realizing is that ftp server is connected before interface server,
Reference picture 3, ftp server and interface server are connected with each other, and interface server and magnanimity platform data quality server are mutual
Connection, magnanimity platform data quality server and magnanimity platform data quality client are interconnected, and magnanimity real time data enters
E file datas are formed in ftp server, the E file datas are sent in interface server, interface server obtains FTP clothes
The data, are then deposited into accumulation layer in real-time data base by E file datas on business device, are equivalent to deposit magnanimity platform data
In the real-time data base installed in quality server, also installation relation database is equivalent in magnanimity platform data quality server
Relational database in accumulation layer, magnanimity platform data quality server obtains data between ftp server and interface server and passed
Defeated state, between ftp server and interface server in data transmission state deposit relational database, magnanimity platform data quality
Quality of data related data is sent in magnanimity platform data quality client by server；Obtained for E document analysis modes
During the mode of mass data, need to carry out data transmission state-detection before quality of data verification, passed by data in engine layers
Defeated detecting and alarm enters data transfer detection service in process layer, during calling data transfer detection service, in interface service
The data transmission state between ftp server and interface server is monitored using FTP service monitor and detection technology in device, and is relied on
JDBC technologies will be reported in relational database in data transmission state record deposit accumulation layer in case being subsequently generated quality of data audit
Accuse, interrupted then and alarm if there is FTP, Field Force should handle warning message at once to keep FTP transmission smooth；Pass through
Data quality checking engine enters data quality checking service in process layer in engine layers, by calling data quality checking service
Detect in accumulation layer in real-time data base quality of data situation and quality of data situation is stored in accumulation layer in relational database；
Quality of data auditing service is entered by quality of data auditing engine in engine layers, quality of data auditing service is called, according to choosing
The data source selected and the quality of data Audit Report in time parameter call relation database simultaneously check related measuring point data quality
Situation；Related data quality condition is shown by presentation layer, related data quality condition is specifically according in the generation quality of data
What the verification rule used in the regular library file of verification was determined, including FTP is interrupted, data are empty, data are 0, data leak source, number
According to repetition, data jump and data burr；Enter the quality of data in process layer by data quality checking engine in engine layers to examine
Survey service specifically includes following steps：The regular library file of quality of data verification is firstly generated, reference picture 4 adopts interface server
The measuring point collected is based on PMS electric network models and builds observation point model tree；Load electric network data observation point model tree；Increase one group of data matter newly
Measure method of calibration, the quality of data method of calibration include data leak source detection method, data be sky detection method, data be 0 detection method,
Data jump detection method, data burr detection method and Data duplication screening method；On measuring point in data source in observation point model tree
Different types of method of calibration is loaded, the regular library file of generation quality of data verification simultaneously inserts quality of data verification rule base
In accumulation layer in relational database；Different types of quality of data method of calibration computational methods are illustrated below：
Data are sky：Inquire about measuring point no data；
Data leak source：The collection period of measuring point is inquired about, formula is applied mechanically：24 (hour) * (60/ data collection cycle (point
Clock))-same day all data record sums, then obtain the leak source number of the same day measuring point；
Data are 0：Data drift is set according to field condition, is 0 if there is data, can be according to the shape of the data
State is judged as whether 0 value is normal, the state recording of the data running status of the corresponding equipment of the data.Equipment is in fortune
Judge whether measuring point data is normal for 0 according to type is measured when row state；During equipment non-operating state, data are only for 0
Normal condition；
Data are jumped：If the data variation of measuring point exceedes the 20% of a upper measuring point, then it is assumed that be the data of saltus step, in saltus step
Data in need to judge whether the data are correct data according to state value in data, data mode value have recorded the data
The running status of corresponding device；
Data burr：If the data variation of measuring point exceedes the 30% of a upper measuring point, then it is assumed that be the data of burr, in saltus step
Data in need to judge whether the data are correct data according to state value in data, data mode value have recorded the data
The running status of corresponding device；
Data duplication：Whether the measuring point of repetition is had in inquiry real-time database；
Data quality checking service in process layer is entered by data quality checking engine in engine layers, the quality of data is called
Detection service, by reading, the quality of data in relational database verifies regular library file and related measuring point is every from real-time data base
Day data proceed by data quality checking, and finally related measuring point data quality record is stored in relational database so as to follow-up
Generate quality of data Audit Report.
Quality of data auditing service is entered by quality of data auditing engine in engine layers, quality of data audit clothes are called
It is engaged in, the quality of data Audit Report in the data source of selection and time parameter call relation database simultaneously checks related data
Quality condition；The related data quality condition and data transmission state that are stored in relational database are generated using JXL technologies
Excel formatted files or using iText technologies by the related data quality condition stored in relational database and data transfer shape
State generates PDF format file, regard the Excel formatted files or PDF format file of generation as quality of data Audit Report, data
Quality auditing result is interrupted including FTP, data are empty, data are 0, data leak source, Data duplication, data jump and data hair
Thorn, selects the quality of data Audit Report in data source and time parameter, call relation database and checks electric in the data source
The quality of data situation for the measuring point that net observation point model seeds have been chosen；
Into presentation layer, presentation layer is used to show related data quality feelings, and displaying is on data consistency, integrality, just
The auditing results such as true property, quality of data auditing result is interrupted including FTP, data are empty, data are 0, data leak source, data weight
Multiple, data jump and data burr.
Real time data checking system function modoularization based on PMS models is obtained to real time data verification as shown in Figure 5
System function module figure, include in the system function module figure menu management component, assembly management, Page Template management,
Quality of data Audit Module, quality of data correction verification module, real time data interactive module, real time data retrieval, data model association
Module and data transfer detection module；By component instance, reference picture 6 selects component, configuration parameter, displaying from Component Gallery
Data model tree structure, selects measuring point from data model tree structure, and parameter configuration is completed, formation component example, configuration
Measuring point related data quality indicator rule.
Embodiment：
Reference picture 3, Fig. 3 show ftp server and interface server is connected with each other, interface server and magnanimity platform number
Interconnected according to quality server, magnanimity platform data quality server and magnanimity platform data quality client are interconnected,
Install in real-time data base and oracle relational databases, relational database and store in magnanimity platform data quality server
Equipment id, PMS models and other configurations information that access network is saved, the side for obtaining mass data using E document analysis modes
Quality of data verification needs log-on data to transmit detection program during formula, the program monitoring interface server and ftp server it is logical
Letter process, if there is FTP interrupt then and alarm and in relational database generate log file in case generation the quality of data
Audit Report, Field Force should handle warning message at once, keep FTP transmission smooth；
Then magnanimity platform data quality indicator system is logged in by magnanimity platform data quality client and opens verification rule
Then the storehouse configuration page, needs the measuring point verified in observation point model Tree Species Selection and the method for configuring verification, such as data leak source is detected
Method, data are that sky detection method, data are 0 detection method, data jump detection method, data burr detection method and Data duplication screening
Method；The regular library file of generation verification simultaneously inserts quality of data verification regular library file in relational database；Subsequent data matter
Measure checking routine and read the regular library file (ctl files) of quality of data verification in relation data database, and from real time data
Extracted in storehouse and meet the quality indicator that proceeds by of checking algorithm, it is the uniformity for verifying the quality of data after the completion of calculating, complete
Property, called for quality of data auditing service in the index such as correctness generation record insertion relational database；
Log in magnanimity platform data quality indicator system and open the quality of data audit page, selection data source and time parameter
Afterwards, the quality of data Audit Report in quality of data auditing service call relation database checks that model seeds are in the data source
The quality report of measuring point through choosing, can select printing.
Described above is only the preferred embodiment of the present invention, it should be pointed out that：Come for those skilled in the art
Say, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improvements and modifications also should be regarded as
Protection scope of the present invention.
Claims (5)
1. a kind of electrical network mass data quality indicator method based on PMS models, it is characterised in that：A kind of quality of data school is provided
Check system, the system includes accumulation layer, engine layers, process layer and the presentation layer interconnected successively, and the accumulation layer includes closing
It is database and real-time data base, the engine layers include data transfer detecting and alarm, data quality checking engine and data matter
Auditing engine is measured, the process layer includes data transfer detection service, data quality checking service and quality of data auditing service；
It the described method comprises the following steps：
S1 data acquisitions：The gathered data by way of FTP transmits E files, specifically data initially enter ftp server and formed
E file datas, then the E file datas enter interface server；
S2 enters accumulation layer：Data subsequently enter accumulation layer and are stored into accumulation layer in real-time data base in interface server；
S3 enters engine layers：Data transfer detecting and alarm is selected by engine layers；
S4 enters process layer：Data transfer detection service is entered according to data transfer detecting and alarm, ftp server is specifically detected
Data transmission state is simultaneously stored in accumulation layer in relational database by data transmission state between interface server；
S5 enters engine layers：Data quality checking engine is selected by engine layers；
S6 enters process layer：Data quality checking service is entered according to data quality checking engine, specifically detected in accumulation layer
Quality of data situation and quality of data situation is stored in real-time data base in accumulation layer in relational database；
S7 enters engine layers：Quality of data auditing engine is selected by engine layers；
S8 enters process layer：Quality of data auditing service is entered according to quality of data auditing engine, according to the data source of selection and
Quality of data Audit Report in time parameter call relation database simultaneously checks related data quality condition；
S9 enters presentation layer：Presentation layer is used to show related data quality condition.
2. the electrical network mass data quality indicator method according to claim 1 based on PMS models, it is characterised in that：Institute
Step S4 is stated to comprise the following steps：
When data transmission state between ftp server and interface server interrupts for FTP, then alarm, Field Force's processing are needed
The warning message is smooth to keep FTP to transmit.
3. the electrical network mass data quality indicator method according to claim 1 based on PMS models, it is characterised in that：Institute
Step S6 is stated to comprise the following steps：
S61 loads electric network data observation point model tree；
S62 chooses the measuring point for needing to detect in batches in observation point model tree；
S63 increases one group of quality of data method of calibration newly, and it is sky that the quality of data method of calibration, which includes data leak source detection method, data,
Detection method, data are 0 detection method, data jump detection method, data burr detection method and Data duplication screening method；
S64 loads the regular library file of different types of quality of data method of calibration generation quality of data verification simultaneously according to choosing measuring point
By in relational database in quality of data verification rule base insertion accumulation layer；
S65 calls data quality checking service, and regular library file is verified and from reality by reading in relational database the quality of data
When database in the related daily data of measuring point proceed by data quality checking, finally related measuring point data quality record is stored in
In relational database.
4. the electrical network mass data quality indicator method according to claim 1 based on PMS models, it is characterised in that：Institute
Step S8 is stated to comprise the following steps：
S81 calls quality of data auditing service, using JXL technologies by the related data quality condition stored in relational database and
Data transmission state generates Excel formatted files or using iText technologies by the related data quality stored in relational database
Situation and data transmission state generation PDF format file, regard the Excel formatted files or PDF format file of generation as data
Quality Audit Report；
S82 selects the quality of data Audit Report in data source and time parameter, call relation database and checks the data source
The quality of data situation for the measuring point that middle power network observation point model seeds have been chosen.
5. the electrical network mass data quality indicator method according to claim 1 based on PMS models, it is characterised in that：Institute
State in step S9, the presentation layer shows related data quality condition, the related data quality condition includes FTP and interrupts, counts
It is 0, data leak source, Data duplication, data jump and data burr according to for empty, data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611265616.7A CN107066500B (en) | 2016-12-30 | 2016-12-30 | Power grid mass data quality verification method based on PMS model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611265616.7A CN107066500B (en) | 2016-12-30 | 2016-12-30 | Power grid mass data quality verification method based on PMS model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107066500A true CN107066500A (en) | 2017-08-18 |
| CN107066500B CN107066500B (en) | 2020-12-04 |
Family
ID=59624351
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611265616.7A Active CN107066500B (en) | 2016-12-30 | 2016-12-30 | Power grid mass data quality verification method based on PMS model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107066500B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609085A (en) * | 2017-09-07 | 2018-01-19 | 国网辽宁省电力有限公司 | A kind of intelligent grid metric data processing method and system based on big data technology |
| CN107958049A (en) * | 2017-11-28 | 2018-04-24 | 航天科工智慧产业发展有限公司 | A kind of quality of data checking and administration system |
| CN108170825A (en) * | 2018-01-05 | 2018-06-15 | 上海电气分布式能源科技有限公司 | Distributed energy data monitoring cleaning method based on cloud platform |
| CN110309131A (en) * | 2019-04-12 | 2019-10-08 | 北京星网锐捷网络技术有限公司 | The method for evaluating quality and device of massive structured data |
| CN110727667A (en) * | 2019-09-25 | 2020-01-24 | 国网安徽省电力有限公司蚌埠供电公司 | Power equipment consistency management system |
| CN112788083A (en) * | 2019-11-11 | 2021-05-11 | 福建天晴数码有限公司 | Method for carrying out engine resource file transmission management based on FTP |
| CN114167198A (en) * | 2021-10-18 | 2022-03-11 | 国网山东省电力公司平原县供电公司 | Method and platform for measuring synchronous line loss data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379667A1 (en) * | 2013-02-12 | 2014-12-25 | International Business Machines Corporation | Data quality assessment |
| CN104391903A (en) * | 2014-11-14 | 2015-03-04 | 广州科腾信息技术有限公司 | Distributed storage and parallel calculation-based power grid data quality detection method |
| CN105550511A (en) * | 2015-12-11 | 2016-05-04 | 北京锐软科技股份有限公司 | Data quality evaluation system and method based on data verification technique |
-
2016
- 2016-12-30 CN CN201611265616.7A patent/CN107066500B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379667A1 (en) * | 2013-02-12 | 2014-12-25 | International Business Machines Corporation | Data quality assessment |
| CN104391903A (en) * | 2014-11-14 | 2015-03-04 | 广州科腾信息技术有限公司 | Distributed storage and parallel calculation-based power grid data quality detection method |
| CN105550511A (en) * | 2015-12-11 | 2016-05-04 | 北京锐软科技股份有限公司 | Data quality evaluation system and method based on data verification technique |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609085A (en) * | 2017-09-07 | 2018-01-19 | 国网辽宁省电力有限公司 | A kind of intelligent grid metric data processing method and system based on big data technology |
| CN107958049A (en) * | 2017-11-28 | 2018-04-24 | 航天科工智慧产业发展有限公司 | A kind of quality of data checking and administration system |
| CN107958049B (en) * | 2017-11-28 | 2021-09-14 | 航天科工智慧产业发展有限公司 | Data quality inspection management system |
| CN108170825A (en) * | 2018-01-05 | 2018-06-15 | 上海电气分布式能源科技有限公司 | Distributed energy data monitoring cleaning method based on cloud platform |
| CN110309131A (en) * | 2019-04-12 | 2019-10-08 | 北京星网锐捷网络技术有限公司 | The method for evaluating quality and device of massive structured data |
| CN110727667A (en) * | 2019-09-25 | 2020-01-24 | 国网安徽省电力有限公司蚌埠供电公司 | Power equipment consistency management system |
| CN110727667B (en) * | 2019-09-25 | 2023-10-17 | 国网安徽省电力有限公司蚌埠供电公司 | Consistency management system for power equipment |
| CN112788083A (en) * | 2019-11-11 | 2021-05-11 | 福建天晴数码有限公司 | Method for carrying out engine resource file transmission management based on FTP |
| CN112788083B (en) * | 2019-11-11 | 2023-09-01 | 福建天晴数码有限公司 | Method for carrying out engine resource file transmission management based on FTP |
| CN114167198A (en) * | 2021-10-18 | 2022-03-11 | 国网山东省电力公司平原县供电公司 | Method and platform for measuring synchronous line loss data |
| CN114167198B (en) * | 2021-10-18 | 2024-03-01 | 国网山东省电力公司平原县供电公司 | Method and platform for measuring synchronous line loss data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107066500B (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107066500A (en) | A kind of electrical network mass data quality indicator method based on PMS models | |
| CN111177101B (en) | Multi-dimensional visualization platform for power distribution network based on big data architecture | |
| CN103762723B (en) | Method and system for managing a power grid | |
| CN102243497B (en) | Networking technology-based remote intelligent analysis service system used for engineering machinery | |
| CN109447480A (en) | A kind of road and bridge engineering equipment safety monitoring system and method based on BIM+GIS modeling | |
| CN109816161A (en) | A kind of power distribution network operation computer-aided decision support System and its application method | |
| CN109658067A (en) | A kind of power engineering project control platform and system | |
| CN107256247A (en) | Big data data administering method and device | |
| CN102147597A (en) | Health monitoring system for structures of great building and bridge | |
| CN107402332A (en) | A kind of Intelligent power distribution terminal multi-epitope automatic checkout system and its detection method | |
| CN106557991A (en) | Voltage monitoring data platform | |
| CN111027834A (en) | Power cable informatization management system and method based on three-dimensional detection technology | |
| CN104777813A (en) | Comprehensive meteorological observation operation monitoring system and monitoring method thereof | |
| CN109284346A (en) | Cloud computing-based power distribution network planning method and device | |
| CN107633050A (en) | A kind of method that stealing probability is judged based on big data analysis electricity consumption behavior | |
| CN106874483A (en) | A kind of device and method of the patterned quality of data evaluation and test based on big data technology | |
| CN107451737A (en) | A kind of big data O2O check of drawings cloud platform management systems examined for engineering construction figure | |
| CN105574698A (en) | Intelligent storage management system based on big data | |
| US20140006398A1 (en) | System and Method for Energy Management Information System Reliability Assurance | |
| CN111524027B (en) | Electric power information statistical method, device, equipment and storage medium | |
| CN111208384A (en) | Smart power grid management system | |
| CN112258159A (en) | BIM + GIS-based hydraulic structure management system and method | |
| CN106646110A (en) | Low-voltage distribution network fault positioning system based on GIS and Petri technologies | |
| CN112050092A (en) | Water supply pipe network leakage information management system, building method and application | |
| CN107194566A (en) | A kind of electric intelligent archive management method based on virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |