CN114253958B - Automatic correction method and device for electric energy data - Google Patents
Automatic correction method and device for electric energy data Download PDFInfo
- Publication number
- CN114253958B CN114253958B CN202111569910.8A CN202111569910A CN114253958B CN 114253958 B CN114253958 B CN 114253958B CN 202111569910 A CN202111569910 A CN 202111569910A CN 114253958 B CN114253958 B CN 114253958B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- electric energy
- complementary
- energy data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012937 correction Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000004458 analytical method Methods 0.000 claims abstract description 52
- 238000013499 data model Methods 0.000 claims abstract description 29
- 230000000295 complement effect Effects 0.000 claims description 230
- 238000000605 extraction Methods 0.000 claims description 137
- 230000005540 biological transmission Effects 0.000 claims description 113
- 238000012795 verification Methods 0.000 claims description 101
- 238000012545 processing Methods 0.000 claims description 78
- 230000002159 abnormal effect Effects 0.000 claims description 44
- 238000007405 data analysis Methods 0.000 claims description 19
- 238000005086 pumping Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 12
- 230000000153 supplemental effect Effects 0.000 claims description 9
- 230000000903 blocking effect Effects 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013480 data collection Methods 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 6
- 230000008014 freezing Effects 0.000 description 6
- 238000007710 freezing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000011835 investigation Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Economics (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Strategic Management (AREA)
- Primary Health Care (AREA)
- Water Supply & Treatment (AREA)
- Human Resources & Organizations (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an automatic correction method and device for electric energy data, wherein the method comprises the following steps: collecting electric energy data to be supplemented, analyzing the electric energy data according to a predetermined intelligent analysis model to obtain an supplemented topic of the electric energy data and a source region corresponding to the electric energy data, calling supplemented topic interfaces matched with the supplemented topic according to the source region corresponding to the electric energy data, issuing supplemented data requirements matched with the supplemented topic to a plurality of determined target service ends through the supplemented topic interfaces, receiving target supplemented data matched with the supplemented data requirements and fed back by at least one target service end through analysis warehouse interfaces included in the supplemented topic interfaces, and executing data correction operation on the target supplemented data and the electric energy data based on a predetermined metering data model to obtain correction data. Therefore, the method and the device can intelligently correct the collected electric energy data to be compensated and acquire correction data meeting the requirements.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to an automatic correction method and apparatus for electric energy data.
Background
In the power industry, processing various data related to electric quantity is the most fundamental item, including data collection, data integration processing, data compression, data transmission, data analysis and the like of electric quantity data, and in the conventional data processing means, uploading data to a cloud network for storage processing is an essential ring. However, in the process of data transmission, due to the problems of channel interference between transmission channels and the like, the finally obtained data has the phenomenon of data loss to a greater or lesser extent, the read data is incomplete, and the accuracy of the data result obtained after the subsequent operations such as analysis or statistics are performed on the missing data is reduced, so that the problem of data loss is particularly important.
Disclosure of Invention
The invention aims to solve the technical problem of providing an automatic correction method and device for electric energy data, which can execute data correction operation on the acquired electric energy data to be supplemented and extracted, reduce the situation that the acquired data cannot be used due to the data loss of the acquired data, and improve the data integrity rate of the electric energy data.
In order to solve the above technical problems, a first aspect of the present invention discloses an automatic correction method for electric energy data, the method comprising:
collecting electric energy data to be compensated, wherein the electric energy data is data which needs to be subjected to data correction operation;
Analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a full-quantity or detail incremental full-quantity extraction theme and a source region corresponding to the electric energy data, the full complement extraction theme is a theme corresponding to first data needing to execute the data correction operation, wherein the first data comprises data with data missing and data without data missing; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute the data correction operation, and the second data is data with data missing;
According to the source region corresponding to the electric energy data, calling a complementary extraction subject interface matched with the complementary extraction subject, wherein the complementary extraction subject interface is an interface which establishes an association relationship with the source region;
issuing a complementary data demand matched with the complementary topic to a plurality of determined target servers through the complementary topic interfaces, wherein the complementary data demand comprises the complementary topic;
receiving target supplementary transmission data matched with the supplementary data requirement and fed back by at least one target server through an analysis and storage interface included in the supplementary extraction subject interface;
And executing the data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data, wherein the correction data comprises the corrected electric energy data and the target supplementary transmission data.
In an optional implementation manner, in a first aspect of the present invention, after the receiving, by the parsing and warehousing interface included in the complementary extraction theme interface, target complementary transmission data that is fed back by at least one target server and matches the complementary extraction data requirement, the method further includes:
Determining target processing operations to be executed on the target complementary transmission data according to a predetermined intelligent computing model, wherein the target processing operations comprise at least one of data statistics, data deduplication, data cleaning and abnormal data analysis;
And executing the target processing operation on the target complementary transmission data based on a data table to obtain a target processing result.
As an alternative embodiment, in the first aspect of the present invention, the method further includes:
Acquiring a task progress corresponding to a first verification task for the electric energy data, wherein the first verification task is used for calculating a data acquisition integrity rate corresponding to the electric energy data;
judging whether the task progress corresponding to the first verification task in a first execution period reaches a preset progress threshold or not;
when judging that the task progress corresponding to the first verification task reaches the progress threshold value in the first execution period, updating the first execution period to a second execution period and updating the first verification task to a second verification task, wherein the ending time corresponding to the first execution period is earlier than the starting time corresponding to the second execution period, and the second verification task is used for calculating the data acquisition integrity rate corresponding to the electric energy data;
when judging that the task progress corresponding to the first verification task does not reach the progress threshold value in the first execution period, calculating the data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data and the electric energy data to obtain a first numerical value;
after the first value is calculated, the task progress corresponding to the first verification task is updated to reach the progress threshold, the first execution period is updated to be a second execution period, and the first verification task is updated to be a second verification task.
In an optional implementation manner, in the first aspect of the present invention, when it is determined that the task progress corresponding to the first verification task in the first execution period reaches the progress threshold, the method further includes:
Judging whether a value corresponding to the data acquisition integrity rate is greater than or equal to a determined integrity rate threshold value or not based on a preset intelligent complementary extraction rule;
When the numerical value is larger than or equal to the integrity rate threshold value, generating a standard mark of the electric energy data and executing the operation of updating the first execution period to a second execution period and updating the first verification task to a second verification task, wherein the standard mark is used for indicating that the numerical value of the electric energy data acquisition integrity rate is larger than or equal to the integrity rate threshold value;
When the numerical value is smaller than the integrity rate threshold value, judging whether a verification mark submitted by a user is detected, wherein the verification mark is used for indicating that the data acquisition integrity rate corresponding to the electric energy data needs to be calculated;
When the verification identifier submitted by the user is judged to be detected, calculating the data acquisition integrity rate corresponding to the electric energy data to obtain a second numerical value;
And executing the operation of judging whether the value corresponding to the data acquisition integrity rate is greater than or equal to the integrity rate threshold value on the second value based on the intelligent complementary extraction rule.
In a first aspect of the present invention, the target service end includes a plurality of sub-target service ends, and the target complementary transmission data includes target complementary transmission data of all the sub-target service ends;
when the target processing operation includes the abnormal data analysis, the executing the target processing operation on the target complementary transmission data based on a data table to obtain a target processing result includes:
Executing the abnormal data analysis operation on the target supplementary transmission data of each sub-target server based on a data table to obtain a data abnormal rate corresponding to the target supplementary transmission data of the sub-target server;
Determining a data anomaly rate higher than a preset anomaly threshold from the data anomaly rates corresponding to the target complementary data of all the sub-target servers to obtain a target data anomaly rate, anomaly complementary data corresponding to the target data anomaly rate and a sub-target server corresponding to the anomaly complementary data;
Marking the abnormal supplementary transmission data, and monitoring a supplementary pumping link between a first interface and a second interface to obtain a data transmission condition corresponding to the supplementary pumping link, wherein the first interface is an interface of a sub-target server corresponding to the abnormal supplementary transmission data, and the second interface is the analysis warehouse-in interface;
When the data transmission condition indicates that the supplemental link does not upload data amount in a preset data uploading time and/or indicates that the supplemental link has a link blocking condition, generating an abnormal identifier corresponding to the supplemental link, wherein the abnormal identifier is used for calling a preset link processing program to process the data transmission condition;
And determining the abnormal identification as a target processing result.
As an alternative embodiment, in the first aspect of the present invention, the method further includes:
comparing target parameters included in a target archive model with standard parameters corresponding to the target archive model in a preset power grid data model to obtain a first comparison result, wherein the power grid data model is used for storing data with the same type as the electric energy data and data with the same type as the target supplementary transmission data;
When the first comparison result shows that the target parameter is not matched with the standard parameter, determining a first parameter which is not matched with the standard parameter in the target parameter;
analyzing the first parameter to obtain a parameter type corresponding to the first parameter, wherein the parameter type comprises at least one of a data table type, a field type, an index type, a trigger type and an interface type;
And executing a model reconstruction operation matched with the parameter type on the first parameter to update the first parameter to the standard parameter.
As an alternative embodiment, in the first aspect of the present invention, the complementary extraction subject is controlled by:
calling an API (application program interface) corresponding to the target server to create complementary extraction subject information, wherein the complementary extraction subject information comprises a naming rule corresponding to the complementary extraction subject and a format corresponding to the content of the complementary extraction subject load;
Acquiring all topics to be detected through the API in a preset period, wherein the topics to be detected are created through the target server;
comparing all the topics to be detected with the complementary extraction topic information to obtain a second comparison result;
When the second comparison result shows that the topic which is not matched with the complementary topic information exists in the topic to be detected, determining and marking the topic which is not matched with the complementary topic information;
and adjusting the topic naming corresponding to the topic and/or adjusting the format corresponding to the load content of the topic according to the complementary topic information.
The second aspect of the invention discloses an automatic correction device for electric energy data, the device comprising:
The acquisition module is used for acquiring electric energy data to be supplemented, wherein the electric energy data are data which need to be subjected to data correction operation;
The analysis module is used for analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a complementary extraction theme of the electric energy data and a source area corresponding to the electric energy data, wherein the complementary extraction theme comprises a full-quantity complementary extraction theme or a detail increment complementary extraction theme, and the full-quantity complementary extraction theme is a theme corresponding to first data needing to execute the data correction operation, wherein the first data comprises data with data loss and data without data loss; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute the data correction operation, and the second data is data with data missing;
the calling module is used for calling a complementary extraction subject interface matched with the complementary extraction subject according to the source region corresponding to the electric energy data obtained through analysis by the analysis module, wherein the complementary extraction subject interface is an interface which has an association relation with the source region;
The data processing module is used for issuing a complementary data requirement matched with the complementary topic to the determined target servers through the complementary topic interfaces, wherein the complementary data requirement comprises the complementary topic;
The data processing module is further used for receiving target supplementary transmission data matched with the supplementary drawing data requirement and fed back by at least one target server through an analysis and storage interface included in the supplementary drawing subject interface;
And the correction module is used for executing the data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data, wherein the correction data comprises the corrected electric energy data and the target supplementary transmission data.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further includes:
The determining module is used for determining target processing operations which need to be executed on the target supplementary transmission data according to a predetermined intelligent calculation model after the data processing module receives target supplementary transmission data which is fed back by at least one target server and matches with the supplementary transmission data requirement through an analysis and storage interface included in the supplementary transmission subject interface, wherein the target processing operations comprise at least one of data statistics, data deduplication, data cleaning and abnormal data analysis;
And the first processing module is used for executing the target processing operation determined by the determining module based on a data table to obtain a target processing result.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further includes:
the acquisition module is used for acquiring a task progress corresponding to a first verification task for the electric energy data, wherein the first verification task is used for calculating a data acquisition integrity rate corresponding to the electric energy data;
The judging module is used for judging whether the task progress corresponding to the first verification task in the first execution period reaches a preset progress threshold value or not;
The updating module is used for updating the first execution period to a second execution period and updating the first verification task to a second verification task when the judging module judges that the task progress corresponding to the first verification task reaches the progress threshold value in the first execution period, the termination time corresponding to the first execution period is earlier than the starting time corresponding to the second execution period, and the second verification task is used for calculating the data acquisition integrity rate corresponding to the electric energy data;
the calculation module is used for calculating the data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data and the electric energy data when judging that the task progress corresponding to the first verification task in the first execution period does not reach the progress threshold value, so as to obtain a first numerical value;
And the updating module is further configured to update the task progress corresponding to the first verification task to reach the progress threshold after the first value is obtained by calculation, update the first execution period to a second execution period, and update the first verification task to a second verification task.
In a second aspect of the present invention, as an optional implementation manner, the determining module is further configured to determine, based on a preset intelligent complementary extraction rule, whether a value corresponding to the data acquisition integrity rate is greater than or equal to a determined integrity rate threshold when it is determined that a task progress corresponding to the first verification task reaches the progress threshold in the first execution period;
The apparatus further comprises:
The generation module is used for generating a standard mark of the electric energy data and triggering the updating module to execute the operation of updating the first execution period into a second execution period and updating the first verification task into a second verification task when the judgment module judges that the numerical value is larger than or equal to the integrity rate threshold, wherein the standard mark is used for indicating that the numerical value of the electric energy data acquisition integrity rate is larger than or equal to the integrity rate threshold;
The judging module is further used for judging whether a verification mark submitted by a user is detected or not when the numerical value is judged to be smaller than the integrity rate threshold value, wherein the verification mark is used for indicating that the data acquisition integrity rate corresponding to the electric energy data needs to be calculated;
the calculation module is further used for calculating a data acquisition integrity rate corresponding to the electric energy data to obtain a second numerical value when the judgment module judges that the verification identifier submitted by the user is detected;
and the judging module is further used for executing the operation of judging whether the value corresponding to the data acquisition integrity rate is greater than or equal to the integrity rate threshold value on the second value based on the intelligent complementary extraction rule.
In a second aspect of the present invention, the target service end includes a plurality of sub-target service ends, and the target complementary transmission data includes target complementary transmission data of all the sub-target service ends;
the first processing module executes the target processing operation on the target complementary transmission data based on a data table, and the mode of obtaining the target processing result specifically includes:
When the target processing operation comprises the abnormal data analysis, executing the abnormal data analysis operation on the target complementary transmission data of each sub-target server based on a data table to obtain a data abnormal rate corresponding to the target complementary transmission data of the sub-target server;
Determining a data anomaly rate higher than a preset anomaly threshold from the data anomaly rates corresponding to the target complementary data of all the sub-target servers to obtain a target data anomaly rate, anomaly complementary data corresponding to the target data anomaly rate and a sub-target server corresponding to the anomaly complementary data;
Marking the abnormal supplementary transmission data, and monitoring a supplementary pumping link between a first interface and a second interface to obtain a data transmission condition corresponding to the supplementary pumping link, wherein the first interface is an interface of a sub-target server corresponding to the abnormal supplementary transmission data, and the second interface is the analysis warehouse-in interface;
When the data transmission condition indicates that the supplemental link does not upload data amount in a preset data uploading time and/or indicates that the supplemental link has a link blocking condition, generating an abnormal identifier corresponding to the supplemental link, wherein the abnormal identifier is used for calling a preset link processing program to process the data transmission condition;
And determining the abnormal identification as a target processing result.
As an alternative embodiment, in the second aspect of the present invention, the apparatus further includes:
the second processing module is used for comparing target parameters included in a target archive model with standard parameters corresponding to the target archive model in a preset power grid data model to obtain a first comparison result, wherein the power grid data model is used for storing data with the same type as the electric energy data and data with the same type as the target supplementary transmission data;
the second processing module is further configured to determine a first parameter that is not matched with the standard parameter in the target parameters when the first comparison result indicates that the target parameters are not matched with the standard parameters;
the analysis module is further configured to analyze the first parameter obtained by the second processing module to obtain a parameter type corresponding to the first parameter, where the parameter type includes at least one of a data table type, a field type, an index type, a trigger type, and an interface type;
The second processing module is further configured to perform a model transformation operation matched with the parameter type on the first parameter obtained by the analysis module, so as to update the first parameter to the standard parameter.
As an alternative embodiment, in the second aspect of the present invention, the complementary extraction subject is controlled by:
calling an API (application program interface) corresponding to the target server to create complementary extraction subject information, wherein the complementary extraction subject information comprises a naming rule corresponding to the complementary extraction subject and a format corresponding to the content of the complementary extraction subject load;
Acquiring all topics to be detected through the API in a preset period, wherein the topics to be detected are created through the target server;
comparing all the topics to be detected with the complementary extraction topic information to obtain a second comparison result;
When the second comparison result shows that the topic which is not matched with the complementary topic information exists in the topic to be detected, determining and marking the topic which is not matched with the complementary topic information;
and adjusting the topic naming corresponding to the topic and/or adjusting the format corresponding to the load content of the topic according to the complementary topic information.
In a third aspect, the present invention discloses another automatic correction device for electric energy data, the device comprising:
a memory storing executable program code;
A processor coupled to the memory;
The processor invokes the executable program code stored in the memory to execute the automatic correction method for electric energy data disclosed in the first aspect of the present invention.
A fourth aspect of the invention discloses a computer storage medium storing computer instructions which, when called, are adapted to perform the method of automatically modifying electrical energy data disclosed in the first aspect of the invention.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
The embodiment of the invention provides an automatic correction method and device for electric energy data, wherein the method comprises the following steps: collecting electric energy data to be supplemented, wherein the electric energy data are data which need to be subjected to data correction operation, analyzing the electric energy data according to a predetermined intelligent analysis model to obtain an supplemented subject of the electric energy data and a source area corresponding to the electric energy data, the supplemented subject comprises a full-quantity supplemented subject or a detail increment supplemented subject, the full-quantity supplemented subject is a subject corresponding to first data which need to be subjected to data correction operation, and the first data comprise data with data loss and data without data loss; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute data correction operation, and the second data is data with data missing; according to the source region corresponding to the electric energy data, calling a complementary extraction subject interface matched with the complementary extraction subject, wherein the complementary extraction subject interface is an interface which has an association relation with the source region; issuing a complementary extraction data demand matched with the complementary extraction theme to the determined target servers through complementary extraction theme interfaces, wherein the complementary extraction data demand comprises the complementary extraction theme; receiving target supplementary transmission data matched with the supplementary data requirement and fed back by at least one target server through an analysis and storage interface included in the supplementary extraction subject interface; and performing data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data, wherein the correction data comprises corrected electric energy data and target supplementary transmission data. Therefore, the invention can intelligently collect the electric energy data which needs to be subjected to data correction operation, thereby being beneficial to improving the data collection efficiency; the electric energy data can be analyzed through the intelligent analysis model, so that the analysis efficiency of the electric energy data is improved; furthermore, after the target supplementary transmission data is received, the electric energy data can be intelligently corrected according to the metering data model and the target supplementary transmission data, so that the condition that the acquired electric energy data has data missing is reduced, and the data integrity rate of the electric energy data is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an automatic correction method for electric energy data according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an automatic correction method for electric energy data according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an automatic correction device for electric energy data according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another automatic correction device for electric energy data according to an embodiment of the present invention;
Fig. 5 is a schematic structural diagram of an automatic correction device for electric energy data according to another embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or article.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The invention discloses an automatic correction method and device for electric energy data, which can intelligently collect the electric energy data needing to execute data correction operation, and is beneficial to improving the data collection efficiency; the electric energy data can be analyzed through the intelligent analysis model, so that the analysis efficiency of the electric energy data is improved; furthermore, after the target supplementary transmission data is received, the electric energy data can be intelligently corrected according to the metering data model and the target supplementary transmission data, so that the condition that the acquired electric energy data has data missing is reduced, and the data integrity rate of the electric energy data is improved. The following will describe in detail.
Example 1
Referring to fig. 1, fig. 1 is a flow chart of an automatic correction method for electric energy data according to an embodiment of the invention. The method for automatically correcting the electric energy data described in fig. 1 can be applied to an automatic correction device for electric energy data, and the embodiment of the invention is not limited. As shown in fig. 1, the automatic correction method of electric energy data may include the following operations:
101. and collecting electric energy data to be supplemented.
In the embodiment of the invention, the electric energy data are data which need to execute data correction operation; the electric energy data can be stored in a local database corresponding to the data center and can be stored in a cloud database corresponding to the cloud server; in addition, the electric energy data may be 714 electric energy data such as a day freezing meter code, a month freezing meter code, a 15 minute clock code, a voltage current curve, a power curve, and the like, which is not limited in the embodiment of the present invention.
102. And analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a complementary extraction theme of the electric energy data and a source region corresponding to the electric energy data.
In the embodiment of the invention, the full-scale complementary extraction theme comprises a full-scale complementary extraction theme or a detail increment complementary extraction theme, wherein the full-scale complementary extraction theme is a theme corresponding to first data needing to execute data correction operation, and the first data comprises data with data loss and data without data loss; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute data correction operation, and the second data is data with data missing;
103. And calling a complementary extraction subject interface matched with the complementary extraction subject according to the source region corresponding to the electric energy data.
In the embodiment of the present invention, the source region corresponding to the electric energy data may include a provincial region (for example, guangdong province) or a municipal region (for example, guangzhou city), and the embodiment of the present invention is not limited, where the complementary extraction subject interface is an interface that establishes an association relationship with the source region;
104. And issuing the complementary extraction data requirements matched with the complementary extraction theme to the determined target servers through the complementary extraction theme interfaces.
In the embodiment of the invention, the data requirement for the complementary extraction comprises a complementary extraction theme, namely when the complementary extraction theme is a full-scale complementary extraction theme, the data requirement for the complementary extraction corresponds to data needing to be received with data loss and without data loss; it should be noted that the target server may be a server of the provincial level metering automation system kafka corresponding to the source region, which is not limited in the embodiment of the present invention.
105. And receiving target complementary transmission data which is fed back by at least one target server and matches with the complementary data requirement through an analysis and storage interface included in the complementary extraction subject interface.
In the embodiment of the invention, a data center corresponding to electric energy data to be compensated sends out compensation data requirements matched with compensation topics to a plurality of determined target service ends through compensation topic interfaces, each target service end judges whether the target service end stores data meeting the compensation data requirements after receiving the compensation data requirements, and after judging that the target service end stores the data meeting the compensation data requirements, the data meeting the compensation data requirements are packaged and fed back to a sending end of the data returning compensation data requirements.
106. And performing data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data.
In the embodiment of the invention, the correction data may include corrected electric energy data and target supplementary transmission data; when the electric energy data is a day-frozen meter code, the analysis and storage interface is a data analysis and storage interface corresponding to the day-frozen meter code, and when the electric energy data is a month-frozen meter code, the analysis and storage interface is a data analysis and storage interface corresponding to the month-frozen meter code, and the embodiment of the invention is not limited; furthermore, the data analysis and storage interface corresponding to the daily/monthly freezing list codes can be used for receiving the daily/monthly freezing list codes uploaded by Kafka clients of the metering automation systems of all provinces, analyzing, converting and storing the daily/monthly freezing list code complement data uploaded by the metering automation systems of all provinces according to the latest metering data model specifications of the data center, so that correction of the daily/monthly freezing list code data is realized.
Therefore, by implementing the automatic correction method of the electric energy data described in fig. 1, the electric energy data required to execute the data correction operation can be intelligently collected, which is beneficial to improving the data collection efficiency; the electric energy data can be analyzed through the intelligent analysis model, so that the analysis efficiency of the electric energy data is improved; furthermore, the matched complementary extraction subject interface can be adaptively called according to the electric energy data source region, so that the situation that the complementary extraction subject interface is not matched to cause the error issuing of the complementary extraction data requirement is reduced, and the lower accuracy of the complementary extraction data requirement is improved; in addition, after the target supplementary transmission data is received, the electric energy data can be intelligently corrected according to the metering data model and the target supplementary transmission data, so that the condition that the acquired electric energy data has data missing is reduced, and the data integrity rate of the electric energy data is improved.
In an alternative embodiment, the method for automatically correcting electric energy data may further include the steps of:
acquiring task progress corresponding to a first verification task for electric energy data, wherein the first verification task is used for calculating data acquisition integrity rate corresponding to the electric energy data;
judging whether the task progress corresponding to the first verification task in the first execution period reaches a preset progress threshold value or not;
When judging that the task progress corresponding to the first verification task in the first execution period reaches a progress threshold, updating the first execution period to a second execution period and updating the first verification task to a second verification task, wherein the ending moment corresponding to the first execution period is earlier than the starting moment corresponding to the second execution period, and the second verification task is used for calculating the data acquisition integrity rate corresponding to the electric energy data;
When judging that the task progress corresponding to the first verification task does not reach the progress threshold value in the first execution period, calculating the data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data and the electric energy data to obtain a first numerical value;
after the first numerical value is obtained through calculation, the task progress corresponding to the first verification task is updated to reach a progress threshold value, the first execution period is updated to be the second execution period, and the first verification task is updated to be the second verification task.
In this alternative embodiment, the first execution period and the second execution period may take 3 days as a period or 5 days as a period in practical application, which is not limited in the embodiment of the present invention.
In the optional embodiment, the task progress corresponding to the first verification task for the electric energy data can be intelligently detected, so that the timeliness of the acquired task progress is improved; and when the task progress does not reach the progress threshold value, the operation of calculating the data acquisition integrity rate of the electric energy data is executed, the situation that the task progress cannot be completed due to factors such as equipment faults is reduced, and the completion quantity of the task progress is further improved.
In another alternative embodiment, when it is determined that the task progress corresponding to the first verification task in the first execution period reaches the progress threshold, the automatic correction method of electric energy data may further include the following steps:
judging whether a value corresponding to the data acquisition integrity rate is greater than or equal to a determined integrity rate threshold value or not based on a preset intelligent complementary extraction rule;
When the value is judged to be greater than or equal to the integrity rate threshold, generating a standard mark of the electric energy data and an operation of updating the first execution period to the second execution period and updating the first verification task to the second verification task, wherein the standard mark is used for representing that the value of the data acquisition integrity rate of the electric energy data is greater than or equal to the integrity rate threshold;
When the value is smaller than the integrity rate threshold value, judging whether a verification mark submitted by a user is detected, wherein the verification mark is used for indicating the data acquisition integrity rate corresponding to the electric energy data to be calculated;
When judging that the verification mark submitted by the user is detected, calculating the data acquisition integrity rate corresponding to the electric energy data to obtain a second value;
and executing the operation of judging whether the value corresponding to the data acquisition integrity rate is greater than or equal to the integrity rate threshold value on the second value based on the intelligent complementary extraction rule.
Therefore, in the alternative embodiment, the calculated data acquisition rate data and the integrity rate threshold value can be intelligently compared, and the matching operation is adaptively executed according to the comparison result, so that the condition that the calculated data acquisition integrity rate corresponding value is smaller than the integrity rate threshold value due to individual data reading errors is less, and the accuracy of the calculation result obtained by calculating the data acquisition integrity rate corresponding value is improved.
In this alternative embodiment, further, when it is determined that the authentication identifier submitted by the user is not detected, the value is marked;
determining a data source region corresponding to the data according to the numerical value to obtain a region which does not reach the standard;
and (3) putting the substandard areas into a key investigation list to increase the investigation frequency of the electric energy data of the substandard areas.
It can be seen that in this alternative embodiment, the matching operation can also be performed for non-compliant regions, helping to analyze the reasons for non-compliant data acquisition integrity.
Example two
Referring to fig. 2, fig. 2 is a flow chart of another automatic correction method for electric energy data according to an embodiment of the invention. The method for automatically correcting the electric energy data described in fig. 2 may be applied to an apparatus for automatically correcting the electric energy data, which is not limited by the embodiment of the present invention. As shown in fig. 2, the automatic correction method of electric energy data may include the steps of:
201. And collecting electric energy data to be supplemented.
202. And analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a complementary extraction theme of the electric energy data and a source region corresponding to the electric energy data.
203. And calling a complementary extraction subject interface matched with the complementary extraction subject according to the source region corresponding to the electric energy data.
204. And issuing the complementary extraction data requirements matched with the complementary extraction theme to the determined target servers through the complementary extraction theme interfaces.
205. And receiving target complementary transmission data which is fed back by at least one target server and matches with the complementary data requirement through an analysis and storage interface included in the complementary extraction subject interface.
206. And performing data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data.
In the embodiment of the present invention, the other descriptions of step 201 to step 206 refer to the other specific descriptions of step 101 to step 106 in the first embodiment, and the description of the embodiment of the present invention is omitted.
207. And determining target processing operations which need to be executed on the target complementary transmission data according to a predetermined intelligent computing model.
In an embodiment of the present invention, the target processing operation includes at least one of data statistics, data deduplication, data cleansing, and exception data analysis.
208. And executing target processing operation on the target complementary transmission data based on the data table to obtain a target processing result.
Therefore, the implementation of the automatic correction method of the electric energy data described in fig. 2 can intelligently collect the electric energy data needed to execute the data correction operation, which is beneficial to improving the data collection efficiency; the electric energy data can be analyzed through the intelligent analysis model, so that the analysis efficiency of the electric energy data is improved; furthermore, the matched complementary extraction subject interface can be adaptively called according to the electric energy data source region, so that the situation that the complementary extraction subject interface is not matched to cause the error issuing of the complementary extraction data requirement is reduced, and the lower accuracy of the complementary extraction data requirement is improved; in addition, after the target supplementary transmission data is received, the electric energy data can be intelligently corrected according to the metering data model and the target supplementary transmission data, so that the condition that the acquired electric energy data has data missing is reduced, and the data integrity rate of the electric energy data is improved; in addition, the target compensation data can be intelligently processed through the intelligent calculation model and the data table, the condition that the target compensation data cannot be used due to the fact that the received target compensation data comprise repeated data and/or abnormal data and the like is reduced, and the accuracy of a correction result obtained after correction operation is carried out on the electric energy data through the target compensation data is improved.
In an alternative embodiment, the target server includes a plurality of sub-target servers, and the target complementary data includes target complementary data of all sub-target servers;
When the target processing operation includes abnormal data analysis, the method for executing the target processing operation based on the data table to obtain the target processing result may specifically include the following steps:
performing abnormal data analysis operation on the target complementary transmission data of each sub-target server based on the data table to obtain the data abnormal rate corresponding to the target complementary transmission data of the sub-target server;
Determining the data anomaly rate higher than a preset anomaly threshold from the data anomaly rates corresponding to the target complementary data of all the sub-target servers to obtain the target data anomaly rate, the anomaly complementary data corresponding to the target data anomaly rate and the sub-target servers corresponding to the anomaly complementary data;
marking abnormal supplementary transmission data, and monitoring a supplementary pumping link between a first interface and a second interface to obtain a data transmission condition corresponding to the supplementary pumping link, wherein the first interface is an interface of a sub-target server corresponding to the abnormal supplementary transmission data, and the second interface is an analysis warehouse-in interface;
When the data transmission condition indicates that the complementary pumping link has no uploading data amount in the preset data uploading time and/or indicates that the complementary pumping link has a link blocking condition, generating an abnormal identifier corresponding to the complementary pumping link, wherein the abnormal identifier is used for calling a preset link processing program to process the data transmission condition;
And determining the exception identification as a target processing result.
Therefore, in this optional embodiment, the data exception handling operation can be performed for the target complementary transmission data, so as to reduce the proportion of the exception data in the received target complementary transmission data, thereby reducing the interference of the exception data on the data correction operation, further causing the situation that the correction result obtained by the data correction operation is wrong, and further improving the accuracy of the data correction operation result.
In another alternative embodiment, the method for automatically correcting electric energy data may further include the steps of:
Comparing target parameters included in the target archive model with standard parameters corresponding to the target archive model in a preset power grid data model to obtain a first comparison result, wherein the power grid data model is used for storing data with the same type as the electric energy data and data with the same type as the target supplementary transmission data;
when the first comparison result shows that the target parameter is not matched with the standard parameter, determining a first parameter which is not matched with the standard parameter in the target parameter;
Analyzing the first parameter to obtain a parameter type corresponding to the first parameter, wherein the parameter type comprises at least one of a data table type, a field type, an index type, a trigger type and an interface type;
and executing the model reconstruction operation matched with the parameter type on the first parameter to update the first parameter as a standard parameter.
It can be seen that, in this alternative embodiment, the matching operation can be intelligently performed according to the comparison result obtained by comparing the target parameter with the standard parameter, so that the target parameter is consistent with the standard parameter, thereby reducing the situation that the target parameter is inconsistent with the standard parameter, and reducing the situation that the accuracy of the result obtained after the relevant data operation is performed through the grid data model due to the inconsistency of the target parameter and the standard parameter.
In yet another alternative embodiment, the replenishment topic is managed by:
calling an API (application program interface) corresponding to a target server to create complementary extraction subject information, wherein the complementary extraction subject information comprises naming rules corresponding to the complementary extraction subject and formats corresponding to the content of the complementary extraction subject load;
Acquiring all topics to be detected through an API interface in a preset period, wherein the topics to be detected are created through a target server;
Comparing all the topics to be detected with the complementary extraction topic information to obtain a second comparison result;
When the second comparison result shows that the topic which is not matched with the complementary extraction topic information exists in the topic to be detected, determining and marking the topic which is not matched with the complementary extraction topic information;
And adjusting the topic name corresponding to the topic and/or adjusting the format corresponding to the load content of the topic according to the complementary topic information.
In this optional embodiment, the method for creating the complementary extraction subject information by calling the API interface corresponding to the target server may specifically include the following steps:
And calling an API interface of a kafka server of the metering automation system of each level in five provinces and two cities through a Web interface to create intelligent complementary extraction subject information.
In this alternative embodiment, the method for automatically correcting electric energy data may further include the steps of:
and generating new adding, modifying and inquiring functions corresponding to the kafka complementary data theme through a Web interface.
Therefore, in the alternative embodiment, the to-be-detected subject and the complementary extraction subject information can be compared regularly, and the matched operation is executed according to the comparison result, so that the processing efficiency when the to-be-detected subject and the complementary extraction subject information are inconsistent is improved, and all created to-be-detected subjects are standardized, so that the processing efficiency when the data correction operation is executed according to the to-be-detected subject is improved.
In another alternative embodiment, the method for automatically correcting electric energy data may further include the steps of:
collecting and analyzing the data feedback rate of each target server for the target complementary transmission data and the data integrity rate of the electric energy data to be complementary extracted;
Drawing a statistical chart corresponding to the data feedback rate and the data integrity rate according to the counted data feedback rate and the data integrity rate, wherein the statistical chart is used for marking ranking information corresponding to the data feedback rate and the data integrity rate of each target server;
Determining an unqualified server with the data feedback rate smaller than a preset feedback rate threshold and/or the data integrity rate smaller than a preset data integrity rate threshold;
a warning flag for a failed server is generated, the warning flag being used to indicate that the frequency of inspection for the failed server needs to be increased.
Therefore, in the alternative embodiment, the data feedback rate of each target server for the target complementary transmission data and the data integrity rate of the electric energy data to be complemented can be intelligently counted, the statistical efficiency of the statistical data feedback rate and the data integrity rate is improved, the matching operation can be executed according to the statistical result, and the data feedback rate and the data integrity rate of each server are further improved to a certain extent.
Example III
Referring to fig. 3, fig. 3 is a schematic structural diagram of an automatic correction device for electric energy data according to an embodiment of the invention. The automatic correction device of the electric energy data may be an automatic correction device terminal of the electric energy data, an automatic correction device of the electric energy data, an automatic correction device system of the electric energy data or an automatic correction device server of the electric energy data, where the automatic correction device server of the electric energy data may be a local server, a remote server, or a cloud server (also called cloud server), and when the automatic correction device server of the electric energy data is a non-cloud server, the non-cloud server may be in communication connection with the cloud server. As shown in fig. 3, the automatic correction device for electric energy data may include an acquisition module 301, an analysis module 302, a calling module 303, a data processing module 304, and a correction module 305, where:
The collection module 301 is configured to collect electrical energy data to be compensated, where the electrical energy data is data that needs to be subjected to a data correction operation.
The analysis module 302 is configured to analyze the electric energy data acquired by the acquisition module 301 according to a predetermined intelligent analysis model, so as to obtain a complementary extraction theme of the electric energy data and a source region corresponding to the electric energy data, where the complementary extraction theme includes a full-scale complementary extraction theme or a detail increment complementary extraction theme, and the full-scale complementary extraction theme is a theme corresponding to first data that needs to perform a data correction operation, where the first data includes data with data loss and data without data loss; the detail increment complement theme is a theme corresponding to second data needing to execute data correction operation, and the second data is data with data missing.
And the calling module 303 is configured to call a complementary extraction subject interface matched with the complementary extraction subject according to the source region corresponding to the electric energy data obtained by the analysis module 302, where the complementary extraction subject interface is an interface that establishes an association relationship with the source region.
The data processing module 304 is configured to issue, to the determined plurality of target servers through the complementary extraction subject interfaces called by the calling module 303, complementary extraction data requirements matching the complementary extraction subject, where the complementary extraction data requirements include the complementary extraction subject.
The data processing module 304 is further configured to receive, through the parsing and warehousing interface included in the complementary extraction subject interface called by the calling module 303, target complementary transmission data that is fed back by the at least one target server and matches with the complementary extraction data requirement.
The correction module 305 is configured to perform a data correction operation on the target supplementary transmission data received by the data processing module 304 and the electric energy data acquired by the acquisition module 301 based on a predetermined metering data model, so as to obtain correction data, where the correction data includes corrected electric energy data and the target supplementary transmission data.
Therefore, the automatic correction device for electric energy data described in fig. 3 can intelligently collect the electric energy data needed to execute the data correction operation, which is beneficial to improving the data collection efficiency; the electric energy data can be analyzed through the intelligent analysis model, so that the analysis efficiency of the electric energy data is improved; furthermore, the matched complementary extraction subject interface can be adaptively called according to the electric energy data source region, so that the situation that the complementary extraction subject interface is not matched to cause the error issuing of the complementary extraction data requirement is reduced, and the lower accuracy of the complementary extraction data requirement is improved; in addition, after the target supplementary transmission data is received, the electric energy data can be intelligently corrected according to the metering data model and the target supplementary transmission data, so that the condition that the acquired electric energy data has data missing is reduced, and the data integrity rate of the electric energy data is improved.
In an alternative embodiment, as shown in fig. 4, the automatic correction device of electric energy data may further include a determining module 306 and a first processing module 307, where:
the determining module 306 is configured to determine, according to a predetermined intelligent computing model, a target processing operation to be performed on the target complementary data after the data processing module 304 receives, through an parsing and warehousing interface included in the complementary theme interface, the target complementary data that is fed back by the at least one target server and matches the complementary data requirement, where the target processing operation includes at least one of data statistics, data deduplication, data cleaning, and abnormal data analysis.
The first processing module 307 is configured to execute the target processing operation determined by the determining module 306 based on the data table, and obtain a target processing result.
Therefore, the automatic correction device for electric energy data described in fig. 4 can intelligently process the target supplementary transmission data through the intelligent calculation model and the data table, so that the situation that the target supplementary transmission data cannot be used due to the fact that the received target supplementary transmission data comprise repeated data and/or abnormal data and the like is reduced, and the accuracy of correction results obtained after correction operation is carried out on the electric energy data through the target supplementary transmission data is improved.
In an alternative embodiment, as shown in fig. 4, the automatic correction device of electric energy data may further include an obtaining module 308, a judging module 309, an updating module 310, and a calculating module 311, where:
the acquiring module 308 is configured to acquire a task progress corresponding to a first verification task for electric energy data, where the first verification task is used to calculate a data acquisition integrity rate corresponding to the electric energy data.
The judging module 309 is configured to judge whether the task progress corresponding to the first verification task obtained by the obtaining module 308 in the first execution period reaches a preset progress threshold.
The updating module 310 is configured to update the first execution period to a second execution period and update the first verification task to a second verification task when the judging module 309 judges that the task progress corresponding to the first verification task in the first execution period reaches the progress threshold, where the ending time corresponding to the first execution period is earlier than the starting time corresponding to the second execution period, and the second verification task is used to calculate the data acquisition integrity rate corresponding to the electric energy data.
The calculating module 311 is configured to calculate, when the determining module 309 determines that the task progress corresponding to the first verification task does not reach the progress threshold in the first execution period, a data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data received by the data processing module 304 and the electric energy data obtained by the acquisition module 301, so as to obtain a first value.
The updating module 310 is further configured to update the task progress corresponding to the first verification task to reach the progress threshold after the first value is calculated by the calculating module 311, update the first execution period to the second execution period, and update the first verification task to the second verification task.
Optionally, after the first value calculated by the calculating module 311 is calculated, a trigger instruction may be generated and sent to the updating module 310, so as to trigger the updating module 310 to perform the above-mentioned operation of updating the task progress corresponding to the first verification task to reach the progress threshold, updating the first execution period to the second execution period, and updating the first verification task to the second verification task.
Therefore, the automatic correction device of the electric energy data described in fig. 4 can be implemented to intelligently detect the task progress corresponding to the first verification task for the electric energy data, so that the timeliness of the acquired task progress is improved; and when the task progress does not reach the progress threshold value, the operation of calculating the data acquisition integrity rate of the electric energy data is executed, the situation that the task progress cannot be completed due to factors such as equipment faults is reduced, and the completion quantity of the task progress is further improved.
In another optional embodiment, the determining module 309 is further configured to determine, when it is determined that the task progress corresponding to the first verification task in the first execution period reaches the progress threshold, whether a value corresponding to the data acquisition integrity rate is greater than or equal to the determined integrity rate threshold based on a preset intelligent complementary extraction rule.
The automatic correction device for electric energy data may further include a generating module 312, where:
the generating module 312 is configured to generate a standard mark of the electric energy data when the judging module 309 judges that the value is greater than or equal to the integrity rate threshold, and trigger the updating module 310 to perform the operation of updating the first execution period to the second execution period and updating the first verification task to the second verification task, where the standard mark is used to indicate that the value of the electric energy data collection integrity rate is greater than or equal to the integrity rate threshold.
The judging module 309 is further configured to, when it is determined that the value is smaller than the integrity rate threshold, judge whether a verification identifier submitted by the user is detected, where the verification identifier is used to indicate that the integrity rate of data collection corresponding to the electric energy data needs to be calculated.
The calculating module 311 is further configured to calculate a data acquisition integrity rate corresponding to the electric energy data to obtain a second value when the judging module 309 judges that the verification identifier submitted by the user is detected.
The judging module 309 is further configured to perform an operation of judging whether the value corresponding to the data acquisition integrity rate is greater than or equal to the integrity rate threshold on the second value obtained by the calculating module 311 based on the intelligent complementary extraction rule.
Therefore, the automatic correction device for electric energy data described in fig. 4 can intelligently compare the calculated data acquisition rate data with the integrity rate threshold value, adaptively execute matching operation according to the comparison result, and reduce the situation that the calculated data acquisition integrity rate corresponding value is smaller than the integrity rate threshold value due to individual data reading errors, thereby improving the accuracy of the calculation result obtained by calculating the data acquisition integrity rate corresponding value.
In yet another alternative embodiment, the target server includes a plurality of sub-target servers, and the target complementary data includes target complementary data for all sub-target servers.
The first processing module 307 performs a target processing operation based on the data table, and the manner of obtaining the target processing result specifically includes:
When the target processing operation comprises abnormal data analysis, performing abnormal data analysis operation on the target complementary transmission data of each sub-target server based on the data table to obtain a data abnormal rate corresponding to the target complementary transmission data of the sub-target server;
Determining the data anomaly rate higher than a preset anomaly threshold from the data anomaly rates corresponding to the target complementary data of all the sub-target servers to obtain the target data anomaly rate, the anomaly complementary data corresponding to the target data anomaly rate and the sub-target servers corresponding to the anomaly complementary data;
marking abnormal supplementary transmission data, and monitoring a supplementary pumping link between a first interface and a second interface to obtain a data transmission condition corresponding to the supplementary pumping link, wherein the first interface is an interface of a sub-target server corresponding to the abnormal supplementary transmission data, and the second interface is an analysis warehouse-in interface;
When the data transmission condition indicates that the complementary pumping link has no uploading data amount in the preset data uploading time and/or indicates that the complementary pumping link has a link blocking condition, generating an abnormal identifier corresponding to the complementary pumping link, wherein the abnormal identifier is used for calling a preset link processing program to process the data transmission condition;
And determining the exception identification as a target processing result.
Therefore, implementing the automatic correction device for electric energy data described in fig. 4 can perform the data exception handling operation for the target supplementary transmission data, thereby reducing the duty ratio of the exception data in the received target supplementary transmission data, and thus reducing the interference of the exception data to the data correction operation, and further resulting in the situation that the correction result obtained by the data correction operation is wrong, and further improving the accuracy of the data correction operation result.
In another alternative embodiment, as shown in fig. 4, the apparatus for automatically correcting electric energy data may further include a second processing module 313, wherein:
the second processing module 313 is configured to compare a target parameter included in the target archive model with a standard parameter corresponding to the target archive model in a preset power grid data model, so as to obtain a first comparison result, where the power grid data model is configured to store data with the same type as the electric energy data to be compensated and data with the same type as the target compensation data.
The second processing module 313 is further configured to determine a first parameter that does not match the standard parameter in the target parameters when the first comparison result indicates that the target parameter does not match the standard parameter.
The analysis module 302 is further configured to analyze the first parameter obtained by the second processing module 313 to obtain a parameter type corresponding to the first parameter, where the parameter type includes at least one of a data table type, a field type, an index type, a trigger type, and an interface type.
The second processing module 313 is further configured to perform a model transformation operation matched with the parameter type on the first parameter obtained by the analysis module 302, so as to update the first parameter to be a standard parameter.
Therefore, the automatic correction device for electric energy data described in fig. 4 can intelligently execute the matching operation according to the comparison result obtained by comparing the target parameter with the standard parameter, so that the target parameter is consistent with the standard parameter, thereby reducing the inconsistent situation of the target parameter and the standard parameter, and reducing the reduced accuracy of the result obtained after the relevant data operation is executed through the power grid data model due to the inconsistent situation of the target parameter and the standard parameter.
In yet another alternative embodiment, the replenishment topic is managed by:
calling an API (application program interface) corresponding to a target server to create complementary extraction subject information, wherein the complementary extraction subject information comprises naming rules corresponding to the complementary extraction subject and formats corresponding to the content of the complementary extraction subject load;
Acquiring all topics to be detected through an API interface in a preset period, wherein the topics to be detected are created through a target server;
Comparing all the topics to be detected with the complementary extraction topic information to obtain a second comparison result;
When the second comparison result shows that the topic which is not matched with the complementary extraction topic information exists in the topic to be detected, determining and marking the topic which is not matched with the complementary extraction topic information;
And adjusting the topic name corresponding to the topic and/or adjusting the format corresponding to the load content of the topic according to the complementary topic information.
Therefore, implementing the automatic correction device for electric energy data described in fig. 4 can periodically compare the to-be-detected subject with the complementary extraction subject information, and execute the matched operation according to the comparison result, so as to improve the processing efficiency when the to-be-detected subject is inconsistent with the complementary extraction subject information, and further standardize all the created to-be-detected subjects, so as to improve the processing efficiency when the data correction operation is executed according to the to-be-detected subject.
Example IV
Referring to fig. 5, fig. 5 is a schematic structural diagram of an automatic correction device for electric energy data according to another embodiment of the present invention. As shown in fig. 5, the automatic correction device of electric energy data may include:
A memory 401 storing executable program codes;
a processor 402 coupled with the memory 401;
the processor 402 invokes executable program codes stored in the memory 401 to perform the steps in the automatic correction method for electric energy data described in the first or second embodiment of the present invention.
Example five
The embodiment of the invention discloses a computer storage medium which stores computer instructions for executing the steps in the automatic correction method of electric energy data described in the first or second embodiment of the invention when the computer instructions are called.
Example six
An embodiment of the present invention discloses a computer program product comprising a non-transitory computer storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the automatic correction method of electric energy data described in the first or second embodiment.
The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer storage medium including Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium readable by a computer that can be used to carry or store data.
Finally, it should be noted that: the embodiment of the invention discloses an automatic correction method and device for electric energy data, which are disclosed by the embodiment of the invention only for illustrating the technical scheme of the invention, but not limiting the technical scheme; although the invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (8)
1. An automatic correction method for electric energy data, characterized in that the method comprises the following steps:
collecting electric energy data to be compensated, wherein the electric energy data is data which needs to be subjected to data correction operation;
Analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a full-quantity or detail incremental full-quantity extraction theme and a source region corresponding to the electric energy data, the full complement extraction theme is a theme corresponding to first data needing to execute the data correction operation, wherein the first data comprises data with data missing and data without data missing; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute the data correction operation, and the second data is data with data missing;
According to the source region corresponding to the electric energy data, calling a complementary extraction subject interface matched with the complementary extraction subject, wherein the complementary extraction subject interface is an interface which establishes an association relationship with the source region;
issuing a complementary data demand matched with the complementary topic to a plurality of determined target servers through the complementary topic interfaces, wherein the complementary data demand comprises the complementary topic;
receiving target supplementary transmission data matched with the supplementary data requirement and fed back by at least one target server through an analysis and storage interface included in the supplementary extraction subject interface;
performing the data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data, wherein the correction data comprises the corrected electric energy data and the target supplementary transmission data;
the method further comprises the steps of:
Acquiring a task progress corresponding to a first verification task for the electric energy data, wherein the first verification task is used for calculating a data acquisition integrity rate corresponding to the electric energy data;
judging whether the task progress corresponding to the first verification task in a first execution period reaches a preset progress threshold or not;
when judging that the task progress corresponding to the first verification task reaches the progress threshold value in the first execution period, updating the first execution period to a second execution period and updating the first verification task to a second verification task, wherein the ending time corresponding to the first execution period is earlier than the starting time corresponding to the second execution period, and the second verification task is used for calculating the data acquisition integrity rate corresponding to the electric energy data;
when judging that the task progress corresponding to the first verification task does not reach the progress threshold value in the first execution period, calculating the data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data and the electric energy data to obtain a first numerical value;
after the first numerical value is obtained through calculation, updating the task progress corresponding to the first verification task to reach the progress threshold, updating the first execution period to a second execution period and updating the first verification task to a second verification task;
the method further comprises the steps of:
comparing target parameters included in a target archive model with standard parameters corresponding to the target archive model in a preset power grid data model to obtain a first comparison result, wherein the power grid data model is used for storing data with the same type as the electric energy data and data with the same type as the target supplementary transmission data;
When the first comparison result shows that the target parameter is not matched with the standard parameter, determining a first parameter which is not matched with the standard parameter in the target parameter;
analyzing the first parameter to obtain a parameter type corresponding to the first parameter, wherein the parameter type comprises at least one of a data table type, a field type, an index type, a trigger type and an interface type;
And executing a model reconstruction operation matched with the parameter type on the first parameter to update the first parameter to the standard parameter.
2. The method for automatically correcting electric energy data according to claim 1, wherein after receiving target complementary transmission data matched with the complementary data requirement and fed back by at least one target server through an analysis and storage interface included in the complementary topic interface, the method further comprises:
Determining target processing operations to be executed on the target complementary transmission data according to a predetermined intelligent computing model, wherein the target processing operations comprise at least one of data statistics, data deduplication, data cleaning and abnormal data analysis;
And executing the target processing operation on the target complementary transmission data based on a data table to obtain a target processing result.
3. The automatic correction method of electric energy data according to claim 1, wherein when it is determined that the task progress corresponding to the first verification task in the first execution period reaches the progress threshold value, the method further comprises:
Judging whether a value corresponding to the data acquisition integrity rate is greater than or equal to a determined integrity rate threshold value or not based on a preset intelligent complementary extraction rule;
When the numerical value is larger than or equal to the integrity rate threshold value, generating a standard mark of the electric energy data and executing the operation of updating the first execution period to a second execution period and updating the first verification task to a second verification task, wherein the standard mark is used for indicating that the numerical value of the electric energy data acquisition integrity rate is larger than or equal to the integrity rate threshold value;
When the numerical value is smaller than the integrity rate threshold value, judging whether a verification mark submitted by a user is detected, wherein the verification mark is used for indicating that the data acquisition integrity rate corresponding to the electric energy data needs to be calculated;
When the verification identifier submitted by the user is judged to be detected, calculating the data acquisition integrity rate corresponding to the electric energy data to obtain a second numerical value;
And executing the operation of judging whether the value corresponding to the data acquisition integrity rate is greater than or equal to the integrity rate threshold value on the second value based on the intelligent complementary extraction rule.
4. The automatic correction method of electric energy data according to claim 2, wherein the target server comprises a plurality of sub-target servers, and the target complementary transmission data comprises target complementary transmission data of all the sub-target servers;
when the target processing operation includes the abnormal data analysis, the executing the target processing operation on the target complementary transmission data based on a data table to obtain a target processing result includes:
Executing the abnormal data analysis operation on the target supplementary transmission data of each sub-target server based on a data table to obtain a data abnormal rate corresponding to the target supplementary transmission data of the sub-target server;
Determining a data anomaly rate higher than a preset anomaly threshold from the data anomaly rates corresponding to the target complementary data of all the sub-target servers to obtain a target data anomaly rate, anomaly complementary data corresponding to the target data anomaly rate and a sub-target server corresponding to the anomaly complementary data;
Marking the abnormal supplementary transmission data, and monitoring a supplementary pumping link between a first interface and a second interface to obtain a data transmission condition corresponding to the supplementary pumping link, wherein the first interface is an interface of a sub-target server corresponding to the abnormal supplementary transmission data, and the second interface is the analysis warehouse-in interface;
When the data transmission condition indicates that the supplemental link does not upload data amount in a preset data uploading time and/or indicates that the supplemental link has a link blocking condition, generating an abnormal identifier corresponding to the supplemental link, wherein the abnormal identifier is used for calling a preset link processing program to process the data transmission condition;
And determining the abnormal identification as a target processing result.
5. The method for automatically modifying electrical energy data according to claim 1, wherein the replenishment topic is managed by:
calling an API (application program interface) corresponding to the target server to create complementary extraction subject information, wherein the complementary extraction subject information comprises a naming rule corresponding to the complementary extraction subject and a format corresponding to the content of the complementary extraction subject load;
Acquiring all topics to be detected through the API in a preset period, wherein the topics to be detected are created through the target server;
comparing all the topics to be detected with the complementary extraction topic information to obtain a second comparison result;
When the second comparison result shows that the topic which is not matched with the complementary topic information exists in the topic to be detected, determining and marking the topic which is not matched with the complementary topic information;
and adjusting the topic naming corresponding to the topic and/or adjusting the format corresponding to the load content of the topic according to the complementary topic information.
6. An automatic correction device for electrical energy data, said device comprising:
The acquisition module is used for acquiring electric energy data to be supplemented, wherein the electric energy data are data which need to be subjected to data correction operation;
The analysis module is used for analyzing the electric energy data according to a predetermined intelligent analysis model to obtain a complementary extraction theme of the electric energy data and a source area corresponding to the electric energy data, wherein the complementary extraction theme comprises a full-quantity complementary extraction theme or a detail increment complementary extraction theme, and the full-quantity complementary extraction theme is a theme corresponding to first data needing to execute the data correction operation, wherein the first data comprises data with data loss and data without data loss; the detail increment supplementary drawing theme is a theme corresponding to second data needing to execute the data correction operation, and the second data is data with data missing;
the calling module is used for calling a complementary extraction subject interface matched with the complementary extraction subject according to the source region corresponding to the electric energy data obtained through analysis by the analysis module, wherein the complementary extraction subject interface is an interface which has an association relation with the source region;
The data processing module is used for issuing a complementary data requirement matched with the complementary topic to the determined target servers through the complementary topic interfaces, wherein the complementary data requirement comprises the complementary topic;
The data processing module is further used for receiving target supplementary transmission data matched with the supplementary drawing data requirement and fed back by at least one target server through an analysis and storage interface included in the supplementary drawing subject interface;
The correction module is used for executing the data correction operation on the target supplementary transmission data and the electric energy data based on a predetermined metering data model to obtain correction data, wherein the correction data comprises corrected electric energy data and target supplementary transmission data;
The apparatus further comprises:
the acquisition module is used for acquiring a task progress corresponding to a first verification task for the electric energy data, wherein the first verification task is used for calculating a data acquisition integrity rate corresponding to the electric energy data;
The judging module is used for judging whether the task progress corresponding to the first verification task in the first execution period reaches a preset progress threshold value or not;
The updating module is used for updating the first execution period to a second execution period and updating the first verification task to a second verification task when the judging module judges that the task progress corresponding to the first verification task reaches the progress threshold value in the first execution period, the termination time corresponding to the first execution period is earlier than the starting time corresponding to the second execution period, and the second verification task is used for calculating the data acquisition integrity rate corresponding to the electric energy data;
the calculation module is used for calculating the data acquisition integrity rate corresponding to the electric energy data in the first execution period according to the target supplementary transmission data and the electric energy data when judging that the task progress corresponding to the first verification task in the first execution period does not reach the progress threshold value, so as to obtain a first numerical value;
The updating module is further configured to update a task progress corresponding to the first verification task to reach the progress threshold after the first value is obtained by calculation, update the first execution period to a second execution period, and update the first verification task to a second verification task;
The apparatus further comprises:
the second processing module is used for comparing target parameters included in a target archive model with standard parameters corresponding to the target archive model in a preset power grid data model to obtain a first comparison result, wherein the power grid data model is used for storing data with the same type as the electric energy data and data with the same type as the target supplementary transmission data;
the second processing module is further configured to determine a first parameter that is not matched with the standard parameter in the target parameters when the first comparison result indicates that the target parameters are not matched with the standard parameters;
the analysis module is further configured to analyze the first parameter obtained by the second processing module to obtain a parameter type corresponding to the first parameter, where the parameter type includes at least one of a data table type, a field type, an index type, a trigger type, and an interface type;
The second processing module is further configured to perform a model transformation operation matched with the parameter type on the first parameter obtained by the analysis module, so as to update the first parameter to the standard parameter.
7. An automatic correction device for electrical energy data, said device comprising:
a memory storing executable program code;
A processor coupled to the memory;
the processor invokes the executable program code stored in the memory to perform the automatic correction method of electrical energy data according to any one of claims 1-5.
8. A computer storage medium storing computer instructions which, when invoked, are adapted to perform the method of automatically modifying electrical energy data according to any one of claims 1 to 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111569910.8A CN114253958B (en) | 2021-12-21 | 2021-12-21 | Automatic correction method and device for electric energy data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111569910.8A CN114253958B (en) | 2021-12-21 | 2021-12-21 | Automatic correction method and device for electric energy data |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114253958A CN114253958A (en) | 2022-03-29 |
CN114253958B true CN114253958B (en) | 2024-06-14 |
Family
ID=80793622
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111569910.8A Active CN114253958B (en) | 2021-12-21 | 2021-12-21 | Automatic correction method and device for electric energy data |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114253958B (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110046145A (en) * | 2019-04-02 | 2019-07-23 | 国网安徽省电力有限公司 | Expert intelligence Analysis Service platform based on electric flux big data research |
CN112104443A (en) * | 2020-09-02 | 2020-12-18 | 广东海启星海洋科技有限公司 | Automatic data supplementary transmission method and device, electronic equipment and storage medium |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113379274A (en) * | 2021-06-22 | 2021-09-10 | 南方电网数字电网研究院有限公司 | Complementary acquisition method and system based on electric energy data and storage medium |
-
2021
- 2021-12-21 CN CN202111569910.8A patent/CN114253958B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110046145A (en) * | 2019-04-02 | 2019-07-23 | 国网安徽省电力有限公司 | Expert intelligence Analysis Service platform based on electric flux big data research |
CN112104443A (en) * | 2020-09-02 | 2020-12-18 | 广东海启星海洋科技有限公司 | Automatic data supplementary transmission method and device, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN114253958A (en) | 2022-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107888397B (en) | Method and device for determining fault type | |
CN111882395A (en) | Monitoring method and system for rental equipment, computer equipment and storage medium | |
CN108805296A (en) | The monitoring method and monitoring device of measuring equipment abnormality | |
CN109743191B (en) | Accurate positioning centralized processing system, method and device for equipment alarm | |
CN108595390B (en) | Method for verifying parameter consistency of SCD and IED of intelligent substation | |
CN113472858B (en) | Buried point data processing method and device and electronic equipment | |
CN115796708A (en) | Intelligent quality inspection method, system and medium for big data for engineering construction | |
CN114253958B (en) | Automatic correction method and device for electric energy data | |
CN115047288A (en) | Transformer area fault positioning method and device based on power acquisition terminal | |
CN114764535A (en) | Power data processing method, device and equipment for simulation and storage medium | |
CN108491432B (en) | Electric power system accumulated quantity storage and extraction method based on message abstract, electronic equipment and storage medium | |
CN111984656B (en) | Method and system for checking basic data key parameters of intelligent substation automation equipment | |
CN116756528B (en) | Method, device, equipment and medium for complementing electricity load data | |
CN111931012A (en) | Data model self-adaption method and device based on fault management subsystem | |
CN113034307A (en) | Data acquisition method for power enterprise | |
CN115603321B (en) | Power load prediction system and method based on power consumption data | |
CN113923101B (en) | Internet of Things equipment network fault analysis method and device | |
CN116821115A (en) | Electric charge accounting service and data problem traceability error correction system and method | |
CN116359658A (en) | Power equipment abnormality judgment method and device, electronic equipment and storage medium | |
CN116304935A (en) | Abnormal data detection method and device and electronic equipment | |
CN108387776A (en) | A kind of method for early warning and device of intelligent electric meter | |
CN112448840B (en) | Communication data quality monitoring method, device, server and storage medium | |
CN113869849A (en) | Method and system for screening and processing abnormal electricity utilization information acquired on same day | |
CN112131210A (en) | Data identification method, device, server and storage medium | |
CN117608912B (en) | Full-automatic log analysis and fault processing system and method based on NLP large model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |