CN110704280A - Main data monitoring system - Google Patents
Main data monitoring system Download PDFInfo
- Publication number
- CN110704280A CN110704280A CN201910960284.1A CN201910960284A CN110704280A CN 110704280 A CN110704280 A CN 110704280A CN 201910960284 A CN201910960284 A CN 201910960284A CN 110704280 A CN110704280 A CN 110704280A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- main data
- main
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 140
- 238000004458 analytical method Methods 0.000 claims abstract description 49
- 238000012546 transfer Methods 0.000 claims abstract description 34
- 238000007726 management method Methods 0.000 claims description 56
- 238000000034 method Methods 0.000 claims description 36
- 230000003993 interaction Effects 0.000 claims description 33
- 230000008569 process Effects 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 9
- 230000010354 integration Effects 0.000 claims description 8
- 230000001360 synchronised effect Effects 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 6
- 230000002085 persistent effect Effects 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 238000001303 quality assessment method Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 abstract description 5
- 230000000007 visual effect Effects 0.000 abstract description 3
- 238000013461 design Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- UGDGKPDPIXAUJL-UHFFFAOYSA-N ethyl n-[4-[benzyl(2-phenylethyl)amino]-2-(4-ethylphenyl)-1h-imidazo[4,5-c]pyridin-6-yl]carbamate Chemical compound N=1C(NC(=O)OCC)=CC=2NC(C=3C=CC(CC)=CC=3)=NC=2C=1N(CC=1C=CC=CC=1)CCC1=CC=CC=C1 UGDGKPDPIXAUJL-UHFFFAOYSA-N 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 238000012558 master data management Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a main data monitoring system, which relates to the technical field of main data application, and comprises a data resource management platform, a weblogic middleware server and a main data server based on cxf-webservice, wherein the weblogic middleware server and the main data server of cxf-webservice are connected with the data resource management platform; the display of the whole network data is realized at different visual angles, the main data stream transfer condition is displayed comprehensively in multiple dimensions, and the visible, searchable, manageable and available targets are achieved; supporting main data service exception rule configuration; supporting monitoring and analysis of main data service abnormity; supporting the distribution of main data exception messages; the three-party consistency rate of the main data is improved to support the main data influence analysis (including two analysis modes of direct influence and indirect influence); when a certain type of main data service transaction fails, early warning is timely given to remind a user of which evaluation index or main data is affected after the data fails.
Description
Technical Field
The invention relates to the technical field of main data application, in particular to a main data monitoring system.
Background
WebLogic is an application server produced by Oracle corporation of America, specifically is middleware based on JAVAEE architecture, and is a Java application server used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application. The dynamic function of Java and the security of the Java Enterprise standard are introduced into the development, integration, deployment and management of large-scale network applications.
The data resource management platform adopts various technical modes such as batch, real-time and quasi-real-time to carry out data interaction with the enterprise-level information management system. Along with the continuous improvement of the practical level of the enterprise-level information management system, the quality requirements of the enterprise-level information management system on the dimensions such as timeliness, consistency and accuracy of main data are continuously improved.
The 'platformized' route is used for meeting the non-functional requirements of basic information system construction such as easy use, cross-platform, stability, safety, integration, expansion and the like. The technical characteristic platformization is to encapsulate the non-functional requirements of the data monitoring into a technical platform, and to stipulate and restrict the main structure inside each functional module through technical standards such as a platform name space, a layer structure and the like. The platform makes all the functional modules in the system automatically inherit all the technical characteristics of the platform, thereby simplifying the technical design work, and developers can put a great deal of saved time and energy on the specific business functional design or the processing of non-functional requirements no matter in the design stage or in the development and maintenance stage in the future.
The defects of the prior art are as follows: in the interaction process of partial main data, various conditions such as data loss, data processing failure and the like can occur, on-site operation and maintenance personnel can only carry out manual analysis through messages of service transactions or communication logs of an information integration platform, the investigation efficiency is not high, the interface is not friendly, and users of basic units cannot know the main data flow transfer condition in time.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to: the method solves the problems that various conditions such as data loss, data processing failure and the like can occur in the interaction process of partial main data, field operation and maintenance personnel can only carry out manual analysis through messages of service transaction or communication logs of an information integration platform, the investigation efficiency is not high, the interface is not friendly, and users of basic units cannot know the main data stream transfer condition in time.
The technical purpose is realized by the following technical scheme that the main data monitoring system comprises a data resource management platform, a weblogic middleware server and a main data server based on cxf-webservice, wherein the weblogic middleware server and the cxf-webservice main data server are connected with the data resource management platform;
the weblogic middleware server is used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application;
the cxf-webservice main data server comprises a main data monitoring module, is connected with the weblogic middleware server, and is used for monitoring the main data side in cooperation with a data stream transfer process and performing application analysis, data interaction process monitoring, main data service exception analysis and statistics, data consistency analysis positioning and system management.
Further, the main data monitoring module comprises an interface layer, a logic layer, a storage layer and an acquisition layer.
Further, the interface layer is used for providing a user login interface, an operation interface and a process approval interface for a platform user, maintaining/managing configuration information and platform use permission of system operation through the platform interface by the user, and providing monitoring processing for a system operation log.
Further, the logic layer is configured to provide core services and functions of the platform, including main data statistics, real-time main data flow transfer monitoring, quasi-real-time main data flow transfer monitoring, synchronous broadcast service flow transfer monitoring, electronic handover single flow transfer monitoring, exception management, statistics report, and transaction query.
Further, the storage layer is used for data storage and is responsible for persistent storage of data, and the persistent storage of the data comprises data file storage, main data storage, data standard information storage, data quality information storage and platform management information storage.
Furthermore, the acquisition layer is used for accessing a data service interface of the data, and the main data monitoring platform provides a real-time data service interface, a quasi-real-time data service interface and a batch data service interface through WebService, JMS or ETL.
Further, the real-time data service interface comprises a real-time service and a real-time broadcast;
the real-time service comprises WebService real-time service access data provided by a peripheral system through a data asset management platform;
the real-time broadcasting comprises that the data asset management platform distributes the change information to a target system in a quasi-real-time manner through a JMS protocol information integration platform;
the quasi-real-time-quasi-real-time receiving comprises data which are sent to a main data monitoring management platform through a JMS protocol asynchronous receiving source system, and when a transaction fails, the data have process rollback and asynchronous processing capacity.
Further, the data interaction process monitoring comprises interaction monitoring of a data provider and main data and interaction monitoring of the main data and a data consumer;
the interactive monitoring of the data provider and the main data comprises batch data flow monitoring and single data flow monitoring;
the interactive monitoring of the main data and the data consumer comprises batch data broadcasting monitoring and single data broadcasting monitoring;
the data interaction process monitoring comprises the steps of carrying out data real-time statistics and message queue monitoring on data of interaction monitoring of a data provider and main data, data of interaction monitoring of the main data and a data consumer, and data interaction abnormity capturing.
Further, the main data service anomaly analysis and statistics comprise reason analysis, service anomaly statistics and influence analysis;
the service abnormity statistics can be used for counting data providers and data consumers, and can be used for counting main data types, error reasons and transaction time, and counting the range of data influence.
Further, the system management comprises user management, authority management and data synchronization management;
the data synchronization management can cooperate with the main data quality problem detail, cooperate with the main data quality score, cooperate with the main data quality assessment rule and the main data service transaction log.
In conclusion, the invention has the following technical effects:
the display of the whole network data is realized at different visual angles, the main data stream transfer condition is displayed comprehensively in multiple dimensions, and the visible, searchable, manageable and available targets are achieved; supporting main data service exception rule configuration; supporting monitoring and analysis of main data service abnormity; supporting the distribution of main data exception messages; the three-party consistency rate of the main data is improved to support the main data influence analysis (including two analysis modes of direct influence and indirect influence); when a certain type of main data service transaction fails, early warning is timely carried out to remind a user of which evaluation index or main data is affected after the data fails; the method supports the configuration of the consistency check rule of the main data, supports the consistency check scheduling of the main data and supports the consistency check analysis of the main data.
Drawings
Fig. 1 is an integrated schematic diagram of monitoring and application analysis of a cooperative data stream transfer process at a main data side of a data resource management platform according to an embodiment;
FIG. 2 is an integrated schematic diagram of a primary data monitoring module according to an embodiment;
FIG. 3 is a schematic diagram of the overall integration of the primary data monitoring according to the embodiment;
FIG. 4 is a schematic view of an embodiment of a primary data monitoring process;
1. analyzing and counting main data service abnormity; 11. analyzing and positioning data consistency; 12. monitoring a data interaction process; 121. interactive monitoring of a data provider and main data; 1211. monitoring batch data circulation; 1212. monitoring single data flow; 122. interactive monitoring of the primary data and the data consumers; 1221. monitoring single data broadcast; 1222. monitoring batch data broadcast; 13. managing a system; 14. analyzing reasons; 15. analyzing the influence; 16. counting the service abnormity; 17. data synchronization; 2. carrying out real-time statistics on data; 3. monitoring a message queue; 4. capturing data interaction abnormity; 5. an interfacial layer; 6. a logic layer; 7. a storage layer; 8. an acquisition layer; 9. a data resource management platform; 91. managing main data; 92. managing the data quality; 93. a business system; 94. a soa service; 95. a message queue.
Detailed Description
In the first embodiment, the first step is,
as shown in fig. 1 and fig. 4, a master data monitoring system includes a data resource management platform 9, and further includes a weblogic middleware server and a master data server based on cxf-webservice, where the weblogic middleware server and the cxf-webservice master data server are connected to the data resource management platform 9;
the weblog middleware server is used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application;
the cxf-webservice main data server comprises a main data monitoring module, which is connected with the weblog middleware server and used for monitoring and application analysis of a main data side collaborative data stream transfer process, performing data interaction process monitoring 12, main data service exception analysis and statistics 1, data consistency analysis and positioning 11 and system management 13, wherein the main data side collaborative data stream transfer process monitoring and application analysis are mainly shown in fig. 1.
In this embodiment, the data resource management platform 9 includes a master data management 91 and a data quality management 92 in parallel with the data management 91, where the master data management 91 is configured to receive all information fed back by a master data server managing cxf-web service, and the data quality management 92 is configured to approve information fed back by a master data server managing cxf-web service.
In the embodiment, the configuration of the main data service exception rule is supported; supporting monitoring and analysis of main data service abnormity; supporting the distribution of main data exception messages; the three-party consistency rate of the main data is improved to support the main data influence analysis (including two analysis modes of direct influence and indirect influence); when a certain type of main data service transaction fails, early warning is timely carried out to remind a user of which evaluation index or main data is affected after the data fails; the method supports the configuration of the consistency check rule of the main data, supports the consistency check scheduling of the main data and supports the consistency check analysis of the main data.
In the second embodiment, the first embodiment of the method,
as shown in fig. 1 and fig. 4, a master data monitoring system includes a data resource management platform 9, and further includes a weblogic middleware server and a master data server based on cxf-webservice, where the weblogic middleware server and the cxf-webservice master data server are connected to the data resource management platform 9;
the weblog middleware server is used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application;
the cxf-webservice main data server comprises a main data monitoring module, which is connected with the weblog middleware server and used for monitoring and application analysis of a main data side collaborative data stream transfer process, performing data interaction process monitoring 12, main data service exception analysis and statistics 1, data consistency analysis and positioning 11 and system management 13, wherein the main data side collaborative data stream transfer process monitoring and application analysis are mainly shown in fig. 1.
In the embodiment, the configuration of the main data service exception rule is supported; supporting monitoring and analysis of main data service abnormity; supporting the distribution of main data exception messages; the three-party consistency rate of the main data is improved to support the main data influence analysis (including two analysis modes of direct influence and indirect influence); when a certain type of main data service transaction fails, early warning is timely carried out to remind a user of which evaluation index or main data is affected after the data fails; the method supports the configuration of the consistency check rule of the main data, supports the consistency check scheduling of the main data and supports the consistency check analysis of the main data.
As shown in fig. 2, the master data monitoring module comprises an interface layer 5, a logic layer 6, a storage layer 7 and an acquisition layer 8.
The interface layer 5 is used for providing a user login interface, an operation interface and a process approval interface for a platform user, maintaining/managing configuration information and platform use permission of system operation through the platform interface by the user, and providing monitoring processing for a system operation log.
The logic layer 6 is used for providing core services and functions of the platform, and comprises main data statistics, real-time main data flow transfer monitoring, quasi-real-time main data flow transfer monitoring, synchronous broadcast service flow transfer monitoring, electronic transfer single flow transfer monitoring, exception management, statistical reports and transaction query.
The storage layer 7 is used for data storage and is responsible for persistent storage of data, and the persistent storage comprises data file storage, main data storage, data standard information storage, data quality information storage and platform management information storage.
The acquisition layer 8 is used for accessing a data service interface of data, and the main data monitoring platform provides a real-time, quasi-real-time and batch data service interface through WebService, JMS and ETL.
The real-time comprises real-time service and real-time broadcast;
the peripheral system accesses data through WebService real-time service provided by the data asset management platform;
the real-time broadcasting and data asset management platform distributes the change information to a target system in quasi-real time through a JMS protocol information integration platform;
the quasi real-time is received in quasi real-time, the data sent to the main data monitoring management platform through the JMS protocol asynchronous receiving source system, and when the transaction fails, the data has flow rollback and asynchronous processing capacity.
Wherein, JSP: the full name is Java Server Pages, called Java Server Pages in chinese, which is fundamentally a simplified Servlet design, which is a dynamic web technology standard created by many companies participating together, advocated by Sun Microsystems, JSP technology is somewhat similar to ASP technology, which inserts Java program segments (Scriptlet) and JSP tags (tag) into traditional web HTML (a subset of standard universal markup language) files (@, HTML) to form JSP files, named suffix (@. JSP). The Web application developed by JSP is cross-platform and can be run not only under Linux, but also on other operating systems.
And Css: cascading Style Sheets (english-language full name: Cascading Style Sheets) is a computer language used to represent file styles such as HTML (an application of standard general markup language) or XML (a subset of standard general markup language).
In the present embodiment, the batch mode in the acquisition layer 8 realizes batch transmission of data by an ETL method.
In this embodiment, the main data real-time main data flow monitoring, quasi-real-time service flow monitoring, synchronous broadcast service flow monitoring, and electronic handover single flow monitoring are used for monitoring each link of the main data, and can monitor the failure number of each link and display the failure node data and the corresponding message information.
The method can display the success number, the failure number and the total number of the overall situation of the main data whole network, the situation of an electronic transfer single, the situation of each service system and the situation of each main data transmission mode in real time to realize the monitoring of the main data, and can check the data with different values according to the real main data flow transfer monitoring, the quasi-vertical service flow monitoring, the synchronous broadcast service flow transfer monitoring, the electronic transfer single flow transfer monitoring, the abnormity Portal 29637, the report statistics and the transaction information.
The main data real-time main data flow monitoring, quasi-real-time service flow monitoring, synchronous broadcast service flow monitoring and electronic transfer single flow monitoring are used for monitoring all links of the main data, and the failure number of each link can be monitored, and failure node data and corresponding message information can be displayed.
In this embodiment, the main data may rebroadcast the monitored abnormal information for each service, the rebroadcast may modify the broadcast state of the main data, the component may broadcast the data information in which the broadcast state is modified in the data, and the data information may be redistributed to each main data after the rebroadcast.
In the third embodiment, the first step is that,
as shown in fig. 1 and fig. 4, a master data monitoring system includes a data resource management platform 9, and further includes a weblogic middleware server and a master data server based on cxf-webservice, where the weblogic middleware server and the cxf-webservice master data server are connected to the data resource management platform 9;
the weblog middleware server is used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application;
the cxf-webservice main data server comprises a main data monitoring module, which is connected with the weblog middleware server and used for monitoring and application analysis of a main data side collaborative data stream transfer process, performing data interaction process monitoring 12, main data service exception analysis and statistics 1, data consistency analysis and positioning 11 and system management 13, wherein the main data side collaborative data stream transfer process monitoring and application analysis are mainly shown in fig. 1.
As shown in fig. 3 and 4, the data interaction process monitoring includes interaction monitoring 121 of data providers and main data and interaction monitoring 12 of main data and data consumers; the data provider and main data interaction monitoring 121 comprises a batch data circulation monitoring 1211 and a single data circulation monitoring 1212; the interaction monitoring 12 of the main data and the data consumer comprises batch data broadcast monitoring 1222 and single data broadcast monitoring 1221; the data interaction process monitoring comprises the steps of carrying out data real-time statistics 2, message queue monitoring 3 and data interaction abnormity capturing 4 on data of interaction monitoring of a data provider and main data and interaction monitoring of the main data and a data consumer, wherein the main data service abnormity analysis and statistics 1 comprises reason analysis 141, service abnormity statistics 16 and influence analysis 15; the service anomaly statistics 16 can be used for statistics of data providers and data consumers, main data category statistics, error reasons and transaction time statistics, and can be used for statistics of the range of data influence, and the system management 13 comprises user management, authority management and data synchronization management; the data synchronization management 17 may coordinate with the master data quality issue details, with the master data quality scores, with the master data quality assessment rules and with the master data service transaction logs.
In this embodiment, the main contents of the project of the main data monitoring and managing platform include main data statistics, real-time main data traffic monitoring, quasi-real-time main data traffic monitoring, synchronous broadcast service traffic monitoring, electronic handover single traffic monitoring, exception management, statistical report forms, and design and development of a transaction query module.
In this embodiment, as shown in fig. 4, the main data service may receive the business system 93 and the data fed back to the soa service 94 by the business system 93, and the main data service may feed back the data to the main data platform library 9, the main data platform library 9 divides the data into the message queue 95 after receiving the information, and after feeding back the information to the internal soa service 94, the soa service 94 returns a record after receiving the information of the information, compares the record with the main data side data to determine whether the record is consistent, and stores the record in the main data platform library.
The soa service 94 is used, among other things, for primary data monitoring reception and business service system data.
In conclusion, the invention has the advantages that: the display of the whole network data is realized at different visual angles, the main data stream transfer condition is displayed comprehensively in multiple dimensions, and the visible, searchable, manageable and available targets are achieved; supporting main data service exception rule configuration; supporting monitoring and analysis of main data service abnormity; supporting the distribution of main data exception messages; the three-party consistency rate of the main data is improved to support the main data influence analysis (including two analysis modes of direct influence and indirect influence); when a certain type of main data service transaction fails, early warning is timely carried out to remind a user of which evaluation index or main data is affected after the data fails; the method supports the configuration of the consistency check rule of the main data, supports the consistency check scheduling of the main data and supports the consistency check analysis of the main data.
The present embodiment is only for explaining the present invention, and it is not limited to the present invention, and those skilled in the art can make modifications of the present embodiment without inventive contribution as needed after reading the present specification, but all of them are protected by patent law within the scope of the claims of the present invention.
Claims (10)
1. A main data monitoring system comprises a data resource management platform and is characterized in that: the weblogic middleware server and a main data server based on cxf-webservice are further included, and the weblogic middleware server and the main data server of cxf-webservice are connected with the data resource management platform;
the weblogic middleware server is used for developing, integrating, deploying and managing large-scale distributed Web application, network application and database application;
the cxf-webservice main data server comprises a main data monitoring module, is connected with the weblogic middleware server, and is used for monitoring the main data side in cooperation with a data stream transfer process and performing application analysis, data interaction process monitoring, main data service exception analysis and statistics, data consistency analysis positioning and system management.
2. A master data monitoring system according to claim 1, wherein: the main data monitoring module comprises an interface layer, a logic layer, a storage layer and an acquisition layer.
3. A master data monitoring system according to claim 2, wherein: the interface layer is used for providing a user login interface, an operation interface and a process approval interface for a platform user, maintaining/managing configuration information and platform use permission of system operation through the platform interface by the user, and providing monitoring processing for a system operation log.
4. A master data monitoring system according to claim 2, wherein: the logic layer is used for providing core services and functions of the platform, and comprises main data statistics, real-time main data flow transfer monitoring, quasi-real-time main data flow transfer monitoring, synchronous broadcast service flow transfer monitoring, electronic transfer single flow transfer monitoring, exception management, statistical reports and transaction query.
5. A master data monitoring system according to claim 2, wherein: the storage layer is used for data storage and is responsible for persistent storage of data, and the persistent storage comprises data file storage, main data storage, data standard information storage, data quality information storage and platform management information storage.
6. A master data monitoring system according to claim 2, wherein: the data monitoring system comprises a collection layer and a main data monitoring platform, wherein the collection layer is used for accessing a data service interface of data, and the main data monitoring platform provides a real-time data service interface, a quasi-real-time data service interface and a batch data service interface through WebService, JMS or ETL.
7. A master data monitoring system according to claim 6, wherein: the real-time data service interface comprises real-time service and real-time broadcast;
the real-time service comprises WebService real-time service access data provided by a peripheral system through a data asset management platform;
the real-time broadcasting comprises that the data asset management platform distributes the change information to a target system in a quasi-real-time manner through a JMS protocol information integration platform;
the quasi-real-time-quasi-real-time receiving comprises data which are sent to a main data monitoring management platform through a JMS protocol asynchronous receiving source system, and when a transaction fails, the data have process rollback and asynchronous processing capacity.
8. A master data monitoring system according to claim 1, wherein: the data interaction process monitoring comprises interaction monitoring of a data provider and main data and interaction monitoring of the main data and a data consumer;
the interactive monitoring of the data provider and the main data comprises batch data flow monitoring and single data flow monitoring;
the interactive monitoring of the main data and the data consumer comprises batch data broadcasting monitoring and single data broadcasting monitoring;
the data interaction process monitoring comprises the steps of carrying out data real-time statistics and message queue monitoring on data of interaction monitoring of a data provider and main data, data of interaction monitoring of the main data and a data consumer, and data interaction abnormity capturing.
9. A master data monitoring system according to claim 1, wherein: the main data service abnormity analysis and statistics comprise reason analysis, service abnormity statistics and influence analysis;
the service abnormity statistics can be used for counting data providers and data consumers, and can be used for counting main data types, error reasons and transaction time, and counting the range of data influence.
10. A master data monitoring system according to claim 1, wherein: the system management comprises user management, authority management and data synchronization management;
the data synchronization management can cooperate with the main data quality problem detail, cooperate with the main data quality score, cooperate with the main data quality assessment rule and the main data service transaction log.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910960284.1A CN110704280A (en) | 2019-10-10 | 2019-10-10 | Main data monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910960284.1A CN110704280A (en) | 2019-10-10 | 2019-10-10 | Main data monitoring system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110704280A true CN110704280A (en) | 2020-01-17 |
Family
ID=69199193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910960284.1A Pending CN110704280A (en) | 2019-10-10 | 2019-10-10 | Main data monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110704280A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111368165A (en) * | 2020-02-25 | 2020-07-03 | 江苏物联网研究发展中心 | Spatio-temporal streaming data integration platform |
| CN111563018A (en) * | 2020-04-28 | 2020-08-21 | 北京航空航天大学 | Resource management and monitoring method of man-machine-object fusion cloud computing platform |
| CN112231362A (en) * | 2020-10-29 | 2021-01-15 | 广东电网有限责任公司广州供电局 | Visual monitoring application system based on power grid main data service log |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009230167A (en) * | 2008-03-19 | 2009-10-08 | Nec Infrontia Corp | Master data management system, master data management method, master data management program and program recording medium |
| CN103631595A (en) * | 2013-12-03 | 2014-03-12 | 大连大学 | Method for developing Web Service based on CXF (CeltiXfire) |
| CN104915765A (en) * | 2015-05-29 | 2015-09-16 | 中国南方电网有限责任公司 | Marketing, dispatching, map operation and modeling integrated management system based on SOA framework |
| CN105871605A (en) * | 2016-03-30 | 2016-08-17 | 国网江西省电力科学研究院 | Operation and maintenance monitoring platform based on big power marketing data |
| CN106649040A (en) * | 2016-12-26 | 2017-05-10 | 上海新炬网络信息技术有限公司 | Automatic monitoring method and device for performance of Weblogic middleware |
| CN104361424B (en) * | 2014-10-11 | 2018-05-04 | 中国电子科技集团公司第十研究所 | Main data system integrated approach based on Enterprise Service Bus |
-
2019
- 2019-10-10 CN CN201910960284.1A patent/CN110704280A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009230167A (en) * | 2008-03-19 | 2009-10-08 | Nec Infrontia Corp | Master data management system, master data management method, master data management program and program recording medium |
| CN103631595A (en) * | 2013-12-03 | 2014-03-12 | 大连大学 | Method for developing Web Service based on CXF (CeltiXfire) |
| CN104361424B (en) * | 2014-10-11 | 2018-05-04 | 中国电子科技集团公司第十研究所 | Main data system integrated approach based on Enterprise Service Bus |
| CN104915765A (en) * | 2015-05-29 | 2015-09-16 | 中国南方电网有限责任公司 | Marketing, dispatching, map operation and modeling integrated management system based on SOA framework |
| CN105871605A (en) * | 2016-03-30 | 2016-08-17 | 国网江西省电力科学研究院 | Operation and maintenance monitoring platform based on big power marketing data |
| CN106649040A (en) * | 2016-12-26 | 2017-05-10 | 上海新炬网络信息技术有限公司 | Automatic monitoring method and device for performance of Weblogic middleware |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111368165A (en) * | 2020-02-25 | 2020-07-03 | 江苏物联网研究发展中心 | Spatio-temporal streaming data integration platform |
| CN111563018A (en) * | 2020-04-28 | 2020-08-21 | 北京航空航天大学 | Resource management and monitoring method of man-machine-object fusion cloud computing platform |
| CN111563018B (en) * | 2020-04-28 | 2021-11-12 | 北京航空航天大学 | Resource management and monitoring method of man-machine-object fusion cloud computing platform |
| CN112231362A (en) * | 2020-10-29 | 2021-01-15 | 广东电网有限责任公司广州供电局 | Visual monitoring application system based on power grid main data service log |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110263054A (en) | Auditing system, checking method, device and the computer equipment of SQL work order | |
| CN104657903A (en) | Compressive guarantee system for equipment based on IETM | |
| CN106570728A (en) | Internet plus inspection and detection service method and platform | |
| CN111866121B (en) | Safety monitoring and management cloud platform for large crane equipment | |
| CN110704280A (en) | Main data monitoring system | |
| CN113094385B (en) | Data sharing fusion platform and method based on software defined open tool set | |
| CN101807816A (en) | Realization method and system of electric power data public access service interfaces in heterogeneous environment | |
| CN111639114A (en) | Distributed data fusion management system based on Internet of things platform | |
| CN101526906A (en) | Management system for running maintenance services | |
| CN116467389A (en) | Data synchronization method for coal mine two-stage data center | |
| CN112580199A (en) | Electric power system multidimensional data unified construction system based on CIM model | |
| CN113238736A (en) | Method for generating integrated platform | |
| CN103136161A (en) | On-line management method of checking and tracking of job schedule abnormalities | |
| CN103295164A (en) | Power grid safety monitoring management information system and management method | |
| CN112241367A (en) | Data line testing method and device | |
| CN103106555A (en) | Maintenance and management system and method based on online and offline integration | |
| CN110827172A (en) | Wisdom water affairs cloud service platform | |
| CN116821106A (en) | Enterprise data digital management system and method | |
| CN115796806A (en) | System construction method based on micro-service | |
| CN115033574A (en) | Information generation method, information generation device, electronic device, and storage medium | |
| KR20140115267A (en) | A centrally managed and accessed system and method for performing data processing on multiple independent servers and datasets | |
| CN114219199A (en) | Supplier credit management method and supplier credit management system | |
| CN114490854A (en) | Report generation method and device, electronic equipment and computer readable medium | |
| CN201590107U (en) | Taxation management system based on Web GIS | |
| Zhang et al. | Intelligent Grid Operation and Maintenance Management and Command Platform Based on Computer Distributed Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200117 |