CN111487920A - Data acquisition and processing system - Google Patents
Data acquisition and processing system Download PDFInfo
- Publication number
- CN111487920A CN111487920A CN202010456198.XA CN202010456198A CN111487920A CN 111487920 A CN111487920 A CN 111487920A CN 202010456198 A CN202010456198 A CN 202010456198A CN 111487920 A CN111487920 A CN 111487920A
- Authority
- CN
- China
- Prior art keywords
- data acquisition
- data
- module
- dtu
- plan
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims abstract description 7
- 238000012544 monitoring process Methods 0.000 claims description 19
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 abstract description 9
- 238000011156 evaluation Methods 0.000 abstract description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/05—Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
- G05B19/054—Input/output
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention discloses a data acquisition processing system which comprises a plurality of DTU data acquisition units and a data processing unit, wherein the data processing unit is respectively and remotely connected with each DTU data acquisition unit. Through the technical scheme, concurrent connection, control management, automatic online and processing condition evaluation of the data acquisition unit and the data processing unit can be realized, so that full automation of data acquisition and preprocessing processes is realized, convenience is brought to a user for data acquisition and preprocessing, and the use experience of the user is improved.
Description
Technical Field
The invention relates to the field of sensor data acquisition and preprocessing, in particular to a data acquisition and processing system.
Background
Data acquisition, also known as data acquisition, utilizes a device to acquire data from outside the system and input it to an interface within the system. The current data acquisition technology is widely applied in various fields.
The current existing data acquisition system is difficult to realize intelligent remote control, the control of each remote data unit is often mechanical and needs manual control intervention, and meanwhile, the data preprocessing and error checking cannot be realized, so that a data acquisition processing system capable of meeting the application requirements is urgently needed.
Disclosure of Invention
To solve the above problems in the prior art, a data acquisition and processing system is provided, and the specific technical scheme is as follows:
a data acquisition processing system comprises a plurality of DTU data acquisition units and a data processing unit, wherein the data processing unit is respectively and remotely connected with each DTU data acquisition unit;
each DTU data acquisition unit is connected with at least one data acquisition equipment group, and the DTU data acquisition units acquire data through the data acquisition equipment groups;
the data processing unit specifically comprises:
the system comprises a planning module, a data acquisition module and a data processing module, wherein the planning module is used for generating a batch execution plan, the batch execution plan comprises a plurality of sub-plans which are linearly arranged, and each sub-plan corresponds to a DTU data acquisition unit;
the first monitoring module is connected with the plan module and is used for continuously monitoring the execution condition of the batch execution plan;
the request module is connected with the first monitoring module and used for sending a request instruction to the DTU data acquisition unit executing the sub-plan when the sub-plan is executed according to the execution condition;
the first processing module is used for receiving the data to be analyzed returned by the DTU data acquisition unit, analyzing the data and outputting the data to be stored;
the first transmission module is respectively connected with the first processing module and the first monitoring module and used for caching the data to be stored and storing all the data to be stored to an external database when the batch execution plan is executed;
the DTU data acquisition unit acquires data according to the batch execution plan, and specifically comprises the following steps:
the second monitoring module is used for continuously monitoring the execution condition of the sub-plan;
and the second processing module is connected with the second monitoring module, processes the acquired data and outputs the data to be processed when one sub-plan finishes execution according to the execution condition, and then sequentially executes the next sub-plan.
Preferably, in the data acquisition and processing system, the data processing unit further includes a first determining module;
the first judging module is used for judging whether each DTU data acquisition unit is in online connection with the data processing unit or not and outputting a corresponding first judging result.
Preferably, in the data acquisition processing system, the DTU data acquisition unit further includes a heartbeat module, and when the DTU data acquisition unit is powered on, the heartbeat module sends a heartbeat data packet to the first judgment module every a first preset time interval;
the first judgment module judges whether the DTU data acquisition unit is connected with the data processing unit on line according to the collection condition of the heartbeat data packet: and when the first judging unit does not receive the heartbeat data packet for a second preset time, judging that the DTU data acquisition unit is not connected with the data processing unit on line.
Preferably, in the data acquisition processing system, each DTU data acquisition unit further includes a second judgment module;
the second judgment module is used for judging whether the DTU data acquisition unit has the lower control operation or not and outputting a corresponding second judgment result.
Preferably, in the data acquisition and processing system, the second judgment module is connected to the second detection module;
the second judging module starts a receiving window for receiving the lower control operation instruction when a sub-plan is executed according to the execution condition, and judges whether the lower control operation exists according to the receiving condition of the lower control operation instruction.
Preferably, in the data acquisition and processing system, the second judgment module is further connected to the second processing module;
and the second processing module executes the next sub-plan in sequence after waiting for the execution of the lower control operation to be finished when the lower control operation exists according to the second judgment result.
Preferably, in the data acquisition and processing system, the DTU data acquisition unit is concurrently connected to the data processing unit.
Preferably, in the data acquisition and processing system, the first processing module further includes a judgment sub-module;
the judgment submodule is used for judging whether the data to be analyzed is successfully analyzed and outputting a third judgment result;
the first processing module is also connected with the request module, and outputs the successfully analyzed data as the data to be stored when the data to be analyzed is successfully analyzed according to the third judgment result; and when the data to be analyzed cannot be analyzed successfully, the control request module sends the request instruction again.
This technical scheme has following advantage or beneficial effect:
through the technical scheme, concurrent connection, control management, automatic online and processing condition evaluation of the data acquisition unit and the data processing unit can be realized, so that full automation of data acquisition and preprocessing processes is realized, convenience is brought to a user for data acquisition and preprocessing, and the use experience of the user is improved.
Drawings
Fig. 1 is a schematic structural diagram of a data acquisition and processing system according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
To solve the above problems in the prior art, a data acquisition and processing system is provided, and the specific technical scheme is as follows:
a data acquisition processing system, as shown in figure 1, comprises a plurality of DTU data acquisition units 2 and a data processing unit 1, wherein the data processing unit 1 is respectively and remotely connected with each DTU data acquisition unit 2;
each DTU data acquisition unit 2 is connected with at least one data acquisition equipment group, and the DTU data acquisition unit 2 acquires data through the data acquisition equipment group;
the data processing unit 1 specifically includes:
the planning module 11 is configured to generate a batch execution plan, where the batch execution plan includes multiple linearly arranged sub-plans, and each sub-plan corresponds to a DTU data acquisition unit;
the first monitoring module 12 is connected with the planning module 11 and is used for continuously monitoring the execution condition of the batch execution plan;
the request module 13 is connected to the first monitoring module 12, and configured to send a request instruction to the DTU data acquisition unit executing the sub-plan when the sub-plan completes execution according to the execution condition;
the first processing module 14 is configured to receive data to be analyzed returned by the DTU data acquisition unit, analyze the data, and output the data to be stored;
the first transmission module 15 is respectively connected with the first processing module 14 and the first monitoring module 12, and is used for caching data to be stored and storing all the data to be stored to an external database when the batch execution plan is executed;
the DTU data acquisition unit 2 performs data acquisition according to the batch execution plan, and specifically includes:
a second monitoring module 21, configured to continuously monitor an execution condition of the sub-plan;
and the second processing module 22 is connected with the second detection module 21, and according to the execution condition, when one sub-plan is executed, the acquired data is processed and the data to be processed is output, and then the next sub-plan is sequentially executed.
As a preferred embodiment, the data acquisition and processing system, wherein the data processing unit further comprises a first judgment module;
the first judging module is used for judging whether each DTU data acquisition unit is in online connection with the data processing unit or not and outputting a corresponding first judging result.
As a preferred embodiment, in the data acquisition processing system, the DTU data acquisition unit further includes a heartbeat module, and when the DTU data acquisition unit is powered on, the heartbeat module sends a heartbeat data packet to the first determination module every a first preset time interval;
the first judgment module judges whether the DTU data acquisition unit is connected with the data processing unit on line according to the collection condition of the heartbeat data packet: and when the first judging unit does not receive the heartbeat data packet for a second preset time, judging that the DTU data acquisition unit is not connected with the data processing unit on line.
In a preferred embodiment of the present invention, since the DTU data acquisition unit has an automatic heartbeat function, it is able to complete power-on automatic dialing and periodically send heartbeat data packets to the data processing unit, so that it is determined whether the DTU data acquisition unit is in an online state by determining the receiving condition of the data processing unit for the heartbeat data packets, and when the DTU data acquisition unit is in an offline state, the DTU data acquisition unit can be initialized by a remote control method, and the data acquisition progress is re-established so that the DTU data acquisition unit completes automatic reconnection.
In a preferred embodiment, each DTU data acquisition unit further includes a second determining module;
the second judgment module is used for judging whether the DTU data acquisition unit has the lower control operation or not and outputting a corresponding second judgment result.
In a preferred embodiment, the data acquisition and processing system is provided, wherein the second judging module is connected to the second detecting module;
the second judging module starts a receiving window for receiving the lower control operation instruction when a sub-plan is executed according to the execution condition, and judges whether the lower control operation exists according to the receiving condition of the lower control operation instruction.
In a preferred embodiment, the data acquisition and processing system, wherein the second determining module is further connected to the second processing module;
and the second processing module executes the next sub-plan in sequence after waiting for the execution of the lower control operation to be finished when the lower control operation exists according to the second judgment result.
In another preferred embodiment of the present invention, the DTU data acquisition unit may receive the lower control operation command outside the task process, and how to balance the processing sequence between the two without generating conflict requires effective planning of the execution time of the lower control operation command: and if the lower control operation instruction is received, the execution of the next sub-plan is suspended, and the lower control operation is executed before the completion, so that the potential system fault caused by task conflict can be effectively prevented.
In a preferred embodiment, the data acquisition processing system is configured such that the DTU data acquisition unit is concurrently connected to the data processing unit.
In another preferred embodiment of the present invention, the DTU data acquisition unit is remotely connected to the data processing unit in a concurrent access manner, and the data processing unit can control a plurality of DTU data acquisition units to perform data acquisition at the same time, thereby greatly increasing the speed of data acquisition and preprocessing and optimizing the user experience.
In a preferred embodiment, the data acquisition and processing system, wherein the first processing module further comprises a judgment sub-module;
the judgment submodule is used for judging whether the data to be analyzed is successfully analyzed and outputting a third judgment result;
the first processing module is also connected with the request module, and outputs the successfully analyzed data as the data to be stored when the data to be analyzed is successfully analyzed according to the third judgment result; and when the data to be analyzed cannot be analyzed successfully, the control request module sends the request instruction again.
In another preferred embodiment of the present invention, the data acquisition processing system is further capable of determining whether the data analysis processing is successful, outputting a corresponding determination result for the data analysis processing result by determining whether the P L C addresses are consistent, and requesting the DTU data acquisition unit to perform feedback of data acquisition again when the data analysis is unsuccessful.
In conclusion, by the technical scheme, concurrent connection, control management, automatic online and processing condition evaluation of the data acquisition unit and the data processing unit can be realized, so that full automation of data acquisition and preprocessing processes is realized, convenience is brought to a user for data acquisition and preprocessing, and the use experience of the user is improved.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (8)
1. A data acquisition and processing system is characterized by comprising a plurality of DTU data acquisition units and a data processing unit, wherein the data processing unit is respectively and remotely connected with each DTU data acquisition unit;
each DTU data acquisition unit is connected with at least one data acquisition equipment group, and the DTU data acquisition unit acquires data through the data acquisition equipment group;
the data processing unit specifically includes:
the planning module is used for generating a batch execution plan, the batch execution plan comprises a plurality of sub-plans which are linearly arranged, and each sub-plan corresponds to one DTU data acquisition unit;
the first monitoring module is connected with the planning module and is used for continuously monitoring the execution condition of the batch execution plan;
the request module is connected with the first monitoring module and used for sending a request instruction to the DTU data acquisition unit executing the sub-plan when the sub-plan finishes execution according to the execution condition;
the first processing module is used for receiving the data to be analyzed returned by the DTU data acquisition unit, analyzing the data and outputting the data to be stored;
the first transmission module is respectively connected with the processing module and the first monitoring module and used for caching the data to be stored and storing all the data to be stored to an external database when the batch execution plan is executed;
the DTU data acquisition unit acquires data according to the batch execution plan, and specifically comprises:
the second monitoring module is used for continuously monitoring the execution condition of the sub-plan;
and the second processing module is connected with the second monitoring module, processes the acquired data and outputs the data to be processed when one sub-plan finishes execution according to the execution condition, and then sequentially executes the next sub-plan.
2. The data acquisition and processing system as in claim 1, wherein the data processing unit further comprises a first determining module;
the first judging module is used for judging whether each DTU data acquisition unit is in online connection with the data processing unit or not and outputting a corresponding first judging result.
3. The data acquisition and processing system of claim 2, wherein the DTU data acquisition unit further comprises a heartbeat module, and when the DTU data acquisition unit is powered on, the heartbeat module sends a heartbeat data packet to the first determination module every a first preset time interval;
the first judging module judges whether the DTU data acquisition unit is connected with the data processing unit on line according to the collection condition of the heartbeat data packet: and when the first judging unit does not receive the heartbeat data packet for a second preset time, judging that the DTU data acquisition unit is not connected with the data processing unit on line.
4. The data acquisition and processing system of claim 1, wherein each of said DTU data acquisition units further comprises a second determining module;
the second judgment module is used for judging whether the DTU data acquisition unit has the lower control operation or not and outputting a corresponding second judgment result.
5. The data acquisition and processing system as in claim 4, wherein the second determination module is connected to the second detection module;
and the second judging module opens a receiving window for receiving a lower control operation instruction when the execution of the sub-plan is finished according to the execution condition, and judges whether the lower control operation exists according to the receiving condition of the lower control operation instruction.
6. The data acquisition and processing system as claimed in claim 5, wherein said second determining module is further connected to said second processing module;
and the second processing module executes the next sub-plan in sequence after waiting for the execution of the lower control operation to be finished when the lower control operation exists according to the second judgment result.
7. The data acquisition processing system of claim 1, wherein the DTU data acquisition unit accesses the data processing unit concurrently.
8. The data acquisition and processing system of claim 1, wherein the first processing module further comprises a judgment sub-module;
the judgment submodule is used for judging whether the data to be analyzed is successfully analyzed and outputting a third judgment result;
the first processing module is also connected with the request module, and outputs the successfully analyzed data as the data to be stored when the data to be analyzed is successfully analyzed according to the third judgment result; and when the data to be analyzed cannot be analyzed successfully, controlling the request module to send the request instruction again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456198.XA CN111487920A (en) | 2020-05-26 | 2020-05-26 | Data acquisition and processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456198.XA CN111487920A (en) | 2020-05-26 | 2020-05-26 | Data acquisition and processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111487920A true CN111487920A (en) | 2020-08-04 |
Family
ID=71795483
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010456198.XA Pending CN111487920A (en) | 2020-05-26 | 2020-05-26 | Data acquisition and processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111487920A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287000A (en) * | 2020-10-22 | 2021-01-29 | 衡阳大河文化发展有限公司 | Exhibition platform system capable of meeting high access requirement |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101038488A (en) * | 2007-05-09 | 2007-09-19 | 重庆交通大学 | Bridge remote monitoring system |
| CN101854305A (en) * | 2010-06-07 | 2010-10-06 | 中兴通讯股份有限公司 | Flow collection system, and method and device thereof |
| CN102375837A (en) * | 2010-08-19 | 2012-03-14 | 中国移动通信集团公司 | Data acquiring system and method |
| CN103019651A (en) * | 2012-08-02 | 2013-04-03 | 青岛海信传媒网络技术有限公司 | Parallel processing method and device for complex tasks |
| CN103384206A (en) * | 2012-05-02 | 2013-11-06 | 中国科学院计算机网络信息中心 | Concurrent processing method and system for mass data |
| CN103384205A (en) * | 2012-05-02 | 2013-11-06 | 中国科学院计算机网络信息中心 | Massive alarming data parallel acquisition system, device and method |
| CN103514277A (en) * | 2013-09-24 | 2014-01-15 | 国家电网公司 | Task parallel processing method for electricity utilization information collection system |
| CN105451248A (en) * | 2014-08-29 | 2016-03-30 | 展讯通信（上海）有限公司 | Data acquisition system and data acquisition method |
| CN105553774A (en) * | 2015-12-23 | 2016-05-04 | 厦门市美亚柏科信息股份有限公司 | Internet data collection method and system |
| CN106781392A (en) * | 2016-12-29 | 2017-05-31 | 深圳齐环科技有限公司 | A kind of data acquisition and its acquisition system and acquisition method based on two-way http services |
-
2020
- 2020-05-26 CN CN202010456198.XA patent/CN111487920A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101038488A (en) * | 2007-05-09 | 2007-09-19 | 重庆交通大学 | Bridge remote monitoring system |
| CN101854305A (en) * | 2010-06-07 | 2010-10-06 | 中兴通讯股份有限公司 | Flow collection system, and method and device thereof |
| CN102375837A (en) * | 2010-08-19 | 2012-03-14 | 中国移动通信集团公司 | Data acquiring system and method |
| CN103384206A (en) * | 2012-05-02 | 2013-11-06 | 中国科学院计算机网络信息中心 | Concurrent processing method and system for mass data |
| CN103384205A (en) * | 2012-05-02 | 2013-11-06 | 中国科学院计算机网络信息中心 | Massive alarming data parallel acquisition system, device and method |
| CN103019651A (en) * | 2012-08-02 | 2013-04-03 | 青岛海信传媒网络技术有限公司 | Parallel processing method and device for complex tasks |
| CN103514277A (en) * | 2013-09-24 | 2014-01-15 | 国家电网公司 | Task parallel processing method for electricity utilization information collection system |
| CN105451248A (en) * | 2014-08-29 | 2016-03-30 | 展讯通信（上海）有限公司 | Data acquisition system and data acquisition method |
| CN105553774A (en) * | 2015-12-23 | 2016-05-04 | 厦门市美亚柏科信息股份有限公司 | Internet data collection method and system |
| CN106781392A (en) * | 2016-12-29 | 2017-05-31 | 深圳齐环科技有限公司 | A kind of data acquisition and its acquisition system and acquisition method based on two-way http services |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287000A (en) * | 2020-10-22 | 2021-01-29 | 衡阳大河文化发展有限公司 | Exhibition platform system capable of meeting high access requirement |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3575975B1 (en) | Method and apparatus for operating smart network interface card | |
| CN108351707B (en) | Man-machine interaction method and device, terminal equipment and computer readable storage medium | |
| EP3690583B1 (en) | Trace data acquisition system, trace data acquisition method, and program | |
| CN105490876A (en) | Method for automatic testing of server performance through packet sending linkage and concurrent monitoring | |
| CN103034802A (en) | Automatic routing inspection system and method based on intelligent rule matching | |
| CN113382056A (en) | Data reporting method, device, equipment, storage medium and system | |
| CN111435227A (en) | Smart home equipment testing method, device, equipment and medium | |
| CN111835799A (en) | Automatic acquisition system and method for vehicle log | |
| CN117792872A (en) | Intelligent ammeter data acquisition and analysis system based on 5G network | |
| CN111487920A (en) | Data acquisition and processing system | |
| CN116302989A (en) | Pressure testing method and system, storage medium and computer equipment | |
| CN112650765B (en) | Station fault judging method, device, equipment and storage medium | |
| CN115515026A (en) | Ammeter data transmission system, method, device, equipment and medium | |
| CN114095343A (en) | Disaster recovery method, device, equipment and storage medium based on double-active system | |
| CN107465620B (en) | General data router and communication method thereof | |
| CN111376255B (en) | Robot data acquisition method and device and terminal equipment | |
| CN111258765A (en) | Load balancing method and device, computing equipment and storage medium | |
| KR101149514B1 (en) | Apparatus for Real-time monitoring of equipment conditions automation equipments | |
| CN116955232A (en) | Hardware identification method and device based on control system | |
| CN106598599B (en) | A kind of execution method and apparatus of program | |
| CN106528121A (en) | Data collection system applied to software development platform | |
| CN107528728A (en) | A kind of method and device for obtaining user experience data | |
| CN106647628B (en) | G code executes the real-time collecting method of status data in a kind of NC Machining Process | |
| CN111427698A (en) | Azakban-based data synchronization method and device and computer equipment | |
| CN114894239B (en) | Data acquisition method and data acquisition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200804 |
|
| RJ01 | Rejection of invention patent application after publication |