CN111143455A - System and method for high-concurrency data reporting of multiple platforms in butt joint mode - Google Patents

Abstract

The invention provides a system for high-concurrency reporting of data by multiple platforms in a butt joint mode, which comprises a butt joint interface, a management layer, a butt joint platform implementation layer, a network communication service layer, a database operation layer and a basic service layer, wherein the butt joint interface and the management layer are responsible for data management, data pool maintenance and database query updating operation, the butt joint platform implementation layer is specifically implemented by the butt joint of the platforms, the network communication service layer is used for sending data, the database operation layer is responsible for database adding, deleting, modifying and searching operations, an external interface of a database module is used, and the basic service layer comprises an Ini file operation module, an xml file operation module, an information conversion module and a file name conversion module and is responsible for formatting the data. The invention has the beneficial effects that: the occupation of the memory and the frequent memory operation are reduced, and the problem that the recording resources of a single device cannot be shared is solved.

Description

System and method for high-concurrency data reporting of multiple platforms in butt joint mode

Technical Field

The invention belongs to the technical field of intelligent interconnection, and particularly relates to a system and a method for high-concurrency data reporting of multiple butted platforms.

Background

In the current market, data uploaded to the platform by the traffic host in different areas or different businesses is already part of intelligent traffic. However, a single traffic host can only upload the traffic host to a certain platform, and cannot give consideration to a plurality of platforms, so that the utilization rate of the host resources is low.

Disclosure of Invention

In view of this, the present invention aims to provide a system and a method for multi-platform high concurrent reporting of data, so as to solve the above-mentioned problems.

In order to achieve the purpose, the technical scheme of the invention is realized as follows:

a system for high-concurrency reporting data by multiple docking platforms comprises a docking interface, a management layer, a docking platform implementation layer, a network communication service layer, a database operation layer and a basic service layer,

the docking interface and the management layer are responsible for the management of data, the maintenance of a data pool and the query update operation of a database,

the docking platform implementation layer is a concrete implementation of docking platforms, each platform has a class, the class comprises a data conversion and uploading sub-class, the docking platform implementation layer processes data according to the own requirements of each platform and sends the converted data to a designated position by using a server,

the network communication service layer realizes the sending of data, the classes of each platform comprise communication server subclasses which are needed by each platform and are responsible for sending pictures, videos and data files to the appointed positions of third-party manufacturers,

the database operation layer is responsible for the operations of adding, deleting, modifying and searching the database, uses the external interface of the database module,

the basic service layer comprises an Ini file operation module, an xml file operation module, an information conversion module and a file name conversion module and is responsible for formatting data.

The invention also provides another embodiment, a method for a system for high-concurrent data reporting based on docking multiple platforms, comprising the following steps:

A. the real-time data received by the docking module is subjected to formatting transfer, data information is stored in a data pool, and the unique serial number uuid is stored in a real-time data queue; meanwhile, formatting inquiry historical data are stored in a data pool, and the unique serial number uuid is stored in a historical data queue;

B. pushing the real-time data uuid and the historical data uuid to a docking module, and preferentially processing the real-time data;

C. illegal data of a video to be intercepted is sent to a kernel, a callback message of video interception is sent to the kernel, asynchronous video interception operation is carried out, an interception result is asynchronously collected, and video information is pushed to each platform module;

D. the platform module circularly acquires the uuid from the record and video to-be-sent queue, formats, converts and sends the uuid by acquiring data information from the data pool according to the uuid, sets the uploading state corresponding to the platform module in the data pool as uploading or uploading failure after sending is finished, and then the docking module uniformly updates the database;

E. and polling the uploading state of the data in the data pool by the docking module, updating the corresponding uploading state in the host database when platform data are uploaded, performing overtime processing on the data in the data pool, and clearing the data when the platform data are uploaded for a preset overtime time.

Compared with the prior art, the method for the high-concurrency data reporting of the multiple platforms in the butt joint mode has the following advantages:

the system and the method for the multi-platform high-concurrency data reporting establish a data pool in a docking management layer, generate the UUID with the unique number, and each platform module at the lower layer acquires data contents from the data pool through the received unique number to process and upload, so that the occupation of a memory and frequent memory operation are reduced, and the problem that recording resources of a single device cannot be shared is solved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:

FIG. 1 is a schematic diagram of a multi-level access system topology;

FIG. 2 is a module hierarchy diagram;

FIG. 3 is a timing diagram of a data push third party platform.

Detailed Description

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.

The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.

As shown in fig. 1 and fig. 2, a system for high-concurrency reporting data by multiple platforms includes a docking interface and management layer, a docking platform implementation layer, a network communication service layer, a database operation layer, and a basic service layer,

the docking interface and the management layer are responsible for the management of data, the maintenance of a data pool and the query update operation of a database,

the docking platform implementation layer is a concrete implementation of docking platforms, each platform has a class, the class comprises a data conversion and uploading sub-class, the docking platform implementation layer processes data according to the self requirements of each platform, such as file name conversion, compatible conversion of data information, generation of new files and the like, and sends the converted data to a designated position by using servers such as FTP, WebService, MQ and the like,

the network communication service layer realizes the transmission of data, the class of each platform comprises the subclasses of communication servers required by each platform, such as FTP, WebService, MQ and the like, and is responsible for transmitting pictures, videos and data files to the specified position of a third party manufacturer,

the database operation layer is responsible for the operations of adding, deleting, modifying and searching the database, uses the external interface of the database module,

the basic service layer comprises an Ini file operation module, an xml file operation module, an information conversion module and a file name conversion module and is responsible for formatting data.

As shown in fig. 3, another embodiment of the present invention provides a method for a system for reporting data based on docking multiple platforms in a high-concurrency manner, including the following steps:

A. the real-time data received by the docking module is subjected to formatting transfer, data information is stored in a data pool, and the unique serial number uuid is stored in a real-time data queue; meanwhile, formatting inquiry historical data are stored in a data pool, and the unique serial number uuid is stored in a historical data queue;

B. pushing the real-time data uuid and the historical data uuid to a docking module, and preferentially processing the real-time data;

C. illegal data of a video to be intercepted is sent to a kernel, a callback message of video interception is sent to the kernel, asynchronous video interception operation is carried out, an interception result is asynchronously collected, and video information is pushed to each platform module;

D. the platform module circularly acquires the uuid from the record and video to-be-sent queue, formats, converts and sends the uuid by acquiring data information from the data pool according to the uuid, sets the uploading state corresponding to the platform module in the data pool to be the uploading state or other states such as uploading failure after the sending is finished, and then uniformly updates the database by the docking module;

E. the docking module polls the uploading state of the data in the data pool, updates the corresponding uploading state in the host database when platform data are uploaded, processes the data in the data pool overtime at the same time, and cleans the data when the platform data are uploaded for a preset overtime time, so that the occupation of data pool resources is avoided.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (2)

1. A system for docking multi-platform high-concurrency data reporting is characterized in that: comprises a docking interface and management layer, a docking platform implementation layer, a network communication service layer, a database operation layer and a basic service layer,

the docking interface and the management layer are responsible for the management of data, the maintenance of a data pool and the query update operation of a database,

the docking platform implementation layer is a concrete implementation of docking platforms, each platform has a class, the class comprises a data conversion and uploading sub-class, the docking platform implementation layer processes data according to the own requirements of each platform and sends the converted data to a designated position by using a server,

the network communication service layer realizes the sending of data, the classes of each platform comprise communication server subclasses which are needed by each platform and are responsible for sending pictures, videos and data files to the appointed positions of third-party manufacturers,

the database operation layer is responsible for the operations of adding, deleting, modifying and searching the database, uses the external interface of the database module,

the basic service layer comprises an Ini file operation module, an xml file operation module, an information conversion module and a file name conversion module and is responsible for formatting data.

2. The method for interfacing a multi-platform high-concurrency data reporting system according to claim 1, comprising the steps of:

A. the real-time data received by the docking module is subjected to formatting transfer, data information is stored in a data pool, and the unique serial number uuid is stored in a real-time data queue; meanwhile, formatting inquiry historical data are stored in a data pool, and the unique serial number uuid is stored in a historical data queue;

B. pushing the real-time data uuid and the historical data uuid to a docking module, and preferentially processing the real-time data;

C. illegal data of a video to be intercepted is sent to a kernel, a callback message of video interception is sent to the kernel, asynchronous video interception operation is carried out, an interception result is asynchronously collected, and video information is pushed to each platform module;

D. the platform module circularly acquires the uuid from the record and video to-be-sent queue, formats, converts and sends the uuid by acquiring data information from the data pool according to the uuid, sets the uploading state corresponding to the platform module in the data pool as uploading or uploading failure after sending is finished, and then the docking module uniformly updates the database;

E. and polling the uploading state of the data in the data pool by the docking module, updating the corresponding uploading state in the host database when platform data are uploaded, performing overtime processing on the data in the data pool, and clearing the data when the platform data are uploaded for a preset overtime time.

Images