CN115865886B - HTTP-based cross-network data interaction method and device - Google Patents
HTTP-based cross-network data interaction method and device Download PDFInfo
- Publication number
- CN115865886B CN115865886B CN202211498656.1A CN202211498656A CN115865886B CN 115865886 B CN115865886 B CN 115865886B CN 202211498656 A CN202211498656 A CN 202211498656A CN 115865886 B CN115865886 B CN 115865886B
- Authority
- CN
- China
- Prior art keywords
- data
- proxy
- task
- request
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000003993 interaction Effects 0.000 title claims abstract description 21
- 230000001360 synchronised effect Effects 0.000 claims abstract description 51
- 238000004806 packaging method and process Methods 0.000 claims abstract description 6
- 238000005538 encapsulation Methods 0.000 claims description 33
- 230000008569 process Effects 0.000 claims description 19
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000007405 data analysis Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a method and a device for cross-network data interaction based on HTTP, wherein the method comprises the following steps: analyzing the data request, adding a proxy mode mark for the data request and determining a data interface proxy mode; acquiring data interface declaration information of a data processing platform, and packaging the data request added with the proxy mode mark according to the data interface declaration information; according to the determined data interface proxy mode, the encapsulated data request is sent to a proxy server through a proxy client, is processed through a data processing platform, and returns a processing result; the data interface proxy mode comprises a data interface synchronous proxy mode and a data interface asynchronous proxy mode, and the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface, so that parallel processing of a plurality of remote data requests of different types is realized, and processing capacity and speed are improved.
Description
Technical Field
The invention relates to the field of database technology and information technology Internet, in particular to a method and a device for cross-network data interaction based on HTTP.
Background
The application scene of the remote data request comprises two cases, namely, the data request is initiated, and the server can immediately return the data result; and secondly, after the request is initiated, the data result is not necessarily returned immediately, and the processing time of the server is not determined.
Currently, a common way of remote data request is by initiating an HTTP request to a remote server and responding by the remote server. The HTTP request is mainly divided into a request and a response process.
In this way, the process of the client requesting data from the server by initiating the HTTP request must be continuous and uninterrupted, otherwise the HTTP request fails. Meanwhile, in the process of establishing connection with the server to wait for data return, the connection between the client and the server is in a blocking state, and other data request tasks can be processed only after the current data request task is processed, otherwise, other data request tasks can not be processed and can only be waited. This approach is only suitable for application scenarios (such as data query scenarios) where sending a data request can return processing results immediately; an application scenario (such as a data analysis scenario) in which the transmission data request is not necessarily able to return the processing result immediately is not applicable at all. Firstly, the process of requesting data from a server by initiating an HTTP request is not continuous in the scene, and a break exists in the middle, which can cause the failure of the data request based on the HTTP protocol; and secondly, the duration of processing the data request by the server cannot be determined, so that the connection between the client and the server is always in a blocking state, other data request tasks are always queued and cannot be processed, the waiting time of the remote data request is greatly prolonged, and the processing efficiency of the data request tasks is reduced. Therefore, the application scene requirement cannot be fully met simply by the existing HTTP request mode.
Based on this, a new method and system are necessary to be introduced, which can be suitable for application scenarios in which sending a data request is not necessarily capable of immediately returning a processing result, so as to solve the technical problems in the prior art that the data request processing process cannot be interrupted, which results in failure of processing a data request task, and the data request task cannot be processed in parallel, which results in low processing efficiency, thereby realizing parallel processing of multiple remote data request tasks, and hybrid processing of different types of remote data requests in different application scenarios, and improving scene adaptability, processing capacity and response speed of the HTTP-based cross-network data interaction system.
Disclosure of Invention
Aiming at the technical problems, the invention provides a cross-network data interaction method and device based on HTTP, which are based on HTTP protocol, by providing a data interface synchronous proxy mode and a data interface asynchronous proxy mode for different types of remote data requests and calling corresponding data interfaces according to the data interface proxy mode, realize parallel processing of different types of remote data requests, supervision of incomplete data request tasks and processing of completed data request tasks, solve the technical problems that the data request processing process cannot be interrupted to cause processing failure of the data request tasks and the data request tasks cannot be processed in parallel to cause low processing efficiency in the prior art, and further improve scene adaptability, processing capacity and response speed of the cross-network data interaction system based on HTTP.
The invention provides a cross-network data interaction method based on HTTP, which comprises the following steps:
s1, determining a data interface proxy mode: analyzing the data request, adding a proxy mode mark for the data request and determining a data interface proxy mode;
s2, preprocessing a data request: acquiring data interface declaration information of a data processing platform, and packaging the data request added with the proxy mode mark according to the data interface declaration information;
s3, data request processing, namely sending the packaged data request to a proxy server through a proxy client according to the determined data interface proxy mode, processing the data request through a data processing platform, and returning a processing result;
the data interface proxy mode comprises a data interface synchronous proxy mode and a data interface asynchronous proxy mode;
the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface;
the proxy mode marks comprise synchronous proxy mode marks and asynchronous proxy mode marks;
the data interface declaration information comprises a data interface path, a data interface request path and a data interface request parameter;
the data interface request path is generated by the data interface path and the proxy mode flag.
As described above, the steps of analyzing the data request, adding the proxy tag to the data request, and determining the proxy of the data interface are:
determining a request type, wherein the proxy client receives a data request and obtains the data request type according to the data request information;
adding an agent mode mark, adding the agent mode mark for the data request according to the data request type, and determining the data interface path;
determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode;
when the data request type is a data query request, the added proxy mode is marked as the synchronous proxy mode, and the data interface proxy mode is the data interface synchronous proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode mark, and the data interface proxy mode is the data interface asynchronous proxy mode.
As described above, the steps of the data interface synchronization proxy method are as follows:
the proxy client encapsulates the synchronous data request according to the synchronous proxy mode mark and the data interface declaration information;
sending a synchronous data request, wherein the proxy client sends the encapsulated synchronous data request to the proxy server and forwards the encapsulated synchronous data request to the synchronous proxy data interface for processing by the proxy server;
and returning the processing result, wherein the proxy server side acquires the processing result of the synchronous data request through the synchronous proxy data interface and returns the processing result to the proxy client side.
As described above, the steps of the data interface asynchronous proxy method are as follows:
preprocessing a data request: the agent client adds a task ID and a task processing state mark for an asynchronous data request, encapsulates the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous agent mode mark according to the asynchronous agent mode mark and the data interface declaration information, and obtains encapsulation information and an asynchronous result callback path;
the proxy client sends the encapsulation information to the proxy server and forwards the encapsulation information to the asynchronous proxy data interface for processing by the proxy server;
task supervision, wherein after the proxy client sends the encapsulation information to the proxy server, the task processing state of the asynchronous data request is supervised according to the task processing state marking timing;
returning a result and releasing the data processing resources;
wherein the task processing status flag includes complete and incomplete.
As described above, after the proxy client sends the encapsulation information to the proxy server, the step of supervising the task processing state of the asynchronous data request according to the task processing state flag timing includes:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and inquires the task processing state of the unfinished asynchronous data request;
the proxy server receives the task supervision request, returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the processing result exists in the proxy server, the returned task supervision processing result is the processing result; if the proxy server does not have the processing result, the returned task supervision processing result is task incomplete information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be completed if the task supervision processing result is the processing result.
As described above, the steps of returning the result and releasing the data processing resources are:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the task supervision processing result obtained after the proxy client sends the task supervision request to the proxy server is the processing result, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result and sends the task ID of the asynchronous data request to the proxy server;
and deleting the encapsulation information and the asynchronous result callback path according to the task ID by the proxy client and the proxy server respectively, and releasing data processing resources.
Correspondingly, the invention also provides a device for implementing the HTTP-based cross-network data interaction method, which comprises a data proxy client unit and a data proxy server unit;
the data proxy client unit is deployed on a data calling party and is used for receiving and processing a data request of the data calling party, forwarding the processed data request to the data proxy server unit and forwarding a processing result returned by the data proxy server unit to the data calling party;
the data proxy server unit is deployed on a data platform side and is used for receiving the data request forwarded by the data proxy client unit, calling a data interface provided by the data platform side to process the data request, and returning a processing result to the data proxy client unit.
As described above, the data proxy client unit includes a task forwarding module, an incomplete task supervision module, and a completed task processing module; the data proxy server unit comprises a task receiving module, a task checking module, a task state updating module and a task clearing module;
wherein,,
the task forwarding module is configured to perform encapsulation processing on data request information, where the data proxy client unit adds a task ID and a task processing state flag for an asynchronous data request, encapsulates the task ID, the task processing state flag, the data interface request path, the data interface request parameter and the proxy mode flag according to the asynchronous proxy mode flag and the data interface declaration information, obtains encapsulation information and an asynchronous result callback path, and sends the encapsulation information to the data proxy server unit;
the incomplete task supervision module is used for storing the submitted data request task in a local cache after sending the encapsulation information to the data proxy server unit, checking the local incomplete task according to a task processing state mark at fixed time, acquiring the incomplete data request task processing state from the data proxy server unit, returning the result of the data request task if the data request task has a processing result in the data proxy server unit, and storing the result in the local cache so as to be used for the completed task processing module to continuously process the processing result;
the completed task processing module is used for checking whether the result of the incomplete task is updated in the local cache at regular time, if the result exists, returning the processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit, and releasing task processing resources of the data request according to the task ID;
the receiving task module is used for receiving and storing the encapsulation information and the task ID forwarded by the data proxy client unit and forwarding a data request to a data interface of a data platform;
the checking task module is used for inquiring the task processing state of a data request task when receiving the task supervision request sent by the data proxy client unit; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired that the task has a processing result, acquiring and returning the processing result;
the update task state module is used for sending the processing result to the data proxy server unit through an update result interface of the data proxy server unit after the data interface processes the data request, and updating the processing result to a local cache for inquiry;
and the task clearing module is used for receiving a task processing completion notification sent by the data proxy client unit and releasing data processing resources according to the task ID of the completed task.
By applying the technical scheme, the invention realizes the parallel processing of different types of remote data requests, the supervision of incomplete data request tasks and the processing of completed data request tasks based on the HTTP protocol, and the technical problems of failure processing of the data request tasks and low processing efficiency caused by incapability of interrupting the data request processing process and the parallel processing of the data request tasks in the prior art are solved by the synchronous proxy mode of the data interface and the asynchronous proxy mode of the data interface provided for different types of remote data requests and the corresponding data interfaces, and the scene adaptability, processing capacity and response speed of the HTTP-based cross-network data interaction system are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow diagram of a method for interaction of data across networks based on HTTP according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of adding proxy mode marks to data requests and determining a proxy mode of a data interface in a cross-network data interaction method based on HTTP according to an embodiment of the invention;
fig. 3 is a schematic flow diagram of a data interface synchronization proxy mode of a cross-network data interaction method based on HTTP according to an embodiment of the present invention;
fig. 4 is a schematic flow diagram of a data interface asynchronous proxy mode of a cross-network data interaction method based on HTTP according to an embodiment of the present invention;
fig. 5 shows a schematic structural diagram of an HTTP-based cross-network data interaction device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention provides a cross-network data interaction method based on HTTP, as shown in figure 1, comprising the following steps:
s101, determining a data interface proxy mode: analyzing the data request, adding proxy mode marks for the data request and determining the proxy mode of the data interface.
In this embodiment, the data interface proxy mode includes a data interface synchronous proxy mode and a data interface asynchronous proxy mode, and the proxy mode flag includes a synchronous proxy mode flag and an asynchronous proxy mode flag.
S102, preprocessing a data request: and acquiring data interface declaration information of a data processing platform, and packaging the data request added with the proxy mode mark according to the data interface declaration information.
In this embodiment, the data interface includes a synchronous proxy data interface and an asynchronous proxy data interface, and the data interface declaration information includes a data interface path, a data interface request parameter, and a proxy mode flag; the data interface request path is generated by the data interface path and the proxy mode flag.
For example, the synchronous proxy mode is marked as "/sync/", the asynchronous proxy mode is marked as "/async/", the data interface path of the synchronous proxy data interface provided by the data platform side is "/abc", the data interface path of the asynchronous proxy data interface is "/def", the data interface request path calling the synchronous proxy data interface through the synchronous proxy mode is "/sync/abc", and the data interface request path calling the asynchronous proxy data interface through the asynchronous proxy mode is "/async/def".
S103, data request processing, namely sending the packaged data request to a proxy server through a proxy client according to the determined data interface proxy mode, processing through a data processing platform, and returning a processing result.
As shown in fig. 2, the steps of analyzing a data request, adding a proxy flag to the data request, and determining a proxy of a data interface are as follows:
s201, determining a request type, wherein the proxy client receives a data request and obtains the data interface request parameters and the data request type according to data request information.
S202, adding a proxy mode mark, adding the proxy mode mark for the data request according to the data request type, and determining the data interface path.
S203, determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode.
In this embodiment, when the data request type is a data query request, the added proxy mode is marked as the synchronization proxy mode, and the data interface proxy mode is the data interface synchronization proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode mark, and the data interface proxy mode is the data interface asynchronous proxy mode.
For example, the synchronous proxy mode is marked as "/sync/", the asynchronous proxy mode is marked as "/async/", the data interface path of the synchronous proxy data interface provided by the data platform side is "/abc", and the data interface path of the asynchronous proxy data interface is "/def";
when the data request type is a data query request, the added proxy mode is marked as "/sync/", and the proxy client and the proxy server can judge that the proxy mode of the data request is a synchronous proxy mode through "/sync/", and the data interface request path of the called synchronous proxy data interface is "/sync/abc";
when the data request type is a data analysis request, the added proxy mode is marked as "/async/", and the proxy client and the proxy server can judge that the proxy mode of the data request is an asynchronous proxy mode through "/async/", and the data interface request path of the called asynchronous proxy data interface is "/async/def".
As shown in fig. 3, the steps of the data interface synchronization proxy method are as follows:
s301, packaging the data request, wherein the proxy client packages the synchronous data request according to the synchronous proxy mode mark and the data interface declaration information.
S302, sending a synchronous data request, and sending the encapsulated synchronous data request to the proxy server by the proxy client, and forwarding the encapsulated synchronous data request to the synchronous proxy data interface by the proxy server for processing.
S303, returning the processing result, and returning the processing result of the synchronous data request to the proxy client after the proxy server acquires the processing result of the synchronous data request through the synchronous proxy data interface.
As shown in fig. 4, the steps of the data interface asynchronous proxy method are as follows:
s401, preprocessing a data request: the agent client adds a task ID and a task processing state mark for an asynchronous data request, encapsulates the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous agent mode mark according to the asynchronous agent mode mark and the data interface declaration information, and obtains encapsulation information and an asynchronous result callback path;
wherein the task processing status flag includes complete and incomplete.
And S402, sending a request, wherein the proxy client sends the encapsulation information to the proxy server and forwards the encapsulation information to the asynchronous proxy data interface for processing by the proxy server.
S403, task supervision is performed, wherein after the proxy client sends the encapsulation information to the proxy server, task processing states of the asynchronous data requests are supervised according to the task processing state marking timing.
Task supervision for efficient task completion of outstanding data requestsIn some embodiments of the present invention, after the proxy client sends the encapsulation information to the proxy server, the step of supervising the task processing state of the asynchronous data request according to the task processing state flag timing specifically includes:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and inquires the task processing state of the unfinished asynchronous data request;
the proxy server receives the task supervision request, returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the processing result exists in the proxy server, the returned task supervision processing result is the processing result; if the proxy server does not have the processing result, the returned task supervision processing result is task incomplete information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be completed if the task supervision processing result is the processing result.
S404, returning the result and releasing the data processing resource.
To accurately return results and effectively utilize processing resourcesIn some embodiments of the present invention, the step of returning the result and freeing the data processing resources comprisesThe body is as follows:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the task supervision processing result obtained after the proxy client sends the task supervision request to the proxy server is the processing result, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result and sends the task ID of the asynchronous data request to the proxy server;
and deleting the encapsulation information and the asynchronous result callback path according to the task ID by the proxy client and the proxy server respectively, and releasing data processing resources.
By applying the technical scheme, analyzing the data request, adding a proxy mode mark for the data request and determining a data interface proxy mode; acquiring data interface declaration information of a data processing platform, and packaging the data request added with the proxy mode mark according to the data interface declaration information; according to the determined data interface proxy mode, the encapsulated data request is sent to the proxy server through the proxy client, the data processing platform is used for processing, and processing results are returned, so that the technical problems that based on the HTTP protocol, different data interface synchronous proxy modes, data interface asynchronous proxy modes and data interfaces are provided for different types of remote data requests, parallel processing of different types of remote data requests, supervision of incomplete data request tasks and processing of completed data request tasks are realized, the incomplete data request tasks do not influence the processing of other data request tasks, interruption of the data request process does not cause data request processing failure, the processing failure of the data request tasks caused by incapacity of interruption of the data request processing process in the prior art, processing efficiency is low caused by incapacity of parallel processing of the data request tasks are solved, and scene adaptability, processing capacity and response speed of an HTTP-based cross-network data interaction system are improved.
Corresponding to one of the HTTP-based cross-network data interaction methods in the embodiments of the present invention, the present invention also discloses an HTTP-based cross-network data interaction device, as shown in fig. 5, where the device includes a data proxy client unit 501 and a data proxy server unit 502;
the data proxy client unit 501 is disposed on a data caller, and is configured to receive and process a data request of the data caller, forward the processed data request to the data proxy server unit, and forward a processing result returned by the data proxy server unit to the data caller.
The data proxy server unit 502 is disposed on the data platform side, and is configured to receive the data request forwarded by the data proxy client unit, call a data interface provided by the data platform side to process the data request, and return a processing result to the data proxy client unit.
In this embodiment, the data proxy client unit 501 includes a task forwarding module 501-1, an incomplete task supervision module 501-2, and a completed task processing module 501-3; the data proxy server unit includes a receiving task module 502-1, an inspection task module 502-2, an update task status module 502-3, and a clear completed task module 502-4;
the task forwarding module 501-1 is configured to perform encapsulation processing on data request information, where the data proxy client unit adds a task ID and a task processing state flag for an asynchronous data request, encapsulates the task ID, the task processing state flag, the data interface request path, the data interface request parameter, and the proxy mode flag according to the asynchronous proxy mode flag and the data interface declaration information, obtains encapsulation information and an asynchronous result callback path, and sends the encapsulation information to the data proxy server unit;
the incomplete task supervision module 501-2 is configured to send the encapsulation information to the data proxy server unit, store the submitted data request task in a local cache, check the local incomplete task at regular time according to a task processing status flag, and obtain the incomplete data request task processing status from the data proxy server unit, if the data request task already has a processing result in the data proxy server unit, return the result of the data request task, and store the result in the local cache, so that the completed task processing module continues to process the processing result;
the completed task processing module 501-3 is configured to periodically check whether a result of an incomplete task is updated in a local cache, and if the result is present, return a processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit, and release task processing resources of the data request according to the task ID;
the receiving task module 502-1 is configured to receive and store the encapsulation information and the task ID forwarded by the data proxy client unit, and forward a data request to a data interface of a data platform;
the checking task module 502-2 is configured to query a task processing state of a data request task when the task supervision request sent by the data proxy client unit is received; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired that the task has a processing result, acquiring and returning the processing result;
the update task status module 502-3 is configured to send the processing result to the data proxy server unit through an update result interface of the data proxy server unit after the data interface processes the data request, and update the processing result to a local cache for query;
the clear completed task module 502-4 is configured to receive a task processing completion notification sent by the data proxy client unit, and release data processing resources according to the task ID of the completed task.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (8)
1. A method of HTTP-based cross-network data interaction, the method comprising:
s1, determining a data interface proxy mode: analyzing the data request, adding a proxy mode mark for the data request and determining a data interface proxy mode;
s2, preprocessing a data request: acquiring data interface declaration information of a data processing platform, and packaging the data request added with the proxy mode mark according to the data interface declaration information;
s3, data request processing, namely sending the packaged data request to a proxy server through a proxy client according to the determined data interface proxy mode, processing the data request through a data processing platform, and returning a processing result;
the data interface proxy mode is characterized by comprising a data interface synchronous proxy mode and a data interface asynchronous proxy mode;
the data interface comprises a synchronous proxy data interface and an asynchronous proxy data interface;
the proxy mode marks comprise synchronous proxy mode marks and asynchronous proxy mode marks;
the data interface declaration information comprises a data interface path, a data interface request path and a data interface request parameter;
the data interface request path is generated by the data interface path and the proxy mode mark.
2. The method of claim 1, wherein the steps of analyzing the data request, adding a proxy tag to the data request, and determining a data interface proxy are:
determining a request type, wherein the proxy client receives a data request and obtains the data request type according to the data request information;
adding an agent mode mark, adding the agent mode mark for the data request according to the data request type, and determining the data interface path;
determining a proxy mode, generating the data interface request path according to the proxy mode mark and the data interface path, and determining the data interface proxy mode;
when the data request type is a data query request, the added proxy mode is marked as the synchronous proxy mode, and the data interface proxy mode is the data interface synchronous proxy mode; when the data request type is a data analysis request, the added proxy mode is marked as the asynchronous proxy mode mark, and the data interface proxy mode is the data interface asynchronous proxy mode.
3. The method of claim 2, wherein the step of synchronizing the proxy mode for the data interface comprises:
the proxy client encapsulates the synchronous data request according to the synchronous proxy mode mark and the data interface declaration information;
sending a synchronous data request, wherein the proxy client sends the encapsulated synchronous data request to the proxy server and forwards the encapsulated synchronous data request to the synchronous proxy data interface for processing by the proxy server;
and returning the processing result, wherein the proxy server side acquires the processing result of the synchronous data request through the synchronous proxy data interface and returns the processing result to the proxy client side.
4. The method of claim 2, wherein the step of the data interface asynchronous proxy mode is:
preprocessing a data request: the agent client adds a task ID and a task processing state mark for an asynchronous data request, encapsulates the task ID, the task processing state mark, the data interface request path, the data interface request parameter and the asynchronous agent mode mark according to the asynchronous agent mode mark and the data interface declaration information, and obtains encapsulation information and an asynchronous result callback path;
the proxy client sends the encapsulation information to the proxy server and forwards the encapsulation information to the asynchronous proxy data interface for processing by the proxy server;
task supervision, wherein after the proxy client sends the encapsulation information to the proxy server, the task processing state of the asynchronous data request is supervised according to the task processing state marking timing;
returning a result and releasing the data processing resources;
wherein the task processing status flag includes complete and incomplete.
5. The method of claim 4, wherein the step of supervising the task processing state of the asynchronous data request according to the task processing state flag timing after the proxy client transmits the encapsulation information to the proxy server comprises:
according to the task processing state mark, the proxy client side sends a task supervision request to the proxy server side at regular time, and inquires the task processing state of the unfinished asynchronous data request;
the proxy server receives the task supervision request, returns a task supervision processing result to the proxy client according to the processing result of the asynchronous data request, and if the processing result exists in the proxy server, the returned task supervision processing result is the processing result; if the proxy server does not have the processing result, the returned task supervision processing result is task incomplete information;
and the proxy client receives the task supervision processing result, updates the task processing state mark according to the task supervision processing result, and updates the task processing state mark to be completed if the task supervision processing result is the processing result.
6. The method of claim 4, wherein the steps of returning the result and freeing the data processing resources are:
the proxy server side obtains the processing result through the asynchronous proxy data interface;
when the obtained task supervision processing result is the processing result after the proxy client sends a task supervision request to the proxy server, the proxy server returns the processing result to the proxy client through the asynchronous result callback path;
the proxy client receives and caches the processing result and sends the task ID of the asynchronous data request to the proxy server;
and deleting the encapsulation information and the asynchronous result callback path according to the task ID by the proxy client and the proxy server respectively, and releasing data processing resources.
7. An apparatus for implementing the HTTP-based cross-network data interaction method of claim 1, comprising a data proxy client unit and a data proxy server unit;
the data proxy client unit is deployed on a data calling party and is used for receiving and processing a data request of the data calling party, forwarding the processed data request to the data proxy server unit and forwarding a processing result returned by the data proxy server unit to the data calling party;
the data proxy server unit is deployed on a data platform side and is used for receiving the data request forwarded by the data proxy client unit, calling a data interface provided by the data platform side to process the data request, and returning a processing result to the data proxy client unit.
8. The apparatus of claim 7, wherein the data proxy client unit comprises a task forwarding module, an incomplete task supervision module, a completed task processing module; the data proxy server unit comprises a task receiving module, a task checking module, a task state updating module and a task clearing module;
wherein,,
the task forwarding module is configured to perform encapsulation processing on data request information, where the data proxy client unit adds a task ID and a task processing state flag for an asynchronous data request, encapsulates the task ID, the task processing state flag, the data interface request path, the data interface request parameter and the asynchronous proxy mode flag according to the asynchronous proxy mode flag and the data interface declaration information, obtains encapsulation information and an asynchronous result callback path, and sends the encapsulation information to the data proxy server unit;
the incomplete task supervision module is used for storing the submitted data request task in a local cache after sending the encapsulation information to the data proxy server unit, checking the local incomplete task according to a task processing state mark at fixed time, acquiring the incomplete data request task processing state from the data proxy server unit, returning the result of the data request task if the data request task has a processing result in the data proxy server unit, and storing the result in the local cache so as to be used for the completed task processing module to continuously process the processing result;
the completed task processing module is used for checking whether the result of the incomplete task is updated in the local cache at regular time, if the result exists, returning the processing result to the data proxy client unit through a processing result callback interface provided by the data proxy client unit, and releasing task processing resources of the data request according to the task ID;
the receiving task module is used for receiving and storing the encapsulation information and the task ID forwarded by the data proxy client unit and forwarding a data request to a data interface of a data platform;
the checking task module is used for inquiring the task processing state of the data request task when receiving the task supervision request sent by the data proxy client unit; if the task processing state is marked as incomplete, returning incomplete information, and if the data request task is inquired that the task has a processing result, acquiring and returning the processing result;
the update task state module is used for sending the processing result to the data proxy server unit through an update result interface of the data proxy server unit after the data interface processes the data request, and updating the processing result to a local cache for inquiry;
and the task clearing module is used for receiving a task processing completion notification sent by the data proxy client unit and releasing data processing resources according to the task ID of the completed task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211498656.1A CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211498656.1A CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115865886A CN115865886A (en) | 2023-03-28 |
| CN115865886B true CN115865886B (en) | 2023-07-18 |
Family
ID=85666988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211498656.1A Active CN115865886B (en) | 2022-11-28 | 2022-11-28 | HTTP-based cross-network data interaction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115865886B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431316B (en) * | 2023-06-06 | 2023-11-14 | 阿里巴巴（中国）有限公司 | Task processing method, system, platform and automatic question-answering method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722663A (en) * | 2004-07-13 | 2006-01-18 | 联想(北京)有限公司 | A proxy server system and method for realizing proxy communication thereof |
| WO2016192556A1 (en) * | 2015-05-29 | 2016-12-08 | 北京金山安全软件有限公司 | Interface invoking method, device and terminal |
| CN107517254A (en) * | 2017-08-22 | 2017-12-26 | 北京梅泰诺通信技术股份有限公司 | A kind of dynamic data demand processing system and method |
| CN111490997A (en) * | 2019-01-25 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Task processing method, agent system, service system and electronic equipment |
| CN113835911A (en) * | 2021-11-23 | 2021-12-24 | 深圳市明源云科技有限公司 | Intranet penetration agent method, system, host and computer readable storage medium |
-
2022
- 2022-11-28 CN CN202211498656.1A patent/CN115865886B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722663A (en) * | 2004-07-13 | 2006-01-18 | 联想(北京)有限公司 | A proxy server system and method for realizing proxy communication thereof |
| WO2016192556A1 (en) * | 2015-05-29 | 2016-12-08 | 北京金山安全软件有限公司 | Interface invoking method, device and terminal |
| CN107517254A (en) * | 2017-08-22 | 2017-12-26 | 北京梅泰诺通信技术股份有限公司 | A kind of dynamic data demand processing system and method |
| CN111490997A (en) * | 2019-01-25 | 2020-08-04 | 北京京东尚科信息技术有限公司 | Task processing method, agent system, service system and electronic equipment |
| CN113835911A (en) * | 2021-11-23 | 2021-12-24 | 深圳市明源云科技有限公司 | Intranet penetration agent method, system, host and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115865886A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3993347A1 (en) | Method and device for application migration | |
| CN104717186B (en) | A kind of method, apparatus and data transmission system for transmitting data in network system | |
| EP2437456A1 (en) | Method and system for realizing concurrent access of multi-kinds of bearer protocols on machine-to-machine (m2m) platform | |
| EP2079024A1 (en) | Proxy server, communication system, communication method, and program | |
| US8903972B2 (en) | Method and apparatus for sharing contents using information of group change in content oriented network environment | |
| CN111343237A (en) | Server cluster communication method, communication device and computer storage medium | |
| EP2939401B1 (en) | Method for guaranteeing service continuity in a telecommunication network and system thereof | |
| CN114500633B (en) | Data forwarding method, related device, program product and data transmission system | |
| US20150373135A1 (en) | Wide area network optimization | |
| CN116048538B (en) | Service grid deployment method and device for DPU | |
| US20150127837A1 (en) | Relay apparatus and data transfer method | |
| CN115865886B (en) | HTTP-based cross-network data interaction method and device | |
| CN110875887A (en) | MQTT protocol-based communication interaction method and communication interaction system | |
| WO2023221452A1 (en) | Packet processing system and method, device, and storage medium | |
| CN107529229B (en) | Data transmission method, device and system | |
| US9350606B2 (en) | System and method for assigning server to terminal and efficiently delivering messages to the terminal | |
| CN111970092B (en) | Multi-protocol redundant network asynchronous communication method supporting reliability adjustment | |
| CN113986501A (en) | Real-time database API (application program interface) uninterrupted calling method, system, storage medium and server | |
| CN107707546B (en) | Message copying method and novel network adapter supporting message copying function | |
| CN110582090A (en) | CDN scheduling method, scheduling system and MEC network element | |
| CN109120578B (en) | Method and device for realizing link connection processing | |
| CN105306442A (en) | Proxy server and command transmission methods | |
| EP3422674A1 (en) | A method of resolving a domain name by a dns server to a plurality of ip addresses based on location information of the user equipment | |
| US11457050B1 (en) | Ephemeral data stream service | |
| CN115396510A (en) | Communication method and communication module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |