CN117014493A - Data processing method, apparatus, device, storage medium, and program product - Google Patents
Data processing method, apparatus, device, storage medium, and program product Download PDFInfo
- Publication number
- CN117014493A CN117014493A CN202210452452.8A CN202210452452A CN117014493A CN 117014493 A CN117014493 A CN 117014493A CN 202210452452 A CN202210452452 A CN 202210452452A CN 117014493 A CN117014493 A CN 117014493A
- Authority
- CN
- China
- Prior art keywords
- edge node
- cloud server
- service request
- service
- node equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 230000004044 response Effects 0.000 claims abstract description 93
- 238000004891 communication Methods 0.000 claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000004590 computer program Methods 0.000 claims description 20
- 238000011084 recovery Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000002955 isolation Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 239000002071 nanotube Substances 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- BKCJZNIZRWYHBN-UHFFFAOYSA-N Isophosphamide mustard Chemical compound ClCCNP(=O)(O)NCCCl BKCJZNIZRWYHBN-UHFFFAOYSA-N 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/2895—Intermediate processing functionally located close to the data provider application, e.g. reverse proxies
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data processing method, a data processing device, a data processing apparatus, a storage medium and a program product. The method comprises the following steps: receiving a first service request sent by a user terminal; under the condition that the first edge node equipment and the cloud server are in a broken chain, the service request processed by the cloud server is processed by the first edge node equipment through cache data, under the condition that the first edge node equipment and the cloud server are in communication, the service request is forwarded to the cloud server for processing, response data returned by the cloud server are received and cached in a database, therefore, edge autonomy can be realized under the condition that the first edge node equipment and the cloud server at the edge side are broken chain, and the stability of service of the edge node equipment is maintained.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a data processing method, apparatus, device, storage medium, and program product.
Background
In the cloud-edge cooperative edge computing scene, an edge node is in communication connection with a cloud through a public network. However, in the application scenario of computing the edge of the internet of things, because the network environment between the edge side and the cloud side is very complex, the link between the edge node of the edge side and the cloud side is not reliable, and the communication link between the cloud side and the edge node is disconnected, which will cause that the service on the edge node cannot operate normally.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, equipment, a storage medium and a program product, which can realize autonomy of edge node equipment under the condition that the edge node equipment is disconnected from a cloud server.
In a first aspect, an embodiment of the present application provides a data processing method, including:
receiving a first service request sent by a user terminal;
searching first response data corresponding to the first service request in a cache database of the first edge node equipment under the condition that the first edge node equipment and the cloud server are in a broken link state, and sending the first response data to the user terminal;
and under the condition that the first edge node equipment is in a communication state with a cloud server, sending the first service request to the cloud server, sending response data received from the cloud server to the user terminal, and storing the response data in the cache database, wherein the response data is data which is determined by the cloud server and corresponds to the first service request.
In a second aspect, an embodiment of the present application further provides a data processing method, where the method includes: receiving a first service request sent by the edge node equipment under the condition that the first edge node equipment is in a communication state with the cloud server;
Determining second response data corresponding to the first service request in response to the first service request;
transmitting the second response data to the first edge node device,
the first edge node device is used for sending the received second response data to the user terminal and storing the second response data in a cache database; the cache database is configured to provide response data corresponding to a third service request when a link failure state is between the first edge node device and the cloud server and the third service request is received, where the third service request includes the first service request.
In a third aspect, an embodiment of the present application provides a data processing apparatus, applied to a first edge node device, where the apparatus includes:
the first receiving module is used for receiving a first service request sent by the user terminal;
the processing module is used for searching first response data corresponding to the first service request in a cache database of the first edge node equipment under the condition that the first edge node equipment and the cloud server are in a broken link state, and sending the first response data to the user terminal;
The first sending module is configured to send the first service request to the cloud server when the first edge node device is in a connected state with the cloud server, and send response data received from the cloud server to the user terminal and store the response data in the cache database, where the response data is data determined by the cloud server and corresponding to the first service request.
In a fourth aspect, an embodiment of the present application provides another data processing apparatus, applied to a cloud service parent, where the apparatus includes:
the second receiving module is used for receiving a first service request sent by the edge node equipment under the condition that the first edge node equipment is in a communication state with the cloud server;
the determining module is used for responding to the first service request and determining second response data corresponding to the first service request;
a second transmitting module, configured to transmit the second response data to the first edge node device,
the first edge node device is used for sending the received second response data to the user terminal and storing the second response data in a cache database; the cache database is configured to provide response data corresponding to the third service request when the first edge node device is in a broken link state with the cloud server and the third service request is received, where the third service request includes the first service request.
In a fifth aspect, an embodiment of the present application provides a data processing apparatus, including: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the data processing method as described above.
In a sixth aspect, embodiments of the present application provide a computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement a data processing method as above.
In a seventh aspect, embodiments of the present application provide a computer program product comprising computer program instructions which, when executed by a processor, implement a data processing method as above.
The data processing method provided by the application receives a first service request sent by a user terminal; under the condition that the first edge node equipment and the cloud server are in a broken chain, the service request processed by the cloud server is processed by the first edge node equipment through cache data, under the condition that the first edge node equipment and the cloud server are in communication, the service request is forwarded to the cloud server for processing, response data returned by the cloud server are received and cached in a database, therefore, edge autonomy can be realized under the condition that the first edge node equipment and the cloud server at the edge side are broken chain, and the stability of service of the edge node equipment is maintained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present application;
FIG. 2 is a flow chart of a data processing method according to another embodiment of the present application;
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a data processing apparatus according to another embodiment of the present application;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application will be described in detail below, and in order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail below with reference to the accompanying drawings and the detailed embodiments. It should be understood that the particular embodiments described herein are meant to be illustrative of the application only and not limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the application by showing examples of the application.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The embodiments will be described in detail below with reference to the accompanying drawings.
Kubernetes is a brand new distributed architecture leading scheme based on container technology, provides a series of functions of deployment operation, resource scheduling, service discovery, dynamic expansion and contraction and the like for containerized application on the basis of Docker technology, and improves convenience and high availability of large-scale container cluster management. The Kubernetes design requires reliance on a good network, so its deployment of clusters is typically confined to a single data center. For Kubernetes clusters across data centers, a multi-cluster management mode is generally adopted, namely, mutually independent Kubernetes clusters are respectively built in respective data centers, then cluster nanotubes are completed through a multi-cluster management tool, and cluster service management is carried out, for example: kubeFed (cluster federation), karmada, etc. are all relatively similar multi-cluster solutions.
However, the applicant found that the existing multi-cluster management scheme has the following problems: 1. each data center needs to build a complete Kubernetes cluster, and under the condition that the cluster is high-availability, management plane resources of 5 nodes (servers) occupy (3 master nodes+two load balancing nodes). 2. The lack of flexible resource management forms, each data center is an independent cluster, mutual interference among the data centers is avoided, and cross-center service scheduling and management cannot be realized. 3. Each cluster is maintained separately, and in order to realize cooperation among the clusters, a set of multi-cluster management platform is constructed above the Kubernetes cluster, for example: kubeFed (cluster federation), karmada, etc., introduces more cluster management level burden.
Therefore, aiming at the data centers distributed in a star shape, a large number of servers exist in the core data center, and a small number of servers exist in other data centers, the application provides a data processing method, which can realize that the data centers of the small number of servers are directly connected into a cluster in the core data center in a node form, solve the problem that the Kubernetes single cluster depends on a good network, realize the unified nano-tube of the core data center server and the other data center servers, realize the three cloud edge cooperative capacity of the core data center for the management cooperation, the data cooperation and the operation and maintenance cooperation of the other data centers, and promote the scale range of the Kubernetes Shan Jiqun nano-tube on the premise of ensuring that the cluster can stably provide service.
In order to solve the problems in the prior art, embodiments of the present application provide a data processing method, apparatus, device, storage medium, and program product. The following first describes a data processing method provided by an embodiment of the present application.
Fig. 1 is a flow chart of a data processing method according to an embodiment of the present application. The method is applied to an edge server (first edge server), and comprises the following steps S101 to S103:
s101, receiving a first service request sent by a user terminal.
S102, under the condition that the first edge node equipment and the cloud server are in a broken link state, first response data corresponding to the first service request are searched in a cache database of the first edge node equipment, and the first response data are sent to the user terminal.
The first service request is a service request which is initiated by the user terminal and aims at data in the cloud server.
Specifically, after receiving a service request of a user terminal, the communication connection state of a current first edge node and a cloud server needs to be judged first, and in the case that a link failure state exists between first edge node equipment and the cloud server, it is indicated that the first service request cannot be forwarded to the cloud server for processing, in order to ensure timely processing of the first service request, response data corresponding to the first service can be found in a cache database of the first edge node equipment, then the first service request is processed according to the response data corresponding to the first service, and the first response data is sent to the user terminal.
S103, under the condition that the first edge node equipment is in a communication state with the cloud server, a first service request is sent to the cloud server, response data fed back by the cloud server is sent to the user terminal, and the response data is stored in a cache database.
The response data are data which are determined by the cloud server and correspond to the first service request.
Specifically, under the condition that the first edge node equipment and the cloud server are in a communication state, the first service request can be forwarded to the cloud server through the first edge node equipment, second response data corresponding to the first service request is returned after the first service request is processed by the cloud server, the first edge node equipment receives the second response data returned by the cloud server, the second response data is cached in a local database, and therefore under the condition that a link between the first edge node equipment and the cloud server is broken, the service request is normally processed through data in the cache database, and edge autonomy is achieved.
In some embodiments, steps S101 to S103 may be performed by a CNI (Container Network Interface ) plug-in.
In some embodiments, steps S101 to S103 may also be based on HTTP reverse proxy technology for critical data caching, with caching providing Kubernetes critical services offline autonomous capability. But may be implemented in other ways, and the embodiments of the present application are not limited thereto.
In this embodiment, a data processing method is provided, and a first service request sent by a user terminal is received; under the condition that the first edge node equipment and the cloud server are in a broken chain, the service request processed by the cloud server is processed by the first edge node equipment through cache data, under the condition that the first edge node equipment and the cloud server are in communication, the service request is forwarded to the cloud server for processing, response data returned by the cloud server are received and cached in a database, therefore, edge autonomy can be realized under the condition that the first edge node equipment and the cloud server at the edge side are broken chain, and the stability of service of the edge node equipment is maintained.
In some embodiments, after S101, that is, after receiving the first service request sent by the user terminal, the method may further include:
under the condition that the first edge node equipment is in a communication state with the cloud server, the first edge node equipment forwards a first service request to the cloud server through a reverse proxy component;
receiving second response data of the first service request returned by the cloud server through the reverse proxy component;
And caching the second response data to the service processing component of the first edge node device.
Specifically, the first edge node device may receive the first service request sent by the user terminal by using the reverse proxy component, process the first service request to the edge server under the condition that the first edge node device is in a connected state with the cloud server, send the second response data to the user terminal when receiving the second response data corresponding to the first service by using the reverse proxy component, and cache the second response data to the local cache database, so that the first edge node device can process the service processing component of the first service request to the first edge node device under the condition that the first edge node device and the cloud server are in a broken link state, and ensure that the service in the first edge node device can be normally processed under the condition that the first edge node device and the cloud server are broken link, thereby realizing edge autonomy.
In one example, in order to achieve the autonomous capability of the edge service, and ensure that the service instance does not fluctuate due to network influence, especially in the case that the edge node is disconnected from the cloud Server, the node is down and restarted, the service instance can still be normally pulled up, and can be achieved through a remote service (API Server) in the cloud Server, and a proxy service (EdgeProxy) and a client of the edge node device, the specific steps are as follows:
1) API Server requests by clients to access Kubernetes pass through edge proxy edge nodes
2) The EdgeProxy judges whether the network of the remote API Server is reachable, and can go to the flow 3, and can not jump to the flow 6.
3) The EdgeProxy forwards the request to the real API Server (i.e., forwards the first service request to the cloud Server through the reverse proxy component).
4) The real API Server responds to the client request and returns response data to the edge proxy. The edge proxy intercepts data in a request response body (namely, receives second response data of a first service request returned by the cloud server through the reverse proxy component), copies the data stream into two parts, stores a local disk after decoding one part of the data stream, and responds to the client in the other part, and the process is finished.
5) When the network is not reachable, the EdgeProxy forwards the request to the local process.
6) The Edgeproxy reads the local cache data, constructs a response body and responds to the client.
In some embodiments, the first edge node device may cache the response data to the first edge node device when receiving the second response data of the first service request returned by the cloud server through the reverse proxy component.
In one example, the edge node autonomy may be implemented when the first edge node device is disconnected from the cloud server by a network snapshot function, and the snapshot function may be added to the cni plug-in for IPAM, which may be used to allocate ip to the container using data recorded in the snapshot after failure of specifying the inspection condition, and call the actual IPAM for processing when the inspection is successful. And the network snapshot function is added for cloud edge scenes, and the offline autonomous capability of network IP distribution is provided.
The method comprises the following specific steps:
1) Through a configuration mode, the call of the ipam is intercepted, and the snapshot-ipam with snapshot logic is entrusted to carry out the call of the ipam.
2) When the snapshot-ipam calls an interception ADD method, checking, calling the actual ipam to carry out ip allocation if the check passes, and then recording the result to a snapshot path with the file name of: { pod name }, the content is the actual ipam return content, if the actual ipam is abnormal, the exception is returned directly. If the record is not locally recorded, checking failure condition at the same time, directly returning to abnormality
3) When intercepting DEL method call, the snapshot-ipam checks, and when checking, the actual ipam is called to carry out ip recovery, and then the corresponding file under the snapshot Path is deleted
In order to avoid that the cloud server receives a large amount of traffic impact in a short time when the plurality of edge node devices in each area are restored to the connected state by the broken link, in some embodiments, after S101, that is, after receiving the first service request sent by the user terminal, the method may further include:
under the condition that the first edge node equipment and the cloud server are recovered to a communication state from a broken link state and a second service request is received, sending a service recovery request to the cloud server;
Receiving communication permission information returned by the cloud server, wherein the communication permission information is used for indicating service permission states of a plurality of edge node devices in a target area and the cloud server, the plurality of edge node devices comprise first edge node devices, and the service permission states are used for indicating whether the cloud server is allowed to receive service requests sent by the plurality of edge node devices;
and sending a second service request to the cloud server under the condition that the service permission state indicates that the cloud server is permitted to receive the service requests sent by the plurality of edge node devices.
Specifically, when the first edge node device is restored to the connected state from the broken link state and receives the second service request, it is indicated that the first edge node device is in network connection with the cloud server through restoration, service interruption caused by network problems can be restored, firstly, a service restoration request needs to be initiated to the cloud server, communication permission information of the cloud device to the first edge node device is obtained to determine whether the first edge node device can restore service traffic with the cloud device, and when the service permission state indicates that the cloud server allows receiving service requests sent by a plurality of edge node devices, it is indicated that no edge node device in other areas is restoring service connection with the cloud server, namely, the second service request can be sent to the cloud server. Under the condition that the service permission state indicates that the cloud server does not allow to receive service requests sent by a plurality of edge node devices, the fact that the edge node devices in other areas are in service connection with the cloud server is indicated, so that in order to avoid traffic impact on the cloud server, sending of a second service request to the cloud server is forbidden, and the second service request is processed by proxy to the first edge node device through a reverse proxy.
Fig. 2 is a flow chart of a data processing method according to another embodiment of the present application. The method is applied to the cloud server and comprises the following steps of S201 to S203:
s201, receiving a first service request sent by edge node equipment under the condition that the first edge node equipment is in a communication state with a cloud server;
s202, responding to a first service request, and determining second response data corresponding to the first service request;
s203, sending second response data to the first edge node device,
the first edge node equipment is used for sending the received second response data to the user terminal and storing the second response data in the cache database; the cache database is used for providing response data corresponding to a third service request under the condition that the first edge node equipment and the cloud server are in a broken link state and the third service request is received, and the third service request comprises the first service request.
Specifically, a cloud server receives a first service request sent by a first edge node, searches response data (second response data) related to the first service request according to the content of the first service request, then sends the second response data to first edge node equipment, and sends the second response data received by the first edge node equipment to a user terminal and stores the second response data in a cache database; the cache database is used for providing response data corresponding to a third service request under the condition that the first edge node equipment and the cloud server are in a broken link state and the third service request is received, and the third service request comprises the first service request. The method and the device have the advantages that the edge node equipment is directly brought into the cluster to which the cloud server belongs to perform unified nano-tube, so that the first edge node equipment has the edge autonomous function of maintaining the stability of edge business service, and the application scene of Kubernetes is expanded.
In some embodiments, before the step S201, that is, before receiving the first service request sent by the edge node device in the case where the first edge node device is in a connected state with the cloud server, the method may further include:
receiving a service recovery request sent by first edge node equipment under the condition that the link failure state between the first edge node equipment and the cloud server is recovered to a communication state;
and sending communication permission information of a first area to which the first edge node equipment belongs to the first edge node equipment according to the service recovery request, wherein the communication permission information is used for indicating service permission states of a plurality of edge node equipment and a cloud server in the first area, the plurality of edge node equipment comprises the first edge node equipment, and the service permission states are used for indicating whether the cloud server allows receiving service requests sent by the plurality of edge node equipment.
Specifically, in this embodiment, edge node devices within a preset area are managed by adopting a group of edge node devices, when a communication state is between a first edge node device and a cloud server, the first edge node device is directly in communication connection with the cloud server, and when a link failure state is between the first edge node device and the cloud server, the first edge node device performs edge autonomy; under the condition that the connection state is recovered from the broken link state between the first edge node equipment and the cloud server, whether the service can be recovered with the cloud server or not is determined by acquiring the communication permission of the cloud server for a preset area (namely a first area) to which the first edge node equipment belongs. Therefore, under the condition that a plurality of preset areas exist, the traffic impact caused by the fact that the edge node equipment in the plurality of areas simultaneously resumes the service connection with the cloud server is avoided through communication permission, and the traffic control is realized.
In some embodiments, after the sending, according to the service restoration request, the communication permission information of the first area to which the first edge node device belongs to the first edge node device, the method may further include:
under the condition that the service permission state in the communication permission information is that the service requests sent by a plurality of edge node devices in a first area are allowed to be received, acquiring the recovery time of the first edge node device, wherein the recovery time is the time required by the first edge node device to recover the service data communication with the cloud server;
and updating the communication permission information of the second area to inhibit receiving service requests sent by a plurality of edge node devices in the second area according to the recovery time, wherein the coverage area of the second area is different from that of the first area.
In one example, the cloud Server can be prevented from being impacted by flow control, and the flow control is mainly used for recovering network links with the cloud Server in batches after the edge network jitter is handled, and at the moment, a full amount of data needs to be acquired, and larger flow impact is generated on a management component API Server of the cloud Server, so that the flow control is realized for reducing the flow impact on the API Server of the cloud Server, and the link on-off of the edge proxy access Kubernetes API Server can be controlled in a permission mode. The access link with the API Server is recovered in batches by controlling whether the permissions expire or not.
The method comprises the following specific steps:
1) The fault isolation controller detects the network fault in the area and updates the access attribute of the area permission to false
2) The loop detection area is known that the node is recovered, at the moment, the allowed access attribute of the area is designed to be true, and the time is set to be max (the maximum value of all allowed time fields and the recovery time interval at the present time)
3) Before each access to the API Server by the edge proxy, obtaining permission, confirming whether the access is true and the current time is greater than the time permitted by the area or not, if so, forwarding the request to the remote API Server, otherwise, continuing to respond the request locally.
In some embodiments, before receiving the request for connection restoration of the first edge node device in the case where the first edge node device is in a connected state with the cloud server, the method further includes:
acquiring heartbeat data of first edge node equipment;
under the condition that the heartbeat data does not meet the preset condition, acquiring the heartbeat data of second edge node equipment in the first area, wherein the second edge node equipment is other edge node equipment except the first edge node equipment in the first area;
under the condition that the heartbeat data of the second edge node equipment meets the preset condition, determining that the first edge node equipment fails;
And migrating the service in the first edge node equipment to a second edge node equipment closest to the first edge node equipment for processing.
In one example, in the context of a distributed data center, where the network within a single data center is stable and reliable, network quality transmissions across the data center are unstable, area detection techniques may be employed to accurately identify whether a node is truly faulty or communication errors caused by an unstable weak network environment. In the application, the fault isolation system can be realized by arranging the fault isolation system in the cloud server, wherein the fault isolation system is a set of self-developed container cloud fault detection and service expelling system and is mainly used for rapidly processing node faults.
The method comprises the following specific steps:
1) The fault isolation controller detects that the fault isolation Agent has no heartbeat to report.
2) The fault isolation controller carries out ping detection on the node once, if the node can be ping-enabled, the node is considered to be normal, the flow is ended, and otherwise, the next step is continued.
3) And detecting whether the node has the area identifier, if not, judging that the node in the data center is abnormal, and directly expelling.
4) The controller acquires the nodes in the area in the cluster, carries out ping detection on the nodes in the area, determines that the edge node is faulty if other nodes in the area can be ping-enabled, executes node isolation operation (namely, transfers the service in the first edge node equipment to the second edge node equipment closest to the first edge node equipment for processing), otherwise determines that the edge network is faulty (namely, the edge node equipment is disconnected from the cloud server), and the process is ended.
Based on the data processing method provided by the embodiment, correspondingly, the application also provides a specific implementation mode of the data processing device, which is applied to the first edge server. Please refer to the following examples.
Referring first to fig. 3, a data processing apparatus 300 according to an embodiment of the present application includes the following modules:
a first receiving module 301, configured to receive a first service request sent by a user terminal;
the processing module 302 is configured to, when the first edge node device and the cloud server are in a broken link state, search first response data corresponding to the first service request in a cache database of the first edge node device, and send the first response data to the user terminal;
the first sending module 303 is configured to send a first service request to the cloud server when the first edge node device is in a connected state with the cloud server, and send response data received from the cloud server to the user terminal and store the response data in the cache database, where the response data is data corresponding to the first service request determined by the cloud server.
In some embodiments, the data processing apparatus 300 may further include:
The forwarding module is used for forwarding the first service request to the cloud server through the reverse proxy component under the condition that the first edge node equipment is in a communication state with the cloud server;
the third receiving module is used for receiving second response data of the first service request returned by the cloud server through the reverse proxy component;
and the caching module is used for caching the second response data to the service processing component of the first edge node equipment.
In some embodiments, the data processing apparatus 300 may further include:
the third sending module is used for sending a service recovery request to the cloud server under the condition that the first edge node equipment and the cloud server are recovered to a communication state from a broken link state and a second service request is received;
the fourth receiving module is used for receiving communication permission information returned by the cloud server, wherein the communication permission information is used for indicating service permission states of a plurality of edge node devices in the target area and the cloud server, the plurality of edge node devices comprise first edge node devices, and the service permission states are used for indicating whether the cloud server is allowed to receive service requests sent by the plurality of edge node devices;
The fourth sending module is configured to send a second service request to the cloud server when the service permission status indicates that the cloud server is allowed to receive the service requests sent by the plurality of edge node devices.
The application also provides a specific implementation mode of the data processing device. The cloud server is applied to the cloud server. Please refer to the following examples.
Referring first to fig. 4, a data processing apparatus 400 provided in an embodiment of the present application includes the following modules:
the second receiving module 401 is configured to receive, when the first edge node device is in a connected state with the cloud server, a first service request sent by the edge node device;
a determining module 402, configured to determine, in response to the first service request, second response data corresponding to the first service request;
a second sending module 403, configured to send second response data to the first edge node device,
the first edge node equipment is used for sending the received second response data to the user terminal and storing the second response data in the cache database; the cache database is used for providing response data corresponding to a third service request under the condition that the first edge node equipment and the cloud server are in a broken link state and the third service request is received, and the third service request comprises the first service request.
In some embodiments, the data processing apparatus 400 may further include:
the fourth receiving module is used for receiving a service recovery request sent by the first edge node equipment under the condition that the link failure state between the first edge node equipment and the cloud server is recovered to a communication state;
the fourth sending module is configured to send, according to the service restoration request, communication permission information of a first area to which the first edge node device belongs to the first edge node device, where the communication permission information is used to indicate service permission states of a plurality of edge node devices and a cloud server in the first area, the plurality of edge node devices include the first edge node device, and the service permission states are used to indicate whether the cloud server allows receiving service requests sent by the plurality of edge node devices.
In some embodiments, the data processing apparatus 400 may further include:
the first acquisition module is used for acquiring recovery time of the first edge node equipment under the condition that the service permission state in the communication permission information is that the service requests sent by the plurality of edge node equipment in the first area are allowed to be received, wherein the recovery time is time required by the first edge node equipment to recover service data communication with the cloud server;
And the updating module is used for updating the communication permission information of the second area into a service request which is forbidden to be received and sent by a plurality of edge node devices in the second area according to the recovery time, wherein the coverage area of the second area is different from that of the first area.
In some embodiments, the data processing apparatus 400 may further include:
the second acquisition module is used for acquiring heartbeat data of the first edge node equipment;
the third acquisition module is used for acquiring the heartbeat data of second edge node equipment in the first area under the condition that the heartbeat data does not meet the preset condition, wherein the second edge node equipment is other edge node equipment except the first edge node equipment in the first area;
the determining module is used for determining that the first edge node equipment fails under the condition that the heartbeat data of the second edge node equipment meets the preset condition;
and the isolation module is used for migrating the service in the first edge node equipment to a second edge node equipment closest to the first edge node equipment for processing.
The data processing apparatus provided in the embodiment of the present invention can implement each step in the method embodiment of fig. 1 or fig. 2, and in order to avoid repetition, a description is omitted here.
Fig. 5 shows a schematic hardware structure of a data processing device according to an embodiment of the present application.
A processor 501 and a memory 502 storing computer program instructions may be included in the data processing device.
In particular, the processor 501 may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured as one or more integrated circuits that implement embodiments of the present application.
Memory 502 may include mass storage for data or instructions. By way of example, and not limitation, memory 502 may comprise a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or universal serial bus (Universal Serial Bus, USB) Drive, or a combination of two or more of the foregoing. Memory 502 may include removable or non-removable (or fixed) media, where appropriate. Memory 502 may be internal or external to the integrated gateway disaster recovery device, where appropriate. In a particular embodiment, the memory 502 is a non-volatile solid state memory.
The memory may include Read Only Memory (ROM), random Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors) it is operable to perform the operations described with reference to methods in accordance with aspects of the present disclosure.
The processor 501 implements any one of the data processing methods of the above embodiments by reading and executing computer program instructions stored in the memory 502.
In one example, the data processing device may also include a communication interface 503 and a bus 510. As shown in fig. 5, the processor 501, the memory 502, and the communication interface 503 are connected to each other by a bus 510 and perform communication with each other.
The communication interface 503 is mainly used to implement communication between each module, apparatus, unit and/or device in the embodiments of the present application.
Bus 510 includes hardware, software, or both that couple the components of the data processing apparatus to one another. By way of example, and not limitation, the buses may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. Bus 510 may include one or more buses, where appropriate. Although embodiments of the application have been described and illustrated with respect to a particular bus, the application contemplates any suitable bus or interconnect.
The data processing device may be based on the above-described embodiments, thereby implementing the data processing method and apparatus described in connection with fig. 1 to 4.
In addition, in combination with the data processing method in the above embodiment, the embodiment of the present application may be implemented by providing a computer storage medium. The computer storage medium has stored thereon computer program instructions; the computer program instructions, when executed by the processor, implement any one of the data processing methods in the foregoing embodiments, and achieve the same technical effects, and in order to avoid repetition, will not be described herein. The computer readable storage medium may include a non-transitory computer readable storage medium, such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, and the like, but is not limited thereto.
In addition, the embodiment of the application also provides a computer program product, which comprises computer program instructions, wherein the computer program instructions can realize the steps and corresponding contents of the embodiment of the method when being executed by a processor.
It should be understood that the application is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present application.
The functional blocks shown in the above block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this disclosure describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be performed in a different order from the order in the embodiments, or several steps may be performed simultaneously.
Aspects of the present disclosure are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present application, and they should be included in the scope of the present application.
Claims (12)
1. A data processing method, characterized by being applied to a first edge node device, the method comprising:
receiving a first service request sent by a user terminal;
searching first response data corresponding to the first service request in a cache database of the first edge node equipment under the condition that the first edge node equipment and the cloud server are in a broken link state, and sending the first response data to the user terminal;
and under the condition that the first edge node equipment is in a communication state with a cloud server, sending the first service request to the cloud server, sending response data received from the cloud server to the user terminal, and storing the response data in the cache database, wherein the response data is data which is determined by the cloud server and corresponds to the first service request.
2. The method of claim 1, wherein after receiving the first service request sent by the user terminal, further comprising:
under the condition that the first edge node equipment is in a communication state with a cloud server, the first edge node equipment forwards the first service request to the cloud server through a reverse proxy component;
Receiving second response data of the first service request returned by the cloud server through the reverse proxy component;
and caching the second response data to a service processing component of the first edge node device.
3. The method of claim 1, wherein after receiving the first service request sent by the user terminal, further comprising:
under the condition that the first edge node equipment and the cloud server are recovered to a communication state from the broken link state and a second service request is received, sending a service recovery request to the cloud server;
receiving communication permission information returned by the cloud server, wherein the communication permission information is used for indicating service permission states of a plurality of edge node devices in a target area and the cloud server, the plurality of edge node devices comprise the first edge node device, and the service permission states are used for indicating whether the cloud server is allowed to receive service requests sent by the plurality of edge node devices;
and sending the second service request to the cloud server under the condition that the service permission state indicates that the cloud server is allowed to receive the service requests sent by the plurality of edge node devices.
4. A data processing method applied to a cloud server, the method comprising:
receiving a first service request sent by first edge node equipment under the condition that the first edge node equipment is in a communication state with a cloud server;
determining second response data corresponding to the first service request in response to the first service request;
transmitting the second response data to the first edge node device,
the first edge node device is used for sending the received second response data to the user terminal and storing the second response data in a cache database; the cache database is configured to provide response data corresponding to a third service request when a link failure state is between the first edge node device and the cloud server and the third service request is received, where the third service request includes the first service request.
5. The method of claim 4, wherein before receiving the first service request sent by the first edge node device when the first edge node device is in a connected state with the cloud server, further comprises:
Receiving a service recovery request sent by the first edge node device under the condition that the link failure state between the first edge node device and the cloud server is recovered to a communication state;
and sending communication permission information of a first area to which the first edge node equipment belongs to the first edge node equipment according to the service recovery request, wherein the communication permission information is used for indicating service permission states of a plurality of edge node equipment and the cloud server in the first area, the plurality of edge node equipment comprises the first edge node equipment, and the service permission states are used for indicating whether the cloud server allows to receive the service requests sent by the plurality of edge node equipment.
6. The method according to claim 5, wherein after the communication permission information of the first area to which the first edge node device belongs is sent to the first edge node device according to the service restoration request, further comprising:
acquiring recovery time of the first edge node equipment under the condition that the service permission state in the communication permission information is that the service requests sent by a plurality of edge node equipment in the first area are allowed to be received, wherein the recovery time is time required by the first edge node equipment to recover service data communication with the cloud server;
And updating the communication permission information of a second area to inhibit receiving service requests sent by a plurality of edge node devices in the second area according to the recovery time, wherein the coverage area of the second area is different from that of the first area.
7. The method according to claim 5, wherein, before receiving the request for connection restoration of the first edge node device in the case where the first edge node device is in a connected state with the cloud server, the method further comprises:
acquiring heartbeat data of the first edge node device;
acquiring heartbeat data of second edge node equipment in the first area under the condition that the heartbeat data does not meet preset conditions, wherein the second edge node equipment is other edge node equipment except the first edge node equipment in the first area;
determining that the first edge node device fails when the heartbeat data of the second edge node device meets the preset condition;
and migrating the service in the first edge node equipment to the second edge node equipment closest to the first edge node equipment for processing.
8. A data processing apparatus for use with a first edge node device, the apparatus comprising:
the first receiving module is used for receiving a first service request sent by the user terminal;
the processing module is used for searching first response data corresponding to the first service request in a cache database of the first edge node equipment under the condition that the first edge node equipment and the cloud server are in a broken link state, and sending the first response data to the user terminal;
the first sending module is configured to send the first service request to the cloud server when the first edge node device is in a connected state with the cloud server, and send response data received from the cloud server to the user terminal and store the response data in the cache database, where the response data is data determined by the cloud server and corresponding to the first service request.
9. A data processing apparatus for use in a cloud server, the apparatus comprising:
the second receiving module is used for receiving a first service request sent by the edge node equipment under the condition that the first edge node equipment is in a communication state with the cloud server;
The determining module is used for responding to the first service request and determining second response data corresponding to the first service request;
a second transmitting module, configured to transmit the second response data to the first edge node device,
the first edge node device is used for sending the received second response data to the user terminal and storing the second response data in a cache database; the cache database is configured to provide response data corresponding to a third service request when a link failure state is between the first edge node device and the cloud server and the third service request is received, where the third service request includes the first service request.
10. A data processing apparatus, the apparatus comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the data processing method of any of claims 1-7.
11. A computer storage medium, characterized in that it has stored thereon computer program instructions which, when executed by a processor, implement the data processing method according to any of claims 1-7.
12. A computer program product, characterized in that it comprises computer program instructions which, when executed by a processor, implement the data processing method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452452.8A CN117014493A (en) | 2022-04-27 | 2022-04-27 | Data processing method, apparatus, device, storage medium, and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452452.8A CN117014493A (en) | 2022-04-27 | 2022-04-27 | Data processing method, apparatus, device, storage medium, and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117014493A true CN117014493A (en) | 2023-11-07 |
Family
ID=88564032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210452452.8A Pending CN117014493A (en) | 2022-04-27 | 2022-04-27 | Data processing method, apparatus, device, storage medium, and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117014493A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117729585A (en) * | 2023-12-14 | 2024-03-19 | 阳光凯讯(北京)科技股份有限公司 | 5G communication-based space-based information distribution method and system |
-
2022
- 2022-04-27 CN CN202210452452.8A patent/CN117014493A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117729585A (en) * | 2023-12-14 | 2024-03-19 | 阳光凯讯(北京)科技股份有限公司 | 5G communication-based space-based information distribution method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2014166265A1 (en) | Method, terminal, cache server and system for updating webpage data | |
| US20150256622A1 (en) | Connection management device, communication system, connection management method, and computer program product | |
| CN111225401A (en) | Method and related equipment for realizing disaster tolerance | |
| CN116886497B (en) | Service grid business centralized proxy switching method and processing system based on DPU | |
| EP4154474A1 (en) | Service producer health-check | |
| CN104717077A (en) | Data center management method, device and system | |
| CN112764956B (en) | Database exception handling system, database exception handling method and device | |
| CN117014493A (en) | Data processing method, apparatus, device, storage medium, and program product | |
| CN106657187A (en) | Message processing method and apparatus thereof | |
| CN111314443A (en) | Node processing method, device and equipment based on distributed storage system and medium | |
| CN111679925B (en) | Database fault processing method, device, computing equipment and storage medium | |
| CN110737543B (en) | Method, device and storage medium for recovering distributed file system data | |
| CN110519109B (en) | Method, device, computing equipment and medium for detecting node association | |
| US9203760B2 (en) | Communication device and route search method | |
| US11212220B2 (en) | Optimized reconciliation in a controller-switch network | |
| CN112422598A (en) | Resource scheduling method, intelligent front-end equipment, intelligent gateway and distributed system | |
| US20230100543A1 (en) | Time source health monitoring support for network timing resiliency | |
| CN114598594B (en) | Method, system, medium and equipment for processing application faults under multiple clusters | |
| CN111338573B (en) | Terminal data management method and system, server and communication terminal | |
| KR101792331B1 (en) | Method and System for Transaction in the M2M/IoT Platform | |
| CN109495384B (en) | Method and device for improving reliability of network routing | |
| CN113890817A (en) | Communication optimization method and device | |
| CN113300873B (en) | Fault detour method and device based on five-tuple hash path | |
| CN117851244B (en) | Database fault detection method and device, electronic equipment and storage medium | |
| CN111988808A (en) | Core network disaster tolerance backup method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |