CN115208872B

CN115208872B - Edge cloud mirror image data processing method and device, computer equipment and storage medium

Info

Publication number: CN115208872B
Application number: CN202110384026.0A
Authority: CN
Inventors: 程庞钢; 张媛; 邹艳军; 巫勇明; 林楷淇
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2021-04-09
Filing date: 2021-04-09
Publication date: 2024-09-27
Anticipated expiration: 2041-04-09
Also published as: CN115208872A

Abstract

The application relates to an edge cloud mirror image data processing method, an edge cloud mirror image data processing device, computer equipment and a storage medium. The method comprises the following steps: acquiring a service file to be transmitted and a mirror image data transmission strategy data set; sequentially determining the time-consuming time length required by transmitting the service file to an edge warehouse at the edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data; determining target mirror image data transmission strategy data from a mirror image data transmission strategy data set according to each time consuming time duration; and based on the target mirror image data transmission strategy data, the service file is mirrored to an edge warehouse of the edge end through pushing. The method can improve the bandwidth utilization rate of the edge cloud network line.

Description

Edge cloud mirror image data processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of edge cloud technologies, and in particular, to an edge cloud mirror image data processing method, an edge cloud mirror image data processing device, a computer device, and a storage medium.

Background

Based on a series of advantages of container technology delivery in seconds, efficient deployment, flexible expansion, resource scheduling and the like, the use of containers has become a future trend. The mirror image contains software data required for the container running, and when the container runs, the TAG (typically hundreds of megabytes to G) of the corresponding mirror image needs to be pulled (pull) to create the container, so that the mirror image is important for the container.

The current pull mirror TAG scheme generally creates a container (i.e. the command used: mirror name (optional)/mirror name: mirror: TAG (optional), default last) through a pull official or private warehouse, ensuring that the container is functioning properly. Wherein, pull official mirror image realizes acceleration through free accelerator; pull private mirror is typically implemented by self-building a network line and then pulling mirror through the line. However, in the edge cloud application scenario, there is an abnormal edge cloud image data transmission through the edge network line pull image TAG, which results in low utilization rate of the edge cloud network line bandwidth.

Disclosure of Invention

In view of the foregoing, it is desirable to provide an edge cloud mirror image data processing method, apparatus, computer device, and storage medium capable of improving the bandwidth utilization of an edge cloud network line.

An edge cloud image data processing method, the method comprising:

acquiring a service file to be transmitted and a mirror image data transmission strategy data set;

sequentially determining the time-consuming time length required for transmitting the service file to an edge warehouse at the edge end according to the file size of the service file and the network line bandwidth of each piece of mirror image data transmission strategy data;

determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time consuming time period;

And mirroring the service file to an edge warehouse of the edge end by pushing based on the target mirror image data transmission strategy data.

In one embodiment, the mirror image data transmission policy data set includes private network line policy data and public network line policy data, and the determining, in sequence, a time-consuming period required for transmitting the service file to an edge repository at an edge according to a file size of the service file and a network line bandwidth of each mirror image data transmission policy data includes:

determining a first time-consuming time period required by the service file to be transmitted from a central end to an edge warehouse of the edge end through a private network line according to the file size of the service file and the private network bandwidth of the private network line policy data;

And determining a second time-consuming time period required by the service file to be transmitted to the edge warehouse of the edge end through the download server from the central end to the download server cluster through the public network line according to the file size of the service file and the public network bandwidth of the public network line policy data.

In one embodiment, the determining the target mirrored data transmission policy data from the mirrored data transmission policy data set according to each of the time-consuming durations includes:

When the first time-consuming duration is smaller than the second time-consuming duration, determining that the private network line policy data is target mirror image data transmission policy data;

The mirroring the service file to the edge repository of the edge end by pushing based on the target mirror data transmission policy data includes:

based on the private network line policy data, when the file size of the service file is in a first numerical value area, the service file is mirrored to an edge warehouse of the edge through pushing by triggering an idle time window of the edge at fixed time.

In one embodiment, the idle time window has a corresponding idle start time point and an idle end time point, and the mirroring the service file to the edge repository of the edge through pushing by triggering the space time window of the edge at a fixed time includes:

Acquiring a trigger time point of the timing trigger;

When the trigger time point is in the time interval of the idle starting time point and the idle ending time point, triggering the idle time window of the edge end to mirror the service file to the edge warehouse of the edge end through pushing.

In one embodiment, the method further comprises:

and when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and executing the step of triggering the idle time window of the edge end to mirror the service file to the edge warehouse of the edge end through pushing.

when the first time consuming period is longer than the second time consuming period, determining that the public network line policy data is target mirror image data transmission policy data;

Based on the public network line policy data, when the file size of the service file is in a second numerical value area, encrypting and compressing the service file to obtain a compressed file;

Uploading the compressed file to the download server cluster through a public network inlet to generate a download address of the compressed file;

the download address is sent to the edge terminal, the edge terminal is triggered to download the compressed file through the download address and to decompress and decrypt the compressed file, and the service file is obtained;

And mirroring the service file to an edge warehouse of the edge end through pushing.

In one embodiment, the method further comprises:

Based on the public network line policy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file;

Splitting the compressed file to obtain n subfiles;

uploading the n subfiles to the download server cluster through a public network inlet to generate a download address of each subfile;

sending each download address to the edge terminal, triggering the edge terminal to download the subfiles through each download address, and performing decompression decryption and combination processing to obtain the service file;

An edge cloud image data processing apparatus, the apparatus comprising:

the acquisition module is used for acquiring the service file to be transmitted and the mirror image data transmission strategy data set;

the time length determining module is used for sequentially determining time-consuming time length required by transmitting the service file to an edge warehouse of an edge end according to the file size of the service file and the network line bandwidth of each piece of mirror image data transmission strategy data;

the policy data determining module is used for determining target mirror image data transmission policy data from the mirror image data transmission policy data set according to each time-consuming time duration;

And the data transmission module is used for mirroring the service file to the edge warehouse of the edge end by pushing based on the target mirror image data transmission policy data.

A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:

A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

The edge cloud mirror image data processing method, the device, the computer equipment and the storage medium are used for acquiring the service file to be transmitted and the mirror image data transmission strategy data set; sequentially determining time-consuming time required by transmitting the service file to an edge warehouse at an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and determining target mirror image data transmission strategy data from a mirror image data transmission strategy data set according to each time-consuming time; based on target mirror image data transmission strategy data, a business file is mirrored to an edge warehouse of an edge end through pushing; and determining target policy data of the edge file transmission according to the network line bandwidth and the size of the service file, thereby improving the utilization rate of the network line bandwidth and determining the normal transmission of the service file.

Drawings

FIG. 1 is an application environment diagram of an edge cloud image data processing method in one embodiment;

FIG. 2 is a flow chart of a method for processing edge cloud mirror data in one embodiment;

FIG. 3 is a flow chart of a method of processing data based on private network line policy data in one embodiment;

FIG. 4 is a flow diagram of data processing based on the public network line policy data in one embodiment;

FIG. 5 is a flowchart of a method for processing edge cloud image data according to another embodiment;

FIG. 6 is a block diagram of edge cloud mirror data processing between a center end and an edge end in one embodiment;

FIG. 7 is a block diagram of an edge cloud mirror data processing device in one embodiment;

Fig. 8 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The edge cloud mirror image data processing method provided by the application can be applied to an application environment shown in fig. 1. Wherein the edge 102 communicates with the center 104 via a network. The method comprises the steps that a center end obtains a service file and a mirror image data transmission strategy data set to be transmitted to an edge end; sequentially determining the time-consuming time length required by transmitting the service file to an edge warehouse at the edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data; determining target mirror image data transmission strategy data from a mirror image data transmission strategy data set according to each time consuming time duration; and based on the target mirror image data transmission strategy data, the service file is mirrored to an edge warehouse of the edge end through pushing. The edge 102 may be, but not limited to, various personal computers, notebook computers, smart phones, and tablet computers, and the center 104 may be a cloud server, i.e. may be implemented by a server stand alone or a server cluster composed of a plurality of servers.

In one embodiment, as shown in fig. 2, an edge cloud mirror image data processing method is provided, and the method is applied to the center end in fig. 1 for illustration, and includes the following steps:

Step 202, obtaining a service file to be transmitted and a mirror image data transmission strategy data set.

The service file with transmission is mirror image data required by operations such as normal operation of an edge cloud service, service processing and the like, and the edge cloud service can be but is not limited to a private mirror image. The mirror image data transmission strategy data refers to strategy data used for transmitting mirror image data between a center end and an edge end, wherein the mirror image data transmission strategy data comprises a network line type of data transmission and a processing mode of data transmission; the mirror image data between the center end and the edge end are transmitted based on network lines, wherein the network lines comprise private network lines, public network lines and the like; the mirror image data transmission strategy data set at least comprises private network line strategy data and public network line strategy data, wherein the private network line strategy data can be private network delay strategy data, and the public network line strategy data can be public network real-time strategy data.

Specifically, mirror image warehouses are deployed in advance at the center end and each edge end (for example, an edge warehouse is deployed at the edge end, and a center warehouse is deployed at the center end), and a base mirror image referenced by dockerfile (including environment data on which an operating system operates) is pushed to the edge endpoint mirror image warehouse, and a back-end server at the center end acquires a service file to be transmitted according to a file download address to be constructed of the mirror image, namely, acquires mirror image data to be transmitted; and deploying a real-time public network channel and a delay private network channel between the central end and the edge end to obtain a preset mirror image data transmission strategy data set for transmitting mirror image data, namely private network line strategy data and public network line strategy data.

Step 204, determining the time-consuming time period required for transmitting the service file to the edge warehouse at the edge end in sequence according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data.

Specifically, acquiring attribute data of a service file, and determining the file size of the service file in the attribute data, a first bandwidth available for transmitting data by a private network line and a second bandwidth available for transmitting data by a public network line; determining a first time-consuming duration required for transmitting the service file by the private network line policy data according to the file size of the service file, the first bandwidth in the private network line policy data and the second bandwidth in the public network line policy data, and determining a second time-consuming duration required for transmitting the service file by the private network line policy data according to the file size of the service file and the second bandwidth in the public network line policy data.

And step 206, determining target mirror image data transmission strategy data from the mirror image data transmission strategy data set according to each time consuming time duration.

Specifically, by comparing the time-consuming periods required for each mirror data transmission policy data in the mirror data transmission policy data set to transmit the same service file, the target mirror data transmission policy data may be determined from the mirror data transmission policy data set according to the length of the time-consuming period, but not limited to. For example, the mirror image data transmission policy data set includes private network line policy data and public network line policy data, and the image data transmission policy data with the shortest time is selected from a first time-consuming period required by the private network line policy data transmission service file and a second time-consuming period required by the private public network line policy data transmission service file as target mirror image data transmission policy data.

And step 208, based on the target mirror image data transmission policy data, the service file is mirrored to an edge warehouse of the edge end through pushing.

Specifically, when the private network line policy data is target mirror image data transmission policy data, the service file is mirrored to an edge warehouse of the edge end through pushing based on the private network line policy data; based on private network line policy data, when the file size of the service file is in a first numerical value area, the service file is mirrored to an edge warehouse of the edge through pushing by triggering an idle time window of the edge periodically. When the public network line policy data is target mirror image data transmission policy data, the service file is mirrored to an edge warehouse of the edge end through pushing based on the public network line policy data; the method comprises the steps that based on public network line policy data, encryption and compression processing are carried out on a file according to the size of a service file, so that a compressed file is obtained; uploading the compressed file to a download server cluster through a public network inlet to generate a download address of the compressed file; the downloading address is sent to the edge terminal, and the edge terminal is triggered to download the compressed file through the downloading address and to decompress and decrypt the compressed file to obtain the service file; and mirroring the service file to an edge warehouse at the edge end through pushing.

In the edge cloud mirror image data processing method, time consuming time required for transmitting the service file to an edge warehouse at an edge end is determined according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and target mirror image data transmission strategy data is determined from a mirror image data transmission strategy data set according to each time consuming time; based on target mirror image data transmission strategy data, a business file is mirrored to an edge warehouse of an edge end through pushing; and determining target policy data of the edge file transmission according to the network line bandwidth and the size of the service file, avoiding the network line bandwidth waste caused by selecting unsuitable data transmission policy data and abnormal transmission of the service file to be retransmitted, improving the utilization rate of the network line bandwidth and determining the normal transmission of the service file.

In one embodiment, as shown in fig. 3, a data processing method based on private network line policy data is provided, and the method is applied to the central end in fig. 1 for illustration, and includes the following steps:

Step 302, when the file size of the service file is within the first numerical value area, a trigger time point of timing trigger is obtained.

The first numerical area is a file size interval, and may be, but not limited to, (10M, + -infinity), M being a unit of file size, megabits. The idle time window (idle starting time point and idle ending time point are set in the idle time window) is preset at each edge in the private network line policy data, which can be but not limited to an idle time window of every day; by setting the idle time window, the service files are mirrored to the edge warehouses of the edge end through pushing, namely, the service files are delayed and synchronized to the edge warehouses through the idle time window, so that the situation that the special line network for transmitting the files between the edge end and the center end is completely used, the normal transmission of the core service files is interfered, and the utilization rate of the special line network resources is not high is avoided. Optionally, the network speed limit bandwidth can be set in the private network line policy data according to the file size of the service file and/or the importance of the service, so that the network utilization rate is improved under the condition of ensuring normal transmission of the service file.

Specifically, when the size of the service file is in the first numerical value region, executing a service file synchronization task through triggering of a timer according to a pre-established timing task; i.e. the back-end server in the central end informs the edge ends to perform push mirroring (push mirroring) of the service files.

Optionally, when the file size of the service file is not in the first numerical value area, triggering to construct a mirror image of the service file through the private line network, and pushing the mirror image to an edge warehouse of the edge.

Step 304, judging whether the trigger time point is in the time interval of the idle starting time point and the idle ending time point, if so, executing step 306; otherwise, step 308 is performed.

Step 306, triggering the idle time window of the edge to mirror the business file to the edge warehouse of the edge through pushing.

Specifically, when the trigger time point is within the time interval of the idle starting time point and the idle ending time point, the service file is mirrored to the edge warehouse of the edge through pushing based on the idle time window of the edge.

Optionally, when the trigger time point is within the time interval of the idle start time point and the idle end time point, at least two different mirror image TAG synchronization tasks (i.e., at least two different service files) are arranged on the same edge end, queuing is performed according to the sequence of the trigger time points of the tasks, and only one mirror image TAG synchronization task is executed at the same time for ensuring the accuracy rate of data synchronization and the integrity of data.

Step 308, suspending this timing trigger until the next idle start time point, executing step 306.

Specifically, when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and mirroring the service file to an edge warehouse of the edge through pushing based on an idle time window of the edge. The trigger time point is not in the time interval of the idle starting time point and the idle ending time point, and can be but not limited to at least two different mirror image TAG synchronous tasks, and the time point is caused by the idle ending time point when the current task is executed in the queuing process.

In the data processing method based on the private network line policy data, idle time windows are preset at each edge end, and when the file size of the service file is in a first numerical value area, a trigger time point of timing trigger is obtained; setting an idle starting time point and an idle ending time point according to the trigger time point and the idle time window, and mirroring the service file to an edge warehouse of the edge end by pushing based on the idle time window of the edge end, namely, utilizing the bandwidth during service idle, so as to improve the utilization rate of the bandwidth; the method can further reduce the pressure on the public network bandwidth, and has higher safety and ensures the accuracy of data.

In one embodiment, as shown in fig. 4, a data processing method based on public network line policy data is provided, and the method is applied to the central end in fig. 1 for illustration, and includes the following steps:

step 402, based on the public network line policy data, judging whether the file size of the service file is in the second numerical value area or the third numerical value area, and executing step 404 when the file size of the service file is in the second numerical value area; when the file size of the service file is in the third numerical region, step 408 is performed.

The second numerical value area and the third numerical value area are numerical value areas of file sizes preset in the public network line policy data, and the file sizes of the second numerical value area are smaller than those of the third numerical value area; for example, the second numerical region is (10M, 100M), the third numerical region is 100M, ++ infinity A kind of electronic device.

Optionally, when the file size of the service file is not in the second numerical value area or the third numerical value area, triggering to construct the mirror image of the service file through the private line network, and pushing the mirror image to an edge warehouse of the edge end.

And step 404, encrypting and compressing the service file to obtain a compressed file, uploading the compressed file to a download server cluster through a public network inlet, and generating a download address of the compressed file.

Specifically, when the size of the service file is in the second numerical area, performing encryption lossless compression processing on the service file to obtain a complete compressed file, and uploading the compressed file to a central public network download server cluster to obtain a secure download address URL_ID and a file signature value MD5 of the compressed file; the public network download server cluster at the center end is established at a public network inlet at the center end, the public network download server cluster comprises at least 3 public network download servers, the public network download server cluster supports breakpoint continuous transmission of files, and the size of the transmitted files is more than or equal to 10M.

Step 406, the download address is sent to the edge, the edge is triggered to download the compressed file through the download address and to perform decompression and decryption processing, so as to obtain the service file, and step 414 is executed.

Specifically, the central end sends the generated download address to the edge end through a private line network or a virtual private network, and the download address can be one edge end or a plurality of edge ends; generating a triggering instruction, wherein the triggering instruction is used for triggering an edge terminal to download a compressed file through a download address and to decompress and decrypt the compressed file to obtain a service file; when the edge end receives the download address, the compressed file and the corresponding dockerfile template are downloaded through the download address according to the trigger instruction, the file signature value MD5 is verified, and when the verification passes, the compressed file is decompressed and decrypted to obtain the service file to be transmitted.

In step 408, the service file is encrypted and compressed to obtain a compressed file, and the compressed file is split to obtain n subfiles.

Specifically, when the size of the service file is in the third numerical region, performing encryption lossless compression processing on the service file to obtain a complete compressed file, and splitting the compressed file into n subfiles with file sizes not in the third numerical region.

Step 410, uploading the n subfiles to the download server cluster through the public network portal, and generating the download address of each subfile.

Specifically, uploading n subfiles to different download servers in a download server cluster through a public network entry to obtain a secure download address url_id and a file signature value MD5 of each subfile.

Step 412, send each download address to the edge, trigger the edge to download the sub-file through each download address, decompress and decrypt the sub-file, and combine the sub-files to obtain a service file, and execute step 414.

Specifically, the central end sends the generated download address of each sub-file to the edge end through a private line network or a virtual private network, and the download address can be one edge end or a plurality of edge ends; generating a triggering instruction, wherein the triggering instruction is used for triggering an edge terminal to download a corresponding sub-file through a download address, and combining, decompressing and decrypting the sub-file to obtain a service file; when the edge end receives the download address, the download address downloads the sub-files and the corresponding dockerfile templates according to the trigger instruction, the file signature value MD5 is verified, when the verification passes, the sub-files are combined to obtain a compressed file, and decompression and decryption operations are carried out on the compressed file to obtain the service file to be transmitted.

In step 414, the service file is mirrored to the edge repository at the edge end by pushing.

Specifically, the edge end pushes the mirror image to an edge warehouse of the edge end by constructing the mirror image of the service file, generates a message of successful pushing and feeds back the message to the center end; when the central terminal receives the information fed back by the edge terminal, a deleting instruction is generated, and files of the public network download server cluster are deleted according to the deleting instruction.

In the data processing method based on the public network line policy data, the download server of the public network is built, when the file size of the service file is in the second numerical area, the service file is encrypted and compressed to obtain the compressed file, and when the file size of the service file is in the third numerical area, the service file is encrypted and compressed to obtain the compressed file, so that the safety of the mirror image file in the Internet and the utilization rate of network bandwidth are improved; the compressed files or n sub-files are uploaded to a download server set for distribution through a public network entrance, so that huge cost caused by special line capacity expansion due to overlarge files is avoided; and splitting the compressed file to obtain n subfiles, so that the downloading pressure of a single downloading server is reduced, and the bandwidth utilization rate is improved.

In another embodiment, as shown in fig. 5, an edge cloud image data processing method is provided, and the method is applied to the center end in fig. 1 for illustration, and includes the following steps:

Step 502, a service file to be transmitted and a mirror data transmission policy data set are acquired.

Step 504, determining a first time-consuming period required by the service file to be transmitted from the center end to the edge warehouse of the edge end through the private network according to the file size of the service file and the private network bandwidth of the private network line policy data.

Step 506, determining a second time-consuming duration required by the service file to be transmitted from the central end to the download server cluster through the public network line and to be transmitted to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line policy data.

Step 508, determining whether the first time period is less than the second time period, if yes, executing step 510, otherwise, executing step 514.

Step 510, determining that the private network line policy data is target mirror image data transmission policy data.

Step 512, mirror the service file to the edge repository of the edge by pushing based on the private network line policy data.

Specifically, when the file size of the service file is in the first numerical value area, the service file is mirrored to an edge warehouse of the edge end through pushing by triggering an idle time window of the edge end at fixed time; namely, acquiring a trigger time point of timing trigger; when the trigger time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing; when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and executing the idle time window triggering the edge end again to mirror the service file to the edge warehouse of the edge end through pushing.

Step 514, determining the public network line policy data as the target mirror image data transmission policy data.

And step 516, mirroring the service file to an edge warehouse of the edge end by pushing based on the public network line policy data.

Specifically, based on public network line policy data, when the file size of the service file is in a second numerical value area, encrypting and compressing the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network inlet to generate a download address of the compressed file; the downloading address is sent to the edge terminal, and the edge terminal is triggered to download the compressed file through the downloading address and to decompress and decrypt the compressed file to obtain the service file; the business file is mirrored to an edge warehouse at the edge end through pushing; based on the public network line policy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file; splitting the compressed file to obtain n subfiles; uploading n subfiles to a download server cluster through a public network inlet, and generating a download address of each subfile; sending each download address to an edge terminal, triggering the edge terminal to download the sub-files through each download address, and performing decompression decryption and combination processing to obtain service files; and mirroring the service file to an edge warehouse at the edge end through pushing.

The following is a block diagram of an embodiment of edge cloud image data processing between a center and edge as shown in fig. 6. The central end side is provided with a download server of a public network, a back end server and a central warehouse, the download server, the back end server and the central warehouse are connected through an intranet (local area network), a bidirectional private network (2-6M bidirectional access network) and a unidirectional private network (more than 10M+), an edge warehouse A and an edge warehouse B are sequentially arranged between the edge end A and the edge end B, the edge warehouse A in the edge end A, the edge node 1 and the edge node 2 are communicated through an intranet in the edge end, and the edge warehouse B in the edge end B, the edge node 3 and the edge node 4 are communicated through an intranet in the edge end.

Acquiring a service file to be transmitted and a mirror image data transmission strategy data set, and determining a first time-consuming time period required by an edge warehouse of the service file transmitted from a center end to an edge end through a private network line; determining a second time-consuming time required by the service file transmitted from the central end to the download server cluster through the public network line and transmitted to the edge warehouse of the edge end through the download server, and acquiring a trigger time point of timing trigger when the first time-consuming time is smaller than the second time-consuming time and the file size of the service file is in a first numerical area; when the trigger time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing; when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and executing the idle time window triggering the edge end again to mirror the service file to the edge warehouse of the edge end through pushing.

When the first time consuming period is longer than the second time consuming period, based on public network line policy data, and when the file size of the service file is in the second numerical area, encrypting and compressing the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network inlet to generate a download address of the compressed file; the downloading address is sent to the edge terminal, and the edge terminal is triggered to download the compressed file through the downloading address and to decompress and decrypt the compressed file to obtain the service file; the business file is mirrored to an edge warehouse at the edge end through pushing; based on the public network line policy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file; splitting the compressed file to obtain n subfiles; uploading n subfiles to a download server cluster through a public network inlet, and generating a download address of each subfile; sending each download address to an edge terminal, triggering the edge terminal to download the sub-files through each download address, and performing decompression decryption and combination processing to obtain service files; and mirroring the service file to an edge warehouse at the edge end through pushing.

In the edge cloud mirror image data processing method, the service files to be transmitted are acquired, the time-consuming time length required by the public network line policy data and the private network line policy data transmission files is determined in sequence according to the size of the service files to be transmitted, the most suitable mirror image data transmission policy data is determined according to the time-consuming time length, when the policy data are transmitted by the public network line policy data, the security of the mirror image files in the Internet and the utilization rate of network bandwidth are improved by carrying out encryption compression processing on the files, the files are split and uploaded to a downloading server set for distribution, huge cost caused by special line capacity expansion of the files is avoided, the downloading pressure of a single downloading server is reduced, and the bandwidth utilization rate is improved. Transmitting files by using a private network line strategy data, and utilizing the bandwidth when the service is idle, so that the utilization rate of the bandwidth is improved; the method can further reduce the pressure on the public network bandwidth, and has higher safety and ensures the accuracy of data.

It should be understood that, although the steps in the flowcharts of fig. 2-5 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2-5 may include multiple steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor does the order in which the steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the steps or stages in other steps or other steps.

In one embodiment, as shown in fig. 7, there is provided an edge cloud image data processing apparatus, including: an acquisition module 702, a duration determination module 704, a policy data determination module 706, and a data transmission module 708, wherein:

the acquiring module 702 is configured to acquire a service file to be transmitted and a mirror data transmission policy data set.

And the duration determining module 704 is configured to sequentially determine a time-consuming duration required for transmitting the service file to an edge repository at an edge end according to the file size of the service file and the network line bandwidth of each piece of mirror image data transmission policy data.

A policy data determining module 706, configured to determine target mirrored data transmission policy data from the mirrored data transmission policy data set according to each of the time-consuming durations.

And the data transmission module 708 is configured to mirror the service file to an edge repository of the edge by pushing based on the target mirror data transmission policy data.

In the edge cloud mirror image data processing device, a service file to be transmitted and a mirror image data transmission strategy data set are obtained; sequentially determining time-consuming time required by transmitting the service file to an edge warehouse at an edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data, and determining target mirror image data transmission strategy data from a mirror image data transmission strategy data set according to each time-consuming time; based on target mirror image data transmission strategy data, a business file is mirrored to an edge warehouse of an edge end through pushing; and determining target policy data of the edge file transmission according to the network line bandwidth and the size of the service file, thereby improving the utilization rate of the network line bandwidth and determining the normal transmission of the service file.

In one embodiment, an edge cloud image data processing apparatus is provided, which includes, in addition to the acquisition module 702, the duration determination module 704, the policy data determination module 706, and the data transmission module 708:

In one embodiment, the duration determining module 704 is further configured to determine a first time-consuming duration required for the service file to be transmitted from the central end to the edge repository of the edge end through the private network line according to the file size of the service file and the private network bandwidth of the private network line policy data.

In one embodiment, the duration determining module 704 is further configured to determine a second time-consuming duration required for the service file to be transmitted from the central end to the download server cluster via the download server to the edge repository at the edge end according to the file size of the service file and the public network bandwidth of the public network line policy data.

In one embodiment, the policy data determining module 706 is further configured to determine that the private network line policy data is target mirror data transmission policy data when the first time period is less than the second time period.

The data transmission module 708 is further configured to mirror the service file to the edge repository of the edge by pushing the edge by periodically triggering an idle time window of the edge when the file size of the service file is within the first numerical region based on the private network line policy data.

The acquisition module 702 is further configured to acquire a trigger time point of the timing trigger.

And the triggering module is used for triggering the idle time window of the edge end to mirror the service file to the edge warehouse of the edge end through pushing when the triggering time point is in the time interval of the idle starting time point and the idle ending time point.

And the suspension module is used for suspending the timing trigger until the next idle starting time point when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, and executing an edge warehouse step of triggering an idle time window of the edge end to mirror the service file to the edge end through pushing.

In one embodiment, the policy data determining module 706 is further configured to determine that the public network line policy data is target mirror data transmission policy data when the first time period is longer than the second time period.

And the encryption and compression module is used for carrying out encryption and compression processing on the service file to obtain a compressed file when the file size of the service file is in the second numerical region based on the public network line policy data.

And the address generation module is used for uploading the compressed file to the download server cluster through the public network entrance to generate a download address of the compressed file.

The triggering module is also used for sending the download address to the edge terminal, triggering the edge terminal to download the compressed file through the download address and to decompress and decrypt the compressed file to obtain the service file.

The data transfer module 708 is further configured to mirror the business file to an edge repository at the edge via push.

The encryption compression module is further used for carrying out encryption compression processing on the service file to obtain a compressed file when the file size of the service file is in a third numerical value area based on the public network line policy data.

And the file splitting module is used for splitting the compressed file to obtain n subfiles.

And the address generation module is used for uploading the n subfiles to the download server cluster through the public network entrance to generate the download address of each subfile.

The triggering module is also used for sending each download address to the edge terminal, triggering the edge terminal to download the sub-files through each download address, and carrying out decompression decryption and combination processing to obtain the service file.

In one embodiment, a central terminal acquires a service file to be transmitted and a mirror image data transmission strategy data set, and determines a first time-consuming time period required by an edge warehouse of the service file transmitted from the central terminal to the edge terminal through a private network line; determining a second time-consuming time required by the service file transmitted from the central end to the download server cluster through the public network line and transmitted to the edge warehouse of the edge end through the download server, and acquiring a trigger time point of timing trigger when the first time-consuming time is smaller than the second time-consuming time and the file size of the service file is in a first numerical area; when the trigger time point is in the time interval of the idle starting time point and the idle ending time point, triggering an idle time window of the edge end to mirror the service file to an edge warehouse of the edge end through pushing; when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and executing the idle time window triggering the edge end again to mirror the service file to the edge warehouse of the edge end through pushing; when the service is idle, the bandwidth is utilized, so that the utilization rate of the bandwidth is improved; the method can further reduce the pressure on the public network bandwidth, and has higher safety and ensures the accuracy of data.

When the first time consuming period is longer than the second time consuming period, based on public network line policy data, and when the file size of the service file is in the second numerical area, encrypting and compressing the service file to obtain a compressed file; uploading the compressed file to a download server cluster through a public network inlet to generate a download address of the compressed file; the downloading address is sent to the edge terminal, and the edge terminal is triggered to download the compressed file through the downloading address and to decompress and decrypt the compressed file to obtain the service file; the business file is mirrored to an edge warehouse at the edge end through pushing; based on the public network line policy data, when the file size of the service file is in a third numerical value area, encrypting and compressing the service file to obtain a compressed file; splitting the compressed file to obtain n subfiles; uploading n subfiles to a download server cluster through a public network inlet, and generating a download address of each subfile; sending each download address to an edge terminal, triggering the edge terminal to download the sub-files through each download address, and performing decompression decryption and combination processing to obtain service files; the service files are pushed to the edge warehouse of the edge end through the mirror image, namely, the files are encrypted and compressed, so that the safety of the mirror image files in the Internet and the utilization rate of network bandwidth are improved, the files are split and uploaded to a downloading server set for distribution, huge cost caused by special line capacity expansion due to overlarge files is avoided, the downloading pressure of a single downloading server is reduced, and the bandwidth utilization rate is improved.

For specific limitation of the edge cloud image data processing apparatus, reference may be made to the limitation of the edge cloud image data processing method hereinabove, and no further description is given here. The respective modules in the edge cloud image data processing device may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for business file data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements an edge cloud mirror data processing method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 8 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:

Sequentially determining the time-consuming time length required by transmitting the service file to an edge warehouse at the edge end according to the file size of the service file and the network line bandwidth of each mirror image data transmission strategy data;

determining target mirror image data transmission strategy data from a mirror image data transmission strategy data set according to each time consuming time duration;

and based on the target mirror image data transmission strategy data, the service file is mirrored to an edge warehouse of the edge end through pushing.

In one embodiment, the processor when executing the computer program further performs the steps of:

Determining a first time-consuming time length required by an edge warehouse of the service file transmitted from a center end to an edge end through a private network line according to the file size of the service file and the private network bandwidth of private network line policy data;

And determining a second time-consuming time period required by the service file to be transmitted to the download server cluster from the central end through the public network line and to be transmitted to the edge warehouse of the edge end through the download server according to the file size of the service file and the public network bandwidth of the public network line policy data.

based on target mirror image data transmission strategy data, mirroring the service file to an edge warehouse of an edge end through pushing, and the method comprises the following steps:

Based on private network line policy data, when the file size of the service file is in a first numerical value area, the service file is mirrored to an edge warehouse of the edge through pushing by triggering an idle time window of the edge at fixed time.

acquiring a trigger time point of timing trigger;

when the trigger time point is not in the time interval of the idle starting time point and the idle ending time point, suspending the timing trigger until the next idle starting time point, and executing an edge warehouse step of triggering an idle time window of the edge end to mirror the service file to the edge end through pushing.

when the first time consuming duration is longer than the second time consuming duration, determining that the public network line policy data is target mirror image data transmission policy data;

based on the public network line policy data, when the file size of the service file is in the second numerical value area, encrypting and compressing the service file to obtain a compressed file;

uploading the compressed file to a download server cluster through a public network inlet to generate a download address of the compressed file;

The downloading address is sent to the edge terminal, and the edge terminal is triggered to download the compressed file through the downloading address and to decompress and decrypt the compressed file to obtain the service file;

And mirroring the service file to an edge warehouse at the edge end through pushing.

Splitting the compressed file to obtain n subfiles;

Uploading n subfiles to a download server cluster through a public network inlet, and generating a download address of each subfile;

sending each download address to an edge terminal, triggering the edge terminal to download the sub-files through each download address, and performing decompression decryption and combination processing to obtain service files;

In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of:

acquiring a trigger time point of timing trigger;

Splitting the compressed file to obtain n subfiles;

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims

1. An edge cloud mirror data processing method, which is characterized by comprising the following steps:

The mirror image data transmission strategy data set comprises private network line strategy data and public network line strategy data, and a first time consuming time period required by the transmission of the service file from a central end to an edge warehouse of an edge end through a private network line is determined according to the file size of the service file and the private network bandwidth of the private network line strategy data;

Determining a second time-consuming time period required by the service file to be transmitted from a central end to a download server cluster through a public network line according to the file size of the service file and the public network bandwidth of the public network line policy data, wherein the second time-consuming time period is required by the service file to be transmitted to an edge warehouse of the edge end through the download server cluster;

based on the target mirror image data transmission strategy data, the service file is mirrored to an edge warehouse of the edge end through pushing;

If it is determined that the private network line policy data in the mirror image data transmission policy data set is the target mirror image data transmission policy data, the mirroring the service file to the edge repository of the edge end by pushing based on the target mirror image data transmission policy data includes: based on the private network line policy data, when the file size of the service file is in a first numerical value area, the service file is mirrored to an edge warehouse of the edge through pushing by triggering an idle time window of the edge at fixed time.

2. The method according to claim 1, wherein the idle time window has a corresponding idle start time point and idle end time point, and the mirroring the service file to the edge repository of the edge by pushing the space time window of the edge by timing triggering includes:

Acquiring a trigger time point of the timing trigger;

3. The method according to claim 2, wherein the method further comprises:

4. The method of claim 1, wherein the public network line policy data is determined to be target mirrored data transmission policy data when the first time period is longer than the second time period;

5. The method according to claim 4, wherein the method further comprises:

Splitting the compressed file to obtain n subfiles;

6. An edge cloud image data processing apparatus, the apparatus comprising:

The mirror image data transmission strategy data set comprises private network line strategy data and public network line strategy data, and a duration determining module is used for determining a first time-consuming duration required by an edge warehouse of the service file transmitted from a center end to an edge end through a private network line according to the file size of the service file and the private network bandwidth of the private network line strategy data; determining a second time-consuming time period required by the service file to be transmitted to an edge warehouse of the edge end through a download server from a central end to the download server cluster through a public network line according to the file size of the service file and the public network bandwidth of the public network line policy data;

The policy data determining module is used for determining that the private network line policy data is target mirror image data transmission policy data when the first time duration is smaller than the second time duration;

the data transmission module is used for mirroring the service file to the edge warehouse of the edge end by pushing based on the target mirror image data transmission policy data;

And if the private network line policy data in the mirror image data transmission policy data set is determined to be the target mirror image data transmission policy data, the data transmission module is further configured to, based on the private network line policy data, trigger the idle time window of the edge end to mirror the service file to the edge repository of the edge end by pushing when the file size of the service file is in the first numerical region.

7. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.

8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 5.