CN109189749B - File synchronization method and terminal equipment - Google Patents
File synchronization method and terminal equipment Download PDFInfo
- Publication number
- CN109189749B CN109189749B CN201811018211.2A CN201811018211A CN109189749B CN 109189749 B CN109189749 B CN 109189749B CN 201811018211 A CN201811018211 A CN 201811018211A CN 109189749 B CN109189749 B CN 109189749B
- Authority
- CN
- China
- Prior art keywords
- task
- synchronization
- data
- file
- synchronous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention is applicable to the technical field of data processing, and provides a file synchronization method, terminal equipment and a computer readable storage medium, comprising the following steps: identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment; if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment; and inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at the position to be executed of the task queue. The invention realizes the automation of the file synchronization of the internal and external network environment and improves the efficiency of the file synchronization.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a file synchronization method, a terminal device, and a computer readable storage medium.
Background
With the development of technology, the internet brings more and more convenience to people, but the openness of the internet cannot meet the confidentiality requirements of certain data and files of enterprises, so under the requirement of security construction, the enterprises generally divide the network into an intranet environment and an extranet environment, for example, the intranet environment is a local area network environment, and the extranet environment is a wide area network environment.
After the network environment is divided, there is a need for the data file to be generated from the intranet environment and to be released to the extranet environment, for example, after a developer develops a new service category in the intranet environment, the service category needs to be released to the extranet environment where the service person is located. In the prior art, in each link of publishing data from an intranet to an extranet, the configuration difficulty is high for intranet users unfamiliar with the publishing process, so that the efficiency of synchronizing the files of the intranet and the extranet in the prior art is low.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a file synchronization method, a terminal device, and a computer readable storage medium, so as to solve the problem in the prior art that efficiency of file synchronization between an intranet and an extranet is low.
A first aspect of an embodiment of the present invention provides a file synchronization method, including:
identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
And inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at the position to be executed of the task queue.
A second aspect of an embodiment of the present invention provides a terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
and inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at the position to be executed of the task queue.
A third aspect of the embodiments of the present invention provides a computer readable storage medium storing a computer program which when executed by a processor performs the steps of:
Identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
and inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at the position to be executed of the task queue.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
on the premise that the network environment is divided into an intranet environment and an extranet environment, the embodiment of the invention checks the synchronous identification in the configuration file of the intranet environment for the data file in the intranet environment, generates the synchronous task based on the data file and executes the synchronous task to synchronize the data file from the intranet environment to the extranet environment if the synchronous identification indicates that the data file is synchronized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a file synchronization method according to an embodiment of the present invention;
FIG. 2 is a flowchart of a file synchronization method according to a second embodiment of the present invention;
FIG. 3 is a flowchart of a file synchronization method according to a third embodiment of the present invention;
FIG. 4 is a flowchart of a file synchronization method according to a fourth embodiment of the present invention;
fig. 5 is a block diagram of a terminal device according to a fifth embodiment of the present invention;
fig. 6 is a schematic diagram of a terminal device according to a sixth embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to illustrate the technical scheme of the invention, the following description is made by specific examples.
Fig. 1 shows an implementation flow of a file synchronization method provided by an embodiment of the present invention, and details are as follows:
in S101, a synchronization identifier in a configuration file of an intranet environment is identified, where the synchronization identifier is used to indicate whether to perform synchronization on a data file stored in the intranet environment.
In the embodiment of the invention, the network environment is divided into an intranet environment and an extranet environment, wherein the intranet environment is equivalent to a local area network environment, and the extranet environment is equivalent to a wide area network environment. The internal network environment and the external network environment are isolated and communicated with each other: under the isolation condition, a host in an intranet environment and a host in an extranet environment cannot access each other, and file transmission cannot be realized; under the condition of communication, a host in an intranet environment and a host in an extranet environment can carry out file transmission. For example, the intranet environment stores a host of a developer, the extranet environment stores a host of a service person, and in the process of developing a new service by the developer, in order to ensure the security and confidentiality of the development process, the intranet environment and the extranet environment are isolated, so that the host in the intranet environment cannot be communicated with the host in the extranet environment. However, when the development work of the developer on the new service is completed and the new service needs to be sent to the service person, the communication operation is performed on the intranet environment and the extranet environment, so that the host in the intranet environment can send the file to the host in the extranet environment. In the embodiment of the invention, the isolation operation can be realized in various modes, such as disabling the authority of the host in the intranet environment to access the external network, so as to realize isolation; the method can also change the network message sent by the host in the intranet environment to the host in the extranet environment, so that the host in the extranet environment can not correctly answer the received network message, and isolation is realized as well. The communication operation corresponds to the isolation operation, for example, the isolation operation is to disable the host authority of the intranet environment, and the communication operation is to open the host authority of the intranet environment; the isolation operation is to change the network message sent by the intranet environment host, and the connection operation is not to change the network message sent by the intranet environment host.
Because file transmission can be performed only when the intranet environment and the extranet environment are in the communication condition, in the embodiment of the invention, whether the intranet environment and the extranet environment can communicate or not is detected, if the intranet environment and the extranet environment are detected to be in the communication condition, the intranet environment and the extranet environment are judged to be in the communication condition, and synchronous identification in configuration files of the intranet environment is started to be identified, so that file synchronization is performed. The operation of detecting whether the intranet environment is communicated with the extranet environment can be adjusted according to actual application scenes, for example, detection can be real-time detection, detection can be performed in a preset time period, and detection can be started when a storage medium in the intranet environment stores a data file. It should be noted that, in the embodiment of the present invention, the data file is a static file, and the type of the storage medium in which the data file is stored is not limited, for example, the storage medium may be a network attached storage (Network Attached Storage, NAS) or an Object storage system (Object-Based Storage System, OBS), and the storage medium in the intranet environment may be one or more.
The configuration file of the intranet environment refers to a configuration file (file suffix is usually properties) of an upper layer system for managing hosts of the intranet environment, and the upper layer system may be a content management system (Content Management System, CMS). And identifying a synchronization identifier in the configuration file, wherein the synchronization identifier is used for indicating whether to execute synchronization on the data file stored in the intranet environment, and the synchronization identifier can be set according to an actual application scene. For example, the first setting manner of the synchronization identifier is to set the synchronization identifier to a specific file suffix, such as ". Jpeg" suffix, ". Jpg" suffix and ". Png" suffix, which indicates that synchronization is performed only on a data file stored in the intranet environment that conforms to the specific file suffix, that is, when the file suffix of the data file stored in the intranet environment is any one of ". Jpeg" suffix, ". Jpg" suffix and ". Png" suffix, the synchronization identifier indicates that synchronization is performed on the data file; the second setting mode is that the synchronous identification is set as a name identification, and the synchronous identification is indicated to be executed only on the data files which are stored in the intranet environment and conform to the name identification, for example, the name identification can be 'to be synchronous', when a certain data file needs to be synchronous, the 'to be synchronous' is only added in the file name of the data file, and the data file is stored in the intranet environment, and then the upper layer system automatically executes the synchronous on the data file according to the synchronous identification in the configuration file; the third setting mode is to set the synchronization identifier as a specified directory, and instruct to perform synchronization only on the data files of the specified directory in the intranet environment. Of course, the above arrangement is merely exemplary, and is not limiting. In addition to the synchronization identification, a data file size limitation, an output file directory of the data file synchronized to the external network environment, a name of a storage medium managed by the configuration file, and the like may be configured in the configuration file. In addition, a file compression mechanism can be configured in the upper system, and when the data file is stored in a storage medium of an intranet environment, the data file is subjected to file compression, so that the capacity of the data file is reduced, and the system bandwidth is improved.
In S102, if the synchronization identifier indicates that synchronization is performed on the data file, a synchronization task based on the data file is generated, where the synchronization task is used to synchronize the data file to an external network environment.
If the synchronization identification in the configuration file of the upper system indicates that synchronization is performed on the data file in the intranet environment, a data file-based synchronization task is generated, and the synchronization task is used for synchronizing the data file to the extranet environment. It should be noted that, the synchronization task is determined according to a communication mechanism between the intranet environment and the extranet environment, for example, communication is performed between the intranet environment and the extranet environment through a hypertext transfer protocol (HyperText Transfer Protocol, HTTP), and then the synchronization task includes an HTTP request message carrying a data file.
Optionally, the data files of the generated synchronous tasks are recorded, a synchronous file set is generated, and an unsynchronized operation is performed on the data files corresponding to the synchronous file set. In the embodiment of the invention, for the data file which has generated the synchronous task, the data file is recorded, for example, the file name of the data file can be recorded, and the file name is added to the synchronous file set. When judging whether to execute synchronization on the data files in the intranet environment next time, matching the file names of the data files with the file names in the synchronized file set, and if the matching is successful, namely the file names identical to the data files exist in the synchronized file set, not executing the synchronization operation on the data files (whether the synchronization mark indicates that the data files are executed or not executed); if the matching fails, and based on the synchronization identification indicating the execution of the synchronization on the data file, the synchronization is executed on the data file, so that the repeated synchronization on the data file is prevented, and the effective utilization rate of the synchronization resource is improved.
In S103, the synchronization task is inserted into a preset task queue, and when the synchronization task is located at a position to be executed in the task queue, the synchronization task is executed.
Because there may be a plurality of data files to be synchronized in the intranet environment, there may be a plurality of generated synchronization tasks corresponding to the data files one by one, so in order to prevent errors occurring when a plurality of synchronization tasks are concurrently processed, in the embodiment of the present invention, the generated synchronization tasks are inserted into a preset task queue, and when the synchronization tasks are located at positions to be executed of the task queue, the synchronization tasks are taken out from the task queue to be executed, so as to promote the order of executing the plurality of synchronization tasks. It should be noted that the position to be executed may be freely set according to the actual application scenario, for example, may be set as the head of a task queue.
Optionally, the task queue is a Redis task queue. In the embodiment of the invention, the task queue can be a Redis task queue, wherein the Redis is a log type database and a key value type database based on a memory, and the Redis task queue in the embodiment of the invention is constructed based on a list data structure of the Redis. The Redis task queues are in a list form and are ordered according to the insertion sequence, the Redis task queues accord with the first-in first-out principle, the task execution mode based on the Redis task queues is generally to firstly take the task positioned at the head of the Redis task queues for execution, and the task positioned after the head of the Redis task queues is sequentially moved one bit in the Redis task queues, so that the process is repeated continuously. Therefore, after the synchronous task based on the data file is generated, the synchronous task is inserted into the tail of the Redis task queue, and when the synchronous task is arranged to the position to be executed of the Redis task queue, namely the head of the queue, the synchronous task is taken out for execution.
Optionally, if the synchronous task is successfully executed, the execution result is stored in a set based on Redis implementation. The set is a String type unordered set in the Redis, repeated data cannot occur in the unordered set, and the complexity of operations such as data addition, deletion and searching is low, so in the embodiment of the invention, if the result of executing the synchronization task is successful, that is, if the data file is successfully transmitted to the external network environment, the execution result is stored in the set, so that the synchronization condition of the data file is convenient to check.
As can be seen from the embodiment shown in fig. 1, in the embodiment of the present invention, by identifying a synchronization identifier in a configuration file of an intranet environment, where the synchronization identifier is used to indicate whether to perform synchronization on a data file stored in the intranet environment, if the synchronization identifier indicates to perform synchronization on the data file, a synchronization task based on the data file is generated, and the synchronization task is inserted into a preset task queue, and when the synchronization task is queued to a position to be performed in the task queue, the synchronization task is performed.
Fig. 2 shows an implementation method of the first embodiment of the present invention after refining a process of generating a synchronization task based on a data file. The embodiment of the invention provides a realization flow chart of a file synchronization method, as shown in fig. 2, the file synchronization method can comprise the following steps:
In S201, data compression is performed on the data file to obtain a first data digest, and the synchronization task based on the data file and the first data digest is generated, where the synchronization task is used to synchronize the data file and the first data digest to the external network environment.
In the process of transmitting the data file from the intranet environment to the extranet environment, data errors, data damage and the like may occur, so that a host of the extranet environment does not receive the correct data file finally. Therefore, in the embodiment of the present invention, after determining that the synchronization identifier indicates to perform synchronization on the data file, data compression is performed on the data file to obtain the first data digest, where data compression is not equivalent to the above file compression, and refers to compressing the content in the data file into the data digest with a fixed length. The data compression may be implemented based on a message digest algorithm (Message Digest Algorithm, MD) or a secure hash algorithm (Secure Hash Algorithm, SHA), etc., and it is worth mentioning that the data digests obtained by compressing the same content with the data are the same. After the first data digest is obtained, a synchronization task based on the data file and the first data digest is generated, the synchronization task being used to transmit the data file and the first data digest to an extranet environment.
Optionally, encrypting the data file and the first data digest to obtain an encrypted file, and generating the synchronization task based on the encrypted file, where the synchronization task is used to synchronize the encrypted file to the external network environment. In an actual application scenario, it may happen that multiple hosts in the external network environment can all establish communication with the hosts in the internal network environment, but only one or some hosts in the external network environment set the permission to receive the data file. Therefore, in view of the above situation, after the first data digest is calculated in the intranet environment, the data file and the first data digest are encrypted by a preset encryption algorithm to obtain an encrypted file, and a synchronization task based on the encrypted file is generated, where the synchronization task is used to synchronize the encrypted file to the extranet environment, and in correspondence with the intranet environment, a decryption key is distributed in advance for a host in the extranet environment, where receiving authority needs to be configured, and the decryption key is generated by the encryption algorithm. After the host in the intranet environment and the host in the extranet environment perform the synchronous task and transmit the encrypted file to the host in the extranet environment, if the host in the extranet environment is a host with receiving authority, that is, a decryption key is stored, the encrypted file can be decrypted to obtain a data file and a first data abstract; if the host computer in the external network environment does not have the receiving authority, namely the decryption key is not stored, the encrypted file cannot be decrypted, and the security of the data file is improved. It should be noted that the encryption algorithm is not limited by the embodiment of the present invention, for example, the encryption algorithm may be an RSA encryption algorithm, a data encryption standard (Data Encryption Standard, DES) algorithm, an advanced encryption standard (Advanced Encryption Standard, AES) algorithm, or the like.
In S202, after performing data compression on the data file in the extranet environment to obtain a second data digest, a result of matching the first data digest and the second data digest is obtained.
After the host computer in the intranet environment executes the synchronization task based on the data file and the first data abstract, that is, after the host computer in the extranet environment receives the data file, the host computer in the extranet environment compresses the data file to obtain the second data abstract. After the second data abstract is obtained, the first data abstract and the second data abstract are matched in an extranet environment, and the matched result is transmitted to the intranet environment.
In S203, if the obtained result of matching the first data digest and the second data digest is different, the data file is re-synchronized.
In the embodiment of the invention, the first data abstract can be temporarily stored in a storage medium of an intranet environment, and if the result of matching the first data abstract with the second data abstract obtained in the intranet environment is the same, the data file received in an extranet environment is proved to be correct, the first data abstract is deleted in the storage medium of the intranet environment; if the result of matching the first data abstract with the second data abstract obtained in the intranet environment is different, and the data file received by the extranet environment is proved to be different from the data file in the intranet environment, a synchronous task is directly generated based on the data file stored in the intranet environment and the first data abstract so as to resynchronize the data file, and the data file is not compressed again in the intranet environment to obtain the first data abstract.
As can be seen from the embodiment shown in fig. 2, in the embodiment of the present invention, after the synchronization identifier indicates that synchronization is performed on the data file, the data file is compressed to obtain a first data digest, a synchronization task based on the data file and the first data digest is generated, after the synchronization task is performed to transmit the data file and the first data digest to the external network environment, the data file is compressed in the external network environment to obtain a second data digest, the first data digest and the second data digest are matched, and the matched result is sent back to the internal network environment, if the matched result received by the internal network environment is different, the data file is re-synchronized.
Fig. 3 shows an implementation method of the present invention after refining the process after executing the synchronization task. The embodiment of the invention provides a realization flow chart of a file synchronization method, as shown in fig. 3, the file synchronization method can comprise the following steps:
in S301, a result of execution of the synchronization task is acquired.
When executing the synchronous task, the task may fail to execute due to the influence of some factors, such as communication fluctuation between the host in the intranet environment and the host in the extranet environment, so in the embodiment of the invention, an independent log file is set, the execution process of the synchronous task is monitored through the log file, and the execution result of the synchronous task recorded by the log file is obtained. For example, if the sync task includes an HTTP request message, the log file is set to record the HTTP message transferred between the captured intranet environment and the captured extranet environment, and the execution result of the sync task is determined according to the captured HTTP message, for example, the captured HTTP message includes a specified HTTP status code, for example, 400 (indicating an error request), 403 (indicating a reject request), and the execution result of the sub-sync task is determined as an execution failure.
In S302, if the execution result is that the execution fails, the executed times of the synchronization task are obtained.
In the embodiment of the invention, the executed times are preset in the log file, the executed times are set to be zero in the initialization, and when the execution result of the synchronous task is obtained to be the execution failure, the executed times in the log file are added with one operation, and the current executed times are obtained. In addition, if the execution result of the synchronous task is that the execution is successful, the completion of the operation for executing the synchronous task is recorded in the log file.
In S303, if the executed number of times is less than or equal to a preset threshold, the synchronization task is inserted into the task queue, and the synchronization task is re-executed.
In the embodiment of the invention, a retry mechanism is set, in the retry mechanism, if the executed times are smaller than or equal to a preset threshold value, the synchronization of the data files is carried out again, specifically, the synchronous task with failed execution is reserved, the synchronous task is reinserted into a task queue, and when the synchronous task is positioned at the position to be executed of the task queue, the synchronous task is taken out from the task queue for execution.
In S304, if the executed number is greater than the preset threshold, an alarm prompt is output.
In the retry mechanism, if the current executed times are greater than a preset threshold, outputting an alarm prompt based on the synchronous task to prompt a user to check. In addition, the synchronous task is temporarily saved, and when the saved duration is greater than or equal to the preset saving duration, the synchronous task is deleted, so that the storage space is saved.
As can be seen from the embodiment shown in fig. 3, in the embodiment of the present invention, after the synchronous task is executed, an execution result of the synchronous task is obtained, if the execution result is that the execution fails, the executed times of the synchronous task are obtained, if the executed times are less than or equal to a preset threshold, the synchronous task is inserted into a task queue, the synchronous task is re-executed, and if the executed times are greater than the preset threshold, an alarm prompt is output.
Fig. 4 shows an implementation method of the first embodiment of the present invention after refining a process of inserting a synchronous task into a preset task queue. The embodiment of the invention provides a realization flow chart of a file synchronization method, as shown in fig. 4, the file synchronization method can comprise the following steps:
in S401, it is detected whether the synchronization task is generated at every preset time interval.
Since there may be a plurality of data files to be synchronized and one synchronization task is generated based on each data file, in the case that there are many synchronization tasks to be performed, execution blocking easily occurs, resulting in low synchronization efficiency. In the embodiment of the invention, the synchronous task is executed through a thread pool in a Java software development kit (Java Development Kit, JDK), and the thread pool can be realized through a ThreadPoolExecutor class in Java. The thread pool comprises a plurality of execution threads, and in the process of executing the synchronous task, one execution thread in the thread pool is firstly set as a detection thread and is specially used for detecting whether the synchronous task is generated or not at preset time intervals.
In S402, if it is detected that a plurality of synchronous tasks are generated, one execution thread is allocated to each synchronous task.
If the detecting thread detects that the synchronous task is generated, one executing thread in the thread pool is allocated for the synchronous task. Correspondingly, if the detecting thread detects that a plurality of synchronous tasks are generated, one executing thread in the thread pool is allocated to each detected synchronous task. It is worth mentioning that the detection threads in the thread pool are not used for allocation.
Optionally, a thread queue is created in the thread pool, and when all execution threads in the thread pool are allocated and there are unassigned synchronous tasks, the unassigned synchronous tasks are inserted into the thread queue. In an actual application scenario, the number of execution threads in a thread pool is limited and may not be matched with a plurality of synchronization tasks one by one, so in the embodiment of the invention, when all execution threads in the thread pool are allocated, and unallocated synchronization tasks still exist, the unallocated synchronization tasks are inserted into a preset thread queue. When idle execution threads are detected to exist, idle execution threads are allocated to synchronous tasks in a thread queue, and the thread allocation order is improved.
Optionally, when the number of synchronous tasks in the thread queue is greater than or equal to a preset number threshold, the thread Cheng Chizhong is newly built on line and the newly built thread is allocated for the synchronous tasks. In the embodiment of the invention, a number threshold value can be set for the thread queue, and when the number of the synchronous tasks in the thread queue is greater than or equal to the number threshold value and the synchronous tasks to be added to the thread queue exist, the operation of the newly built execution thread is executed in the thread pool. For newly built execution threads, two allocation modes exist, the first is to allocate the newly built execution threads for the synchronous tasks in the thread queue (the newly built execution threads can be allocated for the synchronous tasks positioned at the position to be executed in the thread queue) and add the synchronous tasks to be added to the thread queue; the other is to directly allocate a new execution thread for the synchronous task to be added to the thread queue. According to the method, when the synchronous tasks of the thread queues are too many, the concurrent processing pressure of the synchronous tasks is reduced through newly built execution threads.
In S403, the synchronization task is inserted into the task queue during the execution of the execution thread.
And for the execution thread allocated to the synchronous task, inserting the synchronous task into a task queue in the execution process, and taking out the synchronous task from the task queue by the execution thread for execution when the synchronous task is arranged at a position to be executed of the task queue.
Optionally, acquiring the idle time of the execution thread in the thread pool, and deleting the execution thread in the thread pool when the idle time is greater than or equal to a preset idle threshold. For each execution thread in the thread pool, acquiring the idle time length of the execution thread in the idle state, wherein the acquiring operation can be executed at intervals of preset acquiring intervals. When the idle time length is obtained, the idle time length is compared with a preset idle threshold (for example, 24 hours), if the idle time length is greater than or equal to the idle threshold, the execution thread is deleted on line Cheng Chizhong, so that the maintenance pressure of the execution thread is reduced, and the effective utilization rate of resources is improved.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
Fig. 5 shows a block diagram of a terminal device according to an embodiment of the present invention, where the terminal device includes units for performing the steps in the corresponding embodiment of fig. 1. Please refer to fig. 1 and the related description of the embodiment corresponding to fig. 1. For convenience of explanation, only the portions related to the present embodiment are shown.
Referring to fig. 5, the terminal device includes:
the identifying unit 51 is configured to identify a synchronization identifier in a configuration file of an intranet environment, where the synchronization identifier is used to indicate whether to perform synchronization on a data file stored in the intranet environment;
a generating unit 52, configured to generate a synchronization task based on the data file if the synchronization identifier indicates that synchronization is performed on the data file, where the synchronization task is used to synchronize the data file to an external network environment;
and the execution unit 53 is configured to insert the synchronization task into a preset task queue, and execute the synchronization task when the synchronization task is located at a position to be executed in the task queue.
Optionally, the generating unit 52 includes:
the first compression unit is used for carrying out data compression on the data file to obtain a first data abstract, and generating the synchronization task based on the data file and the first data abstract, wherein the synchronization task is used for synchronizing the data file and the first data abstract to the extranet environment;
The execution unit 53 further includes:
the second compression unit is used for obtaining a result of matching the first data abstract and the second data abstract after carrying out data compression on the data file in the extranet environment to obtain the second data abstract;
and the resynchronization unit is used for resynchronizing the data file if the obtained result of matching the first data abstract and the second data abstract is different.
Optionally, the first compression unit includes:
the encryption unit is used for encrypting the data file and the first data abstract to obtain an encrypted file, and generating the synchronization task based on the encrypted file, wherein the synchronization task is used for synchronizing the encrypted file to the external network environment.
Optionally, the execution unit 53 further includes:
the first acquisition unit is used for acquiring an execution result of the synchronous task;
the second acquisition unit is used for acquiring the executed times of the synchronous task if the execution result is that the execution fails;
the re-executing unit is used for inserting the synchronous task into the task queue and re-executing the synchronous task if the executed times are smaller than or equal to a preset threshold value;
And the output unit is used for outputting an alarm prompt if the executed times are greater than the preset threshold value.
Optionally, the execution unit 53 includes:
the detection unit is used for detecting whether the synchronous task is generated or not at preset time intervals;
the allocation unit is used for allocating an execution thread to each synchronous task if a plurality of synchronous tasks are detected to be generated;
and the inserting unit is used for inserting the synchronous task into the task queue in the execution process of the execution thread.
Therefore, the terminal equipment provided by the embodiment of the invention generates the synchronization task based on the data file by identifying the synchronization identifier in the intranet environment, and executing the synchronization task to synchronize if the synchronization identifier indicates to synchronize the data file in the intranet environment, thereby improving the efficiency and the automation degree of the data file synchronization.
Fig. 6 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 6, the terminal device 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62, such as a file synchronization program, stored in the memory 61 and executable on the processor 60. The processor 60, when executing the computer program 62, implements the steps of the respective file synchronization method embodiments described above, such as steps S101 to S103 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the units in the embodiments of the terminal devices described above, for example, the functions of the units 51 to 53 shown in fig. 5.
Illustratively, the computer program 62 may be partitioned into one or more units that are stored in the memory 61 and executed by the processor 60 to complete the present invention. The one or more units may be a series of computer program instruction segments capable of performing a specific function for describing the execution of the computer program 62 in the terminal device 6. For example, the computer program 62 may be divided into an identification unit, a generation unit and an execution unit, each unit functioning specifically as follows:
the device comprises an identification unit, a synchronization unit and a synchronization unit, wherein the identification unit is used for identifying a synchronization identifier in a configuration file of an intranet environment, and the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
the generation unit is used for generating a synchronization task based on the data file if the synchronization identification indicates to perform synchronization on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
and the execution unit is used for inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at the position to be executed of the task queue.
The terminal device 6 may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The terminal device may include, but is not limited to, a processor 60, a memory 61. It will be appreciated by those skilled in the art that fig. 6 is merely an example of the terminal device 6 and does not constitute a limitation of the terminal device 6, and may include more or less components than illustrated, or may combine certain components, or different components, e.g., the terminal device may further include an input-output device, a network access device, a bus, etc.
The processor 60 may be a central processing unit (Central Processing Unit, CPU), other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or a memory of the terminal device 6. The memory 61 may be an external storage device of the terminal device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used for storing the computer program and other programs and data required by the terminal device. The memory 61 may also be used for temporarily storing data that has been output or is to be output.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units is illustrated, and in practical application, the above-mentioned functional allocation may be performed by different functional units, that is, the internal structure of the terminal device is divided into different functional units, so as to perform all or part of the above-mentioned functions. The functional units in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present application. The specific working process of the units in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed terminal device and method may be implemented in other manners. For example, the above-described terminal device embodiments are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (5)
1. A file synchronization method, characterized in that the file synchronization method comprises:
identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at a position to be executed of the task queue;
The generating the synchronous task based on the data file comprises the following steps:
performing data compression on the data file to obtain a first data abstract, and generating the synchronization task based on the data file and the first data abstract, wherein the synchronization task is used for synchronizing the data file and the first data abstract to the extranet environment;
after the synchronization task is executed, the method further comprises:
after data compression is carried out on the data file in the extranet environment to obtain a second data abstract, a result of matching the first data abstract with the second data abstract is obtained;
if the obtained result of matching the first data abstract and the second data abstract is different, resynchronizing the data file;
the generating the synchronization task based on the data file and the data digest includes:
encrypting the data file and the first data abstract to obtain an encrypted file, and generating the synchronous task based on the encrypted file, wherein the synchronous task is used for synchronizing the encrypted file to the external network environment;
the step of inserting the synchronous task into a preset task queue comprises the following steps:
Detecting whether the synchronous task is generated or not at preset time intervals;
if a plurality of synchronous tasks are detected to be generated, distributing an execution thread to each synchronous task;
and inserting the synchronous task into the task queue in the executing process of the executing thread.
2. The file synchronization method as claimed in claim 1, wherein when the synchronization task is located at a position to be executed in the task queue, after executing the synchronization task, further comprising:
acquiring an execution result of the synchronous task;
if the execution result is that the execution fails, acquiring the executed times of the synchronous task;
if the executed times are smaller than or equal to a preset threshold value, inserting the synchronous task into the task queue and re-executing the synchronous task;
and if the executed times are greater than the preset threshold, outputting an alarm prompt.
3. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
Identifying a synchronization identifier in a configuration file of an intranet environment, wherein the synchronization identifier is used for indicating whether to execute synchronization on a data file stored in the intranet environment;
if the synchronization identifier indicates to execute synchronization on the data file, generating a synchronization task based on the data file, wherein the synchronization task is used for synchronizing the data file to an external network environment;
inserting the synchronous task into a preset task queue, and executing the synchronous task when the synchronous task is positioned at a position to be executed of the task queue;
the generating the synchronous task based on the data file comprises the following steps:
performing data compression on the data file to obtain a first data abstract, and generating the synchronization task based on the data file and the first data abstract, wherein the synchronization task is used for synchronizing the data file and the first data abstract to the extranet environment;
after the synchronization task is executed, the method further comprises:
after data compression is carried out on the data file in the extranet environment to obtain a second data abstract, a result of matching the first data abstract with the second data abstract is obtained;
If the obtained result of matching the first data abstract and the second data abstract is different, resynchronizing the data file;
the generating the synchronization task based on the data file and the data digest includes:
encrypting the data file and the first data abstract to obtain an encrypted file, and generating the synchronous task based on the encrypted file, wherein the synchronous task is used for synchronizing the encrypted file to the external network environment;
the step of inserting the synchronous task into a preset task queue comprises the following steps:
detecting whether the synchronous task is generated or not at preset time intervals;
if a plurality of synchronous tasks are detected to be generated, distributing an execution thread to each synchronous task;
and inserting the synchronous task into the task queue in the executing process of the executing thread.
4. The terminal device of claim 3, wherein after executing the synchronization task when the synchronization task is located at a position to be executed of the task queue, further comprising:
acquiring an execution result of the synchronous task;
if the execution result is that the execution fails, acquiring the executed times of the synchronous task;
If the executed times are smaller than or equal to a preset threshold value, inserting the synchronous task into the task queue and re-executing the synchronous task;
and if the executed times are greater than the preset threshold, outputting an alarm prompt.
5. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the file synchronization method according to any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811018211.2A CN109189749B (en) | 2018-09-03 | 2018-09-03 | File synchronization method and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811018211.2A CN109189749B (en) | 2018-09-03 | 2018-09-03 | File synchronization method and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109189749A CN109189749A (en) | 2019-01-11 |
| CN109189749B true CN109189749B (en) | 2023-08-18 |
Family
ID=64917620
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811018211.2A Active CN109189749B (en) | 2018-09-03 | 2018-09-03 | File synchronization method and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189749B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111478968A (en) * | 2020-04-08 | 2020-07-31 | 腾讯科技（深圳）有限公司 | Data processing method and related device |
| CN111753012A (en) * | 2020-06-19 | 2020-10-09 | 北京字节跳动网络技术有限公司 | Data synchronization method, device, equipment and storage medium |
| CN112000741B (en) * | 2020-08-25 | 2024-05-03 | 中国南方电网有限责任公司 | Internal and external network data exchange system, method, device, computer equipment and medium |
| CN112527756A (en) * | 2020-12-17 | 2021-03-19 | 厦门市美亚柏科信息股份有限公司 | File synchronization operation method, terminal equipment and storage medium |
| CN113011798B (en) * | 2021-05-24 | 2021-08-13 | 江苏荣泽信息科技股份有限公司 | Product detection information processing system based on block chain |
| CN114124431B (en) * | 2021-09-13 | 2024-09-06 | 许昌许继软件技术有限公司 | Real-time automatic file synchronization method and system for isolating device |
| CN114006941B (en) * | 2021-10-30 | 2024-07-09 | 平安国际智慧城市科技股份有限公司 | Data synchronization method, device, computer equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106227822A (en) * | 2016-07-22 | 2016-12-14 | 深圳市中润四方信息技术有限公司 | A kind of data distributing method and data synchronous system |
| CN107609061A (en) * | 2017-08-28 | 2018-01-19 | 武汉奇米网络科技有限公司 | A kind of method and apparatus of data syn-chronization |
| CN107634984A (en) * | 2017-08-07 | 2018-01-26 | 国网河南省电力公司 | A kind of file synchronisation method based on one-way transmission path |
| CN108306847A (en) * | 2017-01-13 | 2018-07-20 | 北京国双科技有限公司 | Communication system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101574339B1 (en) * | 2008-04-28 | 2015-12-03 | 엘지전자 주식회사 | Method and apparatus for synchronizing a data between a mobile communication terminal and a TV |
-
2018
- 2018-09-03 CN CN201811018211.2A patent/CN109189749B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106227822A (en) * | 2016-07-22 | 2016-12-14 | 深圳市中润四方信息技术有限公司 | A kind of data distributing method and data synchronous system |
| CN108306847A (en) * | 2017-01-13 | 2018-07-20 | 北京国双科技有限公司 | Communication system and method |
| CN107634984A (en) * | 2017-08-07 | 2018-01-26 | 国网河南省电力公司 | A kind of file synchronisation method based on one-way transmission path |
| CN107609061A (en) * | 2017-08-28 | 2018-01-19 | 武汉奇米网络科技有限公司 | A kind of method and apparatus of data syn-chronization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109189749A (en) | 2019-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109189749B (en) | File synchronization method and terminal equipment | |
| CN114338651B (en) | File transmission method, device, electronic equipment and readable storage medium | |
| US20140229738A1 (en) | Timestamping system and timestamping program | |
| AU2013243923A1 (en) | Systems and methods for securing and restoring virtual machines | |
| CN109739690B (en) | Backup method and related products | |
| US10802888B2 (en) | Information processing device and cooperative distributed storage system | |
| CN110865905A (en) | Data restoration method and device, computer equipment and storage medium | |
| CN112104627B (en) | Block chain-based data transmission method and device, electronic equipment and storage medium | |
| CN103064757A (en) | Method and system for backing up data | |
| CN112422368A (en) | Database cluster monitoring method and system based on clickhouse | |
| CN112131041A (en) | Method, apparatus and computer program product for managing data placement | |
| CN113726743A (en) | Method, device, equipment and medium for detecting network replay attack | |
| CN112163036A (en) | Block chain information construction and query method and related device | |
| CN114363094B (en) | Data sharing method, device, equipment and storage medium | |
| CN109684860B (en) | Data encryption method and device based on business relation | |
| CN113347035B (en) | Ethernet switch configuration persistence method and device | |
| CN111510304B (en) | Information transmission and information management method, system and device and electronic equipment | |
| CN113472715B (en) | Data transmission method and device | |
| WO2023145057A1 (en) | File exchanging device, system, method, and program | |
| CN112926070A (en) | Domestic CPU and operating system based non-secret-related domain and secret-related domain official document exchange method and system | |
| CN112800130A (en) | Data uplink method, system, device, equipment and storage medium | |
| CN114584556B (en) | File transmission method and device | |
| CN117667788B (en) | Data interaction method, computer system, electronic device and storage medium | |
| CN111090547A (en) | Data backup processing method and device | |
| CN104579741A (en) | Business management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |