CN111198793B - Data deleting method and system, medium and computer system - Google Patents
Data deleting method and system, medium and computer system Download PDFInfo
- Publication number
- CN111198793B CN111198793B CN201811381811.5A CN201811381811A CN111198793B CN 111198793 B CN111198793 B CN 111198793B CN 201811381811 A CN201811381811 A CN 201811381811A CN 111198793 B CN111198793 B CN 111198793B
- Authority
- CN
- China
- Prior art keywords
- data
- task
- information
- cluster
- deletion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012217 deletion Methods 0.000 claims abstract description 107
- 230000037430 deletion Effects 0.000 claims abstract description 107
- 230000015654 memory Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000012544 monitoring process Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000004140 cleaning Methods 0.000 description 3
- 230000004083 survival effect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009472 formulation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Stored Programmes (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure provides a data deletion method, including: acquiring configuration information, wherein the configuration information is at least used for representing data to be deleted; generating at least one deletion task based on the configuration information; and distributing at least one deletion task to the corresponding client, wherein the client can execute the deletion task and delete the data to be deleted, which is characterized by the configuration information. The present disclosure also provides a data deletion system, a computer system, and a computer-readable storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technology, and more particularly, to a data deletion method, a data deletion system, a computer system, and a computer-readable storage medium.
Background
In the big data age, data storage is quite important. Data based on elastic search (hereinafter ES) stores fast response queries while also supporting high concurrent writing, but over time, more and more data is stored, requiring periodic cleaning of expired data. The ES 1.7 version supports the TTL cleaning mode, but the ES 2.0 version and above do not support the cleaning mode any more, and the user needs to perform processing according to the actual situation. In this regard, most users currently take development timing tasks to delete expired data periodically.
In carrying out the disclosed concept, the inventors have found that the following drawbacks exist in the related art: the development of timing tasks requires users to manually write the worker deletion data on a hard-coded basis for each service, which is inconvenient for dynamic adjustment.
Disclosure of Invention
In view of this, the present disclosure provides a data deletion method and system capable of automatically generating a data deletion task based on configuration information of a user.
One aspect of the present disclosure provides a data deletion method, including: acquiring configuration information, wherein the configuration information is at least used for representing data to be deleted; generating at least one deletion task based on the configuration information; and distributing the at least one deletion task to a corresponding terminal device, wherein the terminal device can execute the deletion task and delete the data to be deleted, which is characterized by the configuration information.
According to an embodiment of the present disclosure, the generating at least one deletion task based on the configuration information includes: analyzing the configuration information to determine the following information of the data to be deleted: the name of the ES cluster and the IP address of the node for storing the data to be deleted in the ES cluster; and generating at least one deletion task capable of deleting data corresponding to the information and which has currently expired based on the determined information.
According to an embodiment of the present disclosure, the generating at least one deletion task based on the configuration information includes: analyzing the configuration information to determine the following information of the data to be deleted: the name of the ES cluster, the IP address of the node for storing the data to be deleted in the ES cluster and the appointed time parameter; and generating at least one deletion task capable of deleting data corresponding to the information and having a time-to-live value smaller than the specified time parameter based on the determined information.
According to an embodiment of the present disclosure, the above method further includes: acquiring state information of the ES cluster; generating a task adjustment instruction according to the acquired state information; and adjusting the execution pace of the at least one deletion task on the corresponding terminal device through the task adjustment instruction.
According to an embodiment of the present disclosure, the status information includes one or more of the following information of the ES cluster: load status, memory status, CPU status, and disk status.
According to an embodiment of the present disclosure, the assigning the at least one deletion task to a corresponding terminal device includes: determining the number of available terminal devices and the speed of executing tasks, and the number of the at least one deletion task; and assigning the at least one deletion task to the available terminal device according to the determined number and speed.
Another aspect of the present disclosure provides a data deletion system, including: the first acquisition module is used for acquiring configuration information, wherein the configuration information is at least used for representing data to be deleted; the first generation module is used for generating at least one deletion task based on the configuration information; and an allocation module, configured to allocate the at least one deletion task to a corresponding terminal device, where the terminal device is capable of executing the deletion task and deleting data to be deleted, where the data is represented by the configuration information.
According to an embodiment of the present disclosure, the first generating module includes: the first analyzing unit is used for analyzing the configuration information and determining the following information of the data to be deleted: the name of the ES cluster and the IP address of the node for storing the data to be deleted in the ES cluster; and a first generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data that corresponds to the information and that has currently expired.
According to an embodiment of the present disclosure, the first generating module includes: the second parsing unit is used for parsing the configuration information and determining the following information of the data to be deleted: the name of the ES cluster, the IP address of the node for storing the data to be deleted in the ES cluster and the appointed time parameter; and a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a time-to-live value smaller than the specified time parameter.
According to an embodiment of the present disclosure, the above system further includes: the second acquisition module is used for acquiring the state information of the ES cluster; the second generation module is used for generating a task adjustment instruction according to the acquired state information; and the adjusting module is used for adjusting the execution pace of the at least one deletion task on the corresponding terminal equipment through the task adjusting instruction.
According to an embodiment of the present disclosure, the status information includes one or more of the following information of the ES cluster: load status, memory status, CPU status, and disk status.
According to an embodiment of the present disclosure, the above-mentioned distribution module includes: a determining unit for determining the number of available terminal devices and the speed of executing the tasks, and the number of the at least one deletion task; and an allocation unit configured to allocate the at least one deletion task to the available terminal device according to the determined number and speed.
Another aspect of the present disclosure provides a computer system comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of the preceding claims.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.
Another aspect of the present disclosure provides a computer program comprising computer executable instructions which, when executed, are adapted to carry out the method as described above.
According to the embodiment of the disclosure, because the technical means of setting the Web UI interface for the user to input the configuration information and further enabling the client to automatically generate the data deleting task based on the configuration information output by the user is adopted, the technical problem that when the data stored in the ES is deleted in the related art, the user needs to manually write the hard coding timing task for each service, so that the dynamic adjustment is inconvenient is solved, and the technical effect that the corresponding deleting task can be automatically generated for each service is achieved, so that the dynamic adjustment is convenient is at least partially overcome.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:
FIG. 1 schematically illustrates an exemplary system architecture to which the data deletion methods and systems of the present disclosure may be applied;
FIG. 2 schematically illustrates a flow chart of a data deletion method according to an embodiment of the present disclosure;
fig. 3A to 3B schematically illustrate the composition of configuration information according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates the composition of state information for an ES cluster in accordance with an embodiment of the disclosure;
FIG. 5 schematically illustrates a schematic diagram of dynamically adjusting execution pace of tasks according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a data deletion system according to an embodiment of the present disclosure; and
Fig. 7 schematically illustrates a block diagram of a computer system suitable for implementing the data deletion method and system according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides a data deleting method and a system capable of automatically generating a data deleting task based on configuration information of a user. The method comprises the steps of obtaining configuration information, wherein the configuration information is at least used for representing data to be deleted; generating at least one deletion task based on the configuration information; and distributing at least one deletion task to the corresponding client, wherein the client can execute the deletion task and delete the data to be deleted, which is characterized by the configuration information.
Fig. 1 schematically illustrates an exemplary system architecture to which the data deletion methods and systems of the present disclosure may be applied. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.
As shown in fig. 1, the system architecture according to this embodiment may include an ES cluster, a WEB-UI configuration center, a task generator, terminals 1 and 2, a monitoring center, and a coordinator.
The ES cluster is used to store data and may include a plurality of nodes, each having its own IP address. The WEB-UI configuration center is used for providing a WEB-UI interface through which a user can input configuration information. The task generator may be a client, a server or a server cluster, and is configured to automatically generate a deletion task according to the configuration information, and assign the deletion task to the corresponding terminal devices, such as the terminal 1 and the terminal 2, for execution. The monitoring center may be a client, a server or a server cluster, and is configured to monitor the status of the ES cluster, and send the monitored status information to the coordinator. The coordinator may be a client, a server or a server cluster, for coordinating the execution pace of the deletion task according to the state information of the ES cluster.
The WEB-UI configuration center, the task generator, the monitoring center, and the coordinator may be one device, and specifically, may be one device having an information configuration function, a task generation function, a monitoring function, and a coordination function at the same time. Of course, the WEB-UI configuration center, task generator, monitoring center, and coordinator may also be four different devices.
It should be noted that, the data deletion method provided by the embodiments of the present disclosure may be generally performed by a server/server cluster. Accordingly, the data deletion system provided by embodiments of the present disclosure may be generally disposed in a server/server cluster. Or the data deleting method provided by the embodiment of the disclosure may also be executed by the terminal device. Accordingly, the data deletion system provided by the embodiment of the present disclosure may also be provided in the terminal device.
It should be understood that the number of ES clusters, WEB-UI configuration centers, task generators, terminal devices, monitoring centers, coordinators in fig. 1 are merely illustrative. According to the implementation requirement, the system can be provided with any number of ES clusters, WEB-UI configuration centers, task generators, terminal equipment, monitoring centers and coordinators.
Fig. 2 schematically illustrates a flow chart of a data deletion method according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S230, wherein:
in operation S210, configuration information is acquired, where the configuration information is at least used to characterize data that needs to be deleted.
Specifically, the user may input configuration information such as the name of the ES cluster and the IP address of any one node thereof in the configuration table through the WEB-UI configuration center as shown in fig. 1, and the task generator periodically scans the configuration table input by the user through the timed task and reads meta information of the ES cluster (including the name of the ES cluster and the IP address of the relevant node thereof) according to the configuration information.
In operation S220, at least one deletion task is generated based on the configuration information.
Specifically, as shown in fig. 1, assuming that the user inputs configuration information as shown in table 1 in the configuration table, the task generator may automatically generate two deletion tasks, namely task1 and task2, according to the information, where performing task1 may delete relevant data on node 1 corresponding to the IP1 address in the es_name1 cluster, and performing task2 may delete relevant data on node 2 corresponding to the IP2 address in the es_name1 cluster.
TABLE 1
| Names of ES clusters | IP of node | Time parameter |
| ES_namel | IP1 | |
| ES_namel | IP2 |
It should be noted that, in the embodiment of the present disclosure, the task generator may specifically automatically generate the deletion task by using a technology of dynamically generating byte codes (mainly javaassit, CGLIB).
In operation S230, at least one deletion task is assigned to a corresponding terminal device, wherein the terminal device is capable of executing the deletion task and deleting data to be deleted, which is characterized by the configuration information.
Specifically, as shown in fig. 1, the task generator may randomly assign the deletion task to any one or more terminal device processes in the system architecture. For example, task1 may be assigned to terminal 1, task2 may be assigned to terminal 2, or both task1 and task2 may be assigned to terminal 1, or both task1 and task2 may be assigned to terminal 2.
When the terminal device executes task1, the node 1 corresponding to the IP1 address in the es_name1 cluster can be operated, so that the related data stored in the node 1 can be deleted; when the terminal device executes task2, the node 2 corresponding to the IP2 address in the es_name2 cluster can be operated, so as to delete the relevant data stored therein.
Compared with the prior art that when deleting the data stored in the ES cluster, a user is required to manually write a hard coding timing deletion task for each service, so that the task is inconvenient to dynamically adjust, the embodiment of the disclosure provides a visual WEB-UI, the user can input configuration information, a task generator can automatically generate the timing deletion task, the trouble of manual coding of the user is saved, and the task is convenient to dynamically adjust.
The method of fig. 2 is further described with reference to fig. 3A-3B, and fig. 4-5, in conjunction with specific embodiments.
As an alternative embodiment, generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data to be deleted: the name of the ES cluster and the IP address of the node used for storing the data to be deleted in the ES cluster; and generating at least one deletion task capable of deleting data corresponding to the information and which has currently expired based on the determined information.
As shown in fig. 3A, the configuration information may include only the name of the ES cluster and the IP address of the relevant node in the ES cluster. In other words, the user may not configure the time parameter. In this case, the task generator sets the time parameter as a default parameter, i.e., defaults to delete data that has currently expired, when generating the delete task.
Through the embodiment of the disclosure, a user can only specify a deletion position (namely, the name of the ES cluster and the IP address of the related node therein) when configuring information, does not specify which data to delete specifically (namely, does not specify a time parameter), and at this time, the data which has expired currently can be deleted by default according to rules.
As an alternative embodiment, generating at least one deletion task based on the configuration information includes: analyzing the configuration information, and determining the following information of the data to be deleted: the name of the ES cluster, the IP address of the node for storing the data to be deleted in the ES cluster, and the designated time parameter; and generating at least one deletion task capable of deleting data corresponding to the information and having a time-to-live value less than the specified time parameter based on the determined information.
Slightly different from the configuration information shown in fig. 3A, the configuration information may include a user-specified time parameter in addition to the name of the ES cluster and the IP address of the relevant node in the ES cluster, as shown in fig. 3B. In other words, the user may configure the time parameter. In this case, the task generator sets the time parameter to the user-specified time parameter when generating the deletion task, i.e., defaults to delete data whose survival time is earlier than the time parameter.
By the embodiment of the disclosure, a user can specify a deletion position (namely, the name of the ES cluster and the IP address of the related node therein) and specify which data to delete specifically (namely, specify a time parameter) when configuring information, and at this time, the data with survival time earlier than the time parameter can be deleted by default according to rules.
As an alternative embodiment, the method further comprises: acquiring state information of an ES cluster; generating a task adjustment instruction according to the acquired state information; and adjusting the execution pace of at least one deletion task on the corresponding terminal equipment through the task adjustment instruction.
Specifically, as shown in fig. 1, the monitoring center may capture the state of the ES cluster in real time, including one or more of the load (such as the upper limit value of the load), the memory (such as the utilization rate of the memory), the CPU (such as the utilization rate of the CPU), and the disk (such as the utilization rate of the disk) (i.e., as shown in fig. 4, the state information may include one or more of the load state, the memory state, the CPU state, and the disk state of the ES cluster), and for convenience of user observation, the monitoring center may also visually display the captured state information in real time.
As shown in fig. 5, the coordinator may dynamically read data of the monitoring center, so that the execution pace of each deletion task can be dynamically adjusted according to the state of the ES cluster.
For example, if the monitoring finds that the load of the ES cluster reaches the upper limit value, the execution of the deletion task may be immediately terminated, and the deletion task may be restarted after the ES cluster is stable, thereby reducing the consumption of the cluster load.
For another example, if the monitoring finds that the load of the ES cluster is at a medium level but does not exceed an upper limit, the execution pace of the delete task may be dynamically adjusted to reduce the cluster load.
For another example, if the monitoring finds that the ES cluster load is low, the execution pace of the deleter task may be dynamically increased to improve the deletion efficiency of the data.
Compared with the prior art that the state of the ES cluster cannot be monitored when the deleting task is executed, and the execution pace of each deleting task cannot be dynamically adjusted according to the pressure of the ES cluster, the method and the device can automatically visually monitor the state of the ES cluster in real time and dynamically adjust the execution pace of each deleting task according to the monitoring result.
As an alternative embodiment, assigning at least one deletion task to a corresponding terminal device comprises: determining the number of available terminal devices and the speed of executing tasks, and the number of at least one deletion task; and assigning at least one deletion task to the available terminal devices according to the determined number and speed.
In particular, in connection with fig. 1 and 5, the task generator may be capable of dynamically assigning tasks according to the number of terminal devices performing the deletion tasks, and the number of initiated deletion tasks. For example, suppose that there are 2 clusters, each with 2 indexes to be deleted, and 3 terminal devices to execute tasks, at this time, the task generator will start 4 deletion tasks, and the coordinator is responsible for distributing the 4 tasks to 3 terminal devices to execute, for example, one terminal device may execute 2 tasks, and the other two terminal devices may execute 1 task. If a certain task is executed quickly, 2 tasks are distributed to another idle terminal device for execution in the next round of automatic balancing.
According to the embodiment of the disclosure, the allocation and deletion task can be dynamically adjusted according to the state of the task execution end and the number of the started tasks.
Fig. 6 schematically illustrates a block diagram of a data deletion system according to an embodiment of the present disclosure.
As shown in fig. 6, the data deletion system 600 includes a first acquisition module 610, a first generation module 620, and an allocation module 630.
A first obtaining module 610, configured to obtain configuration information, where the configuration information is at least used to characterize data to be deleted;
a first generating module 620, configured to generate at least one deletion task based on the configuration information; and
And the allocation module 630 is configured to allocate at least one deletion task to a corresponding terminal device, where the terminal device is capable of executing the deletion task and deleting the data to be deleted, which is characterized by the configuration information.
Compared with the prior art that when deleting the data stored in the ES cluster, a user is required to manually write a hard coding timing deletion task for each service, so that the task is inconvenient to dynamically adjust, the embodiment of the disclosure provides a visual WEB-UI, the user can input configuration information, a task generator can automatically generate the timing deletion task, the trouble of manual coding of the user is saved, and the task is convenient to dynamically adjust.
As an optional embodiment, the first generating module includes: the first analyzing unit is used for analyzing the configuration information and determining the following information of the data to be deleted: the name of the ES cluster and the IP address of the node for storing the data to be deleted in the ES cluster; and a first generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data that corresponds to the information and that has currently expired.
Through the embodiment of the disclosure, a user can only specify a deletion position (namely, the name of the ES cluster and the IP address of the related node therein) when configuring information, does not specify which data to delete specifically (namely, does not specify a time parameter), and at this time, the data which has expired currently can be deleted by default according to rules.
As an optional embodiment, the first generating module includes: the second parsing unit is used for parsing the configuration information and determining the following information of the data to be deleted: the name of the ES cluster, the IP address of the node for storing the data to be deleted in the ES cluster and the appointed time parameter; and a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a time-to-live value smaller than the specified time parameter.
By the embodiment of the disclosure, a user can specify a deletion position (namely, the name of the ES cluster and the IP address of the related node therein) and specify which data to delete specifically (namely, specify a time parameter) when configuring information, and at this time, the data with survival time earlier than the time parameter can be deleted by default according to rules.
As an alternative embodiment, the above system further comprises: the second acquisition module is used for acquiring the state information of the ES cluster; the second generation module is used for generating a task adjustment instruction according to the acquired state information; and the adjusting module is used for adjusting the execution pace of the at least one deletion task on the corresponding terminal equipment through the task adjusting instruction.
As an alternative embodiment, the status information includes one or more of the following information of the ES cluster: load status, memory status, CPU status, and disk status.
Compared with the prior art that the state of the ES cluster cannot be monitored when the deleting task is executed, and the execution pace of each deleting task cannot be dynamically adjusted according to the pressure of the ES cluster, the method and the device can automatically visually monitor the state of the ES cluster in real time and dynamically adjust the execution pace of each deleting task according to the monitoring result.
As an alternative embodiment, the allocation module includes: a determining unit for determining the number of available terminal devices and the speed of executing the tasks, and the number of the at least one deletion task; and an allocation unit configured to allocate the at least one deletion task to the available terminal device according to the determined number and speed.
According to the embodiment of the disclosure, the allocation and deletion task can be dynamically adjusted according to the state of the task execution end and the number of the started tasks.
Any number of the modules, units, or at least some of the functionality of any number of the modules, units, or units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuits, or in any one of or in any suitable combination of three of software, hardware, and firmware. Or one or more of the modules, units according to embodiments of the present disclosure may be at least partially implemented as computer program modules which, when executed, may perform the corresponding functions.
For example, any of the first acquisition module 610, the first generation module 620, and the allocation module 630 may be combined in one module/unit/sub-unit or any of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Or at least some of the functionality of one or more of these modules/units/sub-units may be combined with at least some of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to embodiments of the present disclosure, at least one of the first acquisition module 610, the first generation module 620, and the allocation module 630 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or co-components in any other reasonable way of integrating or packaging circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the first acquisition module 610, the first generation module 620 and the allocation module 630 may be at least partially implemented as a computer program module which, when executed, may perform the corresponding functions.
It should be noted that, in the embodiments of the present disclosure, a system portion corresponds to a method portion in the embodiments of the present disclosure, and the description of the system portion specifically refers to the method portion and is not described herein again.
Fig. 7 schematically illustrates a block diagram of a computer system suitable for implementing the data deletion method and system according to an embodiment of the present disclosure. The computer system illustrated in fig. 7 is merely an example, and should not be construed as limiting the functionality and scope of use of the embodiments of the present disclosure.
As shown in fig. 7, a computer system 700 according to an embodiment of the present disclosure includes a processor 701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.
In the RAM 703, various programs and data required for the operation of the system 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. Note that the program may be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the system 700 may further include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The system 700 may also include one or more of the following components connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 702 and/or RAM 703 and/or one or more memories other than ROM 702 and RAM 703 described above.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.
Claims (10)
1. A data deletion method comprising:
acquiring configuration information, wherein the configuration information is at least used for representing data to be deleted;
Generating at least one deletion task based on the configuration information; and
Distributing the at least one deletion task to a corresponding terminal device, wherein the terminal device can execute the deletion task and delete the data to be deleted, which is characterized by the configuration information;
wherein the generating at least one deletion task based on the configuration information includes:
Analyzing the configuration information to determine the following information of the data to be deleted: the name of the ES cluster and the IP address of the node for storing the data to be deleted in the ES cluster;
Generating at least one deletion task capable of deleting data corresponding to the information and which has currently expired, based on the determined information; or alternatively
Analyzing the configuration information, and determining the following information of the data needing to be deleted, namely the name of an ES cluster, the IP address of a node used for storing the data needing to be deleted in the ES cluster and a designated time parameter; and
Based on the determined information, at least one deletion task is generated that is capable of deleting data corresponding to the information and having a time-to-live value less than the specified time parameter.
2. The method of claim 1, wherein the method further comprises:
Acquiring state information of the ES cluster;
Generating a task adjustment instruction according to the acquired state information; and
And adjusting the execution pace of the at least one deletion task on the corresponding terminal equipment through the task adjustment instruction.
3. The method of claim 2, wherein the state information includes one or more of a load state, a memory state, a CPU state, and a disk state of the ES cluster.
4. The method of claim 1, wherein the assigning the at least one deletion task to the corresponding terminal device comprises:
determining the number of available terminal devices and the speed of executing tasks, and the number of the at least one deletion task; and
And distributing the at least one deleting task to the available terminal equipment according to the determined number and the speed.
5. A data deletion system, comprising:
the first acquisition module is used for acquiring configuration information, wherein the configuration information is at least used for representing data to be deleted;
The first generation module is used for generating at least one deletion task based on the configuration information; and
The allocation module is used for allocating the at least one deletion task to the corresponding terminal equipment; the terminal equipment can execute the deleting task and delete the data to be deleted, which are characterized by the configuration information;
Wherein the first generation module comprises:
The first analyzing unit is used for analyzing the configuration information and determining the following information of the data to be deleted: the name of the ES cluster and the IP address of the node for storing the data to be deleted in the ES cluster; and
A first generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data that corresponds to the information and that has currently expired; or alternatively
The second parsing unit is used for parsing the configuration information and determining the following information of the data to be deleted: the method comprises the steps of (1) name of an ES cluster, a guard P address of a node for storing data to be deleted in the ES cluster and a specified time parameter; and
And a second generation unit configured to generate, based on the determined information, at least one deletion task capable of deleting data corresponding to the information and having a time-to-live value smaller than the specified time parameter.
6. The system of claim 5, wherein the system further comprises:
The second acquisition module is used for acquiring the state information of the ES cluster;
The second generation module is used for generating a task adjustment instruction according to the acquired state information;
And
And the adjusting module is used for adjusting the execution pace of the at least one deletion task on the corresponding terminal equipment through the task adjusting instruction.
7. The system of claim 6, wherein the state information includes one or more of a load state, a memory state, a CPU state, and a disk state of the ES cluster.
8. The system of claim 5, wherein the allocation module comprises: a determining unit for determining the number of available terminal devices and the speed of executing the tasks, and the number of the at least one deletion task; and
And the allocation unit is used for allocating the at least one deletion task to the available terminal equipment according to the determined number and speed.
9. A computer system, comprising:
One or more processors;
a memory for storing one or more programs,
Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-4.
10. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381811.5A CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811381811.5A CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111198793A CN111198793A (en) | 2020-05-26 |
| CN111198793B true CN111198793B (en) | 2024-10-18 |
Family
ID=70746855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811381811.5A Active CN111198793B (en) | 2018-11-19 | 2018-11-19 | Data deleting method and system, medium and computer system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111198793B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113395369B (en) * | 2021-06-10 | 2022-12-13 | 北京天融信网络安全技术有限公司 | Cache management method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103874051A (en) * | 2012-12-12 | 2014-06-18 | 中国联合网络通信集团有限公司 | Internet of Things preset resource processing method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4908849B2 (en) * | 2006-01-11 | 2012-04-04 | 富士通セミコンダクター株式会社 | File deletion method, file opening method, file deletion program, and file opening program |
| US20170017671A1 (en) * | 2012-04-25 | 2017-01-19 | International Business Machines Corporation | Deleting data in a dispersed storage network |
| CN106126708A (en) * | 2016-06-30 | 2016-11-16 | 乐视控股（北京）有限公司 | A kind of method and apparatus of the file deleting terminal storage |
| CN106569851B (en) * | 2016-10-21 | 2020-10-02 | 海信视像科技股份有限公司 | Application program processing method and device |
| CN106357811B (en) * | 2016-10-25 | 2019-08-13 | Oppo广东移动通信有限公司 | A kind of delet method of Backup Data, apparatus and system |
| CN108073355B (en) * | 2016-11-15 | 2020-03-17 | 杭州海康威视数字技术股份有限公司 | Data storage and deletion method and device |
| CN107659929B (en) * | 2017-09-20 | 2020-06-16 | 北京佰才邦技术有限公司 | Configuration deleting method, system, terminal equipment and service base station |
-
2018
- 2018-11-19 CN CN201811381811.5A patent/CN111198793B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103874051A (en) * | 2012-12-12 | 2014-06-18 | 中国联合网络通信集团有限公司 | Internet of Things preset resource processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111198793A (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10445140B1 (en) | Serializing duration-limited task executions in an on demand code execution system | |
| US11231955B1 (en) | Dynamically reallocating memory in an on-demand code execution system | |
| US7266823B2 (en) | Apparatus and method of dynamically repartitioning a computer system in response to partition workloads | |
| CN113641457B (en) | Container creation method, device, apparatus, medium, and program product | |
| US9471386B2 (en) | Allocating resources to tasks in a build process | |
| US20160357589A1 (en) | Methods and apparatus to scale application deployments in cloud computing environments using virtual machine pools | |
| CN109408205B (en) | Task scheduling method and device based on hadoop cluster | |
| CN112463375B (en) | Data processing method and device | |
| CN110389903B (en) | Test environment deployment method and device, electronic equipment and readable storage medium | |
| CN107251007B (en) | Cluster computing service ensuring device and method | |
| CN115145695B (en) | Resource scheduling method and device, computer equipment and storage medium | |
| CN106302640A (en) | Data request processing method and device | |
| CN111198793B (en) | Data deleting method and system, medium and computer system | |
| CN110677321B (en) | Elastically-telescopic cloud pressure measuring method, device, equipment and storage medium | |
| US9575795B2 (en) | Reverting a virtual resource to its base configuration using the snapshot image based on frequency the virtual resource is requested | |
| CN108965364A (en) | Resource allocation method, apparatus and system | |
| CN104700255B (en) | Multi-process processing method, device and system | |
| CN107092556B (en) | Test method, device and equipment | |
| CN107045452B (en) | Virtual machine scheduling method and device | |
| CN110046319B (en) | Social media information acquisition method, device, system, equipment and storage medium | |
| JP7574435B2 (en) | Method, system and computer program for scheduling a workflow - Patents.com | |
| CN116360937A (en) | Task scheduling method, device, equipment and storage medium | |
| CN112783757B (en) | Test system, method, computer system, and computer-readable storage medium | |
| US20030131042A1 (en) | Apparatus and method of sharing a device between partitions of a logically partitioned computer system | |
| CN112491066B (en) | Load balancing method, device, system, target load balancing equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |