CN111159179A - Table partitioning method, related device and computer readable storage medium - Google Patents
Table partitioning method, related device and computer readable storage medium Download PDFInfo
- Publication number
- CN111159179A CN111159179A CN201911279598.1A CN201911279598A CN111159179A CN 111159179 A CN111159179 A CN 111159179A CN 201911279598 A CN201911279598 A CN 201911279598A CN 111159179 A CN111159179 A CN 111159179A
- Authority
- CN
- China
- Prior art keywords
- data table
- data
- name
- synchronous
- forward link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000638 solvent extraction Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000001360 synchronised effect Effects 0.000 claims abstract description 93
- 238000005192 partition Methods 0.000 claims abstract description 73
- 238000004891 communication Methods 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 2
- 230000015654 memory Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a table partitioning method, a related device and a computer readable storage medium. The method comprises the following steps: firstly, a first data table and a second data table are determined, wherein the second data table is a blank partition table, a synchronous forward link from the first data table to the second data table is established, data of the second data table is enabled to obtain data of the first data table through the synchronous forward link, and when the data of the second data table is synchronous with the first data table, partitioning of the data in the second data table is completed. By adopting the embodiment of the invention, the stable synchronization and partition of the data table with a large amount of data can be completed by creating the blank partition table and the synchronous link.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a table partitioning method, a related device, and a computer-readable storage medium.
Background
In daily development, a large table is often encountered, and the large table is a table storing millions or even tens of millions of records. Such tables are too large, resulting in a database that takes too long to query and insert, and that performs poorly, if a federated query is involved. The purpose of table partitioning is to reduce the burden of the database and improve the efficiency of the database, and in general, to improve the efficiency of table addition, deletion, and check.
The existing partitioning scheme is either to perform partitioning offline, needs to stop the application, and may support one day or even longer, which results in that the application is unavailable for one day, and thus service functions are not available. Or the partition is established online, but establishing the partition for a large table often causes unexpected mutation, resulting in that the service is not available for a long time. Therefore, how to partition a large table without losing data and without affecting the use of business functions is an ongoing problem for those skilled in the art.
Disclosure of Invention
The embodiment of the invention discloses a table partitioning method, a related device and a computer readable storage medium, which can complete stable synchronization and partitioning of a data table with a large amount of data.
In a first aspect, an embodiment of the present invention provides a table partitioning method, where the method includes:
determining a first data table and a second data table, wherein the second data table is a blank preset partition table, and the table structure of the second data table is the same as that of the first data table;
establishing a synchronous forward link from the first data table to the second data table, wherein the synchronous forward link is used for synchronizing the data of the first data table to a preset partition of the second data table;
and synchronizing the data of the first data table into a preset partition of the second data table through the synchronous forward link to finish table partitioning.
In the method, the device firstly determines a first data table and a second data table, wherein the second data table is a blank partition table, establishes a synchronous forward link from the first data table to the second data table, so that the data of the second data table obtains the data of the first data table through the synchronous forward link, and completes the partition of the data in the second data table when the data of the second data table is synchronous with the first data table; the embodiment of the application completes stable synchronization and partition of the data table with a large amount of data in a mode of creating a blank partition table and a synchronous link.
Based on the first aspect, in an optional implementation manner, after the establishing of the synchronous forward link from the first data table to the second data table, the method includes:
establishing a synchronous reverse link of the first data table to the second data table before synchronizing data of the first data table to a preset partition of the second data table through the synchronous forward link, wherein the synchronous reverse link is used for synchronizing data of the second data table to the first data table.
This implementation establishes a synchronous reverse link of the first data table to the second data table after establishing a synchronous forward link of the first data table to the second data table, in preparation for fault-tolerant rollback of subsequent data to ensure data security.
Based on the first aspect, in an optional implementation manner, the synchronizing, by the synchronization forward link, data of the first data table into a preset partition of the second data table, and completing table partitioning includes:
judging whether the data in the second data table is completely consistent with the data in the first data table;
if the data in the first data table are not consistent with the data in the second data table, continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, selecting a specific time, and performing read-only operation on the first data table to stabilize the data in the first data table;
renaming the name A of the first data table to be the name C, and renaming the name B of the second data table to be the name A.
In this implementation, after the data in the second data table is synchronized with the first data table, the name of the first data table is renamed, the name of the second data table is renamed to the name of the first data table, the second data table with the name of the first data table is a partition table, and at this time, the data partition with the name of the first data table is completed.
Based on the first aspect, in one optional implementation manner, after renaming the name a of the first data table to be the name C and renaming the name B of the second data table to be the name a, the method further includes:
synchronizing data of the second data table and data of the first data packet according to the synchronous reverse link;
and if the second data table has data errors within a preset time period, withdrawing the operation of renaming the name A of the first data table to be the name C and renaming the name B of the second data table to be the name A through the synchronous reverse link.
The implementation mode utilizes the synchronous reverse link to carry out fault-tolerant rollback, and if the data in the second data table is wrong in the preset time period, the renaming operation is withdrawn through the synchronous reverse link, so that the safety of the data is ensured.
Based on the first aspect, in an optional implementation manner, the method further includes:
and if no data error occurs in the second data table after the preset time period is exceeded, deleting the first data table.
According to the implementation mode, the first data packet is deleted after the safety of the second data packet is determined, so that the memory resource is saved, and the space utilization rate of the database is improved.
In a second aspect, an embodiment of the present invention provides a table partitioning apparatus, including:
the device comprises a determining unit, a judging unit and a judging unit, wherein the determining unit is used for determining a first data table and a second data table, the second data table is a blank preset partition table, and the table structure of the second data table is the same as that of the first data table;
the establishing unit is used for establishing a synchronous forward link from the first data table to the second data table, and the synchronous forward link is used for synchronizing the data of the first data table to a preset partition of the second data table;
and the synchronization unit is used for synchronizing the data of the first data table into a preset partition of the second data table through the synchronous forward link to finish table partitioning.
Based on the second aspect, in an optional implementation manner, the apparatus further includes:
the establishing unit is further configured to, after establishing the synchronous forward link from the first data table to the second data table, establish a synchronous reverse link from the first data table to the second data table before synchronizing the data of the first data table into the preset partition of the second data table through the synchronous forward link, where the synchronous reverse link is used to synchronize the data of the second data table into the first data table.
Based on the second aspect, in an optional implementation manner, the synchronization unit includes:
the judging unit is used for judging whether the data in the second data table is completely consistent with the data in the first data table;
if the data in the first data table are not consistent with the data in the second data table, the synchronization unit is further used for continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, the read-only unit is used for selecting specific time and performing read-only operation on the first data table to stabilize the data in the first data table;
and the renaming unit is used for renaming the name A of the first data table to be the name C and renaming the name B of the second data table to be the name A.
Based on the second aspect, in an optional implementation manner, the apparatus further includes:
the synchronization unit is further configured to rename a name a of the first data table to a name C, and synchronize data of the second data table and data of the first data packet according to the synchronization reverse link after renaming a name B of the second data table to the name a;
and the withdrawing unit is used for withdrawing the operation of renaming the name A of the first data table to the name C and renaming the name B of the second data table to the name A through the synchronous reverse link if the second data table has data errors in a preset time period.
Based on the second aspect, in an optional implementation manner, the apparatus further includes:
and the deleting unit is used for deleting the first data table if no data error occurs in the second data table after the preset time period is exceeded.
It should be noted that, for the implementation manner and the corresponding beneficial effects of the second aspect, reference may be made to the description in the first aspect and the corresponding implementation manner, and details are not described herein again.
In a third aspect, an embodiment of the present invention provides a table partitioning device, which is characterized by including a processor, a memory, and a communication device, where the processor, the memory, and the communication device are connected to each other, where the memory is used to store a computer program, and the communication device is used to perform information interaction with an external device; the processor is configured to invoke the computer program to perform the method according to the first aspect.
It should be noted that, for the implementation manner and the corresponding beneficial effects of the third aspect, reference may be made to the description in the first aspect and the corresponding implementation manner, and details are not described herein again.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method of the first aspect.
It should be noted that, for the implementation manner and the corresponding beneficial effects of the fourth aspect, reference may be made to the description in the first aspect and the corresponding implementation manner, and details are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments of the present invention or the background art will be briefly described below.
FIG. 1 is a system architecture diagram of a table partitioning method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a table partitioning method according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a table partitioning apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a table partitioning apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments. As used in this specification, the terms "device," "unit," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a device may be, but is not limited to, a processor, a data processing platform, a computing device, a computer, 2 or more computers, and the like.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
First, some terms in the present application are explained so as to be easily understood by those skilled in the art.
(1) Table structure: a table frame is described, the structure of the table must be designed before the table is built, the table structure is designed to actually define the number of fields forming the table, and the information of the name, data type, field attribute (including field length, format, default value, mandatory field, validity rule, validity text, index and the like), primary key and the like of each field.
(2) And (4) partitioning a table: the data of a large table is divided into many small subsets called partitions, of the types FAT32, NTFST32, NTFS. In addition, the type division of the partition table mainly includes: range, list and hash partitions. The basis for partitioning is mainly according to its table internal properties. At the same time, the partition table may create its unique partition index.
(3) And (3) synchronous link: the Synchronization Link comprises a physical Link and a data Link, wherein the physical Link refers to a Link for connecting two synchronous network nodes, a section of physical line from one node to an adjacent node, and no other switching node is arranged in the middle, is used for bearing timing signals and Synchronization information, and is a Link for transmitting the Synchronization information. Data links are data links that require communication protocols in addition to physical lines to control the transmission of such data, and that are data links in which hardware and software implementing such protocols are added to the links.
In order to better understand the table partitioning method and the related device provided by the embodiment of the present invention, a system architecture of the table partitioning method according to the embodiment of the present invention is described below. As shown in fig. 1, which is a schematic diagram of a system architecture of a table partitioning method provided in an embodiment of the present invention, the system may include one or more servers (a plurality of servers may form a server cluster), and one or more terminals (or devices), where:
the server may include, but is not limited to, a background server, a component server, a data processing server, etc., and the server may communicate with a plurality of terminals through the internet. The server provides table partitioning services for the terminal. The server needs to run a corresponding server-side program to provide corresponding table partitioning services, such as database services, data calculation, decision execution, and the like.
The terminal can install and run the relevant application. An application is a program that corresponds to a server and provides local services to a client. Here, the local service may include, but is not limited to: sending information (e.g., attribute data information) to the server and receiving information sent by the server (e.g., table partition information) and sharing information, and so on. The terminal in the embodiment of the scheme can be packagedIncluding but not limited to any smart operating system based electronic product that can interact with a user through input devices such as keyboards, virtual keyboards, touch pads, touch screens, and voice-activated devices, such as smartphones, tablets, personal computers, etc. The smart operating system includes, but is not limited to, any operating system that enriches device functionality by providing various mobile applications to the mobile device, such as Android (Android)TM)、iOSTM、Windows PhoneTMAnd the like.
The table partitioning method of the present application is described below with reference to fig. 2, as shown in fig. 2, which is a flowchart of a table partitioning method provided in an embodiment of the present application, and the method may be implemented based on the system architecture shown in fig. 1 or other architectures, and the method may include, but is not limited to, the following steps:
step S201: a first data table and a second data table are determined.
Specifically, the device first determines a first data table, wherein the first data table is a data table requiring table partitioning, the device creates a second data table, wherein the second data table is a blank preset partition table, the table structure of the second data table is the same as that of the first data table, a partition is created while the second data table is created, and the partition rule may be a date partition, a place partition, a fixed capacity partition, or the like, for example, the partition rule is a partition by month, that is, data of different months are inserted into different partitions, and subsequently, newly inserted data are also inserted according to the months; partitioning by location, i.e. inserting data of different locations in different zones, and subsequently newly inserted data also being inserted by location.
Step S202: a synchronous forward link is established from the first data table to the second data table.
Specifically, after determining a first data table and a second data table, the device establishes a synchronous forward link from the first data table to the second data table, where the synchronous forward link is used to synchronize data of the first data table into a preset partition of the second data table, the synchronous forward link can stably and quickly transmit massive data, through the synchronous forward link, the data of the first data table can be inserted into a partition of the second data table, the second data table obtains the data in the first data table, the data is allocated into a corresponding partition according to a partition rule of the second data table, after the data of the second data table is synchronized with the data of the first data table, a temporary index and a related data structure of the second data table are obtained, the obtained temporary index of the second data table is the same as the index of the first data table, where the obtaining of the temporary index and the related data structure of the second data table may be created artificially, or the target document can be obtained in an automatic extraction mode after the target document is imported.
In one embodiment, after establishing the synchronous forward link from the first data table to the second data table, establishing a synchronous reverse link from the first data table to the second data table; the synchronous reverse link is used for preparing fault-tolerant rollback, namely if data errors occur in the subsequent process of synchronizing the second data table and the first data table, the synchronous reverse link can be used for rollback, and the data of a user is guaranteed not to be lost.
Step S203: and synchronizing the data of the first data table into the preset partition of the second data table through the synchronous forward link.
Specifically, after the device establishes a synchronous forward link from the first data table to the second data table, the data of the first data table is synchronized into a preset partition of the second data table through the synchronous forward link, the data of the first data table is inserted into the partition of the second data table according to the partition rule of the second data table, at this time, the second data table is equivalent to the partition of the first data table, and the delay of synchronizing the data according to the synchronous forward link can be reduced to the minimum, which is about 1.3 seconds.
In one embodiment, after the device synchronizes the data of the first data table to the preset partition of the second data table through the synchronization forward link, the device renames the name of the first data table, and renames the name of the second data table to the name of the first data table, for example, if the name of the first data table is a and the name of the second data table is B, the data of the second data table is synchronized with the data of the first data table through the synchronization forward link, then renames the name a of the first data table to C, and renames the name B of the second data table to a, and keeps the data synchronization of the second data table and the first data table, when the table named a is the second data table, i.e., the partition table, the partition of the a table is completed.
In one embodiment, the device synchronizes the data of the first data table to a preset partition of the second data table through the synchronous forward link, and determines whether the data in the second data table and the data in the first data table are completely consistent; if the data in the first data table are not consistent with the data in the second data table, continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, selecting a specific time to perform read-only operation on the first data table so as to stabilize the data in the first data table; and then renaming the first data table, renaming a name A of the first data table to a name C, and renaming a name B of the second data table to a name A, wherein the specific time can be a time period with the least database access amount under big data statistics so as to reduce the influence of read-only operation on the first data table on a user, the database comprises the first data table and the second data table, and before the specific time, a database update notice can be output to remind the user to store required data in advance. The renaming process lasts about 30s approximately, the influence on the online application program is reduced to the minimum, after the renaming is completed, namely the name of the second data table is changed into the name of the first data table, the second data table takes the function of the first data table, and the second data table is a partition table, which is equivalent to the completion of the partition of the first data table.
In one embodiment, after the table partitioning is completed, the read-only state of the table a is released, the table a becomes a partition table, and the original first data table (i.e., the table C) is deleted, so that the data space is saved, and the space utilization rate of the database is improved.
In one embodiment, the second data table (a table) and the application are continuously observed and detected, and if a data error occurs in the process of renaming the name a of the first data table to the name C and the name B of the second data table to the name a, for example, an application cannot connect to a database now or a performance problem occurs in the data tables, the renaming operation is withdrawn through the synchronous reverse link, that is, the operation of renaming the name a of the first data table to the name C and renaming the name B of the second data table to the name a is withdrawn. It can be understood that after the renaming operation is performed on the first data table and the second data table, the second data table takes the function of the first data table, the writing-in and writing-out of the first data table is changed into the writing-in and writing-out of the second data table, at this time, the data of the second data table and the data of the first data packet are synchronized according to the synchronous reverse link, the second data table (table a) and the application program are observed and detected within a preset time period, if the data error occurs in the second data table within the preset time period, it is indicated that the second data table is unavailable, the renaming operation is withdrawn through the synchronous reverse link, if the data error does not occur in the second data table within the preset time period, it is indicated that the second data table is available, that is, the second data table normally operates, the first data table is deleted, so as to save memory resources and improve the space utilization rate of the database; wherein the preset time period may be 7 days.
The method comprises the steps of determining a first data table and a second data table, wherein the second data table is a blank partition table, establishing a synchronous forward link and a synchronous reverse link from the first data table to the second data table, enabling the data of the second data table to obtain the data of the first data table according to the synchronous forward link, renaming the name of the first data table after the data of the second data table is synchronous with the first data table, renaming the name of the second data table to the name of the first data table, enabling the second data table with the name of the first data table to be the partition table, and completing table partitioning; and monitoring the second data table after renaming, and if a data error occurs, withdrawing the renaming operation according to the synchronous reverse link to ensure that the data is not lost.
In order to better implement the above solution of the embodiments of the present application, the present application further provides a table partitioning apparatus, which is described in detail below with reference to the accompanying drawings:
as shown in fig. 3, an embodiment of the present application provides a schematic structural diagram of a table partitioning apparatus 30, where the table partitioning apparatus 30 may include: a determining unit 301, a establishing unit 302 and a synchronizing unit 303, wherein,
a determining unit 301, configured to determine a first data table and a second data table, where the second data table is a blank preset partition table, and a table structure of the second data table is the same as that of the first data table;
an establishing unit 302, configured to establish a synchronous forward link from the first data table to the second data table, where the synchronous forward link is used to synchronize data of the first data table into a preset partition of the second data table;
a synchronizing unit 303, configured to synchronize, through the synchronous forward link, data of the first data table into a preset partition of the second data table, so as to complete table partitioning.
In one implementation, the apparatus further includes:
the establishing unit 302 is further configured to, after establishing the synchronous forward link of the first data table to the second data table, establish a synchronous reverse link of the first data table to the second data table before synchronizing the data of the first data table to the preset partition of the second data table through the synchronous forward link, where the synchronous reverse link is used to synchronize the data of the second data table to the first data table.
In one implementation manner, the synchronization unit 303 includes:
a judging unit 304, configured to judge whether data in the second data table and data in the first data table are completely consistent;
if the data in the first data table is inconsistent with the data in the second data table, the synchronization unit 303 is further configured to continue synchronizing the data in the second data table with the data in the first data table, and if the data in the first data table is consistent with the data in the second data table, the read-only unit 305 is configured to select a specific time and perform a read-only operation on the first data table, so that the data in the first data table is stable;
a renaming 306 unit, configured to rename the name a of the first data table to a name C, and rename the name B of the second data table to the name a.
In one implementation, the apparatus further includes:
the synchronizing unit 303 is further configured to rename the name a of the first data table to a name C, and synchronize the data of the second data table and the data of the first data packet according to the synchronous reverse link after renaming the name B of the second data table to the name a;
a withdrawing unit 307, configured to withdraw the operation of renaming the name a of the first data table and renaming the name B of the second data table to name a through the synchronous reverse link if the second data table has a data error within a preset time period.
In one implementation, the apparatus further includes:
a deleting unit 308, configured to delete the first data table if no data error occurs in the second data table after a preset time period is exceeded.
It should be noted that, in the embodiment of the present application, functions of each functional unit in the apparatus described in fig. 3 may refer to the related description of step S201 to step S203 in the embodiment of the method described in fig. 2, and are not described again here.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a simplified table partitioning apparatus according to an embodiment of the present disclosure. For ease of understanding and illustration, one or more of the following components may be included in the table partitioning apparatus 40 of FIG. 4: memory 401, processor 402, communication device 403, and input/output device 404.
Memory 401 may include one or more memory units, each of which may include one or more memories, which may be used to store programs and various data and enable high-speed, automated access to the programs or data during operation of table partitioning apparatus 40. When the table partitioning apparatus 40 shown in fig. 4 performs the method described in fig. 2, the memory may be used to store data tables, table partitioning codes, and other related data, etc.
The communication device 403, which may also be referred to as a transceiver, or transceiver, etc., may include elements for wireless, wired, or other communication. Optionally, a device for implementing a receiving function in part 403 may be regarded as a receiving unit, and a device for implementing a sending function may be regarded as a sending unit, that is, part 403 includes a receiving unit and a sending unit.
determining a first data table and a second data table, wherein the second data table is a blank preset partition table, and the table structure of the second data table is the same as that of the first data table;
establishing a synchronous forward link from the first data table to the second data table, wherein the synchronous forward link is used for synchronizing the data of the first data table to a preset partition of the second data table;
and synchronizing the data of the first data table into a preset partition of the second data table through the synchronous forward link to finish table partitioning.
In one embodiment, after the processor 402 establishes the synchronous forward link from the first data table to the second data table, the method includes:
establishing a synchronous reverse link of the first data table to the second data table before synchronizing data of the first data table to a preset partition of the second data table through the synchronous forward link, wherein the synchronous reverse link is used for synchronizing data of the second data table to the first data table.
In one embodiment, the processor 402 synchronizes the data of the first data table to a preset partition of the second data table through the synchronization forward link, and completing the table partition includes:
judging whether the data in the second data table is completely consistent with the data in the first data table;
if the data in the first data table are not consistent with the data in the second data table, continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, selecting a specific time, and performing read-only operation on the first data table to stabilize the data in the first data table;
renaming the name A of the first data table to be the name C, and renaming the name B of the second data table to be the name A.
In one embodiment, after the processor 402 renames the name a of the first data table to the name C and the name B of the second data table to the name a, the method further includes:
synchronizing data of the second data table and data of the first data packet according to the synchronous reverse link;
and if the second data table has data errors within a preset time period, withdrawing the operation of renaming the name A of the first data table to be the name C and renaming the name B of the second data table to be the name A through the synchronous reverse link.
In one embodiment, if no data error occurs in the second data table after the preset time period is exceeded, the processor 402 deletes the first data table.
It should be noted that, in the embodiment of the present application, reference may be made to the specific implementation manner in the embodiment of fig. 2 in each method embodiment for the step executed by the processor 402 in the table partitioning device 40, and details are not described here again.
Input/output devices 404 may provide an interface between the interfaces of peripheral components of meter-partitioning apparatus 40, such as a keyboard, mouse, display, buttons, similar devices, etc. These buttons may include, but are not limited to: an on/off button, a lock button, a reset button, etc.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. And the aforementioned storage medium includes: a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
In this application, the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. While the present application has been described herein in conjunction with various embodiments, other variations to the disclosed embodiments may be understood and effected by those skilled in the art in practicing the present application as claimed herein.
Claims (10)
1. A method of table partitioning, comprising:
determining a first data table and a second data table, wherein the second data table is a blank preset partition table, and the table structure of the second data table is the same as that of the first data table;
establishing a synchronous forward link from the first data table to the second data table, wherein the synchronous forward link is used for synchronizing the data of the first data table to a preset partition of the second data table;
and synchronizing the data of the first data table into a preset partition of the second data table through the synchronous forward link to finish table partitioning.
2. The method of claim 1, wherein after establishing the synchronous forward link from the first data table to the second data table, the method comprises:
establishing a synchronous reverse link of the first data table to the second data table before synchronizing data of the first data table to a preset partition of the second data table through the synchronous forward link, wherein the synchronous reverse link is used for synchronizing data of the second data table to the first data table.
3. The method of claim 2, wherein the synchronizing data of the first data table into the pre-defined partition of the second data table over the synchronous forward link, wherein completing the table partitioning comprises:
judging whether the data in the second data table is completely consistent with the data in the first data table;
if the data in the first data table are not consistent with the data in the second data table, continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, selecting a specific time, and performing read-only operation on the first data table to stabilize the data in the first data table;
renaming the name A of the first data table to be the name C, and renaming the name B of the second data table to be the name A.
4. The method of claim 3, wherein after renaming the name A of the first data table to the name C and the name B of the second data table to the name A, further comprising:
synchronizing data of the second data table and data of the first data packet according to the synchronous reverse link;
and if the second data table has data errors within a preset time period, withdrawing the operation of renaming the name A of the first data table to be the name C and renaming the name B of the second data table to be the name A through the synchronous reverse link.
5. The method of claim 4, further comprising:
and if no data error occurs in the second data table after the preset time period is exceeded, deleting the first data table.
6. A table partitioning apparatus, comprising:
the device comprises a determining unit, a judging unit and a judging unit, wherein the determining unit is used for determining a first data table and a second data table, the second data table is a blank preset partition table, and the table structure of the second data table is the same as that of the first data table;
the establishing unit is used for establishing a synchronous forward link from the first data table to the second data table, and the synchronous forward link is used for synchronizing the data of the first data table to a preset partition of the second data table;
and the synchronization unit is used for synchronizing the data of the first data table into a preset partition of the second data table through the synchronous forward link to finish table partitioning.
7. The apparatus of claim 6, further comprising:
the establishing unit is further configured to, after establishing the synchronous forward link from the first data table to the second data table, establish a synchronous reverse link from the first data table to the second data table before synchronizing the data of the first data table into the preset partition of the second data table through the synchronous forward link, where the synchronous reverse link is used to synchronize the data of the second data table into the first data table.
8. The apparatus of claim 7, wherein the synchronization unit comprises:
the judging unit is used for judging whether the data in the second data table is completely consistent with the data in the first data table;
if the data in the first data table are not consistent with the data in the second data table, the synchronization unit is further used for continuing to synchronize the data in the second data table with the data in the first data table, and if the data in the first data table are consistent with the data in the second data table, the read-only unit is used for selecting specific time and performing read-only operation on the first data table to stabilize the data in the first data table;
and the renaming unit is used for renaming the name A of the first data table to be the name C and renaming the name B of the second data table to be the name A.
9. The table partitioning device is characterized by comprising a processor and a communication device, wherein the processor and the communication device are connected with each other, and the communication device is used for information interaction with an external device; the processor is configured to invoke the stored program code, to execute the unit of the method according to any of claims 1-5.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores program instructions that, when executed by a processor, cause the processor to perform the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911279598.1A CN111159179A (en) | 2019-12-12 | 2019-12-12 | Table partitioning method, related device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911279598.1A CN111159179A (en) | 2019-12-12 | 2019-12-12 | Table partitioning method, related device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111159179A true CN111159179A (en) | 2020-05-15 |
Family
ID=70557246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911279598.1A Pending CN111159179A (en) | 2019-12-12 | 2019-12-12 | Table partitioning method, related device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111159179A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590613A (en) * | 2021-07-13 | 2021-11-02 | 上海一谈网络科技有限公司 | Data table partitioning method and device, computer equipment and storage medium |
| CN114185896A (en) * | 2021-12-14 | 2022-03-15 | 中国平安财产保险股份有限公司 | Data processing method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106534226A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Processing method and apparatus for data migration |
| CN108255909A (en) * | 2017-07-27 | 2018-07-06 | 平安科技（深圳）有限公司 | Tables of data backup method and server based on oracle database |
| CN110489426A (en) * | 2019-08-26 | 2019-11-22 | 杭州安恒信息技术股份有限公司 | A kind of automation partition method, device and the equipment of database table |
-
2019
- 2019-12-12 CN CN201911279598.1A patent/CN111159179A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106534226A (en) * | 2015-09-09 | 2017-03-22 | 阿里巴巴集团控股有限公司 | Processing method and apparatus for data migration |
| CN108255909A (en) * | 2017-07-27 | 2018-07-06 | 平安科技（深圳）有限公司 | Tables of data backup method and server based on oracle database |
| CN110489426A (en) * | 2019-08-26 | 2019-11-22 | 杭州安恒信息技术股份有限公司 | A kind of automation partition method, device and the equipment of database table |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590613A (en) * | 2021-07-13 | 2021-11-02 | 上海一谈网络科技有限公司 | Data table partitioning method and device, computer equipment and storage medium |
| CN114185896A (en) * | 2021-12-14 | 2022-03-15 | 中国平安财产保险股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN114185896B (en) * | 2021-12-14 | 2024-07-09 | 中国平安财产保险股份有限公司 | Data processing method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102392944B1 (en) | Data backup methods, storage media and computing devices | |
| WO2018219178A1 (en) | Data synchronization method and apparatus, server, and storage medium | |
| EP2596425B1 (en) | Membership tracking and data eviction in mobile middleware scenarios | |
| RU2438263C2 (en) | Methods and apparatus for dataset synchronisation in wireless environment | |
| CN106802932B (en) | Routing method and device of database and database system | |
| JP2023546249A (en) | Transaction processing methods, devices, computer equipment and computer programs | |
| TW200842627A (en) | Techniques to cross-synchronize data | |
| CN105095489A (en) | Distributed file deletion method, device and system | |
| WO2014150590A1 (en) | Local store data versioning | |
| EP2603867B1 (en) | Sharing data on mobile devices | |
| CN110888858A (en) | Database operation method and device, storage medium and electronic device | |
| CN111159179A (en) | Table partitioning method, related device and computer readable storage medium | |
| US8756194B1 (en) | Cloud-based data replication for web applications with replica identifier reassignment feature | |
| AU2016101498B4 (en) | Data Interchange System | |
| US9614932B2 (en) | Managing and implementing web application data snapshots | |
| CN109165259B (en) | Index table updating method based on network attached storage, processor and storage device | |
| CN103491113B (en) | A kind of synchronous method, the apparatus and system of information fusion file | |
| CN112989773B (en) | Method, apparatus, device and computer readable medium for synchronizing update data | |
| CN116974983A (en) | Data processing method, device, computer readable medium and electronic equipment | |
| CN112632294B (en) | Method and system for synchronizing Neo4j data to search server | |
| US11334455B2 (en) | Systems and methods for repairing a data store of a mirror node | |
| CN113468190B (en) | Configuration updating method and device for MongoDB database instance | |
| CN112055057B (en) | Method, system and equipment for dynamic expansion of Web system | |
| CN114817392A (en) | Method, device, equipment and computer readable medium for displaying data | |
| CN114722125A (en) | Database transaction processing method, device, equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |