CN109388524B - Virtual disk format addressing method and computer readable storage medium - Google Patents
Virtual disk format addressing method and computer readable storage medium Download PDFInfo
- Publication number
- CN109388524B CN109388524B CN201811172062.5A CN201811172062A CN109388524B CN 109388524 B CN109388524 B CN 109388524B CN 201811172062 A CN201811172062 A CN 201811172062A CN 109388524 B CN109388524 B CN 109388524B
- Authority
- CN
- China
- Prior art keywords
- data
- copy
- block
- l2table
- blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 74
- 238000004590 computer program Methods 0.000 claims description 19
- 230000008859 change Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1479—Generic software techniques for error detection or fault masking
- G06F11/1482—Generic software techniques for error detection or fault masking by means of middleware or OS functionality
- G06F11/1484—Generic software techniques for error detection or fault masking by means of middleware or OS functionality involving virtual machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Quality & Reliability (AREA)
- Bioethics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an addressing method of a virtual disk format and a computer readable storage medium. The method comprises the following steps: when an addressing request of a qcow2 file is received, reading an L1 table and/or an L2table in the qcow2 file; when the data of the L1 table or the L2table are detected to be damaged, acquiring a copy of the damaged table stored in the qcow2 file in advance; reading the data of the copy; and responding to the addressing request according to the copy data.
Description
Technical Field
The present invention relates to the field of information processing, and in particular, to a virtual disk format addressing method and a computer readable storage medium.
Background
The qcow2 disk format is a disk image format supported by a qemu simulator, and represents a fixed-size virtual disk in a file manner. Since files are employed, they can be easily stored on various backend file systems/block devices regardless of the backend storage media used. Compared with the normal raw format, the qcow2 format has the following advantages:
1. the method naturally supports the simplified configuration even if a back-end file system does not support sparse files;
2. copy-on-write is supported;
3. support for snapshots;
4. supporting a disk chain, wherein a plurality of qcow2 files are superposed to represent a logical virtual disk;
5. compression and encryption are supported.
Therefore, qcow2 is widely used in the virtualization and cloud computing field, one qcow2 file is composed of several blocks (cluster) with the same size, and the disk blocks seen inside the virtual machine are mapped one by one to each block of qcow 2.
Fig. 1 is a diagram illustrating a qcow2 file layout in the prior art. Given a disk offset address, its location in qcow2 can be located by L1- > L2- > cluster offset as shown in FIG. 1.
Fig. 2 is a schematic diagram of addressing logic of a qcow2 file in the prior art, and the inventors found that at least the following problems exist in the prior art by the addressing method shown in the schematic diagram:
when the L1 table and/or the L2table are destroyed, the addressing operation cannot be continued, which often causes the problem that the entire disk cannot be recovered.
Disclosure of Invention
In order to solve the technical problem, the invention provides an addressing method in a virtual disk format and a computer readable storage medium, which can improve the success rate of addressing.
In order to achieve the object of the present invention, the present invention provides an addressing method in a virtual disk format, which is characterized by comprising:
when an addressing request of a qcow2 file is received, reading an L1 table and/or an L2table in the qcow2 file;
when the data of the L1 table or the L2table are detected to be damaged, acquiring a copy of the damaged table stored in the qcow2 file in advance;
reading the data of the copy;
and responding to the addressing request according to the copy data.
Wherein, the method also has the following characteristics: before the obtaining of the copy of the table with the damage in the qcow2 file stored in advance, the method further includes:
acquiring a data block i required for storing data of an L1 table, wherein i is an integer greater than or equal to 1;
from MnStarting a block, and storing data of an L1 table as a copy of an L1 table in i consecutive data blocks, wherein M is a prime number greater than or equal to 11, and n is an integer greater than or equal to 1;
the storage location of the copy of the L1 table is recorded.
Wherein, the method also has the following characteristics: the value of M is 17.
Wherein, the method also has the following characteristics: the recording the storage location of the copy of the L1 table includes:
and adding the information of the storage position of the copy of the L1 table to a qcow2 file header.
Wherein, the method also has the following characteristics: the M thnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
when the fact that the qcow2 file is expanded is detected, whether a target block with the number meeting the integral power of M exists in blocks newly added in the expanding operation or not is searched;
if there is a target block whose number corresponds to the integer power of M, storing the data of the L1 table in the continuous i data blocks from the target block;
the storage location of the newly added copy of the L1 table is recorded.
Wherein, the method also has the following characteristics: the M thnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
upon detecting that the data in the L1 table is updated, synchronizing the updated data in the L1 table to a block number MnWherein n ∈ [1, k ] in the block](ii) a Wherein:
k＝round_up(logM bMax)-1；
wherein, bMax is the value of the maximum block number of the qcow2 file, and round _ up represents rounding up.
Wherein, the method also has the following characteristics: before the obtaining of the copy of the table with the damage in the qcow2 file stored in advance, the method further includes:
selecting two blocks as the copies of the L2table according to a preset number selection strategy;
the storage location of the copy of the L2table is recorded.
Wherein, the method also has the following characteristics: the selecting two blocks as the copies of the L2table according to the preset number selection policy comprises:
storing data of an L2table in an available free block with even number as a main block of the L2table copy;
the data of the L2table stored in the available free block with the number of (a + b/2) mod b is selected as the slave block of the L2table copy, wherein a is the number of the master block of the L2table copy, and b is the maximum value of the power of 2 which is not more than bMax, wherein bMax is the maximum block number of the qcow2 file.
Wherein, the method also has the following characteristics: according to a preset number selection strategy, two blocks are selected as copies of an L2table, and the method further comprises the following steps:
if no free block exists in the even numbered blocks in the qcow2 file, adding a new block at the tail of the qcow2 file, and storing the data of the L2table in the newly added block when the number of the new block is even.
Wherein, the method also has the following characteristics: the recording the storage location of the copy of the L1 table includes:
adding information of the storage location of the copy of the L2table to the L1 table.
Wherein, the method also has the following characteristics: after selecting two blocks as the copies of the L2table according to the preset number selection policy, the method further includes:
when the qcow2 file is detected to be expanded, judging whether the two blocks selected according to the number selection strategy change according to the total block number of the qcow2 file;
if the change occurs, recalculating the numbers of the blocks corresponding to the copies of all the L2 tables to obtain a newly determined block;
migrating data in the L2table from an original storage location to the newly determined block, and updating a storage location of a copy of the L2 table.
Wherein, the method also has the following characteristics: after the migrating the data in the L2table from the original storage location to the newly determined block, the method further includes:
and marking the use state of the block of the original storage position as idle.
Wherein, the method also has the following characteristics: after selecting two blocks as the copies of the L2table according to the preset number selection policy, the method further includes:
upon detecting that the data in the L2table is updated, synchronizing the updated data in the L2table into the two blocks.
Wherein, the method also has the following characteristics: the reading the data of the copy comprises:
when a plurality of copies of the damaged table are available, detecting the data integrity of the copies according to a preset copy reading sequence;
and when the data of the copy is detected to be complete, reading the data of the copy, otherwise, selecting the next copy for data integrity detection until the copy with complete data is detected.
To achieve the object of the present invention, the present invention also provides a computer-readable storage medium storing a computer program executed by a processor to implement operations comprising:
when an addressing request of a qcow2 file is received, reading an L1 table and/or an L2table in the qcow2 file;
when the data of the L1 table or the L2table are detected to be damaged, acquiring a copy of the damaged table stored in the qcow2 file in advance;
reading the data of the copy;
and responding to the addressing request according to the copy data.
Wherein the computer-readable storage medium is further characterized by: before the obtaining of the pre-stored copy of the table with the corruption in the qcow2 file, the computer program is further executed by a processor to perform operations comprising:
acquiring a data block i required for storing data of an L1 table, wherein i is an integer greater than or equal to 1;
storing data of an L1 table as a copy of an L1 table in i consecutive data blocks starting from the Mn-th block, wherein M is a prime number equal to or greater than 11, and n is an integer equal to or greater than 1;
the storage location of the copy of the L1 table is recorded.
Wherein the computer-readable storage medium is further characterized by: the value of M is 17.
Wherein the computer-readable storage medium is further characterized by: the recording the storage location of the copy of the L1 table includes:
and adding the information of the storage position of the copy of the L1 table to a qcow2 file header.
Wherein the computer-readable storage medium is further characterized by: after storing the data of the L1 table in i consecutive data blocks starting from the Mn-th block, the computer program is further executed by the processor to implement operations including:
when the fact that the qcow2 file is expanded is detected, whether a target block with the number meeting the integral power of M exists in blocks newly added in the expanding operation or not is searched;
if there is a target block whose number corresponds to the integer power of M, storing the data of the L1 table in the continuous i data blocks from the target block;
the storage location of the newly added copy of the L1 table is recorded.
Wherein the computer-readable storage medium is further characterized by: after storing the data of the L1 table in i consecutive data blocks starting from the Mn-th block, the computer program is further executed by the processor to implement operations further comprising:
upon detecting that the data in the L1 table is updated, synchronizing the updated data in the L1 table to a block number MnWherein n ∈ [1, k ] in the block](ii) a Wherein:
k＝round_up(logM bMax)-1；
wherein, bMax is the value of the maximum block number of the qcow2 file, and round _ up represents rounding up.
Wherein the computer-readable storage medium is further characterized by: before obtaining the copy of the table with the damage in the qcow2 file stored in advance, the computer-readable storage medium further performs the following operations, including:
selecting two blocks as the copies of the L2table according to a preset number selection strategy;
the storage location of the copy of the L2table is recorded.
Wherein the computer-readable storage medium is further characterized by: the selecting two blocks as the copies of the L2table according to the preset number selection policy comprises:
storing data of an L2table in an available free block with even number as a main block of the L2table copy;
the data of the L2table stored in the available free block with the number of (a + b/2) mod b is selected as the slave block of the L2table copy, wherein a is the number of the master block of the L2table copy, and b is the maximum value of the power of 2 which is not more than bMax, wherein bMax is the maximum block number of the qcow2 file.
Wherein the computer-readable storage medium is further characterized by: the selecting two blocks as copies of the L2table according to a preset number selection policy, the computer program further executed by the processor to perform operations comprising:
if no free block exists in the even numbered blocks in the qcow2 file, adding a new block at the tail of the qcow2 file, and storing the data of the L2table in the newly added block when the number of the new block is even.
Wherein the computer-readable storage medium is further characterized by: the recording the storage location of the copy of the L1 table includes:
adding information of the storage location of the copy of the L2table to the L1 table.
In an embodiment of the computer-readable storage medium provided by the present invention, after selecting two blocks as copies of the L2table according to the preset number selection policy, the computer program is further executed by a processor to implement operations including:
when the qcow2 file is detected to be expanded, judging whether the two blocks selected according to the number selection strategy change according to the total block number of the qcow2 file;
if the change occurs, recalculating the numbers of the blocks corresponding to the copies of all the L2 tables to obtain a newly determined block;
migrating data in the L2table from an original storage location to the newly determined block, and updating a storage location of a copy of the L2 table.
Wherein the computer-readable storage medium is further characterized by: after the migrating of the data in the L2table from the original storage location to the newly determined block, the computer program is further executed by a processor to perform operations comprising:
and marking the use state of the block of the original storage position as idle.
In an embodiment of the computer-readable storage medium provided by the present invention, after selecting two blocks as copies of the L2table according to the preset number selection policy, the computer program is further executed by a processor to implement operations including:
upon detecting that the data in the L2table is updated, synchronizing the updated data in the L2table into the two blocks.
Wherein the computer-readable storage medium is further characterized by: the reading the data of the copy comprises:
when a plurality of copies of the damaged table are available, detecting the data integrity of the copies according to a preset copy reading sequence;
and when the data of the copy is detected to be complete, reading the data of the copy, otherwise, selecting the next copy for data integrity detection until the copy with complete data is detected.
According to the embodiment provided by the invention, when an addressing request of a qcow2 file is received, an L1 table and/or an L2table in the qcow2 file are/is read, if data of the L1 table or the L2table are damaged, a copy of the damaged table stored in the qcow2 file in advance is obtained, the aim of repairing addressing information through the copy of the data is fulfilled by reading the data of the copy and responding to the addressing request, the recovery capability of the qcow2 format is improved, and the data security of the whole virtualization system is further improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a diagram of a prior art qcow2 file layout;
FIG. 2 is a diagram of addressing logic for a qcow2 file in the prior art;
fig. 3 is a flowchart of an addressing method of a virtual disk format according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Fig. 3 is a flowchart of an addressing method of a virtual disk format according to the present invention. The method shown in fig. 3 comprises:
and step 304, responding to the addressing request according to the copy data.
According to the method embodiment provided by the invention, when an addressing request of a qcow2 file is received, an L1 table and/or an L2table in the qcow2 file are/is read, if data of the L1 table or the L2table are damaged, a copy of the damaged table stored in the qcow2 file in advance is obtained, the aim of repairing addressing information through the copy of the data is fulfilled by reading the data of the copy and responding to the addressing request, the recovery capability of the qcow2 format is improved, and the data security of the whole virtualization system is further improved.
The following examples of the method provided by the present invention are further illustrated:
according to the qcow2 specification, the maximum number of L1/L2 tables for a fixed-size virtual disk (e.g., 1TB) can be determined. For each 1TB virtual machine disk, the size and number of the L1 table and the L2table are specifically shown in table 1, which is as follows:
| cluster-size(＝L2-table-size) | Max.L2 size per TB | Max.L1 size per TB |
| 64KB | 128MB(2048tables) | 16KB(1table) |
| 128KB | 64MB(512tables) | 8KB |
| 256KB | 32MB(128tables) | 4KB |
| 512KB | 16MB(32tables) | 2KB |
| 1MB | 8MB(8tables) | 1KB |
| 2MB | 4MB(2tables) | 512B |
TABLE 1
For qcow2 disk addressing, the importance of L1 is greater than that of L2, L1 corruption means that the entire disk is not addressable, and L2table corruption means that a portion of the disk is not addressable.
In order to improve the recovery capability of the qcow2 format, the invention designs a copy storage mode of L1/L2, and once the qcow2 file L1/L2 is damaged, the restoration can be carried out through the data copy, so that the data security of the whole virtualization system is improved. The specific implementation process is as follows:
selection of L1 blocks
The first three blocks (0-2) of the Qcow2 file have special uses according to the general usage rules of the protocol, and the L1 table defaults to starting from block 3, and the required L1 table size is determined for a given disk size, see Table 1.
Since the L1 table is extremely important for qcow2, it is preferable to store multiple copies of L1.
In an embodiment of the method provided by the present invention, before acquiring the copy of the table where the damage occurs in the qcow2 file, the method further includes:
acquiring a data block i required for storing data of an L1 table, wherein i is an integer greater than or equal to 1;
from MnStarting a block, and storing data of an L1 table as a copy of an L1 table in i consecutive data blocks, wherein M is a prime number greater than or equal to 11, and n is an integer greater than or equal to 1;
the storage location of the copy of the L1 table is recorded.
Specifically, if the data of the L1 table needs 2 blocks to be carried, the mth block needs to be selected when making the copynM and Mn+1 block to carry the data of the L1 table.
The prime number of 11 or more is selected mainly because the protocol in the format of the qcow2 file is specified to be configured in advance from the 0 th block to the 10 th block, undefined blocks can be selected for making copies, and the prime number is selected mainly because the increase of copies does not affect the occupation of other data on the blocks in file expansion, thereby reducing the conflict of the occupation of storage space.
Of course, in view of both of the above aspects, it is preferable that M be 17.
For example, the present invention describes the retention of duplicate blocks by taking the L1 table as an example of occupying a complete block, wherein a plurality of consecutive L1 table blocks are the same. Since the L1 table is extremely important for qcow2, it is preferable to store multiple copies of the L1 table, taking the block number 17nThe block (n ═ 1,2,3, 4 …) is dedicated to storing a copy of the L1 table, i.e., the 17 th and 289 th blocks … are necessarily blocks of the L1 table. For example, for a qcow2 file with a block size of 1MB and a disk size of 1TB, there are 4 copies of the L1 table at 17 th, 289 th, 4913 th and 83521 th blocks, respectively.
In order to comply with the existing addressing procedure of the protocol and improve the recovery capability of the addressing information, the recording the storage location of the copy of the L1 table includes:
and adding the information of the storage position of the copy of the L1 table to a qcow2 file header.
Of course, in practical applications, the purpose of recording can also be achieved by maintaining a list of copies of the L1 table, but compared with a method of directly storing the storage location information in the header, storing the storage location information in the header can better be compatible with the existing addressing procedure, and the cost of information maintenance is reduced.
Specifically, when creating a qcow2 file, at least creating a 289 th block, marking a 17 th block as a main L1 block and the 289 second block as a slave block, and ensuring that an L1 table has at least two copies during creation; therein, the main L1 block always starts with 17 blocks and is recorded inside the qcow2 header.
Wherein, if the data of the L1 table is carried by at least two blocks, at the time of making the copy, 17 th ordernEach data block being the starting block of the copy, through 17nThe next consecutive data block of the data block continues to carry the data of the L1 table.
In the file management process, the number of copies of the L1 table and the stored content are changed, and the following description is made respectively:
(1) the number of copies of the L1 table varies:
at the M thnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
when the fact that the qcow2 file is expanded is detected, whether a target block with the number meeting the integral power of M exists in blocks newly added in the expanding operation or not is searched;
if there is a target block whose number corresponds to the integer power of M, storing the data of the L1 table in the continuous i data blocks from the target block;
the storage location of the newly added copy of the L1 table is recorded.
Specifically, the following flow is added when expanding the qcow2 file:
considering the reduced provisioning of qcow2, increasing as data increases, when expanding a qcow2 file, if the expanded block number is an integer power of 17, the data of the L1 table is synchronized to the block while using the next block for conventional use, otherwise, the block is directly used for conventional use.
When the file is expanded, a new copy is added in time, so that normal use of addressing capability can be ensured, and the occurrence probability of unavailable addressing information is reduced.
(2) The data of the L1 table changes:
at the M thnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
upon detecting that the data in the L1 table is updated, synchronizing the updated data in the L1 table to a block number MnWherein n ∈ [1, k ] in the block](ii) a Wherein:
k＝round_up(logM bMax)-1；
wherein, bMax is the value of the maximum block number of the qcow2 file, and round _ up represents rounding up.
Specifically, the following flow is added when the L1 table of qcow2 is updated:
a. if the maximum block number of the qcow2 file is bMax, determining the value of k, wherein k is round _ up (log)17 bMax)；
b. For n ∈ [1, k)]Synchronizing the L1 data to block number 17nAll of the blocks of (1).
By synchronously updating the duplicate data, the consistency of the data is ensured, the correctness of the data is effectively ensured, and the normal use of the addressing capability is ensured.
Selection of master-slave blocks by L2
As can be seen from Table 1, the number of L2 blocks is determined after the block size and disk size are determined. For example, if the block size is 512KB and the disk size is 1TB, a maximum of 32 blocks are required to store the L2table, i.e., there are 32 different L2 tables.
Referring to table 1 above, in one qcow2 file, the number of L1 tables is one, and the number of L2 tables is at least two, the amount of data stored in the L2table, the number of L2 tables and the utilization rate of storage space are considered comprehensively, different from the unlimited number of copies in the L1 table, the number of copies in the L2table is set to 2.
Specifically, before obtaining the damaged table from the copy pre-stored in the qcow2 file, the method further includes:
selecting two blocks as the copies of the L2table according to a preset number selection strategy;
the storage location of the copy of the L2table is recorded.
The number selection policy can be set according to actual needs, and the following management policies are proposed in the invention, specifically as follows:
wherein, according to a preset number selection policy, selecting two blocks as the copies of the L2table includes:
storing data of an L2table in an available free block with even number as a main block of the L2table copy;
the data of the L2table stored in the available free block with the number of (a + b/2) mod b is selected as the slave block of the L2table copy, wherein a is the number of the master block of the L2table copy, and b is the maximum value of the power of 2 which is not more than bMax, wherein bMax is the maximum block number of the qcow2 file.
Further, if there is no free block in the even numbered blocks in the qcow2 file, a new block is added at the end of the qcow2 file, and when the number of the new block is even, the data of the L2table is stored in the newly added block.
Specifically, the position of the main block of the L2table selects an available free block with an even current block number, and if the available free block does not exist, the free block is added at the end of the file, so that the free block can be always found, and the creation of the main block of the L2table is ensured.
After determining the copy master block position of the L2table, if the master block number of a copy of a certain L2table is a, the current maximum block number of the qcow2 file is bMax, and b is the maximum value of the power of 2 not greater than bMax, the copy slave block number is defined as (a + b/2) mod b, for example, a is 10, bMax is 945, and at this time, b is 512, then the slave block of the copy of the L2table is located at the 266 th block.
According to the acquisition mode of the copy slave block of the L2table, the slave block changes along with the increase of the bMax, and at this time, the slave block of the L2 needs to be migrated to a new position. By using the b value, more migration from the block can be avoided.
In combination with the selection of the L1 block, it can be found that the duplicate position of the L1 table does not collide with the duplicate position of the L2 table.
In order to comply with the existing addressing procedure of the protocol and improve the recovery capability of the addressing information, the recording the storage location of the copy of the L2table includes:
adding information of the storage location of the copy of the L2table to the L1 table.
Of course, in practical applications, the purpose of recording can also be achieved by maintaining a list of copies of the L2table, but compared with a method of directly storing the storage location information in the header, storing the storage location information in the header can better be compatible with the existing addressing procedure, and the cost of information maintenance is reduced.
After selecting two blocks as the copies of the L2table according to the preset number selection policy, the method further includes:
when the qcow2 file is detected to be expanded, judging whether the two blocks selected according to the number selection strategy change according to the total block number of the qcow2 file;
if the change occurs, recalculating the numbers of the blocks corresponding to the copies of all the L2 tables to obtain a newly determined block;
migrating data in the L2table from an original storage location to the newly determined block, and updating a storage location of a copy of the L2 table.
Specifically, when the qcow2 file is expanded, whether the b value before and after expansion changes is calculated;
if the b value changes, acquiring a copy of the L2table, for example, acquiring the copy from the L1 table, and calculating the positions of slave blocks of the copies of all L2 tables; and finishing data migration according to the re-determined position, so that the storage position of the copy conforms to a preset number selection rule, and the accuracy of the data storage position is ensured.
Optionally, after the migrating the data in the L2table from the original storage location to the newly determined block, the method further includes:
and marking the use state of the block of the original storage position as idle.
Specifically, in order to accurately maintain the use state of the blocks in the file, the blocks of the table data of the block storage L2, which are originally used as copies, are marked as free after the data are migrated, so that other uses can be performed subsequently, and accurate management of the storage resources is realized.
In order to ensure consistency of the copy data content and the L2table data and better maintain addressability, after selecting two blocks as the copies of the L2table according to the preset number selection policy, the method further comprises:
upon detecting that the data in the L2table is updated, synchronizing the updated data in the L2table into the two blocks.
By synchronously updating the duplicate data, the consistency of the data is ensured, the correctness of the data is effectively ensured, and the normal use of the addressing capability is ensured.
Data recovery for L1/L2 tables
The reading the data of the copy comprises:
when a plurality of copies of the damaged table are available, detecting the data integrity of the copies according to a preset copy reading sequence;
and when the data of the copy is detected to be complete, reading the data of the copy, otherwise, selecting the next copy for data integrity detection until the copy with complete data is detected.
Specifically, according to the slave block position definition of L1/L2, the duplicate data can be conveniently found, and the master block data is covered by the duplicate data.
In summary, the invention provides an improved qcow2 file format, which improves the recovery capability of qcow2 by storing multiple copies of L1/L2table in a file, and after a problem occurs in a certain L1/L2, the data of the copy can be recovered, and the recovery capability of the qcow2 file format is improved by data redundancy. In contrast, once the L1 table entry is damaged, the conventional qcow2 format causes large-area disk data loss and is difficult to recover. Therefore, the invention provides a data block copy selection strategy of L1/L2, improves the availability of the key data of L1/L2 by using a series of copy data, improves the reliability of key data in a qcow2 format, greatly improves the recovery capability of the qcow2 after the key data are damaged in a severe environment, and realizes the recovery of the key data to the maximum extent.
The present invention provides a computer-readable storage medium storing a computer program for execution by a processor to perform operations comprising:
when an addressing request of a qcow2 file is received, reading an L1 table and/or an L2table in the qcow2 file;
when the data of the L1 table or the L2table are detected to be damaged, acquiring a copy of the damaged table stored in the qcow2 file in advance;
reading the data of the copy;
and responding to the addressing request according to the copy data.
In an embodiment of the computer-readable storage medium provided by the present invention, before obtaining the copy of the table where the damage occurs in the qcow2 file, the computer program is further executed by a processor to implement the following operations, including:
acquiring a data block i required for storing data of an L1 table, wherein i is an integer greater than or equal to 1;
from MnStarting a block, and storing data of an L1 table as a copy of an L1 table in i consecutive data blocks, wherein M is a prime number greater than or equal to 11, and n is an integer greater than or equal to 1;
the storage location of the copy of the L1 table is recorded.
In an embodiment of the computer-readable storage medium provided by the present invention, a value of M is 17.
In an embodiment of the computer-readable storage medium provided by the present invention, the recording a storage location of the copy of the L1 table includes:
and adding the information of the storage position of the copy of the L1 table to a qcow2 file header.
In an embodiment of the computer-readable storage medium provided by the present invention, the M-th datanStarting with a block, after storing the data of the L1 table in consecutive i data blocks, the computer program is further executed by the processor to perform operations comprising:
when the fact that the qcow2 file is expanded is detected, whether a target block with the number meeting the integral power of M exists in blocks newly added in the expanding operation or not is searched;
if there is a target block whose number corresponds to the integer power of M, storing the data of the L1 table in the continuous i data blocks from the target block;
the storage location of the newly added copy of the L1 table is recorded.
In an embodiment of the computer-readable storage medium provided by the present invention, the M-th datanStarting with a block, after storing the data of the L1 table in consecutive i data blocks, the computer program is further executed by the processor to perform operations comprising:
upon detecting that the data in the L1 table is updated, synchronizing the updated data in the L1 table to a block number MnWherein n ∈ [1, k ] in the block](ii) a Wherein:
k＝round_up(logM bMax)-1；
wherein, bMax is the value of the maximum block number of the qcow2 file, and round _ up represents rounding up.
In an embodiment of the computer-readable storage medium provided by the present invention, before obtaining the copy of the table where the damage occurs in the qcow2 file, the computer program is further executed by a processor to implement the following operations, including:
selecting two blocks as the copies of the L2table according to a preset number selection strategy;
the storage location of the copy of the L2table is recorded.
In an embodiment of the computer-readable storage medium provided by the present invention, the selecting two blocks as the copies of the L2table according to the preset number selection policy includes:
storing data of an L2table in an available free block with even number as a main block of the L2table copy;
the data of the L2table stored in the available free block with the number of (a + b/2) mod b is selected as the slave block of the L2table copy, wherein a is the number of the master block of the L2table copy, and b is the maximum value of the power of 2 which is not more than bMax, wherein bMax is the maximum block number of the qcow2 file.
In an embodiment of the computer-readable storage medium provided by the present invention, the selecting two blocks as the copies of the L2table according to the preset number selection policy, the computer program further being executed by a processor to implement operations comprising:
if no free block exists in the even numbered blocks in the qcow2 file, adding a new block at the tail of the qcow2 file, and storing the data of the L2table in the newly added block when the number of the new block is even.
In an embodiment of the computer-readable storage medium provided by the present invention, the recording a storage location of the copy of the L1 table includes:
adding information of the storage location of the copy of the L2table to the L1 table.
In an embodiment of the computer-readable storage medium provided by the present invention, after selecting two blocks as copies of the L2table according to the preset number selection policy, the computer program is further executed by a processor to implement operations including:
when the qcow2 file is detected to be expanded, judging whether the two blocks selected according to the number selection strategy change according to the total block number of the qcow2 file;
if the change occurs, recalculating the numbers of the blocks corresponding to the copies of all the L2 tables to obtain a newly determined block;
migrating data in the L2table from an original storage location to the newly determined block, and updating a storage location of a copy of the L2 table.
In an embodiment of the computer readable storage medium provided by the present invention, after the migrating the data in the L2table from the original storage location to the newly determined block, the computer program is further executed by a processor to implement operations comprising:
and marking the use state of the block of the original storage position as idle.
In an embodiment of the computer-readable storage medium provided by the present invention, after selecting two blocks as copies of the L2table according to the preset number selection policy, the computer program is further executed by a processor to implement operations including:
upon detecting that the data in the L2table is updated, synchronizing the updated data in the L2table into the two blocks.
In an embodiment of the computer-readable storage medium provided by the present invention, the reading the data of the copy includes:
when a plurality of copies of the damaged table are available, detecting the data integrity of the copies according to a preset copy reading sequence;
and when the data of the copy is detected to be complete, reading the data of the copy, otherwise, selecting the next copy for data integrity detection until the copy with complete data is detected.
According to the embodiment of the computer-readable storage medium provided by the invention, when an addressing request of a qcow2 file is received, an L1 table and/or an L2table in the qcow2 file are/is read, if data of the L1 table or the L2table are damaged, a copy of the damaged table stored in the qcow2 file in advance is obtained, the purpose of repairing addressing information through the copy of the data is realized by reading the data of the copy and responding to the addressing request, the recovery capability of the qcow2 format is improved, and the data security of the whole virtualization system is further improved.
It will be understood by those of ordinary skill in the art that all or part of the steps of the above embodiments may be implemented using a computer program flow, which may be stored in a computer readable storage medium and executed on a corresponding hardware platform (e.g., system, apparatus, device, etc.), and when executed, includes one or a combination of the steps of the method embodiments.
Alternatively, all or part of the steps of the above embodiments may be implemented by using an integrated circuit, and the steps may be respectively manufactured as an integrated circuit module, or a plurality of the blocks or steps may be manufactured as a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The devices/functional modules/functional units in the above embodiments may be implemented by general-purpose computing devices, and they may be centralized on a single computing device or distributed on a network formed by a plurality of computing devices.
Each device/function module/function unit in the above embodiments may be implemented in the form of a software function module and may be stored in a computer-readable storage medium when being sold or used as a separate product. The computer readable storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (14)
1. An addressing method of a virtual disk format, comprising:
when an addressing request of a qcow2 file is received, reading an L1 table and/or an L2table in the qcow2 file;
when the data of the L1 table or the L2table are detected to be damaged, recording the storage position of the copy of the L1 table;
acquiring a copy of the damaged table pre-stored in the qcow2 file;
reading the data of the copy;
responding to the addressing request according to the copy data;
before the obtaining of the copy of the table with the damage in the qcow2 file stored in advance, the method further includes:
acquiring a data block i required for storing data of an L1 table, wherein i is an integer greater than or equal to 1;
from MnStarting a block, and storing data of an L1 table as a copy of an L1 table in i consecutive data blocks, wherein M is a prime number greater than or equal to 11, and n is 1,2,3 … …, k;
recording the storage location of the copy of the L1 table;
where k is round _ up (log)MbMax)-1；
Wherein, bMax is the value of the maximum block number of the qcow2 file, and round _ up represents rounding up.
2. The method of claim 1, wherein M has a value of 17.
3. The method according to claim 1 or 2, wherein the recording the storage location of the copy of the L1 table comprises:
and adding the information of the storage position of the copy of the L1 table to a qcow2 file header.
4. The method of claim 1 or 2, wherein the M-th order is selected from the group consisting ofnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
when the fact that the qcow2 file is expanded is detected, whether a target block with the number meeting the integral power of M exists in blocks newly added in the expanding operation or not is searched;
if there is a target block whose number corresponds to the integer power of M, storing the data of the L1 table in the continuous i data blocks from the target block;
the storage location of the newly added copy of the L1 table is recorded.
5. The method of claim 1 or 2, wherein the M-th order is selected from the group consisting ofnStarting from blocks, after storing data of the L1 table in consecutive i data blocks, the method further comprises:
upon detecting that the data in the L1 table is updated, synchronizing the updated data in the L1 table to a block number MnAll of the blocks of (1).
6. The method of claim 1, wherein the obtaining the table with the damage is prior to the copy pre-stored in the qcow2 file, and the method further comprises:
selecting two blocks as the copies of the L2table according to a preset number selection strategy;
the storage location of the copy of the L2table is recorded.
7. The method of claim 6, wherein selecting two blocks as copies of an L2table according to a preset number selection policy comprises:
storing data of an L2table in an available free block with even number as a main block of the L2table copy;
the data of the L2table stored in the available free block with the number of (a + b/2) mod b is selected as the slave block of the L2table copy, wherein a is the number of the master block of the L2table copy, and b is the maximum value of the power of 2 which is not more than bMax, wherein bMax is the maximum block number of the qcow2 file.
8. The method of claim 7, wherein two blocks are selected as copies of an L2table according to a preset number selection policy, the method further comprising:
if no free block exists in the even numbered blocks in the qcow2 file, adding a new block at the tail of the qcow2 file, and storing the data of the L2table in the newly added block when the number of the new block is even.
9. The method according to any one of claims 6 to 8, wherein the recording the storage location of the copy of the L1 table comprises:
adding information of the storage location of the copy of the L2table to the L1 table.
10. The method according to any one of claims 6 to 8, wherein after selecting two blocks as copies of the L2table according to the preset number selection policy, the method further comprises:
when the qcow2 file is detected to be expanded, judging whether the two blocks selected according to the number selection strategy change according to the total block number of the qcow2 file;
if the change occurs, recalculating the numbers of the blocks corresponding to the copies of all the L2 tables to obtain a newly determined block;
migrating data in the L2table from an original storage location to the newly determined block, and updating a storage location of a copy of the L2 table.
11. The method of claim 10, wherein after migrating the data in the L2table from the original storage location to the newly determined block, the method further comprises:
and marking the use state of the block of the original storage position as idle.
12. The method according to any one of claims 6 to 8, wherein after selecting two blocks as copies of the L2table according to the preset number selection policy, the method further comprises:
upon detecting that the data in the L2table is updated, synchronizing the updated data in the L2table into the two blocks.
13. The method of claim 1, wherein reading the data of the copy comprises:
when a plurality of copies of the damaged table are available, detecting the data integrity of the copies according to a preset copy reading sequence;
and when the data of the copy is detected to be complete, reading the data of the copy, otherwise, selecting the next copy for data integrity detection until the copy with complete data is detected.
14. A computer-readable storage medium storing a computer program for execution by a processor to perform the method of any one of claims 1 to 13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811172062.5A CN109388524B (en) | 2018-10-09 | 2018-10-09 | Virtual disk format addressing method and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811172062.5A CN109388524B (en) | 2018-10-09 | 2018-10-09 | Virtual disk format addressing method and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109388524A CN109388524A (en) | 2019-02-26 |
| CN109388524B true CN109388524B (en) | 2021-10-01 |
Family
ID=65426742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811172062.5A Active CN109388524B (en) | 2018-10-09 | 2018-10-09 | Virtual disk format addressing method and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109388524B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1612112A (en) * | 2003-10-30 | 2005-05-04 | 微软公司 | Shadow page tables for address translation control |
| US8112396B2 (en) * | 2006-06-07 | 2012-02-07 | Emc Corporation | Backup and recovery of integrated linked databases |
| US8386733B1 (en) * | 2008-02-15 | 2013-02-26 | Symantec Corporation | Method and apparatus for performing file-level restoration from a block-based backup file stored on a sequential storage device |
| CN103907099A (en) * | 2011-11-02 | 2014-07-02 | 华为技术有限公司 | Uncached static short address translation table in cache coherent computer system |
| US9135118B2 (en) * | 2011-03-07 | 2015-09-15 | Aptare, Inc. | System to catalog and search point-in-time instances of a file system |
| CN105224478A (en) * | 2015-09-25 | 2016-01-06 | 联想(北京)有限公司 | A kind of formation of mapping table, renewal and restoration methods and electronic equipment |
| CN106959927A (en) * | 2016-01-08 | 2017-07-18 | 阿里巴巴集团控股有限公司 | The method and device of the positional information of logical partition in acquisition virtual machine |
| CN107544870A (en) * | 2017-06-14 | 2018-01-05 | 新华三云计算技术有限公司 | A kind of disk backup method of virtue machine and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999433B (en) * | 2012-11-21 | 2015-06-17 | 北京航空航天大学 | Redundant data deletion method and system of virtual disks |
| US10437681B2 (en) * | 2013-10-30 | 2019-10-08 | Trilio Data, Inc. | Method and apparatus of managing application workloads on backup and recovery system |
| CN103699459A (en) * | 2013-12-31 | 2014-04-02 | 汉柏科技有限公司 | Method and system for incremental backup of virtual machine data based on Qcow2 snapshots |
| CN106033454B (en) * | 2015-03-17 | 2020-01-31 | 联想(北京)有限公司 | Formatting method, processing method and device of virtual file system |
| CN106598496A (en) * | 2016-12-08 | 2017-04-26 | 蓝信工场（北京）科技有限公司 | Method and device for constructing virtual magnetic disk and processing data |
| CN107544871B (en) * | 2017-07-21 | 2020-10-02 | 新华三云计算技术有限公司 | Virtual machine disk backup method and device |
-
2018
- 2018-10-09 CN CN201811172062.5A patent/CN109388524B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1612112A (en) * | 2003-10-30 | 2005-05-04 | 微软公司 | Shadow page tables for address translation control |
| US8112396B2 (en) * | 2006-06-07 | 2012-02-07 | Emc Corporation | Backup and recovery of integrated linked databases |
| US8386733B1 (en) * | 2008-02-15 | 2013-02-26 | Symantec Corporation | Method and apparatus for performing file-level restoration from a block-based backup file stored on a sequential storage device |
| US9135118B2 (en) * | 2011-03-07 | 2015-09-15 | Aptare, Inc. | System to catalog and search point-in-time instances of a file system |
| CN103907099A (en) * | 2011-11-02 | 2014-07-02 | 华为技术有限公司 | Uncached static short address translation table in cache coherent computer system |
| CN105224478A (en) * | 2015-09-25 | 2016-01-06 | 联想(北京)有限公司 | A kind of formation of mapping table, renewal and restoration methods and electronic equipment |
| CN106959927A (en) * | 2016-01-08 | 2017-07-18 | 阿里巴巴集团控股有限公司 | The method and device of the positional information of logical partition in acquisition virtual machine |
| CN107544870A (en) * | 2017-06-14 | 2018-01-05 | 新华三云计算技术有限公司 | A kind of disk backup method of virtue machine and device |
Non-Patent Citations (1)
| Title |
|---|
| 几种虚拟机镜像格式及其性能测评;许艳军，姜进磊，王博，杨广文;《计算机应用》;20130630;第23页第2.2节、图2 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109388524A (en) | 2019-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12117911B2 (en) | Remote data replication method and system | |
| US11249834B2 (en) | Storage system with coordinated recovery across multiple input-output journals of different types | |
| US10983955B2 (en) | Data unit cloning in memory-based file systems | |
| CN102331949B (en) | Methods for generating and restoring memory snapshot of virtual machine, device and system | |
| CN105718217B (en) | A kind of method and device of simplify configuration storage pool data sign processing | |
| CN105190573A (en) | Reduced redundancy in stored data | |
| US8572338B1 (en) | Systems and methods for creating space-saving snapshots | |
| CN107729536B (en) | Data storage method and device | |
| CN109522154B (en) | Data recovery method and related equipment and system | |
| CN104077380A (en) | Method and device for deleting duplicated data and system | |
| US8799595B1 (en) | Eliminating duplicate data in storage systems with boot consolidation | |
| CN107133132B (en) | Data sending method, data receiving method and storage device | |
| US10572335B2 (en) | Metadata recovery method and apparatus | |
| CN108073352B (en) | Virtual disk processing method and device | |
| CN111522502B (en) | Data deduplication method and device, electronic equipment and computer-readable storage medium | |
| CN104572656A (en) | Method and device for recycling space of disk mirror image | |
| WO2016029743A1 (en) | Method and device for generating logical disk of virtual machine | |
| CN105446664A (en) | Memory file storage method and storage apparatus | |
| CN104965835A (en) | Method and apparatus for reading and writing files of a distributed file system | |
| CN116009761A (en) | Data writing method and related equipment | |
| WO2019037587A1 (en) | Data restoration method and device | |
| CN109388524B (en) | Virtual disk format addressing method and computer readable storage medium | |
| CN105068896A (en) | Data processing method and device based on RAID backup | |
| CN108334457B (en) | IO processing method and device | |
| CN108769123B (en) | Data system and data processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |