[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2018054079A1 - 一种存储文件的方法、第一虚拟机及名称节点 - Google Patents

一种存储文件的方法、第一虚拟机及名称节点 Download PDF

Info

Publication number
WO2018054079A1
WO2018054079A1 PCT/CN2017/085351 CN2017085351W WO2018054079A1 WO 2018054079 A1 WO2018054079 A1 WO 2018054079A1 CN 2017085351 W CN2017085351 W CN 2017085351W WO 2018054079 A1 WO2018054079 A1 WO 2018054079A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual machine
data
written
storage area
virtual
Prior art date
Application number
PCT/CN2017/085351
Other languages
English (en)
French (fr)
Inventor
李亿
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018054079A1 publication Critical patent/WO2018054079A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a method for storing a file, a first virtual machine, and a name node.
  • a distributed file system includes a client, a data node, and a name node; wherein the data node is used to store files and the name node is used to manage files stored on the data nodes.
  • the client can query the files stored in each data node through the name node and obtain the address of each data node, so as to read the file from the data node or write the file to the data node.
  • a data node in a distributed file system can be either a physical server or a virtual machine.
  • the virtual hard disk of the virtual machine is provided by the distributed block storage system, and writing the file to the virtual machine actually writes the file to the virtual hard disk of the virtual machine and writes to the virtual hard disk.
  • the file is implemented by writing files to the physical hard disk managed by the distributed block storage system.
  • the distributed file system uses the file copy mechanism when storing files on the virtual hard disk, and saves the same file in N (N is an integer greater than 1) virtual hard disk in the distributed file system;
  • the block storage system also uses the file copy mechanism to save the files in the same virtual hard disk in M (M is an integer greater than 1) physical hard disk. Because the file copy mechanism is adopted in both the distributed file system and the distributed block storage system, the number of files actually saved in the same file on the physical hard disk is N*M, resulting in redundancy of file shares. The redundancy of the number of copies of the file saved in the same file wastes storage space and affects the processing performance of the system.
  • the first method is, for a file to be stored, only in a virtual machine of the distributed file system. Store the file in .
  • the file can be accessed only through the virtual machine. If the virtual machine fails, the virtual machine needs to wait for the virtual machine to return to normal before providing the file read and write service to the client again, resulting in a distributed file system.
  • the second method is to use the hot standby mechanism of the virtual machine, that is, to configure the hot standby virtual machine corresponding to the primary virtual machine, and the hot standby virtual machine writes files synchronously with the primary virtual machine.
  • the distributed file system switches to the hot standby virtual machine to continue to provide file read and write services to the client.
  • the distributed file system needs a certain waiting time when switching to the hot standby virtual machine, which causes the distributed file system to fail to provide file read and write services to the client during the waiting time, so that the distributed file system is available. Reduced; and the hot standby virtual machine does not provide external services before switching to the primary virtual machine, resulting in wasted resources.
  • the existing methods for solving the problem of file number redundancy in a distributed file system may result in low availability of the distributed file system and inability to better solve the problem of file number redundancy.
  • the embodiment of the invention provides a method for storing a file, a first virtual machine and a name node, which are used to solve the problem of redundancy of the number of files existing when the distributed file system stores the file, and improve the availability of the system.
  • an embodiment of the present invention provides a method for storing a file in a distributed file system, where
  • the distributed file system includes a name node and a plurality of virtual machines as data nodes, wherein the plurality of virtual machines share the same storage area; the method includes:
  • the first virtual machine receives the data to be written sent by the client, the address of the second virtual machine, and then writes the received data to be written to the storage area shared by the plurality of virtual machines, and generates or updates the metadata of the data to be written;
  • the first virtual machine sends the metadata generated or updated by the first virtual machine to the second virtual machine according to the received address of the second virtual machine.
  • the first virtual machine is a virtual machine in the plurality of virtual machines that has the right to write data to the storage area by the name node
  • the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines
  • the metadata of the data to be written includes, but is not limited to, a storage location of the data to be written, a file name of the data to be written, and a file directory of the data to be written.
  • the data to be written written by the first virtual machine in the storage area is only saved in the storage area.
  • the data to be written only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no saved file due to the file copy mechanism adopted by both the distributed file system and the distributed block storage system. The problem of number of copies.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has the slave
  • the storage area reads the permission to write data.
  • the method before the first virtual machine writes the data to be written to the storage area, the method further includes: the first virtual machine receives the write permission identifier of the first virtual machine sent by the client, and the write permission identifier is a name node.
  • the client requests the name node to write the data to be written to the distributed file system, it is sent to the client, and is used to specify that the first virtual machine has the right to write the data to be written to the storage area.
  • a plurality of virtual machines share a storage area.
  • the multiple virtual machines can be mounted on the same virtual hard disk provided by the distributed block storage system.
  • the virtual hard disk includes a storage area shared by multiple virtual machines.
  • the metadata of the data to be written sent by the first virtual machine to the second virtual machine has the following two purposes:
  • the metadata is used by the second virtual machine to generate or update file information recorded in its own operating system, and the file information is used for operating system storage.
  • the data to be written is read in the area.
  • the metadata is used by the second virtual machine to read the data to be written from the storage area.
  • the second virtual machine can read the data to be written in the storage area shared by the multiple virtual machines according to the metadata of the data to be written sent by the first virtual machine.
  • the second virtual machine may be designated by the name node to have the right to read the data to be written from the storage area.
  • an embodiment of the present invention provides a method for storing a file in a distributed file system, where the distributed file system includes a name node and a plurality of virtual machines as data nodes, and the plurality of virtual machines share the same storage area; Methods include:
  • the name node After receiving the request message that the client requests to write the data to be written to the distributed file system, the name node sends a response message corresponding to the request message to the client.
  • the response message sent by the name node to the client includes the address of the first virtual machine and the address of the second virtual machine.
  • the response message further indicates that the first virtual machine has written to the storage area for multiple virtual machines.
  • a virtual machine of the authority of the data, the second virtual machine being a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the response message sent by the name node specifies that one of the plurality of virtual machines has data to be written into the shared storage area. Permissions, so data written to a shared storage area will only be saved in that storage area. For the data written to the shared storage area, only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no file copy mechanism due to the distributed file system and the distributed block storage system. The resulting problem of redundant copies of saved files.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has a slave
  • the storage area reads the permission to write data.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • the name node may indicate the rights of the first virtual machine and the rights of the second virtual machine to the client by using a response message in the following two manners:
  • the response message sent by the name node to the client further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, wherein the write permission identifier and the read permission identifier respectively indicate the first virtual machine Permissions and permissions of the second virtual machine, that is, the write permission identifier is used to specify that the first virtual machine has the right to write data to be written to the storage area, and the read permission identifier is used to specify that the second virtual machine has read from the storage area. Permission to write data.
  • the address of the first virtual machine and the address of the second virtual machine in the response message sent by the name node to the client are arranged according to a preset rule, and the preset rule indicates the authority of the first virtual machine and the second virtual
  • the authority of the machine that is, the first virtual machine has the right to write the data to be written to the storage area, and the second virtual machine has the right to read the data to be written from the storage area.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • how the name node handles when the virtual machine fails can include the following two situations:
  • the name node sends the first update information to the client, where
  • the first update information includes an address of the updated first virtual machine
  • the first update information further specifies another virtual machine of the plurality of virtual machines other than the failed first virtual machine as the updated first virtual machine, that is, the specified One of the second virtual machines is the updated first virtual machine, and the updated first virtual machine has the right to write data to the storage area.
  • the name node sends the second update information to the client
  • the second update information includes the address of the updated second virtual machine
  • the second update information further specifies multiple virtual Another virtual machine other than the machine serves as the updated second virtual machine, and the updated second virtual machine has the right to read the data to be written from the storage area.
  • the name node specifies another virtual machine to replace the failed virtual machine, thereby occurring in the first virtual machine and/or the second virtual machine.
  • the distributed file system can still provide services for the client to read and write data, further improving the availability of the distributed file system.
  • an embodiment of the present invention provides a method for storing a file in a distributed file system, where the distributed file system includes a name node, multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; include:
  • the client sends a request message to the name node to write the data to be written to the distributed file system, and then receives the response message corresponding to the request message sent by the name node.
  • the response message includes an address of the first virtual machine and an address of the second virtual machine.
  • the response message further indicates that the first virtual machine is a virtual machine having permission to write data to the storage area among the plurality of virtual machines,
  • the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the client sends the to-be-written data and the address of the second virtual machine to the first virtual machine according to the address of the first virtual machine included in the response message, and indicates the first virtual machine: writing the data to be written, generating or updating the data to be written. Metadata, and transmitting metadata of the data to be written to the second virtual machine according to the address of the second virtual machine.
  • the distributed file system includes multiple virtual machines sharing the same storage area. Therefore, in the distributed file system, the client instructs the first virtual machine to write the data to be written in the shared storage area only in the storage area. Store one copy. For the data to be written, only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no saved file due to the file copy mechanism adopted by both the distributed file system and the distributed block storage system. The problem of number of copies.
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has read from the storage area. Take permission to write data.
  • the number of virtual machines available in a distributed file system that can be used to provide clients with services for reading and writing data to be written is multiple.
  • other virtual machines can provide services for the client to read and write data to be written, so that the availability of the distributed file system is improved, and the hot standby of the virtual machine is avoided in the prior art. There is a waste of resources in the mechanism.
  • the client can obtain the rights of the first virtual machine and the rights of the second virtual machine by using the response message sent by the received name node in the following two manners:
  • the response message received by the client further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, wherein the write permission identifier and the read permission identifier respectively indicate the permissions of the first virtual machine
  • the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area.
  • the read permission identifier is used to specify that the second virtual machine has the right to read the data to be written from the storage area.
  • the address of the first virtual machine and the address of the second virtual machine in the response message received by the client are arranged according to a preset rule, where the preset rule indicates the authority of the first virtual machine and the second virtual machine. Permissions, that is, the first virtual machine has the right to write data to be written to the storage area, and the second virtual machine has the right to read the data to be written from the storage area.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • an embodiment of the present invention provides a method for storing a file in a distributed file system, where the distributed file system includes a name node and a plurality of virtual machines as data nodes, and the plurality of virtual machines share the same storage area; include:
  • the second virtual machine receives the metadata sent by the first virtual machine.
  • the first virtual machine is a virtual machine in which a name node has a right to write data to the storage area
  • the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the metadata is metadata of the data to be written generated or updated after the first virtual machine writes the data to be written to the storage area.
  • the shared storage is performed.
  • the data written in the area will only be saved in the storage area.
  • For the data written to the shared storage area only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no file copy mechanism due to the distributed file system and the distributed block storage system. The resulting problem of redundant copies of saved files.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has read from the storage area. Permission to write data.
  • the number of virtual machines available in a distributed file system that can be used to provide clients with services for reading and writing data to be written is multiple.
  • other virtual machines can provide services for the client to read and write data to be written, so that the availability of the distributed file system is improved, and the hot standby of the virtual machine is avoided in the prior art. There is a waste of resources in the mechanism.
  • the second virtual machine may obtain the read permission identifier of the second virtual machine sent by the client before receiving the metadata sent by the first virtual machine.
  • the read permission identifier is sent to the client when the client requests the name node to write the data to be written to the distributed file system, and the read permission identifier is used to specify that the second virtual machine has read the data to be written from the storage area. permission.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the data to be written in the storage area shared by the plurality of virtual machines may be read according to the received metadata, and the following manner may be adopted:
  • the first type if the second virtual machine reads the data to be written through its own operating system, the second virtual machine is based on the number of elements. According to the file information recorded in the operating system of the own generation or update, the file information can be used by the operating system to read the data to be written from the storage area.
  • the second virtual machine reads the data to be written, the second virtual machine reads the data to be written from the storage area according to the metadata.
  • the second virtual machine can read the data to be written in the storage area shared by the multiple virtual machines according to the metadata of the data to be written sent by the first virtual machine.
  • the second virtual machine is designated by the name node to have the right to read the data to be written from the storage area.
  • an embodiment of the present invention provides a first virtual machine in a distributed file system, where the distributed file system includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area.
  • a virtual machine specifies, by a name node, a virtual machine having a right to write data to the storage area among the plurality of virtual machines; the first virtual machine includes:
  • a receiving module configured to receive data to be written by the client, and an address of the second virtual machine, where the second virtual machine is a virtual machine other than the first virtual machine;
  • a processing module configured to write, to the storage area, data to be written received by the receiving module, and generate or update metadata of the data to be written
  • a sending module configured to send, to the second virtual machine, the processing module to generate or update the metadata according to the address of the second virtual machine received by the receiving module.
  • the metadata of the data to be written includes, but is not limited to, a storage location of the data to be written, a file name of the data to be written, and a file directory of the data to be written.
  • the data to be written written by the processing module to the storage area is only saved in the storage area.
  • the data to be written only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no saved file due to the file copy mechanism adopted by both the distributed file system and the distributed block storage system. The problem of number of copies.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has the slave
  • the storage area reads the permission to write data.
  • the receiving module is further configured to: before the processing module writes the data to be written to the storage area, receive a write permission identifier of the first virtual machine sent by the client, where the write permission identifier is a name node at the client.
  • the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area.
  • Multiple virtual machines share a storage area.
  • the following methods can be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the metadata of the data to be written sent by the sending module to the second virtual machine has the following two purposes:
  • the metadata is used by the second virtual machine to generate or update file information recorded in its own operating system, and the file information is used for operating system storage.
  • the data to be written is read in the area.
  • the metadata is used by the second virtual machine to read the data to be written from the storage area.
  • the second virtual machine can read the data to be written in the storage area shared by the plurality of virtual machines according to the metadata of the data to be written sent by the sending module.
  • the second virtual machine is designated by the name node to have the right to read the data to be written from the storage area.
  • an embodiment of the present invention provides a name node in a distributed file system, where the distributed file system includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; include:
  • a receiving module configured to receive a request message that the client requests to write data to be written to the distributed file system
  • a sending module configured to send, to the client, a response message corresponding to the request message received by the receiving module, where the response message includes an address of the first virtual machine and an address of the second virtual machine, and further, the response message further indicates that the first virtual machine is One of the plurality of virtual machines having the authority to write data to the storage area, and the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the response message sent by the name node specifies that one of the plurality of virtual machines has data to be written into the shared storage area. Permissions, so data written by the processing module to the shared storage area will only be saved in the storage area. For the data written to the shared storage area, only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no file copy mechanism due to the distributed file system and the distributed block storage system. The resulting problem of redundant copies of saved files.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has a slave
  • the storage area reads the permission to write data.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • the response message sent by the sending module may indicate the rights of the first virtual machine and the rights of the second virtual machine to the client in the following two manners:
  • the response message sent by the sending module to the client further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, wherein the write permission identifier and the read permission identifier respectively indicate the authority of the first virtual machine and the second
  • the permission of the virtual machine that is, the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area
  • the read permission identifier is used to specify that the second virtual machine has the right to read the data to be written from the storage area.
  • the address of the first virtual machine and the address of the second virtual machine in the response message sent by the sending module to the client are arranged according to a preset rule, where the preset rule indicates the right of the first virtual machine and the right of the second virtual machine, that is, The first virtual machine has the right to write data to be written to the storage area, and the second virtual machine has the right to read the data to be written from the storage area.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • how the sending module handles when the virtual machine fails may include the following two situations:
  • the first update information is sent to the client, where the first update information includes an address of the updated first virtual machine, and the first update information is also specified in multiple virtual machines. Another virtual machine other than the failed first virtual machine serves as the updated first virtual machine, and the updated first virtual machine has the right to write data to the storage area.
  • the second update information is sent to the client, where the second update information includes the address of the updated second virtual machine, and the second update information is further specified by multiple virtual machines.
  • Another virtual machine is the updated second virtual machine, and the updated second virtual machine has the right to read the data to be written from the storage area.
  • the sending module specifies another virtual machine to replace the failed virtual machine, thereby occurring in the first virtual machine and/or the second virtual machine.
  • the distributed file system can still provide services for the client to read and write data, further improving the availability of the distributed file system.
  • the embodiment of the present invention provides a client, where the distributed file system includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; the client includes:
  • a sending module configured to send, to the name node, a request message for writing a data to be written to the distributed file system
  • a receiving module configured to receive a response message corresponding to the request message sent by the name node
  • the response message includes an address of the first virtual machine and an address of the second virtual machine.
  • the response message further indicates that the first virtual machine is a virtual machine having permission to write data to the storage area among the plurality of virtual machines,
  • the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines;
  • the sending module is further configured to send the to-be-written data and the address of the second virtual machine to the first virtual machine according to the address of the first virtual machine included in the response message received by the receiving module, and instruct the first virtual machine to: write the data to be written Generating or updating metadata of the data to be written, and transmitting metadata of the data to be written to the second virtual machine according to the address of the second virtual machine included in the response message received by the receiving module.
  • the distributed file system includes multiple virtual machines sharing the same storage area. Therefore, in the distributed file system, the sending module instructs the first virtual machine to write the data to be written in the shared storage area only in the storage area. Store one copy. For the data to be written, only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no saved file due to the file copy mechanism adopted by both the distributed file system and the distributed block storage system. The problem of number of copies.
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has read from the storage area. Take permission to write data.
  • the number of virtual machines available in a distributed file system that can be used to provide clients with services for reading and writing data to be written is multiple.
  • other virtual machines can provide services for the client to read and write data to be written, so that the availability of the distributed file system is improved, and the hot standby of the virtual machine is avoided in the prior art. There is a waste of resources in the mechanism.
  • the receiving module can obtain the rights of the first virtual machine and the rights of the second virtual machine by using the response message sent by the name node:
  • the response message received by the receiving module further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, wherein the write permission identifier and the read permission identifier respectively indicate the permissions of the first virtual machine
  • the permission of the second virtual machine that is, the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area
  • the read permission identifier is used to specify that the second virtual machine has read from the storage area to be written. Permissions for data.
  • the address of the first virtual machine and the address of the second virtual machine in the response message received by the receiving module are arranged according to a preset rule, where the preset rule indicates the authority of the first virtual machine and the second virtual machine. Permissions, that is, the first virtual machine has the right to write data to be written to the storage area, and the second virtual machine has the right to read the data to be written from the storage area.
  • the receiving module provides two ways of obtaining the permission of the first virtual machine and the authority of the second virtual machine by receiving the response message.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • An eighth aspect of the present invention provides a second virtual machine in a distributed file system, where the distributed file system includes a name node and a plurality of virtual machines as data nodes, and the plurality of virtual machines share the same storage area;
  • the second virtual machine includes:
  • the receiving module is configured to receive metadata sent by the first virtual machine.
  • the first virtual machine is a virtual machine in which a name node has a right to write data to the storage area
  • the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the metadata is metadata of the data to be written generated or updated after the first virtual machine writes the data to be written to the storage area.
  • the shared storage is performed.
  • the data written in the area will only be saved in the storage area.
  • For the data written to the shared storage area only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no file copy mechanism due to the distributed file system and the distributed block storage system. The resulting problem of redundant copies of saved files.
  • the first virtual machine of the plurality of virtual machines included in the distributed file system has the right to write data to the storage area
  • the second virtual machine of the plurality of virtual machines except the first virtual machine has read from the storage area. Permission to write data.
  • the number of virtual machines available in a distributed file system that can be used to provide clients with services for reading and writing data to be written is multiple.
  • the other virtual machine can provide the client with read and write data to be written.
  • the service improves the availability of the distributed file system, and also avoids the waste of resources in the prior art when the virtual machine hot standby mechanism is adopted.
  • the receiving module may obtain the permission of the second virtual machine in the following manner: the receiving module receives the read permission identifier of the second virtual machine sent by the client before receiving the metadata sent by the first virtual machine.
  • the read permission identifier is sent to the client when the client requests the name node to write the data to be written to the distributed file system, and the read permission identifier is used to specify that the second virtual machine has read the data to be written from the storage area. permission.
  • a plurality of virtual machines share a storage area.
  • the following manners may be adopted: multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the second virtual machine further includes a processing module.
  • the processing module may read the data to be written in the storage area shared by the plurality of virtual machines according to the received metadata, and the processing module may adopt the following manner:
  • the processing module After the receiving module receives the metadata sent by the first virtual machine, if the second virtual machine reads the data to be written through its own operating system, the processing module generates or updates the record recorded in its own operating system according to the metadata. File information, which is used by the operating system to read data to be written from the storage area.
  • the processing module reads the data to be written from the storage area according to the metadata.
  • the processing module can read the data to be written in the storage area shared by the plurality of virtual machines according to the metadata of the data to be written sent by the first virtual machine.
  • the second virtual machine is designated by the name node to have the right to read the data to be written from the storage area.
  • a computer readable storage medium where computer execution instructions are stored, and when at least one processor of a computing node executes the computer to execute an instruction, the computing node executes the first aspect or the first Various possible aspects of the aspects are provided by the method provided, or the method provided by the various possible designs of the second or second aspect described above, or the methods provided by the various possible designs of the third or third aspect described above.
  • a computer program product comprising computer executed instructions stored in a computer readable storage medium.
  • At least one processor of the computing node can read the computer-executable instructions from a computer-readable storage medium, the at least one processor executing the computer-executing instructions, such that the computing node implements the first aspect or the methods provided by the various possible designs of the first aspect Or the method provided by the various possible designs of the second aspect or the second aspect described above, or the method provided by the various possible designs of the third aspect or the third aspect described above.
  • FIG. 1 is a schematic diagram of a connection relationship between a name node, a client, and multiple data nodes in a distributed file system according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a connection relationship between a distributed file system and a distributed block storage system according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for storing a file in a distributed file system according to an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a distributed file system and a distributed block storage system using the method for storing files shown in FIG. 3;
  • FIG. 5 is a schematic structural diagram of a first virtual machine according to an embodiment
  • FIG. 6 is a schematic structural diagram of another first virtual machine according to an embodiment
  • FIG. 7 is a schematic structural diagram of a name node according to an embodiment
  • FIG. 8 is a schematic structural diagram of another name node according to an embodiment
  • FIG. 9 is a schematic structural diagram of a client provided by the embodiment.
  • FIG. 10 is a schematic structural diagram of another client provided by the embodiment.
  • FIG. 11 is a schematic structural diagram of a second virtual machine according to an embodiment
  • FIG. 12 is a schematic structural diagram of another second virtual machine according to an embodiment
  • FIG. 13 is a schematic structural diagram of a distributed file system according to an embodiment of the present invention.
  • the embodiment of the invention relates to a distributed file system.
  • the distributed file system is described in detail below.
  • a distributed file system can include a name node and multiple data nodes.
  • the name node may also be referred to as a master server or other name.
  • the data node may also be referred to as a data server or other name. It should be noted that only the distributed file system includes a client scenario in FIG. 1 . In practice, a distributed file system may include multiple clients.
  • the name node is used to manage multiple data nodes, the name node records information of files stored in each data node (such as metadata files), the service status of each data node, etc.; the data node is used to store files when the client When the file is read and written, the client first requests the name node to obtain the index information of the data node, and then accesses the corresponding data node according to the requested index information to perform file reading and writing.
  • Files may be synchronized between multiple data nodes. For example, when a file needs to be written in two data nodes, one of the data nodes can be written first, and then the data node synchronizes the file to another data node.
  • information interaction between the name node and the data node is also possible.
  • the name node, the data node, and the client may configure the corresponding function implementation on any of the following computing devices.
  • the computing capable device may be a physical device or a virtual device; for example, the physical device may be a personal computer, a notebook computer, a mainframe, a networked computer, a handheld computer, a personal digital assistant, a workstation, etc., and the virtual device may be deployed in a physical device. A virtual machine or container in the device.
  • the virtual hard disk of the virtual machine is provided by the distributed block storage system, and the distributed block storage system manages multiple physical hard disks, and the file written to the virtual hard disk of the virtual machine is actually distributed. The file is written to the physical hard disk managed by the block storage system.
  • the distributed file system in order to ensure its own reliability, the distributed file system generally uses a file copy mechanism when storing files.
  • the file when storing a file, the file is stored on two data nodes, that is, stored in a virtual On the machine 1 and the virtual machine 2; in order to ensure the reliability of the distributed block storage system, the file copy mechanism is also adopted when storing the files of the virtual machine, for example, when the file of the virtual machine 1 is stored, respectively, in the physical The physical hard disk of the server 1, the physical hard disk 3 of the physical server 2, and the physical hard disk 5 of the physical server 3 store the file, and are stored in the physical hard disk 2 of the physical server 1 when the file of the virtual machine 2 is stored.
  • the file is stored on the physical hard disk 4 of the physical server 2 and the physical hard disk 6 of the physical server 3.
  • the file is saved in six copies on the physical hard disk managed by the distributed block storage system. Obviously, the redundancy of the file number saved for the same file will waste storage space and affect the processing performance of the system.
  • the distributed file system can store multiple files, so the number of virtual machines included in the distributed file system is not limited, and the number of virtual hard disks included in each virtual machine is not limited; The number of physical servers included in the block storage system is not limited, and the number of physical hard disks included in each physical server is not limited.
  • the embodiment of the present invention provides a method for storing a file in a distributed file system, where the distributed file system includes a name node and multiple virtual machines as data nodes. Among them, multiple virtual machines share the same storage area. As shown in FIG. 3, the method includes:
  • S301 The client sends a request message to the name node to write to the distributed file system to write data to be written.
  • the data to be written may be video data, audio data, document data or other binary data.
  • the granularity of the data to be written can be a file, a data block, or other granularity.
  • the number of data to be written may be one or more. As long as one or more data is written to the distributed file system after the method shown in FIG. 3 is executed once, the one or more data can be regarded as to be written. data.
  • S302 The name node sends a response message corresponding to the request message to the client.
  • the response message includes an address of the first virtual machine and an address of the second virtual machine, and the response message further indicates that the first virtual machine is a virtual machine having the right to write data to the storage area among the plurality of virtual machines, and the second virtual The machine is a virtual machine other than the first virtual machine among the plurality of virtual machines.
  • the number of the first virtual machines must be one; the number of the second virtual machines may be one or more, and the number of the second virtual machines is not limited in the embodiment of the present invention.
  • only one virtual machine of the plurality of virtual machines has the right to write data to the storage area, because the reason is: if there are multiple virtual machines for writing data to be written, then when the client wants to When the distributed file system writes data to be written, multiple virtual machines receive an instruction to write data to be written; since multiple virtual machines share the same storage area, the write data to be written received by multiple virtual machines
  • the instruction will instruct multiple virtual machines to write the data to be written to the same storage area at the same time. This will cause the instruction to write the data to be written to be indistinguishable from which virtual machine should write the data to be written, resulting in writing the data to be written.
  • the instructions cannot be executed.
  • the reason why the number of the second virtual machines in the embodiment of the present invention may be multiple is that when there are multiple clients to read the data to be written, the second virtual machine can be read to improve the client reading. The efficiency of the data to be written.
  • the second virtual machine may directly access the distributed block storage system. The data to be written is read by the system to avoid the situation that the prior art fails to read the data to be written when the first virtual machine fails.
  • the response message sent by the name node to the client may only indicate that the first virtual machine has the right to write data to the storage area, without indicating the second virtual machine.
  • the reason is: since only one virtual machine of the plurality of virtual machines has the right to write data to the storage area, when the response message indicates that the first virtual machine of the plurality of virtual machines has the data written to the storage area Permission, then the second virtual machine other than the first virtual machine among the plurality of virtual machines has the right to read the data to be written from the storage area by default.
  • the response message further indicates that the second virtual machine has permission to read data to be written from the stored area.
  • the rights of the first virtual machine indicated by the response message and the rights of the second virtual machine are only for the data to be written.
  • the first virtual machine and the second virtual machine share a storage area in which data to be written is written.
  • the response message indicates that the virtual machine 1 is the first virtual machine, the virtual machine 2 is the second virtual machine, and the virtual machine 1 writes the data to be written 1 to the virtual machine 1
  • the metadata of the data to be written 1 is then sent to the virtual machine 2, wherein the virtual machine 2 shares the storage area 1 with the virtual machine 1
  • the response The message indicates that the virtual machine 1 is the first virtual machine, and the virtual machine 3 is the second virtual machine.
  • the virtual machine 1 writes the data to be written 2 into the storage area 2 of the virtual machine 1, and then sends the metadata of the data to be written 2 to The virtual machine 3, wherein the virtual machine 3 shares the storage area 2 with the virtual machine 1.
  • S303 The client sends the to-be-written data and the address of the second virtual machine to the first virtual machine according to the address of the first virtual machine.
  • the client sends the to-be-written data to the first virtual machine and the address of the second virtual machine is used to instruct the first virtual machine to write the data to be written, generate or update the metadata to be written, and according to the second virtual machine.
  • the address sends the metadata of the data to be written to the second virtual machine.
  • S304 The first virtual machine writes data to be written to the storage area shared by the plurality of virtual machines, and generates or updates metadata of the data to be written.
  • the metadata of the data to be written may be used by the first virtual machine and the second virtual machine to read data to be written from the storage area shared by the plurality of virtual machines according to the metadata; the metadata of the data to be written includes but is not limited to: The storage location of the write data, the name of the data to be written, and the directory of the data to be written.
  • S305 The first virtual machine sends the generated or updated metadata to the second virtual machine according to the address of the second virtual machine.
  • the distributed file system may generally include a client or may not include a client. If the distributed file system includes a client, the number of clients includes, but is not limited to, one. In the embodiment of the present invention, in order to more clearly describe the interaction between the client, the name node, the first virtual machine, and the second virtual machine, the client is included in the distributed file system. In actual implementation, the distributed file system may not include the client. In this case, the embodiment of the present invention may be regarded as interaction between the client and the distributed file system.
  • the distributed file system includes multiple virtual machines that mount the same virtual hard disk provided by the distributed block storage system, where the virtual hard disk includes a storage area shared by multiple virtual machines.
  • the data to be written is only in the storage area.
  • For the data to be written only multiple copies are saved due to the file copy mechanism adopted by the distributed block storage system, and there is no saved file due to the file copy mechanism adopted by both the distributed file system and the distributed block storage system. The problem of number of copies.
  • the first virtual machine among the plurality of virtual machines included in the distributed file system has the authority to write data to the storage area, and the plurality of virtual machines The second virtual machine other than the first virtual machine has the right to read the data to be written from the storage area.
  • the number of virtual machines available in a distributed file system that can be used to provide clients with services for reading and writing data to be written is multiple.
  • other virtual machines can provide services for the client to read and write data to be written, so that the availability of the distributed file system is improved, and the hot standby of the virtual machine is avoided in the prior art. There is a waste of resources in the mechanism.
  • a distributed file system and a distributed block storage system using the method shown in FIG. 3 can be as shown in FIG.
  • the distributed file system shown in FIG. 4 includes a first virtual machine, a second virtual machine, a client, and a name node. In actual implementation, there is no limit to the number of second virtual machines and the number of clients.
  • the distributed block storage system shown in Figure 4 contains three physical servers, each containing two physical hard disks.
  • the first virtual machine has the right to write data to the storage area
  • the second virtual machine has the right to read the data to be written from the storage area. Since the first virtual machine and the second virtual machine share the same storage area, the first virtual machine and the second virtual share can share the same virtual hard disk 1.
  • the first virtual machine can write the data to be written to the virtual hard disk 1.
  • the second virtual machine can read the data to be written from the virtual hard disk 1. Therefore, the data to be written is only stored in the distributed file system, that is, stored in the virtual hard disk 1.
  • the data to be written can be stored in the distributed block storage system, for example, the physical hard disk 1 of the physical server 1 is separately stored.
  • the data to be written is only saved in the physical hard disk by three copies.
  • the same file in the distributed file system and the distributed block storage system shown in FIG. 2 is stored in the physical hard disk by six copies.
  • the distributed file system shown in FIG. 4 is used after the method shown in FIG.
  • the data to be written in the distributed block storage system is only saved in the physical hard disk by three copies, thereby greatly reducing the number of copies of the file and solving the problem of redundancy of the file number saved in the distributed file system.
  • the first virtual machine can be used to write data to be written and read data to be written, and the second virtual machine can be used to read data to be written, so that when one of the virtual machines fails, the other virtual machine can be used.
  • a non-failed virtual machine provides the client with services to read and write data to be written, improving system availability.
  • the data is written by the operating system of the first virtual machine, and the client needs to read the data to be written by using the second virtual machine.
  • the operating system of the second virtual machine needs to be read or updated according to the metadata of the data to be written, so that the operating system of the second virtual machine can be implemented from the storage area.
  • the file information is used by an operating system of the second virtual machine to read the to-be-written data from the storage area. There are two ways to update the file information in the operating system of the second virtual machine. First, if the operating system of the second virtual machine can learn the change of the data in the storage area, the file of the data to be written can be updated by itself. The second operating system of the second virtual machine can update the file information of the data to be written according to the metadata of the data to be written sent by the first virtual machine.
  • the storage area is directly written, instead of being written by the operating system of the first virtual machine, the client can pass the second virtual machine.
  • the data to be written in the shared storage area is directly read without being read by the operating system of the second virtual machine.
  • the second virtual machine does not need to generate or update the file information recorded in the operating system according to the metadata of the data to be written, and the second virtual machine can read from the storage area according to the metadata of the data to be written. Waiting for data to be written.
  • the name node needs to indicate, by the corresponding message, that the first virtual machine is a virtual machine having the right to write data to the storage area among the plurality of virtual machines, and the second virtual machine is the first virtual machine among the plurality of virtual machines.
  • the virtual machine other than the virtual machine that is, after the operation of the above S302, the client can obtain not only the address of the first virtual machine and the address of the second virtual machine, but also the first virtual machine has the right to write data to the storage area.
  • the second virtual machine has the right to read data to be written from the storage area.
  • the manner in which the name node notifies the client of the rights of the first virtual machine and the second virtual machine includes but is not limited to the following two types:
  • the response message sent by the name node to the client further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, where the write permission identifier is used to specify that the first virtual machine has a write to the storage area.
  • the permission to write data, the read permission identifier is used to specify that the second virtual machine has the right to read the data to be written from the storage area.
  • the client learns the rights of the first virtual machine and the second virtual machine according to the write permission identifier and the read permission identifier, and sends the write permission identifier to the first virtual machine, and sends the read permission identifier to the second virtual machine, that is, multiple virtual machines The respective permissions of the machine are sent to the corresponding virtual machine.
  • the process of sending the write permission identifier to the first virtual machine by the client may be performed before S303 or after S303, or simultaneously with S303, that is, the write permission identifier, the data to be written, and the second virtual The address of the machine is sent to the first virtual machine.
  • the embodiment of the present invention does not limit the order of execution of the two steps.
  • the client can send the read permission identifier to the second virtual machine.
  • the address of the first virtual machine and the address of the second virtual machine are arranged according to a preset rule in the response message, where the preset rule is used to specify that the first virtual machine has the right to write the data to be written to the storage area, and the second virtual The machine has the right to read data to be written from the storage area.
  • the preset rule may be an order of addresses of virtual machines included in the response message.
  • the name node and the client may agree in advance that the first address sent by the name node to the client is the address of the first virtual machine, and the client may determine after receiving the addresses of the multiple virtual machines included in the response message.
  • the first address is the address of the first virtual machine having the right to write data to the storage area
  • the remaining addresses are the addresses of the second virtual machine having the right to read the data to be written from the storage area.
  • the method for detecting a failure of a virtual machine includes but is not limited to the following three types: first, the name node detects that a virtual machine has failed; second, the client reads through a virtual machine. If the data is read or written, if the data read/write process cannot be completed, it is determined that the virtual machine is faulty, and the client reports the failure of the virtual machine to the name node. Third, the virtual machine periodically performs a self-test. When a virtual machine is discovered When the fault occurs, the fault message is reported to the name node or reported to the name node through the client. Therefore, when a virtual machine in the distributed file system fails, the name node can learn the failure message of the virtual machine in the above three ways, and then take corresponding operations to avoid the distributed file system cannot provide the client. The case of data read and write services.
  • the failure of the virtual machine of the distributed file system may be classified into the following two cases:
  • the name node sends a first update message to the client, where the first update message includes an updated address of the first virtual machine, where the first update message specifies a failure of the plurality of virtual machines.
  • Another virtual machine other than the first virtual machine serves as the updated first virtual machine, that is, the first virtual machine indicating the update has the right to write data to the storage area shared by the plurality of virtual machines; thus, the client needs to write data.
  • it can be written by the updated first virtual machine.
  • the name node specifies another virtual machine other than the failed first virtual machine among the plurality of virtual machines as the updated first virtual machine, that is, the first indicating the update.
  • the virtual machine has the right to write data to the storage area shared by multiple virtual machines.
  • the client wants to write data
  • the virtual machine can write through the updated first virtual machine.
  • the virtual machine can pass the first The second virtual machine reads or reads through the updated first virtual machine.
  • the client does not affect the writing or reading of data by the client, thereby improving the availability of the system.
  • the number of the second virtual machines may be one or more.
  • the following method may be further performed after executing the foregoing method.
  • the name node specifies one or more updated second virtual machines in addition to the plurality of virtual machines included in the distributed file system, and the updated second virtual machine has the right to read the data to be written from the storage area, and the updated second The virtual machine shares the same storage area with the plurality of virtual machines included in the distributed file system; the name node indicates to the client that the updated second virtual machine has the right to read the data to be written from the storage area.
  • the client After receiving the indication of the name node, the client notifies the updated first virtual machine: the metadata of the data to be written is sent to the updated second virtual machine. In this way, when the client wants to read the data to be written, it can be read not only by the second virtual machine but also by the updated second virtual machine.
  • the name node specifies the updated second virtual machine outside the plurality of virtual machines included in the distributed file system to have the right to read the data to be written from the storage area.
  • the client Not only can be read by the second virtual machine, but also read by the updated second virtual machine, which improves the efficiency of the client reading the data to be written.
  • the name node sends the second update information to the client, the second update information includes the address of the updated second virtual machine, and the second update information specifies another virtual machine other than the plurality of virtual machines.
  • the updated second virtual machine has the right to read the data to be written from the storage area.
  • the updated second virtual machine shares the same storage area as the plurality of virtual machines included in the distributed file system.
  • the client notifies the first virtual machine according to the indication of the name node: sending the metadata of the data to be written to the updated Two virtual machines.
  • the first virtual machine may send the metadata of the data to be written to the updated second virtual machine according to the indication of the notification message. In this way, when the client wants to read the data to be written, it can not only pass The second virtual machine that has not failed can be read or read by the first virtual machine, and can also be read by the updated second virtual machine.
  • the name node specifies that the updated second virtual machine has the right to read the data to be written from the storage area, and then the first time when the client wants to write the data to be written
  • the virtual machine writes when the client wants to read the data to be written, can be read not only by the first virtual machine and the second virtual machine that has not failed, but also by the updated second virtual machine.
  • the client when the first virtual machine fails, the client does not affect the writing or reading of the data to be written by the client, thereby improving the availability of the system.
  • the method for storing files in the distributed file system provided by the embodiment of the present invention can solve the problem of redundancy of files stored in the distributed file system.
  • the method for storing files in the distributed file system provided by the embodiment of the present invention does not affect the client to read or write files, thereby improving system availability. .
  • the embodiment of the present invention provides a first virtual machine in a distributed file system, where the distributed file system includes a name node and a plurality of virtual machines as data nodes, and the plurality of virtual machines share the same storage area, and the first virtual machine is A virtual machine in a plurality of virtual machines that is designated by a name node to have data to write to the storage area.
  • the first virtual machine 500 includes:
  • the receiving module 501 is configured to receive data to be written sent by the client, and an address of the second virtual machine, where the second virtual machine is a virtual machine other than the first virtual machine.
  • the processing module 502 is configured to write, to the storage area, the data to be written received by the receiving module 501, and generate or update metadata of the data to be written;
  • the sending module 503 is configured to send or update the metadata to the second virtual machine sending processing module 502 according to the address of the second virtual machine received by the receiving module 501.
  • the receiving module 501 is further configured to: before the processing module 502 writes the data to be written to the storage area, receive the write permission identifier of the first virtual machine sent by the client.
  • the write permission identifier is sent to the client when the client requests the name node to write the data to be written to the distributed file system, and the write permission identifier is used to specify that the first virtual machine has written to the storage area to be written. Permissions for data.
  • multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the metadata is used by the second virtual machine to generate or update file information recorded in its own operating system, and the file information is used by the operating system from the storage.
  • the data to be written is read in the area; or if the second virtual machine reads the data to be written, the metadata is used by the second virtual machine to read the data to be written from the storage area.
  • the second virtual machine is designated by the name node to have read access to the data to be written from the storage area.
  • the first virtual machine 500 provided by the embodiment of the present invention can solve the problem of redundancy of files stored in the distributed file system.
  • the operation of the first virtual machine 500 provided by the embodiment of the present invention may cause the failure of the virtual machine to not affect the client to read or write the file. Increased system availability.
  • first virtual machine 500 provided by the embodiment of the present invention may be used to perform operations performed by the first virtual machine in the method for storing files in the distributed file system shown in FIG. 3, and the first virtual machine 500 does not explain in detail. Description For the implementation, refer to the related description in the method for storing files in the distributed file system shown in FIG. 3.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the embodiment of the present invention further provides a first virtual machine, and the first virtual machine may perform the method provided by the embodiment corresponding to FIG. 3, which may be the same as the first virtual machine 500 shown in FIG. 5.
  • the device where the first virtual machine 600 is located includes at least one processor 601, a memory 602, and a communication interface 603; the at least one processor 601, the memory 602, and the communication interface 603 are all connected by a bus 604;
  • the memory 602 is configured to store a computer execution instruction
  • the at least one processor 601 is configured to execute a computer execution instruction stored by the memory 602, so that the first virtual machine 600 performs data interaction with other devices in the distributed file system through the communication interface 603 to perform the foregoing.
  • the method for storing a file in a distributed file system provided by an embodiment, or causing the first virtual machine 600 to perform data interaction with other devices in the distributed file system through the communication interface 603 to implement a part of the distributed file system or All features.
  • the at least one processor 601 may include different types of processors 601, or include the same type of processor 601; the processor 601 may be any one of the following: a central processing unit (CPU), an ARM processor , Field Programmable Gate Array (FPGA), dedicated processor and other devices with computational processing capabilities. In an optional implementation manner, the at least one processor 601 may also be integrated into a many-core processor.
  • processors 601 may include different types of processors 601, or include the same type of processor 601; the processor 601 may be any one of the following: a central processing unit (CPU), an ARM processor , Field Programmable Gate Array (FPGA), dedicated processor and other devices with computational processing capabilities.
  • the at least one processor 601 may also be integrated into a many-core processor.
  • the memory 602 may be any one or any combination of the following: a random access memory (RAM), a read only memory (ROM), a non-volatile memory (non-volatile memory). (NVM), Solid State Drives (SSD), mechanical hard disks, disks, disk arrays and other storage media.
  • RAM random access memory
  • ROM read only memory
  • NVM non-volatile memory
  • SSD Solid State Drives
  • the communication interface 603 is used by the first virtual machine 600 to perform data interaction with other devices, such as other devices in a distributed file system.
  • the communication interface 603 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, or the like having a network access function.
  • the bus 604 can include an address bus, a data bus, a control bus, etc., for ease of representation, Figure 6 shows the bus with a thick line.
  • the bus 604 can be any one or any combination of the following: an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, and an extended industry standard structure ( Extended Industry Standard Architecture (EISA) bus and other devices for wired data transmission.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the embodiment of the present invention provides a name node in a distributed file system, where the distributed file system includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; as shown in FIG. Node 700 includes:
  • the receiving module 701 is configured to receive a request message that the client requests to write data to be written to the distributed file system;
  • the sending module 702 is configured to send, to the client, a response message corresponding to the request message received by the receiving module 701,
  • the response message includes an address of the first virtual machine and an address of the second virtual machine, and the response message indicates that the first virtual machine is a virtual machine having the right to write data to the storage area among the plurality of virtual machines, and the second virtual machine is more A virtual machine other than the first virtual machine in the virtual machine.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • the response message further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, where the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area, and the read permission The identifier is used to specify that the second virtual machine has the right to read data to be written from the storage area.
  • the address of the first virtual machine and the address of the second virtual machine in the response message are arranged according to a preset rule, where the preset rule is used to specify that the first virtual machine has the right to write the data to be written to the storage area, and the designated The second virtual machine has the right to read data to be written from the storage area.
  • multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the sending module 702 is further configured to: when the first virtual machine fails, send the first update information to the client, where the first update information includes an address of the updated first virtual machine, and the first update information specifies multiple Another virtual machine other than the failed first virtual machine in the virtual machine as the updated first virtual machine, the updated first virtual machine has the right to write data to the storage area; and/or when the second virtual machine occurs
  • the second update information is sent to the client, the second update information includes the address of the updated second virtual machine, and the second update information specifies another virtual machine other than the plurality of virtual machines as the updated second virtual machine, and is updated.
  • the second virtual machine has the right to read data to be written from the storage area.
  • the name node 700 provided by the embodiment of the present invention can solve the problem of redundancy of file shares stored in the distributed file system.
  • the operation of the name node 700 provided by the embodiment of the present invention can make the failure of the virtual machine not affect the client to read or write the file, thereby improving the System availability.
  • name node 700 provided by the embodiment of the present invention can be used to perform the operations performed by the name node in the method for storing files in the distributed file system shown in FIG. 3, and the implementation manner of the name node 700 not explained and described in detail can be referred to.
  • the embodiment of the present invention further provides a name node, which can perform the method provided by the embodiment corresponding to FIG. 3, and can be the same as the name node 700 shown in FIG. 7.
  • the name node 800 includes at least one processor 801, a memory 802, and a communication interface 803; the at least one processor 801, the memory 802, and the communication interface 803 are each connected by a bus 804;
  • the memory 802 is configured to store a computer execution instruction
  • the at least one processor 801 is configured to execute a computer execution instruction stored by the memory 802, so that the name node 800 performs data interaction with other devices in the distributed file system through the communication interface 803 to execute the foregoing embodiment.
  • the at least one processor 801 may include different types of processors 801 or include the same type of processor 801; the processor 801 may be any of the following: a CPU, an ARM processor, an FPGA, a dedicated processor, etc. having a processing process Capable device. In an optional implementation manner, the at least one processor 801 can also be integrated into a multi-core processing. Device.
  • the memory 802 may be any one or any combination of the following: a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • Communication interface 803 is used by name node 800 to interact with other devices, such as other devices in a distributed file system.
  • the communication interface 803 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, and the like having a network access function.
  • the bus 804 can include an address bus, a data bus, a control bus, etc., for ease of representation, Figure 8 shows the bus with a thick line.
  • the bus 804 may be any one or any combination of the following: a device for wired data transmission such as an ISA bus, a PCI bus, or an EISA bus.
  • the embodiment of the present invention provides a client, where the distributed file system of the client includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; as shown in FIG. 9, the client 900 include:
  • the sending module 901 is configured to send, to the name node, a request message that requests to write data to be written to the distributed file system;
  • the receiving module 902 is configured to receive a response message corresponding to the request message sent by the name node, where the response message includes an address of the first virtual machine and an address of the second virtual machine, where the response message indicates that the first virtual machine has a plurality of virtual machines. a virtual machine that stores the right to write data in the storage area, and the second virtual machine is a virtual machine other than the first virtual machine among the plurality of virtual machines;
  • the sending module 901 is further configured to send, to the first virtual machine, the to-be-written data, the address of the second virtual machine, according to the address of the first virtual machine included in the response message received by the receiving module 902, and instruct the first virtual machine to write the to-be-written
  • the data, the metadata for generating or updating the data to be written, and the metadata of the data to be written are transmitted to the second virtual machine according to the address of the second virtual machine included in the response message received by the receiving module 902.
  • the response message further includes a write permission identifier of the first virtual machine and a read permission identifier of the second virtual machine, where the write permission identifier is used to specify that the first virtual machine has the right to write the data to be written to the storage area, and the read permission The identifier is used to specify that the second virtual machine has the right to read data to be written from the storage area.
  • the address of the first virtual machine and the address of the second virtual machine included in the response message are arranged according to a preset rule, where the preset rule is used to specify that the first virtual machine has the right to write the data to be written to the storage area, and the specified The second virtual machine has the right to read data to be written from the storage area.
  • multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the response message further indicates that the second virtual machine has the right to read the data to be written from the storage area.
  • the client 900 provided by the embodiment of the present invention can solve the problem of redundancy of files stored in the distributed file system.
  • the operation of the client 900 provided by the embodiment of the present invention can make the failure of the virtual machine not affect the client to read or write files, thereby improving the System availability.
  • the client 900 provided by the embodiment of the present invention may be used to perform operations performed by a client in a method for storing a file in the distributed file system shown in FIG. 3, and an implementation manner not specifically explained and described by the client 900 may be referred to.
  • the embodiment of the present invention further provides a client, where the client can perform the corresponding figure in FIG. 3 .
  • the method provided by the embodiment can be the same as the client 900 shown in FIG.
  • the device where the client 1000 is located includes at least one processor 1001, a memory 1002, and a communication interface 1003; the at least one processor 1001, the memory 1002, and the communication interface 1003 are all connected by a bus 1004;
  • the memory 1002 is configured to store a computer execution instruction
  • the at least one processor 1001 is configured to execute a computer execution instruction stored by the memory 1002, so that the client 1000 performs data interaction with a device in a distributed file system through the communication interface 1003 to perform the foregoing embodiment.
  • the at least one processor 1001 may include different types of processors 1001, or include the same type of processor 1001; the processor 1001 may be any of the following: a CPU, an ARM processor, an FPGA, a dedicated processor, etc. have computational processing Capable device. In an optional implementation manner, the at least one processor 1001 may also be integrated into a many-core processor.
  • the memory 1002 may be any one or any combination of the following: a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • Communication interface 1003 is used by client 1000 to perform data interaction with other devices, such as other devices in a distributed file system.
  • the communication interface 1003 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, and the like having a network access function.
  • the bus 1004 can include an address bus, a data bus, a control bus, etc., for ease of representation, Figure 10 shows the bus with a thick line.
  • the bus 1004 may be any one or any combination of the following: a device for wired data transmission such as an ISA bus, a PCI bus, or an EISA bus.
  • the embodiment of the present invention provides a second virtual machine in a distributed file system.
  • the distributed file system includes a name node and multiple virtual machines as data nodes, and multiple virtual machines share the same storage area; as shown in FIG.
  • the second virtual machine 1100 includes:
  • the receiving module 1101 is configured to receive metadata sent by the first virtual machine, where the first virtual machine is a virtual machine that is designated by the name node to have the right to write data to the storage area, and the second virtual machine is The virtual machine other than the first virtual machine in the virtual machine, the metadata is the metadata of the data to be written generated or updated after the first virtual machine writes the data to be written to the storage area.
  • the receiving module 1101 is further configured to: before receiving the metadata sent by the first virtual machine, receive a read permission identifier of the second virtual machine sent by the client, where the read permission identifier is a name node requesting from the client node to the name node
  • the read permission identifier is used to specify that the second virtual machine has the right to read the data to be written from the storage area when the data to be written is written to the distributed file system.
  • multiple virtual machines mount the same virtual hard disk provided by the distributed block storage system, and the virtual hard disk includes a storage area.
  • the second virtual machine further includes: a processing module 1102, configured to: after the receiving module 1101 receives the metadata sent by the first virtual machine, if the second virtual machine reads the data to be written by using the operating system of the second virtual machine, Metadata generates or updates file information recorded in its own operating system, the file information is used by the operating system to read data to be written from the storage area; or if the second virtual machine reads data to be written, according to metadata from the storage area Read the data to be written.
  • a processing module 1102 configured to: after the receiving module 1101 receives the metadata sent by the first virtual machine, if the second virtual machine reads the data to be written by using the operating system of the second virtual machine, Metadata generates or updates file information recorded in its own operating system, the file information is used by the operating system to read data to be written from the storage area; or if the second virtual machine reads data to be written, according to metadata from the storage area Read the data to be written.
  • the second virtual machine is designated by the name node to have read access to the data to be written from the storage area.
  • the second virtual machine 1100 provided by the embodiment of the present invention can solve the problem of redundancy of file shares stored in the distributed file system.
  • the operation of the second virtual machine 1100 provided by the embodiment of the present invention may cause the failure of the virtual machine to not affect the client to read or write the file. Increased system availability.
  • the second virtual machine 1100 provided by the embodiment of the present invention may be used to perform operations performed by the second virtual machine in the method for storing files in the distributed file system shown in FIG. 3, and the second virtual machine 1100 does not explain in detail.
  • the embodiment of the present invention further provides a second virtual machine, which may perform the method provided by the embodiment corresponding to FIG. 3, and may be the same as the second virtual machine 1100 shown in FIG.
  • the device where the second virtual machine 1200 is located includes at least one processor 1201, a memory 1202, and a communication interface 1203; the at least one processor 1201, the memory 1202, and the communication interface 1203 are all connected by a bus 1204;
  • the memory 1202 is configured to store a computer execution instruction
  • the at least one processor 1201 is configured to execute a computer execution instruction stored by the memory 1202, so that the second virtual machine 1200 performs data interaction with other devices in the distributed file system through the communication interface 1203 to perform the foregoing.
  • the method for storing a file in a distributed file system provided by an embodiment, or causing the second virtual machine 1200 to perform data interaction with other devices in the distributed file system through the communication interface 1203 to implement a part of the distributed file system or All features.
  • At least one processor 1201 may include different types of processors 1201, or include the same type of processor 1201; the processor 1201 may be any of the following: CPU, ARM processor, FPGA, dedicated processor, etc. with calculation processing Capable device. In an optional implementation manner, the at least one processor 1201 may also be integrated into a many-core processor.
  • the memory 1202 may be any one or any combination of the following: a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • a storage medium such as a RAM, a ROM, an NVM, an SSD, a mechanical hard disk, a magnetic disk, a disk array, or the like.
  • Communication interface 1203 is used by second virtual machine 1200 to perform data interaction with other devices, such as other devices in a distributed file system.
  • the communication interface 1203 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, and the like having a network access function.
  • the bus 1204 can include an address bus, a data bus, a control bus, etc., for ease of representation, Figure 12 shows the bus with a thick line.
  • the bus 1204 may be any one or any combination of the following: a device for wired data transmission such as an ISA bus, a PCI bus, or an EISA bus.
  • the embodiment of the present invention provides a distributed file system.
  • the distributed file system 1300 includes a first virtual machine 1301, a name node 1302, a client 1303, and a second virtual machine 1304.
  • the first virtual machine 1301 in the distributed file system 1300 can be used to perform the related operations performed by the first virtual machine in the method for storing files in the distributed file system shown in FIG. 3 , and the specific implementation may be FIG. 5 .
  • the first virtual machine 500 shown or the first virtual machine 600 shown in FIG. 6; the name node 1302 in the distributed file system 1300 can be used to execute the name node in the method for storing files in the distributed file system shown in FIG.
  • the specific implementation of the related operations may be the name node 700 shown in FIG. 7 or the name node 800 shown in FIG. 8; the client 1303 in the distributed file system 1300 may be used to perform the distributed process shown in FIG.
  • File storage in the file system The specific operation performed by the client in the method may be the client 900 shown in FIG. 9 or the client 1000 shown in FIG. 10; the second virtual machine 1304 in the distributed file system 1300 may be used to execute The related operations performed by the second virtual machine in the method for storing files in the distributed file system shown in FIG. 3 may be the second virtual machine 1100 shown in FIG. 11 or the second virtual device shown in FIG. Machine 1200.
  • the data to be written is saved only in a storage area shared by a plurality of virtual machines, which solves the problem of redundancy of files stored in the distributed file system.
  • the client can still perform a write operation or a read operation on the write data by the non-failed virtual machine in the distributed file system 1300, thereby improving the distribution.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种存储文件的方法、第一虚拟机及名称节点,用以解决分布式文件系统存储文件时存在的文件份数冗余问题,并提高系统的可用性。方法包括:客户端向名称节点发送请求向分布式文件系统写入待写数据的请求消息;名称节点向客户端发送请求消息对应的响应消息,响应消息包括第一虚拟机的地址和第二虚拟机的地址,并指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;客户端向第一虚拟机发送待写数据和第二虚拟机的地址;第一虚拟机向多个虚拟机共享的存储区域写入待写数据,并生成或更新待写数据的元数据;第一虚拟机向第二虚拟机发送生成或更新的元数据。

Description

一种存储文件的方法、第一虚拟机及名称节点 技术领域
本发明涉及计算机技术领域,尤其涉及一种存储文件的方法、第一虚拟机及名称节点。
背景技术
分布式文件系统包括客户端(client)、数据节点(datanode)和名称节点(namenode);其中,数据节点用于存储文件,名称节点用于管理数据节点上存储的文件。客户端可通过名称节点查询每个数据节点中存储的文件并获得每个数据节点的地址,从而实现从数据节点中读取文件或者将文件写入数据节点。分布式文件系统中的数据节点可以是物理服务器,也可以是虚拟机。
当分布式文件系统中的数据节点为虚拟机时,该虚拟机的虚拟硬盘是由分布式块存储系统提供的,向虚拟机写文件实际是向虚拟机的虚拟硬盘写文件,向虚拟硬盘写文件的实现方式是向分布式块存储系统管理的物理硬盘写文件。
分布式文件系统为了保证文件的可靠性,在虚拟硬盘存储文件时会采用文件副本机制,将同一个文件保存在分布式文件系统中N个(N为大于1的整数)虚拟硬盘中;而分布式块存储系统为了保证文件的可靠性,也会采用文件副本机制,将同一个虚拟硬盘中的文件在M个(M为大于1的整数)物理硬盘中保存。由于分布式文件系统和分布式块存储系统均采用文件副本机制,会导致同一个文件在物理硬盘中实际保存的文件份数为N*M,造成了文件份数冗余。同一个文件保存的文件份数冗余会浪费存储空间,影响系统的处理性能。
现有技术中为了解决分布式文件系统中文件份数冗余的问题,通常采用以下两种方法:第一种方法是,对于需要存储的一份文件,仅在分布式文件系统的一个虚拟机中存储该文件。采用第一种方法,只能通过该虚拟机才能访问该文件,若该虚拟机发生故障,则需要等待该虚拟机恢复正常后才能再次为客户端提供文件读写服务,导致分布式文件系统的可用性降低;第二种方法是,采用虚拟机的热备机制,即配置主虚拟机对应的热备虚拟机,该热备虚拟机与主虚拟机同步写入文件。当主虚拟机发生故障时,分布式文件系统切换到热备虚拟机继续为客户端提供文件读写服务。采用第二种方法,分布式文件系统切换到热备虚拟机时需要一定的等待时长,导致分布式文件系统在该等待时长内无法为客户端提供文件读写服务,使分布式文件系统的可用性降低;并且,热备虚拟机在切换至主虚拟机之前不对外提供服务,导致资源浪费。
综上,现有的解决分布式文件系统中文件份数冗余问题的方法会导致分布式文件系统的可用性低、无法较好地解决文件份数冗余问题。
发明内容
本发明实施例提供一种存储文件的方法、第一虚拟机及名称节点,用以解决分布式文件系统存储文件时存在的文件份数冗余的问题,并提高系统的可用性。
第一方面,本发明实施例提供一种分布式文件系统中存储文件的方法,该方法中, 分布式文件系统包括名称节点、多个作为数据节点的虚拟机,其中的多个虚拟机共享同一存储区域;该方法包括:
第一虚拟机接收客户端发送的待写数据、第二虚拟机的地址,然后向多个虚拟机共享的存储区域写入接收到的待写数据,并生成或更新待写数据的元数据;第一虚拟机根据接收到的第二虚拟机的地址向第二虚拟机发送第一虚拟机生成或更新的元数据。
其中,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;待写数据的元数据包括但不限于:待写数据的存储位置、待写数据的文件名称以及待写数据的文件目录。
采用上述方法,由于分布式文件系统包括的多个虚拟机共享同一存储区域,因而在分布式文件系统中,第一虚拟机写入该存储区域的待写数据在该存储区域中仅保存一份。对于待写数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,采用上述方案,分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,第一虚拟机向存储区域写入待写数据之前,还包括:第一虚拟机接收客户端发送的第一虚拟机的写权限标识,写权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,用于指定第一虚拟机具有向存储区域写入待写数据的权限。
采用上述方案,提供了一种客户端向第一虚拟机指示第一虚拟机的权限的方式。
多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,该虚拟硬盘包括多个虚拟机共享的存储区域。
第一虚拟机向第二虚拟机发送的待写数据的元数据有以下两种用途:
第一种,若第二虚拟机通过自身的操作系统读取待写数据,则元数据用于第二虚拟机生成或更新自身的操作系统中记录的文件信息,文件信息用于操作系统从存储区域中读取待写数据。
第二种,若第二虚拟机读取待写数据,则元数据用于第二虚拟机从存储区域中读取待写数据。
采用上述方案,第二虚拟机可以根据第一虚拟机发送的待写数据的元数据读取多个虚拟机共享的存储区域中的待写数据。
在一种可能的实现方式中,第二虚拟机可以被名称节点指定具有从存储区域读取待写数据的权限。
第二方面,本发明实施例提供一种分布式文件系统中存储文件的方法,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该 方法包括:
名称节点接收客户端请求向分布式文件系统写入待写数据的请求消息后,向客户端发送该请求消息对应的响应消息。
其中,名称节点向客户端发送的响应消息中包括第一虚拟机的地址和第二虚拟机的地址,此外,该响应消息还指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,而名称节点发送的响应消息指定多个虚拟机中的一个第一虚拟机具有向共享的存储区域中写入数据的权限,因而向共享的存储区域中写入的数据仅会在该存储区域中保存一份。对于写入共享的存储区域的数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,响应消息中指示分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
在一种可能的实现方式中,名称节点通过响应消息向客户端指示第一虚拟机的权限和第二虚拟机的权限时可采用如下两种方式:
第一种方式,名称节点向客户端发送的响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,其中的写权限标识和读权限标识分别指示了第一虚拟机的权限和第二虚拟机的权限,即:写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
第二种方式,名称节点向客户端发送的响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,该预设规则指示了第一虚拟机的权限和第二虚拟机的权限,即:第一虚拟机具有向存储区域写入待写数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。
采用这两种方式,提供了名称节点通过响应消息向客户端指示第一虚拟机的权限和第二虚拟机的权限的两种方式。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
由于分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个,因而当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务。具体实现时,名称节点在虚拟机发生故障时通过何种方式处理可包含以下两种情况:
第一种情况,当第一虚拟机发生故障时,名称节点向客户端发送第一更新信息,该 第一更新信息包括更新的第一虚拟机的地址,第一更新信息还指定了多个虚拟机中除发生故障的第一虚拟机以外的另一个虚拟机作为更新的第一虚拟机,即指定第二虚拟机中的某一个虚拟机作为更新的第一虚拟机,更新的第一虚拟机具有向存储区域写入数据的权限。
第二种情况,当第二虚拟机发生故障时,名称节点向客户端发送第二更新信息,该第二更新信息包括更新的第二虚拟机的地址,第二更新信息还指定了多个虚拟机以外的另一个虚拟机作为更新的第二虚拟机,更新的第二虚拟机具有从存储区域读取待写数据的权限。
采用上述方案,无论是第一虚拟机发生故障还是第二虚拟机发生故障,名称节点均指定了其他虚拟机来替代发生故障的虚拟机,从而在第一虚拟机和/或第二虚拟机发生故障的情况下,分布式文件系统仍能为客户端提供读写数据的服务,进一步提高了分布式文件系统的可用性。
第三方面,本发明实施例提供一种分布式文件系统中存储文件的方法,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该方法包括:
客户端向名称节点发送请求向分布式文件系统写入待写数据的请求消息,后接收名称节点发送的请求消息对应的响应消息。
其中,响应消息包括第一虚拟机的地址和第二虚拟机的地址,此外,响应消息还指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机。
客户端根据响应消息包括的第一虚拟机的地址向第一虚拟机发送待写数据和第二虚拟机的地址,并指示第一虚拟机:写入待写数据、生成或更新待写数据的元数据、并根据第二虚拟机的地址向第二虚拟机发送待写数据的元数据。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,因而在分布式文件系统中,客户端指示第一虚拟机写入该共享存储区域的待写数据只在该存储区域中存储一份。对于待写数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,由于分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,客户端通过接收到的名称节点发送的响应消息来获知第一虚拟机的权限和第二虚拟机的权限时可采用如下两种方式:
第一种方式,客户端接收到的响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,其中的写权限标识和读权限标识分别指示了第一虚拟机的权限和第二虚拟机的权限,即:写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权 限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
第二种方式,客户端接收到的响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,该预设规则指示了第一虚拟机的权限和第二虚拟机的权限,即:第一虚拟机具有向存储区域写入待写数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。
采用这两种方式,提供了客户端通过接收响应消息获知第一虚拟机的权限和第二虚拟机的权限的两种方式。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
在一种可能的实现方式中,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
第四方面,本发明实施例提供一种分布式文件系统中存储文件的方法,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该方法包括:
第二虚拟机接收第一虚拟机发送的元数据。其中,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机,元数据为第一虚拟机向存储区域写入待写数据后生成或更新的待写数据的元数据。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,而多个虚拟机中的一个第一虚拟机具有向共享的存储区域中写入数据的权限,因而向共享的存储区域中写入的数据仅会在该存储区域中保存一份。对于写入共享的存储区域的数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,第二虚拟机获知身权限可采用如下方式:第二虚拟机在接收第一虚拟机发送的元数据之前,接收客户端发送的第二虚拟机的读权限标识,读权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
第二虚拟机接收第一虚拟机发送的元数据之后,可根据接收到的元数据读取多个虚拟机共享的存储区域中的待写数据,具体可采用以下方式:
第一种,若第二虚拟机通过自身的操作系统读取待写数据,则第二虚拟机根据元数 据生成或更新自身的操作系统中记录的文件信息,该文件信息可用于操作系统从存储区域中读取待写数据。
第二种,若第二虚拟机读取待写数据,则第二虚拟机根据元数据从存储区域中读取待写数据。
采用上述方案,第二虚拟机可以根据第一虚拟机发送的待写数据的元数据读取多个虚拟机共享的存储区域中的待写数据。
在一种可能的实现方式中,第二虚拟机被名称节点指定具有从存储区域读取待写数据的权限。
第五方面,本发明实施例提供一种分布式文件系统中的第一虚拟机,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机;该第一虚拟机包括:
接收模块,用于接收客户端发送的待写数据、第二虚拟机的地址,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;
处理模块,用于向存储区域写入接收模块接收的待写数据,并生成或更新待写数据的元数据;
发送模块,用于根据接收模块接收的第二虚拟机的地址向第二虚拟机发送处理模块生成或更新元数据。
其中,待写数据的元数据包括但不限于:待写数据的存储位置、待写数据的文件名称以及待写数据的文件目录。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,因而在分布式文件系统中,处理模块写入该存储区域的待写数据在该存储区域中仅保存一份。对于待写数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,采用上述方法,分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,接收模块还用于:在处理模块向存储区域写入待写数据之前,接收客户端发送的第一虚拟机的写权限标识,写权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,该写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限。
采用上述方案,提供了一种第一虚拟机从客户端获知自身权限的方式。
多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
发送模块向第二虚拟机发送的待写数据的元数据有以下两种用途:
第一种,若第二虚拟机通过自身的操作系统读取待写数据,则元数据用于第二虚拟机生成或更新自身的操作系统中记录的文件信息,文件信息用于操作系统从存储区域中读取待写数据。
第二种,若第二虚拟机读取待写数据,则元数据用于第二虚拟机从存储区域中读取待写数据。
采用上述方案,第二虚拟机可以根据发送模块发送的待写数据的元数据读取多个虚拟机共享的存储区域中的待写数据。
在一种可能的实现方式中,第二虚拟机被名称节点指定具有从存储区域读取待写数据的权限。
第六方面,本发明实施例提供一种分布式文件系统中的名称节点,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该名称节点包括:
接收模块,用于接收客户端请求向分布式文件系统写入待写数据的请求消息;
发送模块,用于向客户端发送接收模块接收的请求消息对应的响应消息,该响应消息包括第一虚拟机的地址和第二虚拟机的地址,此外,该响应消息还指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,而名称节点发送的响应消息指定多个虚拟机中的一个第一虚拟机具有向共享的存储区域中写入数据的权限,因而处理模块向共享的存储区域中写入的数据仅会在该存储区域中保存一份。对于写入共享的存储区域的数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,响应消息中指示分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
在一种可能的实现方式中,发送模块发送的响应消息向客户端指示第一虚拟机的权限和第二虚拟机的权限时可采用如下两种方式:
第一种方式
发送模块向客户端发送的响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,其中的写权限标识和读权限标识分别指示了第一虚拟机的权限和第二虚拟机的权限,即:写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
第二种方式
发送模块向客户端发送的响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,该预设规则指示了第一虚拟机的权限和第二虚拟机的权限,即:第一虚拟机具有向存储区域写入待写数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。
采用这两种方式,提供了发送模块发送的响应消息向客户端指示第一虚拟机的权限和第二虚拟机的权限的两种方式。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
由于分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个,因而当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务。具体实现时,发送模块在虚拟机发生故障时通过何种方式处理可包含以下两种情况:
第一种情况,当第一虚拟机发生故障时,向客户端发送第一更新信息,该第一更新信息包括更新的第一虚拟机的地址,第一更新信息还指定了多个虚拟机中除发生故障的第一虚拟机以外的另一个虚拟机作为更新的第一虚拟机,更新的第一虚拟机具有向存储区域写入数据的权限。
第二种情况,当第二虚拟机发生故障时,向客户端发送第二更新信息,该第二更新信息包括更新的第二虚拟机的地址,第二更新信息还指定了多个虚拟机以外的另一个虚拟机作为更新的第二虚拟机,更新的第二虚拟机具有从存储区域读取待写数据的权限。
采用上述方案,无论是第一虚拟机发生故障还是第二虚拟机发生故障,发送模块均指定了其他虚拟机来替代发生故障的虚拟机,从而在第一虚拟机和/或第二虚拟机发生故障的情况下,分布式文件系统仍能为客户端提供读写数据的服务,进一步提高了分布式文件系统的可用性。
第七方面,本发明实施例提供一种客户端,该客户端所在的分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该客户端包括:
发送模块,用于向名称节点发送请求向分布式文件系统写入待写数据的请求消息;
接收模块,用于接收名称节点发送的请求消息对应的响应消息;
其中,响应消息包括第一虚拟机的地址和第二虚拟机的地址,此外,响应消息还指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;
发送模块,还用于根据接收模块接收的响应消息包括的第一虚拟机的地址向第一虚拟机发送待写数据、第二虚拟机的地址,并指示第一虚拟机:写入待写数据、生成或更新待写数据的元数据、以及根据接收模块接收的响应消息包括的第二虚拟机的地址向第二虚拟机发送待写数据的元数据。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,因而在分布式文件系统中,发送模块指示第一虚拟机写入该共享存储区域的待写数据只在该存储区域中存储一份。对于待写数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,由于分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,接收模块通过接收到的名称节点发送的响应消息来获知第一虚拟机的权限和第二虚拟机的权限时可采用如下两种方式:
第一种方式,接收模块接收到的响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,其中的写权限标识和读权限标识分别指示了第一虚拟机的权限和第二虚拟机的权限,即:写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
第二种方式,接收模块接收到的响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,该预设规则指示了第一虚拟机的权限和第二虚拟机的权限,即:第一虚拟机具有向存储区域写入待写数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。
采用这两种方式,提供了接收模块通过接收响应消息获知第一虚拟机的权限和第二虚拟机的权限的两种方式。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
在一种可能的实现方式中,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
第八方面,本发明实施例一种分布式文件系统中的第二虚拟机,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;该第二虚拟机包括:
接收模块,用于接收第一虚拟机发送的元数据。其中,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机,元数据为第一虚拟机向存储区域写入待写数据后生成或更新的待写数据的元数据。
采用上述方案,由于分布式文件系统包括的多个虚拟机共享同一存储区域,而多个虚拟机中的一个第一虚拟机具有向共享的存储区域中写入数据的权限,因而向共享的存储区域中写入的数据仅会在该存储区域中保存一份。对于写入共享的存储区域的数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的 服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
在一种可能的实现方式中,接收模块获知第二虚拟机的权限可采用如下方式:接收模块在接收第一虚拟机发送的元数据之前,接收客户端发送的第二虚拟机的读权限标识,读权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
在一种可能的实现方式中,多个虚拟机共享一个存储区域在具体实现时可采用如下方式:多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
在一种可能的实现方式中,第二虚拟机还包括处理模块。接收模块接收第一虚拟机发送的元数据之后,处理模块可根据接收到的元数据读取多个虚拟机共享的存储区域中的待写数据,处理模块在具体可采用以下方式:
第一种,在接收模块接收第一虚拟机发送的元数据之后,若第二虚拟机通过自身的操作系统读取待写数据,则处理模块根据元数据生成或更新自身的操作系统中记录的文件信息,文件信息用于操作系统从存储区域中读取待写数据。
第二种,若第二虚拟机读取待写数据,则处理模块根据元数据从存储区域中读取待写数据。
采用上述方案,处理模块可以根据第一虚拟机发送的待写数据的元数据读取多个虚拟机共享的存储区域中的待写数据。
在一种可能的实现方式中,第二虚拟机被名称节点指定具有从存储区域读取待写数据的权限。
第九方面,提供一种计算机可读存储介质,计算机可读存储介质中存储有计算机执行指令,当计算节点的至少一个处理器执行该计算机执行指令时,计算节点执行上述第一方面或者第一方面的各种可能设计提供的方法、或者执行上述第二方面或者第二方面的各种可能设计提供的方法、或者执行上述第三方面或者第三方面的各种可能设计提供的方法。
第十方面,提供一种计算机程序产品,该计算机程序产品包括计算机执行指令,该计算机执行指令存储在计算机可读存储介质中。计算节点的至少一个处理器可以从计算机可读存储介质读取该计算机执行指令,至少一个处理器执行该计算机执行指令使得计算节点实施上述第一方面或者第一方面的各种可能设计提供的方法、或者执行上述第二方面或者第二方面的各种可能设计提供的方法、或者执行上述第三方面或者第三方面的各种可能设计提供的方法。
附图说明
图1为本发明实施例提供的分布式文件系统中名称节点、客户端和多个数据节点的连接关系的示意图;
图2为本发明实施例提供的分布式文件系统与分布式块存储系统的连接关系的示意图;
图3为本发明实施例提供的分布式文件系统中存储文件的方法的流程示意图;
图4为采用图3所示的存储文件的方法的分布式文件系统和分布式块存储系统的结构示意图;
图5为明实施例提供的一种第一虚拟机的结构示意图;
图6为明实施例提供的另一种第一虚拟机的结构示意图;
图7为明实施例提供的一种名称节点的结构示意图;
图8为明实施例提供的另一种名称节点的结构示意图;
图9为明实施例提供的一种客户端的结构示意图;
图10为明实施例提供的另一种客户端的结构示意图;
图11为明实施例提供的一种第二虚拟机的结构示意图;
图12为明实施例提供的另一种第二虚拟机的结构示意图;
图13为本发明实施例提供的一种分布式文件系统的结构示意图。
具体实施方式
为了更好地理解本发明实施例的上述目的、方案和优势,下文提供了详细描述。该详细描述通过使用框图、流程图等附图和/或示例,阐明了装置和/或方法的各种实施方式。在这些框图、流程图和/或示例中,包含一个或多个功能和/或操作。本领域技术人员将理解到:这些框图、流程图或示例内的各个功能和/或操作,能够通过各种各样的硬件、软件、固件单独或共同实施,或者通过硬件、软件和固件的任意组合实施。
本发明实施例涉及分布式文件系统,下面对分布式文件系统进行详细介绍。
如图1所示,分布式文件系统可包含名称节点和多个数据节点。根据分布式文件系统的应用场合不同,名称节点又可以称为主控服务器或者其他名称,对应地,数据节点又可以称为数据服务器或者其他名称。需要说明的是,图1中仅示出分布式文件系统包含一个客户端的场景,实际中,分布式文件系统中可包含多个客户端。
其中,名称节点用于管理多个数据节点,名称节点记录每个数据节点中存储的文件的信息(例如元数据文件)、每个数据节点的服务状态等;数据节点用于存储文件,当客户端进行文件读写操作时,客户端首先向名称节点请求获取数据节点的索引信息,然后再根据请求到的索引信息访问相应的数据节点来进行文件读写。多个数据节点之间可能会同步文件。比如当某个文件需要在两个数据节点中写入时,可先写入其中一个数据节点,再由该数据节点将文件同步给另一个数据节点。此外,名称节点和数据节点间也可以直接进行信息交互。
其中,名称节点、数据节点、客户端可以在以下任一种具有计算能力的设备上配置对应功能实现。该具有计算能力的设备可以是物理设备或虚拟设备;例如,物理设备可以是个人计算机、笔记本计算机、大型机、联网计算机、手持式计算机、个人数字助理、工作站等,虚拟设备可以是部署在物理设备中的虚拟机或容器等。
参见图2,当数据节点为虚拟机时,虚拟机的虚拟硬盘由分布式块存储系统提供,分布式块存储系统管理有多个物理硬盘,向虚拟机的虚拟硬盘写入文件实际是向分布式块存储系统管理的物理硬盘中写入文件。
参见图2,分布式文件系统为了保证自身的可靠性,在存储文件时一般会采用文件副本机制,比如在存储某个文件时,将该文件分别存储在两个数据节点上,即存储在虚拟 机1和虚拟机2上;分布式块存储系统为了保证自身的可靠性,具体在存储虚拟机的文件时也会采用文件副本机制,比如在实现虚拟机1的该文件的存储时分别在物理服务器1的物理硬盘1、物理服务器2的物理硬盘3和物理服务器3的物理硬盘5上存储该文件,以及在实现虚拟机2的该文件的存储时分别存储在物理服务器1的物理硬盘2、物理服务器2的物理硬盘4和物理服务器3的物理硬盘6上存储该文件。这样,由于分布式文件系统和分布式块存储系统均采用文件副本机制,导致该文件在分布式块存储系统管理的物理硬盘中保存六份。显然,对于同一个文件保存的文件份数冗余会浪费存储空间,影响系统的处理性能。
需要说明的是,为了解释分布式文件系统在存储某个文件时的操作,图2的分布式文件系统中仅示出了两个虚拟机,每个虚拟机包含一个虚拟硬盘;图2的分布式块存储系统中仅示出了三个物理服务器,每个物理服务器包含两个物理硬盘。实际实现时,分布式文件系统可以存储有多个文件,因而对分布式文件系统包含的虚拟机的数量不做限制,对每个虚拟机包含的虚拟硬盘的数量不做限制;同时,对分布式块存储系统包含的物理服务器的数量不做限制,对每个物理服务器包含的物理硬盘的数量也不做限制。
为了解决分布式文件系统存在的文件份数冗余的问题,本发明实施例提供一种分布式文件系统中存储文件的方法,分布式文件系统包括名称节点、多个作为数据节点的虚拟机,其中,多个虚拟机共享同一存储区域。如图3所示,该方法包括:
S301:客户端向名称节点发送请求向分布式文件系统写入待写数据的请求消息。
待写数据可以是视频数据、音频数据、文档数据或者其他二进制数据。待写数据的粒度可以是文件、数据块或者其他粒度。待写数据的数量可以是一个或多个,只要将图3所示方法执行一次后,某一个或多个数据被写入了分布式文件系统,该一个或多个数据即可视为待写数据。
S302:名称节点向客户端发送请求消息对应的响应消息。
其中,响应消息包括第一虚拟机的地址和第二虚拟机的地址,响应消息还指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机。
其中,第一虚拟机的数量必须为一个;第二虚拟机的数量可以是一个,也可以是多个,本发明实施例中对第二虚拟机的数量不做限制。
本发明实施例中,多个虚拟机中仅有一个虚拟机具有向存储区域写入数据的权限,其原因是:如果有多个虚拟机用于写入待写数据,那么当客户端要向分布式文件系统写入待写数据时,会有多个虚拟机接收到写入待写数据的指令;由于多个虚拟机共享同一存储区域,那么多个虚拟机接收到的写入待写数据的指令会在同一时刻指示多个虚拟机将待写数据写入同一存储区域,这样会造成写入待写数据的指令无法区分应该通过哪个虚拟机写入待写数据,导致写入待写数据的指令无法执行。另外,只有一个虚拟机写入待写数据,保证在分布式系统写入数据阶段只写了一份待写数据,相对于现有技术在分布式系统写入数据阶段是写入多份待写数据,减少了数据冗余。
本发明实施例中第二虚拟机的数量可以为多个的原因是:当有多个客户端要读取待写数据时,可通过多个第二虚拟机进行读取,提高客户端读取待写数据的效率。另外,第二虚拟机得到第一虚拟机发送的该待写数据的元数据后,可以直接从分布式块存储系 统读取该待写数据,避免现有技术在第一虚拟机出现故障而无法读取该待写数据的情况。
根据本发明实施例中对第一虚拟机的数量的限制,名称节点发送给客户端的响应消息中可仅指示第一虚拟机具有向存储区域写入数据的权限,而不必指示第二虚拟机的权限,其原因是:由于多个虚拟机中仅有一个虚拟机具有向存储区域写入数据的权限,当响应消息中指示了多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,那么多个虚拟机中除第一虚拟机以外的第二虚拟机即默认具有从存储区域读取待写数据的权限。
可选地,响应消息还指示第二虚拟机具有从所存储区域读取待写数据的权限。
S302中,响应消息指示的第一虚拟机的权限和第二虚拟机的权限仅针对待写数据。第一虚拟机与第二虚拟机共享写入待写数据的存储区域。比如,在将待写数据1存储到分布式文件系统时,响应消息指示虚拟机1为第一虚拟机,虚拟机2为第二虚拟机,虚拟机1将待写数据1写入虚拟机1的存储区域1中,然后将待写数据1的元数据发送给虚拟机2,其中,虚拟机2与虚拟机1共享存储区域1;在将待写数据2存储到分布式文件系统时,响应消息指示虚拟机1为第一虚拟机,虚拟机3为第二虚拟机,虚拟机1将待写数据2写入虚拟机1的存储区域2中,然后将待写数据2的元数据发送给虚拟机3,其中,虚拟机3与虚拟机1共享存储区域2。
S303:客户端根据第一虚拟机的地址向第一虚拟机发送待写数据和第二虚拟机的地址。
其中,客户端向第一虚拟机发送待写数据和第二虚拟机的地址是用于指示第一虚拟机写入待写数据、生成或更新待写数据的元数据、以及根据第二虚拟机的地址向第二虚拟机发送待写数据的元数据。
S304:第一虚拟机向多个虚拟机共享的存储区域写入待写数据,并生成或更新待写数据的元数据。
其中,待写数据的元数据可用于第一虚拟机和第二虚拟机根据该元数据从多个虚拟机共享的存储区域读取待写数据;待写数据的元数据包括但不限于:待写数据的存储位置、待写数据的名称以及待写数据的目录。
S305:第一虚拟机根据第二虚拟机的地址向第二虚拟机发送生成或更新的元数据。
需要说明的是,分布式文件系统一般可包括客户端,也可不包括客户端。若分布式文件系统包括客户端,客户端的数量包括但不限于一个。本发明实施例中,为了更清楚地描述客户端、名称节点、第一虚拟机和第二虚拟机之间的交互,将客户端包括在分布式文件系统中。实际实现时,分布式文件系统也可不包括客户端,此时本发明实施例可视为客户端和分布式文件系统的交互。
可选地,分布式文件系统包括的多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,该虚拟硬盘包括多个虚拟机共享的存储区域。
采用图3所示的分布式文件系统中存储文件的方法,由于分布式文件系统包括的多个虚拟机共享同一存储区域,因而在分布式文件系统中,该待写数据只在该存储区域中存储一份。对于待写数据来说,仅会由于分布式块存储系统采用的文件副本机制而保存多份,而不存在由于分布式文件系统和分布式块存储系统均采用文件副本机制而导致的保存的文件份数冗余的问题。
此外,由于在图3所示的分布式文件系统中存储文件的方法中,分布式文件系统包括的多个虚拟机中的第一虚拟机具有向存储区域写入数据的权限,多个虚拟机中除第一虚拟机以外的第二虚拟机具有从存储区域读取待写数据的权限。因而,分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,使分布式文件系统的可用性得到了提高,同时也避免了现有技术中采用虚拟机热备机制时存在的资源浪费问题。
为了更形象地解释图3所示方法如何解决存储文件份数冗余问题、同时提高系统的可用性,现将图3所示方法应用在分布式文件系统和分布式块存储系统中加以说明。采用图3所示方法的分布式文件系统和分布式块存储系统可如图4所示。图4所示的分布式文件系统包含第一虚拟机、第二虚拟机、客户端和名称节点。实际实现时,对第二虚拟机的数量和客户端的数量不做限制。图4所示的分布式块存储系统包含三个物理服务器,每个物理服务器包含两个物理硬盘。
其中,第一虚拟机具有向存储区域写入数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。由于第一虚拟机和第二虚拟机共享同一存储区域,可视为第一虚拟机和第二虚拟共享同一个虚拟硬盘1,第一虚拟机可向虚拟硬盘1中写入待写数据,第二虚拟机可从虚拟硬盘1中读取待写数据。因而待写数据在分布式文件系统中仅存储了一份,即存储在虚拟硬盘1中,待写数据在分布式块存储系统中可以存储三份,例如分别存储在物理服务器1的物理硬盘1、物理服务器2的物理硬盘3和物理服务器3的物理硬盘5上。这样,待写数据在物理硬盘中仅保存了三份。图2所示的分布式文件系统和分布式块存储系统中同一个文件在物理硬盘中保存了六份,相比之下,采用图3所示方法后,图4所示的分布式文件系统和分布式块存储系统中待写数据仅在物理硬盘中保存了三份,从而大大减少了文件保存的份数,解决了分布式文件系统中保存的文件份数冗余的问题。
此外,在图4中,第一虚拟机可用于写入待写数据和读取待写数据,第二虚拟机可用于读取待写数据,因而当其中某个虚拟机发生故障时,可由另一个未发生故障的虚拟机为客户端提供读写待写数据的服务,提高了系统的可用性。
进一步地,第二虚拟机在接收到第一虚拟机发送的待写数据的元数据之后,是否需要生成或更新自身的操作系统中记录的文件信息可分为以下两种情况:
第一种情况
若第一虚拟机向多个虚拟机共享的存储区域写入待写数据时是通过第一虚拟机的操作系统写入的,并且客户端通过第二虚拟机读取待写数据也需要通过第二虚拟机的操作系统读取,此时,第二虚拟机需要根据待写数据的元数据生成或更新自身的操作系统中记录的文件信息,才能实现第二虚拟机的操作系统从该存储区域中读取待写数据。其中,该文件信息用于第二虚拟机的操作系统从存储区域中读取所述待写数据。更新第二虚拟机的操作系统中的文件信息的方式可以有两种;第一种,如果第二虚拟机的操作系统能够获知存储区域中数据的变化,则可以自己更新该待写数据的文件信息;第二种,第二虚拟机的操作系统能根据第一虚拟机发送的该待写数据的元数据,更新该待写数据的文件信息。
第二种情况
当第一虚拟机向多个虚拟机共享的存储区域写入待写数据时是直接写入该存储区域,而不是通过第一虚拟机的操作系统写入时,客户端可以通过第二虚拟机直接读取该共享的存储区域中的待写数据,而不需要通过第二虚拟机的操作系统读取。此时,第二虚拟机不需要根据待写数据的元数据生成或更新自身的操作系统中记录的文件信息,第二虚拟机仅根据待写数据的元数据即可从该存储区域中读取待写数据。
在S302中,名称节点需要通过相应消息指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机,即通过如上S302的操作后,客户端不仅可以获取第一虚拟机的地址和第二虚拟机的地址,还可以获知第一虚拟机具有向存储区域写入数据的权限,第二虚拟机具有从存储区域读取待写数据的权限。名称节点向客户端通知第一虚拟机和第二虚拟机的权限的方式包括但不限于以下两种:
第一种方式
在执行S302中,名称节点向客户端发送的响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。客户端根据写权限标识和读权限标识获知第一虚拟机和第二虚拟机的权限,并将写权限标识发送给第一虚拟机,将读权限标识发送给第二虚拟机,即将多个虚拟机各自的权限下发给相应的虚拟机。
其中,客户端将写权限标识发送给第一虚拟机的过程可以在执行S303之前,也可以在执行S303之后,也可以与S303同时执行,即同时将写权限标识、待写数据和第二虚拟机的地址发送给第一虚拟机。本发明实施例对这两个步骤的执行顺序不做限制。类似地,客户端可以将读权限标识发送给第二虚拟机。
第二种方式
在响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,预设规则用于指定第一虚拟机具有向存储区域写入待写数据的权限,以及指定第二虚拟机具有从存储区域读取待写数据的权限。
其中,预设规则可以是响应消息包括的虚拟机的地址的顺序。比如,名称节点和客户端可以事先约定:名称节点向客户端发送的第一个地址为第一虚拟机的地址,那么客户端在接收到响应消息包括的多个虚拟机的地址后,可以确定第一个地址为具有向存储区域写入数据的权限的第一虚拟机的地址,其余的地址为具有从存储区域读取待写数据的权限的第二虚拟机的地址。
前面已经提到,由于分布式文件系统中可用于为客户端提供读写待写数据的服务的虚拟机的数量为多个。当某个虚拟机发生故障时,可以由其他虚拟机为客户端提供读写待写数据的服务,从而提高了分布式文件系统的可用性。下面将对分布式文件系统的虚拟机发生故障后如何操作做详细解释。
在分布式文件系统中,检测到虚拟机发生故障的方式包括但不限于以下三种:第一种,名称节点检测到某个虚拟机发生故障;第二种,客户端通过某个虚拟机读取数据或写入数据时,若无法完成数据读写过程,则确定该虚拟机发生故障,客户端将该虚拟机发生故障的消息上报给名称节点;第三种,虚拟机会定期进行自检,当某个虚拟机发现 自身发生故障时,会将自身发生故障的消息直接上报给名称节点,或者通过客户端上报给名称节点。因此,当分布式文件系统中的某个虚拟机发生故障时,名称节点可以上述三种途径获知该虚拟机发生故障的消息,继而可采取相应操作,避免出现分布式文件系统无法为客户端提供数据读写服务的情况。
本发明实施例中,分布式文件系统的虚拟机发生故障可分为以下两种情况:
第一种情况:第一虚拟机发生故障
当第一虚拟机发生故障时,名称节点向客户端发送第一更新消息,该第一更新消息包括更新的第一虚拟机的地址,该第一更新消息指定多个虚拟机中除发生故障的第一虚拟机以外的另一个虚拟机作为更新的第一虚拟机,即指示更新的第一虚拟机具有向多个虚拟机共享的存储区域写入数据的权限;因而,客户端需要写入数据待写时,可以通过更新的第一虚拟机写入。
采用这种方法,当第一虚拟机发生故障时,名称节点指定多个虚拟机中除发生故障的第一虚拟机以外的另一个虚拟机作为更新的第一虚拟机,即指示更新的第一虚拟机具有向多个虚拟机共享的存储区域写入数据的权限,那么当客户端要写入数据时可通过更新的第一虚拟机写入,当客户端要读取数据时,可通过第二虚拟机读取或者通过更新的第一虚拟机读取。
综上,采用这种方法,在第一虚拟机发生故障时不会影响客户端写入或读取数据,提高了系统的可用性。
如前所述,第二虚拟机的数量可以是一个或者多个,当第二虚拟机的数量为一个时,若第一虚拟机发生故障,则在执行上述方法之后还可以进一步地执行如下方法:名称节点在分布式文件系统包括的多个虚拟机之外指定一个或多个更新的第二虚拟机,更新的第二虚拟机具有从存储区域读取待写数据的权限,更新的第二虚拟机与分布式文件系统包括的多个虚拟机共享同一存储区域;名称节点向客户端指示更新的第二虚拟机具有从存储区域读取待写数据的权限。客户端接收到名称节点的指示后,通知更新的第一虚拟机:将待写数据的元数据发送给更新的第二虚拟机。这样,客户端要读取待写数据时,不仅可以通过第二虚拟机读取,还可通过更新的第二虚拟机读取。
名称节点在分布式文件系统包括的多个虚拟机之外指定更新的第二虚拟机具有从存储区域读取待写数据的权限,当存在多个客户端需要读取待写数据时,客户端不仅可以通过第二虚拟机读取,还可以通过更新的第二虚拟机读取,提高了客户端读取待写数据的效率。
第二种情况:第二虚拟机发生故障
当第二虚拟机发生故障时,名称节点向客户端发送第二更新信息,第二更新信息包括更新的第二虚拟机的地址,第二更新信息指定多个虚拟机以外的另一个虚拟机作为更新的第二虚拟机,更新的第二虚拟机具有从存储区域读取待写数据的权限。其中,更新的第二虚拟机与分布式文件系统包括的多个虚拟机共享上述同一存储区域。名称节点向客户端指示更新的第二虚拟机具有从存储区域读取待写数据的权限后,客户端根据名称节点的指示通知第一虚拟机:将待写数据的元数据发送给更新的第二虚拟机。
其中,第一虚拟机在接收到客户端发送的通知消息后可根据通知消息的指示将待写数据的元数据发送给更新的第二虚拟机。这样,客户端要读取待写数据时,不仅可以通 过未发生故障的第二虚拟机读取或通过第一虚拟机读取,还可通过更新的第二虚拟机读取。
采用这种方法,当第二虚拟机发生故障时,名称节点指定更新的第二虚拟机具有从存储区域读取待写数据的权限,那么当客户端要写入待写数据时可通过第一虚拟机写入,当客户端要读取待写数据时,不仅可通过第一虚拟机和未发生故障的第二虚拟机读取,也可以通过更新的第二虚拟机读取。综上,采用这种方法,在第一虚拟机发生故障时不会影响客户端写入或读取待写数据,提高了系统的可用性。
采用本发明实施例提供的分布式文件系统中存储文件的方法,可以解决分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统中的某个虚拟机发生故障时,采用本发明实施例提供的分布式文件系统中存储文件的方法不会影响客户端读取或写入文件,提高了系统的可用性。
本发明实施例提供一种分布式文件系统中的第一虚拟机,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机。如图5所示,该第一虚拟机500包括:
接收模块501,用于接收客户端发送的待写数据、第二虚拟机的地址,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;
处理模块502,用于向存储区域写入接收模块501接收的待写数据,并生成或更新待写数据的元数据;
发送模块503,用于根据接收模块501接收的第二虚拟机的地址向第二虚拟机发送处理模块502生成或更新元数据。
可选地,接收模块501还用于:在处理模块502向存储区域写入待写数据之前,接收客户端发送的第一虚拟机的写权限标识。
其中,写权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限。
可选地,多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
可选地,若第二虚拟机通过自身的操作系统读取待写数据,则元数据用于第二虚拟机生成或更新自身的操作系统中记录的文件信息,文件信息用于操作系统从存储区域中读取待写数据;或者若第二虚拟机读取待写数据,则元数据用于第二虚拟机从存储区域中读取待写数据。
可选地,第二虚拟机被名称节点指定具有从存储区域读取待写数据的权限。
采用本发明实施例提供的第一虚拟机500,可以解决分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统中的某个虚拟机发生故障时,通过本发明实施例提供的第一虚拟机500的操作,可以使得虚拟机的故障不会影响客户端读取或写入文件,提高了系统的可用性。
需要说明的是本发明实施例提供的第一虚拟机500可用于执行图3所示的分布式文件系统中存储文件的方法中第一虚拟机执行的操作,第一虚拟机500未详细解释和描述 的实现方式可参考图3所示的分布式文件系统中存储文件的方法中的相关描述。
需要说明的是,本发明实施例中对模块的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。另外,在本申请各个实施例中的各功能模块可以集成在一个处理模块中,也可以是各个模块单独物理存在,也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。
基于以上实施例,本发明实施例还提供了一种第一虚拟机,该第一虚拟机可以执行图3对应的实施例提供的方法,可以与图5所示的第一虚拟机500相同。
参见图6,第一虚拟机600所在的设备包括至少一个处理器601、存储器602和通信接口603;所述至少一个处理器601、所述存储器602和所述通信接口603均通过总线604连接;
所述存储器602,用于存储计算机执行指令;
所述至少一个处理器601,用于执行所述存储器602存储的计算机执行指令,使得所述第一虚拟机600通过所述通信接口603与分布式文件系统中的其它设备进行数据交互来执行上述实施例提供的分布式文件系统中存储文件的方法,或者使得所述第一虚拟机600通过所述通信接口603与分布式文件系统中的其它设备进行数据交互来实现分布式文件系统的部分或者全部功能。
至少一个处理器601,可以包括不同类型的处理器601,或者包括相同类型的处理器601;处理器601可以是以下的任一种:中央处理器(Central Processing Unit,简称CPU)、ARM处理器、现场可编程门阵列(Field Programmable Gate Array,简称FPGA)、专用处理器等具有计算处理能力的器件。一种可选实施方式,所述至少一个处理器601还可以集成为众核处理器。
存储器602可以是以下的任一种或任一种组合:随机存取存储器(Random Access Memory,简称RAM)、只读存储器(read only memory,简称ROM)、非易失性存储器(non-volatile memory,简称NVM)、固态硬盘(Solid State Drives,简称SSD)、机械硬盘、磁盘、磁盘阵列等存储介质。
通信接口603用于第一虚拟机600与其他设备(例如分布式文件系统中的其他设备)进行数据交互。通信接口603可以是以下的任一种或任一种组合:网络接口(例如以太网接口)、无线网卡等具有网络接入功能的器件。
该总线604可以包括地址总线、数据总线、控制总线等,为便于表示,图6用一条粗线表示该总线。总线604可以是以下的任一种或任一种组合:工业标准体系结构(Industry Standard Architecture,简称ISA)总线、外设组件互连标准(Peripheral Component Interconnect,简称PCI)总线、扩展工业标准结构(Extended Industry Standard Architecture,简称EISA)总线等有线数据传输的器件。
本发明实施例提供一种分布式文件系统中的名称节点,该分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;如图7所示,名称节点700包括:
接收模块701,用于接收客户端请求向分布式文件系统写入待写数据的请求消息;
发送模块702,用于向客户端发送接收模块701接收的请求消息对应的响应消息,响 应消息包括第一虚拟机的地址和第二虚拟机的地址,响应消息指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机。
可选地,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
可选地,响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
可选地,响应消息中第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,预设规则用于指定第一虚拟机具有向存储区域写入待写数据的权限,以及指定第二虚拟机具有从存储区域读取待写数据的权限。
可选地,多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
可选地,发送模块702还用于:当第一虚拟机发生故障时,向客户端发送第一更新信息,第一更新信息包括更新的第一虚拟机的地址,第一更新信息指定多个虚拟机中除发生故障的第一虚拟机以外的另一个虚拟机作为更新的第一虚拟机,更新的第一虚拟机具有向存储区域写入数据的权限;和/或当第二虚拟机发生故障时,向客户端发送第二更新信息,第二更新信息包括更新的第二虚拟机的地址,第二更新信息指定多个虚拟机以外的另一个虚拟机作为更新的第二虚拟机,更新的第二虚拟机具有从存储区域读取待写数据的权限。
采用本发明实施例提供的名称节点700,可以解决分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统中的某个虚拟机发生故障时,通过本发明实施例提供的名称节点700的操作,可以使得虚拟机的故障不会影响客户端读取或写入文件,提高了系统的可用性。
需要说明的是本发明实施例提供的名称节点700可用于执行图3所示的分布式文件系统中存储文件的方法中名称节点执行的操作,名称节点700未详细解释和描述的实现方式可参考图3所示的分布式文件系统中存储文件的方法中的相关描述。
基于以上实施例,本发明实施例还提供了一种名称节点,该名称节点可执行图3对应的实施例提供的方法,可以与图7所示的名称节点700相同。
参见图8,名称节点800包括至少一个处理器801、存储器802和通信接口803;所述至少一个处理器801、所述存储器802和所述通信接口803均通过总线804连接;
所述存储器802,用于存储计算机执行指令;
所述至少一个处理器801,用于执行所述存储器802存储的计算机执行指令,使得所述名称节点800通过所述通信接口803与分布式文件系统中的其它设备进行数据交互来执行上述实施例提供的分布式文件系统中存储文件的方法,或者使得所述名称节点800通过所述通信接口803与分布式文件系统中的其它设备进行数据交互来实现分布式文件系统的部分或者全部功能。
至少一个处理器801,可以包括不同类型的处理器801,或者包括相同类型的处理器801;处理器801可以是以下的任一种:CPU、ARM处理器、FPGA、专用处理器等具有计算处理能力的器件。一种可选实施方式,所述至少一个处理器801还可以集成为众核处理 器。
存储器802可以是以下的任一种或任一种组合:RAM、ROM、NVM、SSD、机械硬盘、磁盘、磁盘阵列等存储介质。
通信接口803用于名称节点800与其他设备(例如分布式文件系统中的其他设备)进行数据交互。通信接口803可以是以下的任一种或任一种组合:网络接口(例如以太网接口)、无线网卡等具有网络接入功能的器件。
该总线804可以包括地址总线、数据总线、控制总线等,为便于表示,图8用一条粗线表示该总线。总线804可以是以下的任一种或任一种组合:ISA总线、PCI总线、EISA总线等有线数据传输的器件。
本发明实施例提供一种客户端,该客户端所在的分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;如图9所示,客户端900包括:
发送模块901,用于向名称节点发送请求向分布式文件系统写入待写数据的请求消息;
接收模块902,用于接收名称节点发送的请求消息对应的响应消息,响应消息包括第一虚拟机的地址和第二虚拟机的地址,响应消息指示第一虚拟机为多个虚拟机中具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机;
发送模块901,还用于根据接收模块902接收的响应消息包括的第一虚拟机的地址向第一虚拟机发送待写数据、第二虚拟机的地址,并指示第一虚拟机写入待写数据、生成或更新待写数据的元数据、以及根据接收模块902接收的响应消息包括的第二虚拟机的地址向第二虚拟机发送待写数据的元数据。
可选地,响应消息还包括第一虚拟机的写权限标识和第二虚拟机的读权限标识,写权限标识用于指定第一虚拟机具有向存储区域写入待写数据的权限,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
可选地,响应消息包括的第一虚拟机的地址和第二虚拟机的地址按照预设规则排列,预设规则用于指定第一虚拟机具有向存储区域写入待写数据的权限以及指定第二虚拟机具有从存储区域读取待写数据的权限。
可选地,多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
可选地,响应消息还指示第二虚拟机具有从存储区域读取待写数据的权限。
采用本发明实施例提供的客户端900,可以解决分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统中的某个虚拟机发生故障时,通过本发明实施例提供的客户端900的操作,可以使得虚拟机的故障不会影响客户端读取或写入文件,提高了系统的可用性。
需要说明的是本发明实施例提供的客户端900可用于执行图3所示的分布式文件系统中存储文件的方法中客户端执行的操作,客户端900未详细解释和描述的实现方式可参考图3所示的分布式文件系统中存储文件的方法中的相关描述。
基于以上实施例,本发明实施例还提供了一种客户端,该客户端可以执行图3对应 的实施例提供的方法,可以与图9所示的客户端900相同。
参见图10,客户端1000所在的设备包括至少一个处理器1001、存储器1002和通信接口1003;所述至少一个处理器1001、所述存储器1002和所述通信接口1003均通过总线1004连接;
所述存储器1002,用于存储计算机执行指令;
所述至少一个处理器1001,用于执行所述存储器1002存储的计算机执行指令,使得所述客户端1000通过所述通信接口1003与分布式文件系统中的设备进行数据交互来执行上述实施例提供的分布式文件系统中存储文件的方法,或者使得所述客户端1000通过所述通信接口1003与分布式文件系统中的设备进行数据交互来实现分布式文件系统的部分或者全部功能。
至少一个处理器1001,可以包括不同类型的处理器1001,或者包括相同类型的处理器1001;处理器1001可以是以下的任一种:CPU、ARM处理器、FPGA、专用处理器等具有计算处理能力的器件。一种可选实施方式,所述至少一个处理器1001还可以集成为众核处理器。
存储器1002可以是以下的任一种或任一种组合:RAM、ROM、NVM、SSD、机械硬盘、磁盘、磁盘阵列等存储介质。
通信接口1003用于客户端1000与其他设备(例如分布式文件系统中的其他设备)进行数据交互。通信接口1003可以是以下的任一种或任一种组合:网络接口(例如以太网接口)、无线网卡等具有网络接入功能的器件。
该总线1004可以包括地址总线、数据总线、控制总线等,为便于表示,图10用一条粗线表示该总线。总线1004可以是以下的任一种或任一种组合:ISA总线、PCI总线、EISA总线等有线数据传输的器件。
本发明实施例提供一种分布式文件系统中的第二虚拟机,分布式文件系统包括名称节点、多个作为数据节点的虚拟机,多个虚拟机共享同一存储区域;如图11所示,第二虚拟机1100包括:
接收模块1101,用于接收第一虚拟机发送的元数据,第一虚拟机为多个虚拟机中被名称节点指定具有向存储区域写入数据的权限的一个虚拟机,第二虚拟机为多个虚拟机中除第一虚拟机以外的虚拟机,元数据为第一虚拟机向存储区域写入待写数据后生成或更新的待写数据的元数据。
可选地,接收模块1101还用于:在接收第一虚拟机发送的元数据之前,接收客户端发送的第二虚拟机的读权限标识,读权限标识是名称节点在客户端向名称节点请求向分布式文件系统写入待写数据时向客户端发送的,读权限标识用于指定第二虚拟机具有从存储区域读取待写数据的权限。
可选地,多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,虚拟硬盘包括存储区域。
可选地,第二虚拟机还包括:处理模块1102,用于在接收模块1101接收第一虚拟机发送的元数据之后,若第二虚拟机通过自身的操作系统读取待写数据,则根据元数据生成或更新自身的操作系统中记录的文件信息,文件信息用于操作系统从存储区域中读取待写数据;或者若第二虚拟机读取待写数据,则根据元数据从存储区域中读取待写数据。
可选地,第二虚拟机被名称节点指定具有从存储区域读取待写数据的权限。
采用本发明实施例提供的第二虚拟机1100,可以解决分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统中的某个虚拟机发生故障时,通过本发明实施例提供的第二虚拟机1100的操作,可以使得虚拟机的故障不会影响客户端读取或写入文件,提高了系统的可用性。
需要说明的是本发明实施例提供的第二虚拟机1100可用于执行图3所示的分布式文件系统中存储文件的方法中第二虚拟机执行的操作,第二虚拟机1100未详细解释和描述的实现方式可参考图3所示的分布式文件系统中存储文件的方法中的相关描述。
基于以上实施例,本发明实施例还提供了一种第二虚拟机,该第二虚拟机可以执行图3对应的实施例提供的方法,可以与图11所示的第二虚拟机1100相同。
参见图12,第二虚拟机1200所在的设备包括至少一个处理器1201、存储器1202和通信接口1203;所述至少一个处理器1201、所述存储器1202和所述通信接口1203均通过总线1204连接;
所述存储器1202,用于存储计算机执行指令;
所述至少一个处理器1201,用于执行所述存储器1202存储的计算机执行指令,使得所述第二虚拟机1200通过所述通信接口1203与分布式文件系统中的其它设备进行数据交互来执行上述实施例提供的分布式文件系统中存储文件的方法,或者使得所述第二虚拟机1200通过所述通信接口1203与分布式文件系统中的其它设备进行数据交互来实现分布式文件系统的部分或者全部功能。
至少一个处理器1201,可以包括不同类型的处理器1201,或者包括相同类型的处理器1201;处理器1201可以是以下的任一种:CPU、ARM处理器、FPGA、专用处理器等具有计算处理能力的器件。一种可选实施方式,所述至少一个处理器1201还可以集成为众核处理器。
存储器1202可以是以下的任一种或任一种组合:RAM、ROM、NVM、SSD、机械硬盘、磁盘、磁盘阵列等存储介质。
通信接口1203用于第二虚拟机1200与其他设备(例如分布式文件系统中的其他设备)进行数据交互。通信接口1203可以是以下的任一种或任一种组合:网络接口(例如以太网接口)、无线网卡等具有网络接入功能的器件。
该总线1204可以包括地址总线、数据总线、控制总线等,为便于表示,图12用一条粗线表示该总线。总线1204可以是以下的任一种或任一种组合:ISA总线、PCI总线、EISA总线等有线数据传输的器件。
本发明实施例提供一种分布式文件系统,如图13所示,分布式文件系统1300包括:第一虚拟机1301、名称节点1302、客户端1303和第二虚拟机1304。
其中,分布式文件系统1300中的第一虚拟机1301可用于执行图3所示的分布式文件系统中存储文件的方法中第一虚拟机所执行的相关操作,其具体实现形式可以是图5所示的第一虚拟机500或图6所示的第一虚拟机600;分布式文件系统1300中的名称节点1302可用于执行图3所示的分布式文件系统中存储文件的方法中名称节点所执行的相关操作,其具体实现形式可以是图7所示的名称节点700或图8所示的名称节点800;分布式文件系统1300中的客户端1303可用于执行图3所示的分布式文件系统中存储文件 的方法中客户端所执行的相关操作,其具体实现形式可以是图9所示的客户端900或图10所示的客户端1000;分布式文件系统1300中的第二虚拟机1304可用于执行图3所示的分布式文件系统中存储文件的方法中第二虚拟机所执行的相关操作,其具体实现形式可以是图11所示的第二虚拟机1100或图12所示的第二虚拟机1200。
在分布式文件系统1300中,待写数据仅在多个虚拟机共享的存储区域中保存一份,解决了分布式文件系统中存储的文件份数冗余的问题。此外,当分布式文件系统1300中的某个虚拟机发生故障时,客户端仍可通过分布式文件系统1300中未发生故障的虚拟机对待写数据进行写入操作或读取操作,提高了分布式文件系统的可用性。
本领域内的技术人员应明白,本发明的实施例可提供为方法、系统、或计算机程序产品。因此,本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例,但本领域内的技术人员一旦得知了基本创造性概念,则可对这些实施例做出另外的变更和修改。所以,所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然,本领域的技术人员可以对本发明实施例进行各种改动和变型而不脱离本发明实施例的精神和范围。这样,倘若本发明实施例的这些修改和变型属于本发明权利要求及其等同技术的范围之内,则本发明也意图包含这些改动和变型在内。

Claims (22)

  1. 一种分布式文件系统中存储文件的方法,其特征在于,所述分布式文件系统包括名称节点、多个作为数据节点的虚拟机,所述多个虚拟机共享同一存储区域;所述方法包括:
    第一虚拟机接收客户端发送的待写数据、第二虚拟机的地址,所述第一虚拟机为所述多个虚拟机中被所述名称节点指定具有向所述存储区域写入数据的权限的一个虚拟机,所述第二虚拟机为所述多个虚拟机中除所述第一虚拟机以外的虚拟机;
    所述第一虚拟机向所述存储区域写入所述待写数据,并生成或更新所述待写数据的元数据;
    所述第一虚拟机根据所述第二虚拟机的地址向所述第二虚拟机发送所述元数据。
  2. 如权利要求1所述的方法,其特征在于,所述第一虚拟机向所述存储区域写入所述待写数据之前,还包括:
    所述第一虚拟机接收所述客户端发送的所述第一虚拟机的写权限标识,所述写权限标识是所述名称节点在所述客户端向所述名称节点请求向分布式文件系统写入所述待写数据时向所述客户端发送的,所述写权限标识用于指定所述第一虚拟机具有向所述存储区域写入所述待写数据的权限。
  3. 如权利要求1或2所述的方法,其特征在于,所述多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,所述虚拟硬盘包括所述存储区域。
  4. 如权利要求1至3任一项所述的方法,其特征在于,
    若所述第二虚拟机通过自身的操作系统读取所述待写数据,则所述元数据用于所述第二虚拟机生成或更新自身的操作系统中记录的文件信息,所述文件信息用于所述操作系统从所述存储区域中读取所述待写数据;或者
    若所述第二虚拟机读取所述待写数据,则所述元数据用于所述第二虚拟机从所述存储区域中读取所述待写数据。
  5. 如权利要求1至4任一项所述的方法,其特征在于,所述第二虚拟机被所述名称节点指定具有从所述存储区域读取所述待写数据的权限。
  6. 一种分布式文件系统中存储文件的方法,其特征在于,所述分布式文件系统包括名称节点、多个作为数据节点的虚拟机,所述多个虚拟机共享同一存储区域;所述方法包括:
    所述名称节点接收客户端请求向所述分布式文件系统写入待写数据的请求消息;
    所述名称节点向所述客户端发送所述请求消息对应的响应消息,所述响应消息包括第一虚拟机的地址和第二虚拟机的地址,所述响应消息指示所述第一虚拟机为所述多个虚拟机中具有向所述存储区域写入数据的权限的一个虚拟机,所述第二虚拟机为所述多个虚拟机中除所述第一虚拟机以外的虚拟机。
  7. 如权利要求6所述的方法,其特征在于,所述响应消息还指示所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  8. 如权利要求6或7所述的方法,其特征在于,所述响应消息还包括所述第一虚拟机的写权限标识和所述第二虚拟机的读权限标识,所述写权限标识用于指定所述第一 虚拟机具有向所述存储区域写入所述待写数据的权限,所述读权限标识用于指定所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  9. 如权利要求6或7所述的方法,其特征在于,所述响应消息中所述第一虚拟机的地址和所述第二虚拟机的地址按照预设规则排列,所述预设规则用于指定所述第一虚拟机具有向所述存储区域写入所述待写数据的权限,以及指定所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  10. 如权利要求6至9任一项所述的方法,其特征在于,所述多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,所述虚拟硬盘包括所述存储区域。
  11. 如权利要求6至10任一项所述的方法,其特征在于,所述方法还包括:
    当所述第一虚拟机发生故障时,所述名称节点向所述客户端发送第一更新信息,所述第一更新信息包括更新的第一虚拟机的地址,所述第一更新信息指定所述多个虚拟机中除发生故障的所述第一虚拟机以外的另一个虚拟机作为所述更新的第一虚拟机,所述更新的第一虚拟机具有向所述存储区域写入数据的权限;和/或当所述第二虚拟机发生故障时,所述名称节点向所述客户端发送第二更新信息,所述第二更新信息包括更新的第二虚拟机的地址,所述第二更新信息指定所述多个虚拟机以外的另一个虚拟机作为所述更新的第二虚拟机,所述更新的第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  12. 一种分布式文件系统中的第一虚拟机,其特征在于,所述分布式文件系统包括名称节点、多个作为数据节点的虚拟机,所述多个虚拟机共享同一存储区域,所述第一虚拟机为所述多个虚拟机中被所述名称节点指定具有向所述存储区域写入数据的权限的一个虚拟机;所述第一虚拟机包括:
    接收模块,用于接收客户端发送的待写数据、第二虚拟机的地址,所述第二虚拟机为所述多个虚拟机中除所述第一虚拟机以外的虚拟机;
    处理模块,用于向所述存储区域写入所述接收模块接收的所述待写数据,并生成或更新所述待写数据的元数据;
    发送模块,用于根据所述接收模块接收的所述第二虚拟机的地址向所述第二虚拟机发送所述处理模块生成或更新所述元数据。
  13. 如权利要求12所述的第一虚拟机,其特征在于,所述接收模块还用于:
    在所述处理模块向所述存储区域写入所述待写数据之前,接收所述客户端发送的所述第一虚拟机的写权限标识,所述写权限标识是所述名称节点在所述客户端向所述名称节点请求向分布式文件系统写入所述待写数据时向所述客户端发送的,所述写权限标识用于指定所述第一虚拟机具有向所述存储区域写入所述待写数据的权限。
  14. 如权利要求12或13所述的第一虚拟机,其特征在于,所述多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,所述虚拟硬盘包括所述存储区域。
  15. 如权利要求12至14任一项所述的第一虚拟机,其特征在于,
    若所述第二虚拟机通过自身的操作系统读取所述待写数据,则所述元数据用于所述第二虚拟机生成或更新自身的操作系统中记录的文件信息,所述文件信息用于所述操作系统从所述存储区域中读取所述待写数据;或者
    若所述第二虚拟机读取所述待写数据,则所述元数据用于所述第二虚拟机从所述存储区域中读取所述待写数据。
  16. 如权利要求12至15任一项所述的第一虚拟机,其特征在于,所述第二虚拟机被所述名称节点指定具有从所述存储区域读取所述待写数据的权限。
  17. 一种分布式文件系统中的名称节点,其特征在于,所述分布式文件系统包括所述名称节点、多个作为数据节点的虚拟机,所述多个虚拟机共享同一存储区域;所述名称节点包括:
    接收模块,用于接收客户端请求向所述分布式文件系统写入待写数据的请求消息;
    发送模块,用于向所述客户端发送所述接收模块接收的所述请求消息对应的响应消息,所述响应消息包括第一虚拟机的地址和第二虚拟机的地址,所述响应消息指示所述第一虚拟机为所述多个虚拟机中具有向所述存储区域写入数据的权限的一个虚拟机,所述第二虚拟机为所述多个虚拟机中除所述第一虚拟机以外的虚拟机。
  18. 如权利要求17所述的名称节点,其特征在于,所述响应消息还指示所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  19. 如权利要求17或18所述的名称节点,其特征在于,所述响应消息还包括所述第一虚拟机的写权限标识和所述第二虚拟机的读权限标识,所述写权限标识用于指定所述第一虚拟机具有向所述存储区域写入所述待写数据的权限,所述读权限标识用于指定所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  20. 如权利要求17或18所述的名称节点,其特征在于,所述响应消息中所述第一虚拟机的地址和所述第二虚拟机的地址按照预设规则排列,所述预设规则用于指定所述第一虚拟机具有向所述存储区域写入所述待写数据的权限,以及指定所述第二虚拟机具有从所述存储区域读取所述待写数据的权限。
  21. 如权利要求17至20任一项所述的名称节点,其特征在于,所述多个虚拟机挂载分布式块存储系统提供的同一虚拟硬盘,所述虚拟硬盘包括所述存储区域。
  22. 如权利要求17至21任一项所述的名称节点,其特征在于,所述发送模块还用于:
    当所述第一虚拟机发生故障时,向所述客户端发送第一更新信息,所述第一更新信息包括更新的第一虚拟机的地址,所述第一更新信息指定所述多个虚拟机中除发生故障的所述第一虚拟机以外的另一个虚拟机作为所述更新的第一虚拟机,所述更新的第一虚拟机具有向所述存储区域写入数据的权限;和/或
    当所述第二虚拟机发生故障时,向所述客户端发送第二更新信息,所述第二更新信息包括更新的第二虚拟机的地址,所述第二更新信息指定所述多个虚拟机以外的另一个虚拟机作为所述更新的第二虚拟机,所述更新的第二虚拟机具有从所述存储区域读取所述待写数据的权限。
PCT/CN2017/085351 2016-09-23 2017-05-22 一种存储文件的方法、第一虚拟机及名称节点 WO2018054079A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610846967.0A CN106446159B (zh) 2016-09-23 2016-09-23 一种存储文件的方法、第一虚拟机及名称节点
CN201610846967.0 2016-09-23

Publications (1)

Publication Number Publication Date
WO2018054079A1 true WO2018054079A1 (zh) 2018-03-29

Family

ID=58167356

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/085351 WO2018054079A1 (zh) 2016-09-23 2017-05-22 一种存储文件的方法、第一虚拟机及名称节点

Country Status (2)

Country Link
CN (1) CN106446159B (zh)
WO (1) WO2018054079A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111443872A (zh) * 2020-03-26 2020-07-24 深信服科技股份有限公司 分布式存储系统构建方法、装置、设备、介质
CN113641467A (zh) * 2021-10-19 2021-11-12 杭州优云科技有限公司 一种虚拟机的分布式块存储实现方法

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106446159B (zh) * 2016-09-23 2019-11-12 华为技术有限公司 一种存储文件的方法、第一虚拟机及名称节点
CN107704596B (zh) * 2017-10-13 2021-06-29 郑州云海信息技术有限公司 一种读取文件的方法、装置及设备
CN109753226A (zh) * 2017-11-07 2019-05-14 阿里巴巴集团控股有限公司 数据处理系统、方法及电子设备
CN110110003A (zh) * 2018-01-26 2019-08-09 广州中国科学院计算机网络信息中心 M2m平台的数据存储控制方法及装置
CN110688194B (zh) * 2018-07-06 2023-03-17 中兴通讯股份有限公司 基于云桌面的磁盘管理方法、虚拟机及存储介质
CN113037569A (zh) * 2021-04-19 2021-06-25 杭州和利时自动化有限公司 一种基于双服务器的冗余服务方法、装置、设备及介质
CN114138737B (zh) * 2022-02-08 2022-07-12 亿次网联(杭州)科技有限公司 文件存储方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130325812A1 (en) * 2012-05-30 2013-12-05 Spectra Logic Corporation System and method for archive in a distributed file system
CN103729250A (zh) * 2012-10-11 2014-04-16 国际商业机器公司 用于选择被配置为满足一组要求的数据节点的方法和系统
CN104731691A (zh) * 2013-12-18 2015-06-24 国际商业机器公司 动态调整分布式文件系统内文件副本数目的方法和系统
CN104838374A (zh) * 2012-12-06 2015-08-12 英派尔科技开发有限公司 分散hadoop集群
CN106446159A (zh) * 2016-09-23 2017-02-22 华为技术有限公司 一种存储文件的方法、第一虚拟机及名称节点

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521063B (zh) * 2011-11-30 2013-12-25 广东电子工业研究院有限公司 一种适用于虚拟机迁移和容错的共享存储方法
CN103797770B (zh) * 2012-12-31 2015-12-02 华为技术有限公司 一种共享存储资源的方法和系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130325812A1 (en) * 2012-05-30 2013-12-05 Spectra Logic Corporation System and method for archive in a distributed file system
CN103729250A (zh) * 2012-10-11 2014-04-16 国际商业机器公司 用于选择被配置为满足一组要求的数据节点的方法和系统
CN104838374A (zh) * 2012-12-06 2015-08-12 英派尔科技开发有限公司 分散hadoop集群
CN104731691A (zh) * 2013-12-18 2015-06-24 国际商业机器公司 动态调整分布式文件系统内文件副本数目的方法和系统
CN106446159A (zh) * 2016-09-23 2017-02-22 华为技术有限公司 一种存储文件的方法、第一虚拟机及名称节点

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111443872A (zh) * 2020-03-26 2020-07-24 深信服科技股份有限公司 分布式存储系统构建方法、装置、设备、介质
CN113641467A (zh) * 2021-10-19 2021-11-12 杭州优云科技有限公司 一种虚拟机的分布式块存储实现方法
CN113641467B (zh) * 2021-10-19 2022-02-11 杭州优云科技有限公司 一种虚拟机的分布式块存储实现方法

Also Published As

Publication number Publication date
CN106446159B (zh) 2019-11-12
CN106446159A (zh) 2017-02-22

Similar Documents

Publication Publication Date Title
WO2018054079A1 (zh) 一种存储文件的方法、第一虚拟机及名称节点
US11354336B2 (en) Fault-tolerant key management system
US10747673B2 (en) System and method for facilitating cluster-level cache and memory space
US11157457B2 (en) File management in thin provisioning storage environments
US10382540B2 (en) Synchronizing storage state information
WO2020263765A1 (en) Orchestrator for orchestrating operations between a computing environment hosting virtual machines and storage environment
JP2019101703A (ja) 記憶システム及び制御ソフトウェア配置方法
WO2019148841A1 (zh) 一种分布式存储系统、数据处理方法和存储节点
WO2019061352A1 (zh) 数据加载方法及装置
WO2016045428A1 (zh) 一种创建虚拟机的方法和创建虚拟机的装置
US20160306550A1 (en) Constructing a scalable storage device, and scaled storage device
WO2021057108A1 (zh) 一种读数据方法、写数据方法及服务器
CN114514500A (zh) 跨区复制块存储装置
WO2018157605A1 (zh) 一种集群文件系统中消息传输的方法及装置
US9910808B2 (en) Reflective memory bridge for external computing nodes
EP3479256A2 (en) Fabric encapsulated resilient storage
US10552067B2 (en) Method and system for delivering message in storage system
JPWO2015141219A1 (ja) ストレージシステム、制御装置、データアクセス方法およびプログラム
WO2012171363A1 (zh) 分布式缓存系统中的数据操作方法和装置
CN116389233A (zh) 容器云管理平台主备切换系统、方法、装置和计算机设备
WO2016046951A1 (ja) 計算機システム及びそのファイル管理方法
US20200371849A1 (en) Systems and methods for efficient management of advanced functions in software defined storage systems
US11288004B1 (en) Consensus-based authority selection in replicated network-accessible block storage devices
US10908982B2 (en) Method and system for processing data
US10762011B2 (en) Reflective memory bridge for external computing nodes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17852153

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17852153

Country of ref document: EP

Kind code of ref document: A1