WO2016065610A1

WO2016065610A1 - Method for accessing files, distributed storage system and storage node

Info

Publication number: WO2016065610A1
Application number: PCT/CN2014/090017
Authority: WO
Inventors: 朱冠宇; 徐君; 刘海燕
Original assignee: 华为技术有限公司
Priority date: 2014-10-31
Filing date: 2014-10-31
Publication date: 2016-05-06
Also published as: CN105765542A; CN105765542B

Abstract

A method for accessing files, a distributed storage system, and a storage node. The method comprises: a first storage node receives a host access request forwarded by a network device, said access request being used for accessing a first virtual space of a target file, said first virtual space being a space in the global virtual address space allocated for the target file by a management node, and the target address of the access request being the global virtual address of said first virtual space (S210), the first storage node being a storage node used for storing said target file; the first storage node, on the basis of the correspondence between the target address and the local virtual address of the first storage node, determines the local virtual address corresponding to the target address of the access request (S220); and the first storage node, on the basis of the local virtual address, accesses the target file (S230). The efficiency of file access can thereby be improved.

Description

Method of accessing files, distributed storage systems, and storage nodes

Technical field

The present invention relates to the field of storage technologies, and in particular, to a method for accessing a file, a distributed storage system, and a storage node.

Background technique

New non-violate memory (NVM) media, such as Phase Change Memory (PCM), Resistive RAM (ReRAM), Magnetic Random Access Memory (Magnetic Random Access Memory) , MRAM), etc., with similar dynamic random access memory (DRAM) read and write characteristics, can be hung on the memory bus for access. At present, storage systems based on non-volatile storage media are hotspots in the industry.

Existing methods for remotely accessing a file system include using a Network File System (NFS) or a Common Internet File System (CIFS) protocol. These traditional ways of accessing files are essentially based on Remote Procedure Call (RPC), which is based on the Transmission Control Protocol/Internet Protocol (TCP/IP). . For an NVM-based storage system, if the traditional remote access file system access method is still used, it still needs to be processed by the host's kernel TCP/IP protocol stack, and the access efficiency is low.

Summary of the invention

The embodiment of the invention provides a method for accessing a file, a distributed storage system and a storage node, which can improve the efficiency of file access.

In a first aspect, a method for accessing a file is provided, the method being applied to a distributed storage system, the distributed storage system comprising a management node, a plurality of storage nodes, and a network device, each storage node in the distributed storage system The local virtual address space corresponds to a portion of the global virtual address space of the distributed storage system, and the method includes:

Receiving, by the first storage node, an access request of the host forwarded by the network device, where the access request is used to access a first virtual space of the target file, where the first virtual space is the management node allocates the target file in the global virtual address space. Space, the destination address of the access request is the first virtual a global virtual address of the space, the first storage node being a storage node of the plurality of storage nodes for storing the target file;

Determining, by the first storage node, the local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the first storage node;

The first storage node accesses the target file according to the local virtual address.

With reference to the first aspect, in a first possible implementation, the number of bits of the destination address is 2N bits, where N is the number of bits of the local virtual address, and the high N bits of the destination address include the first storage node The device information, the lower N bits of the destination address is the local virtual address, and the access request is forwarded by the network device to the first storage node according to the device information of the first storage node.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner, when the access request is a write request, the first storage node accesses the target file according to the local virtual address. ,include:

If the first storage node determines that there is a local physical space address corresponding to the local virtual address, the first storage node writes the data in the write request to the local physical space corresponding to the local physical space address; or

If the first storage node determines that there is no local physical space address corresponding to the local virtual address, the first storage node allocates a local physical space for the target file, and writes the data in the write request to the allocated local physical In space.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a third possible implementation, when the access request is a read request, the first storage node accesses the target file according to the local virtual address. ,include:

The first storage node determines a local physical space address corresponding to the local virtual address, and reads data in the local physical space corresponding to the local physical space address.

With reference to the first aspect or any one of the first to the third possible implementation manners of the first aspect, in a fourth possible implementation, the first storage node receives the network device forwarding Before the host's access request, the method also includes:

Receiving, by the first storage node, a space allocation request sent by the management node, where the space allocation request is used to allocate a local virtual address space for the target file, where the space allocation request carries a global virtual address of the first virtual space;

The first storage node allocates the local virtual address space to the target file according to the space allocation request;

The first storage node establishes a correspondence between the destination address and a local virtual address of the local virtual address space.

In conjunction with the fourth possible implementation of the first aspect, in a fifth possible implementation, the method further includes:

Receiving, by the first storage node, a space reclaiming message sent by the management node, where the space reclaiming message is used to reclaim the local virtual address space, where the space reclaiming message carries a global virtual address of the first virtual space;

The first storage node deletes the correspondence between the destination address and the local virtual address according to the space collection message, and recovers the local virtual address space;

The first storage node reclaims the local physical space corresponding to the local virtual address space.

In a second aspect, a distributed storage system is provided, including: a management node, a plurality of storage nodes, and a network device;

A local virtual address space of each of the plurality of storage nodes corresponds to a portion of a global virtual address space of the distributed storage system;

The management node is configured to allocate a first virtual space to the target file in the global virtual address space;

The network device is configured to forward an access request of the host to the first storage node, where the access request is used to access the first virtual space, and the destination address of the access request is a global virtual address of the first virtual space, the first storage node a storage node for storing the target file among the plurality of storage nodes;

The first storage node is configured to receive the access request by the network device, and determine the local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the first storage node, according to the The local virtual address accesses the target file.

With reference to the second aspect, in a first possible implementation manner, the number of bits of the destination address is 2N bits, N is the number of bits of the local virtual address, and the high N bits of the destination address include the device of the first storage node. Information, the lower N bits of the destination address are the local virtual address;

The network device is specifically configured to forward the access request to the first storage node according to the device information of the first storage node.

With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner, when the access request is a write request, the first storage node is specifically configured to determine the presence and the local The local physical space address corresponding to the virtual address, the data in the write request is written to the The local physical space corresponding to the local physical space address; or, if it is determined that there is no local physical space address corresponding to the local virtual address, allocate a local physical space for the target file, and write the data in the write request to the allocated In the local physical space.

With reference to the second aspect, or the first possible implementation manner of the second aspect, in a third possible implementation, when the access request is a read request, the first storage node is specifically configured to determine the local virtual address Corresponding local physical space address, reading data in the local physical space corresponding to the local physical space address.

With reference to the second aspect or any one of the first to the third possible implementation manners of the second aspect, in a fourth possible implementation, the first storage node is further configured to receive the management node a space allocation request for allocating a local virtual address space for the target file, the space allocation request carrying a global virtual address of the first virtual space;

Allocating the local virtual address space to the target file according to the space allocation request;

Establish a correspondence between the destination address and a local virtual address of the local virtual address space.

In conjunction with the fourth possible implementation of the second aspect, in a fifth possible implementation, the first storage node is further configured to receive a space collection message sent by the management node, where the space recovery message is used to recover the local a virtual address space, where the space reclaim message carries a global virtual address of the first virtual space;

Deleting the correspondence between the destination address and the local virtual address according to the space collection message, and recovering the local virtual address space;

Reclaims the local physical space corresponding to the local virtual address space.

In a third aspect, a storage node is provided, including:

a receiving module, configured to receive an access request of a host forwarded by a network device in the distributed storage system, where the access request is used to access a first virtual space of the target file, where the first virtual space is a management node in the distributed storage system a space allocated for the target file in the global virtual address space of the distributed storage system, where a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space, and the destination address of the access request a global virtual address for the first virtual space;

The processing module is configured to determine the local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the storage node, and access the target file according to the local virtual address.

In combination with the third aspect, in the first possible implementation, the number of bits of the destination address is 2N. Bit, N is the number of bits of the local virtual address, the high N bits of the destination address contain device information of the storage node, and the lower N bits of the destination address are the local virtual address, and the access request is determined by the network device according to the storage The device information of the node is forwarded to the storage node.

With reference to the third aspect or the first possible implementation manner of the third aspect, in a second possible implementation manner, when the access request is a write request, the processing module is specifically configured to:

If it is determined that the local physical space address corresponding to the local virtual address exists, the data in the write request is written to the local physical space corresponding to the local physical space address; or

If it is determined that there is no local physical space address corresponding to the local virtual address, the local physical space is allocated to the target file, and the data in the write request is written into the allocated local physical space.

With reference to the third aspect, or the first possible implementation manner of the third aspect, in a third possible implementation, when the access request is a read request, the processing module is specifically configured to determine, corresponding to the local virtual address. The local physical space address reads the data in the local physical space corresponding to the local physical space address.

With reference to the third aspect, or any one of the first to the third possible implementation manners of the third aspect, in a fourth possible implementation, the receiving module is further configured to receive the sending by the management node a space allocation request, the space allocation request is used to allocate a local virtual address space for the target file, where the space allocation request carries a global virtual address of the first virtual space;

The processing module is further configured to allocate the local virtual address space to the target file according to the space allocation request, and establish a correspondence between the destination address and a local virtual address of the local virtual address space.

In conjunction with the fourth possible implementation of the third aspect, in a fifth possible implementation, the receiving module is further configured to receive a space collection message sent by the management node, where the space recovery message is used to recover the local virtual address. a space, the space reclaim message carrying a global virtual address of the first virtual space;

The processing module is further configured to delete the correspondence between the destination address and the local virtual address according to the space collection message, recover the local virtual address space, and recover the local physical space corresponding to the local virtual address space.

In a fourth aspect, a computer program product is provided, comprising a computer readable storage medium storing program code, the program code comprising instructions for performing the method of the first aspect described above.

Based on the foregoing technical solution, the method for accessing a file, the distributed storage system, and the storage node in the embodiment of the present invention access the distributed storage system according to the local virtual address corresponding to the destination address by receiving an access request whose destination address is a global virtual address. The files in the file enable the host to directly access files in the distributed storage system, thereby improving the efficiency of file access.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings in the following description are only attached to some embodiments of the present invention. Figure.

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention.

FIG. 2 is a schematic flowchart of a method for accessing a file according to an embodiment of the present invention.

3 is a schematic block diagram of a distributed storage system in accordance with an embodiment of the present invention.

4 is a schematic block diagram of a storage node in accordance with an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of a storage node according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments.

It should be understood that the technical solutions of the embodiments of the present invention can be applied to various storage systems, especially distributed storage systems based on non-volatile storage media.

It should also be understood that in the embodiment of the present invention, the term "virtual address space" may also be expressed as "virtual space", that is, "virtual address space" and "virtual space" are common; similarly, "local physical storage space" "Common with "local physical space."

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention. In FIG. 1, host 130, storage node 120, and management node 110 are interconnected by network 140. Storage node 120, management node 110, and network 140 form a distributed storage system.

The host 130 can be a computer or other user device, and the host 130 can communicate with the distributed storage system through a network. Host 130 may include any computing device known in the art, such as a server, desktop computer, etc., with an operating system and other applications installed in host 130, and there may be multiple hosts 130. Host 130 can access files of the distributed storage system. E.g, The host 130 can provide a file access interface to an application on the host through the file interface module.

The storage node 120 is configured to store file data and process access requests to the file. Storage node 120 may include at least one type of memory, for example, a new type of NVM having the ability to address and access in bytes, such as PCM, ReRAM, MRAM, and the like. In the embodiment of the present invention, the file is stored in the NVM memory, and the storage node 120 can implement access to the new NVM through the memory bus.

The management node 110 is configured to manage metadata of the file, process metadata query, update request, and the like of the host to the file. The metadata of the file may include information such as the starting address of the file, the size of the allocated space, and the actual size of the file. The management node 110 can also be used to manage the global virtual address space of the distributed storage system.

The management node 110 and the storage node 120 may also be combined, that is, one node may have the functions of both the management node and the storage node.

In the embodiment of the present invention, the management node may be independently set (for example, the metadata management node 110 in FIG. 1), or may be a node having the functions of the metadata management node and the storage node at the same time. For convenience of description, the management node 110 will be described below as an example.

The physical layer and link layer of the network 140 may be based on Ethernet, a PCIE switching network, an InfiniBand network, and an optical network. The network 140 is used to route access requests and access responses. Specifically, the routes may be performed by a network device such as a router or a switch. For convenience of description, the following describes a network device as an example.

FIG. 2 shows a schematic flow diagram of a method 200 of accessing a file in accordance with an embodiment of the present invention.

The method 200 is applied to a distributed storage system including a management node, a plurality of storage nodes, and a network device. The local virtual address space of each storage node in the distributed storage system corresponds to a portion of the global virtual address space of the distributed storage system.

S210, the first storage node receives an access request of the host forwarded by the network device, where the access request is used to access a first virtual space of the target file, where the first virtual space is the management node in the global virtual address space. a space allocated by the file, the destination address of the access request is a global virtual address of the first virtual space, and the first storage node is a storage node of the plurality of storage nodes for storing the target file.

The global virtual address space of a distributed storage system is the space that is virtualized for managing files in the distributed storage system. The local virtual address space of the different storage nodes in the distributed storage system corresponds to different portions of the global virtual address space of the distributed storage system. For example, global virtual Mapping a first portion of the virtual address space to a local virtual address space of the first storage node in the distributed storage system, and mapping a second portion of the global virtual address space to a second storage node in the distributed storage system The local virtual address space, wherein the first part of the space is different from the second part of the space. In the embodiment of the present invention, the management node may allocate a continuous space (represented as the first virtual space) from the global virtual address space of the distributed storage system to the target file. Since the local virtual address space of each storage node in the distributed storage system corresponds to a portion of the global virtual address space of the distributed storage system, the first virtual space can be mapped to the local virtual address space of the storage node.

The local virtual address space may use an unused interval of the virtual address space of an operating system (OS) OS. For example, the 64-bit OS has a virtual address space size of 2 to the 64th power. In addition to the user-state virtual address space and the kernel virtual address space, there is also a reserved virtual address space. The local virtual address space can use the virtual address reserved by the OS. space. You can also re-divide the virtual address space of the operating system as needed, and divide a virtual address space into a local virtual address space.

The number of bits in the address of the local virtual address space (ie, the local virtual address) is the number of bits in the OS. For example, for a 64-bit OS, the number of bits in the local virtual address is 64 bits.

The address of the global virtual address space is the global virtual address. The number of bits in the global virtual address can be greater than the number of bits in the local virtual address. Optionally, the number of bits of the global virtual address may be twice the number of bits of the local virtual address, that is, if the number of bits of the local virtual address is N, the number of bits of the global virtual address is 2N. For example, for a 64-bit OS, the global virtual address can have 128 bits.

The global virtual address space of the distributed storage system can be mapped to the local virtual address space of the storage node by the correspondence between the global virtual address and the local virtual address. For example, in the case where the number of bits of the local virtual address is N and the number of bits of the global virtual address is 2N, the high N bits of the global virtual address may correspond to the device information of the node (eg, ID, or ID and node type), globally. The lower N bits of the virtual address may correspond to a local virtual address within the node, such that the global virtual address space may correspond to the local virtual address space of each storage node. It should be understood that only a portion of the high N bits may carry the device information of the storage node (eg, the highest 8 bits are the ID of the first storage node), and the remaining bits may be reserved or reserved for other purposes.

The correspondence between the global virtual address and the local virtual address may be established in advance, or may be established when space is allocated for the target file.

If a pre-uniform way is established, you can configure the locals of all storage nodes on the management node. Corresponding relationship between the local virtual address and the global virtual address (which may be referred to as a total mapping table), and the mapping between the local virtual address of the storage node and the global virtual address (which may be referred to as a sub-mapping table) is configured on each storage node. The total mapping table can also be configured on each storage node. In this manner, the local virtual address of all available local virtual address spaces of each storage node is already in the global virtual address and local virtual address correspondence. When the management node allocates the first virtual space to the target file, the global virtual address in the corresponding relationship is allocated. Based on the correspondence, the first virtual space is automatically mapped to the local virtual address space of the storage node, and further mapped through the local virtual address space. Go to the local physical storage space.

If the management node allocates the first virtual space to the target file, the global virtual address in the global virtual address space is allocated, and a space allocation request is sent to the storage node, and the allocated global virtual is carried. The storage node allocates a local virtual address space for the target file, and associates the local virtual address of the allocated local virtual address space with the global virtual address in the space allocation request, that is, establishes the global virtual address of the first virtual space and the allocated local address. The correspondence between the local virtual addresses of the virtual address space. The storage node may also return the local virtual address of the local virtual address space allocated for the target file to the management node, and the management node establishes a correspondence between the global virtual address of the first virtual space and the local virtual address on the management node.

When the host accesses the target file in the distributed storage system, the host sends an access request to the network device to access the first virtual space of the target file. The destination address of the access request is the global virtual address of the first virtual space. The network device forwards the access request to the first storage node.

Optionally, in an embodiment of the present invention, the number of bits of the destination address is 2N, where N is the number of bits of the local virtual address in the first storage node corresponding to the destination address, and the high N bit of the destination address includes the The device information of the first storage node, the lower N bits of the destination address is the local virtual address, and the access request is forwarded by the network device to the first storage node according to the device information of the first storage node.

That is, in this embodiment, the number of bits of the destination address of the access request is 2N, and the high N bits of the destination address of the access request include device information (eg, ID, or ID and node type) of the first storage node, The lower N bits of the destination address of the access request are the local virtual addresses in the first storage node for storing the local virtual address space of the target file. The network device may forward the access request of the host to the first storage node according to the device information of the first storage node.

S220. The first storage node determines a local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the first storage node.

After receiving the access request sent by the network device, the first storage node converts the destination address of the access request into a local virtual address in the first storage node.

In this embodiment, the correspondence between the destination address and the local virtual address may be in the form of a correspondence table, that is, the correspondence relationship table is used to record the destination address corresponding to the local virtual address, in this case, the first storage. The node determines a local virtual address corresponding to the destination address according to the correspondence relationship table; the correspondence between the destination address and the local virtual address may also be intangible, that is, pre-defining a part of the destination address (for example, a low of 2N bits) The N-bit is the local virtual address corresponding to the destination address. In this case, the first storage node directly determines the corresponding bit in the destination address as the local virtual address corresponding to the destination address.

S230. The first storage node accesses the target file according to the local virtual address.

The first storage node accesses the target file according to the local virtual address obtained in S220.

Optionally, in an embodiment of the present invention, when the access request is a read request, the first storage node determines a local physical space address corresponding to the local virtual address, and reads a local physical medium corresponding to the local physical space address. Data in space.

Specifically, when the access request is a read request, the first storage node queries the local physical space address corresponding to the local virtual address, and if there is no corresponding local physical space address, the access error is generated, so an access error is sent to the network device. Access response. If there is a corresponding local physical space address, the data in the local physical space corresponding to the local physical space address is read, and the data is carried in the access response and sent to the network device.

Optionally, in another embodiment of the present invention, when the access request is a write request, in a scenario, if the first storage node determines that there is a local physical space address corresponding to the local virtual address, The data in the write request is written to the local physical space corresponding to the local physical space address. In another case, if the first storage node determines that there is no local physical space address corresponding to the local virtual address, the first storage node allocates the local physical space for the target file, and writes the local physical space. The data in the request is written to the allocated local physical space.

Specifically, when the access request is a write request, the first storage node queries the local physical space address corresponding to the local virtual address, and if the corresponding local physical space address does not exist, the local physical space is not allocated to the target file. Allocating a local physical space for the target file, then writing the data in the write request to the allocated local physical space, and sending a successful access response to the network device; if there is a corresponding local physical space address, the request is written The data in the local data space is written in the local physical space corresponding to the local physical space address, and the access to the network device is successfully sent. should.

The method for accessing a file in the embodiment of the present invention, by receiving an access request whose destination address is a global virtual address, accessing a file in the distributed storage system according to a local virtual address corresponding to the destination address, enables the host to directly access the distributed storage. Files in the system, which can improve the efficiency of file access.

In the embodiment of the present invention, optionally, before the first storage node receives the access request of the host forwarded by the network device, the method 200 further includes:

Optionally, the space allocation request may carry a starting global virtual address and size of the first virtual space of the target file. When receiving the space allocation request, the first storage node may first determine whether the local virtual address space satisfies the size requirement, and if not, return an error response; if yes, allocate a local virtual address space for the target file, and establish a corresponding The corresponding relationship returns a successful response.

In the embodiment of the present invention, optionally, the method 200 further includes:

When the target file space is reclaimed, the management node sends a space reclamation message to the first storage node. The space reclamation message may carry the global virtual address of the first virtual space. The first storage node may first determine whether there is a local virtual address corresponding to the global virtual address, and if not, return an error response; if yes, delete the correspondence between the global virtual address and the local virtual address, and recycle the first The local virtual address space corresponding to the virtual space, and the local physical space corresponding to the local virtual address space is recovered, and a successful response is returned.

It is to be understood that the specific embodiments of the present invention are not intended to limit the scope of the embodiments of the invention.

It should also be understood that in various embodiments of the present invention, the size of the sequence numbers of the above processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be implemented by the present invention. The implementation of the examples constitutes any limitation.

A method of accessing a file according to an embodiment of the present invention is described in detail above, and a distributed storage system and a storage node according to an embodiment of the present invention will be described below.

FIG. 3 shows a schematic block diagram of a distributed storage system 300 in accordance with an embodiment of the present invention. As shown in FIG. 3, the distributed storage system 300 includes a management node 310, a plurality of storage nodes 320, and a network device 330.

The local virtual address space of each of the plurality of storage nodes 320 corresponds to a portion of the global virtual address space of the distributed storage system 300.

The management node 310 is configured to allocate a first virtual space for the target file in the global virtual address space.

The network device 330 is configured to forward an access request of the host to the first storage node, where the access request is used to access the first virtual space, and the destination address of the access request is a global virtual address of the first virtual space, the first storage The node is a storage node of the plurality of storage nodes 320 for storing the target file;

The first storage node is configured to receive the access request by the network device 330, and determine the local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the first storage node. The target file is accessed according to the local virtual address.

The distributed storage system of the embodiment of the present invention can directly access the distributed storage by accessing the file in the distributed storage system according to the local virtual address corresponding to the destination address by receiving the access request with the destination address being the global virtual address. Files in the system, which can improve the efficiency of file access.

In the embodiment of the present invention, optionally, the number of bits of the destination address is 2N, where N is the number of bits of the local virtual address, and the high N bits of the destination address include device information of the first storage node, and the destination address The lower N bits are the local virtual address;

The network device 330 is specifically configured to forward the access request to the first storage node according to the device information of the first storage node.

In the embodiment of the present invention, optionally, when the access request is a write request, in one case, If the first storage node determines that there is a local physical space address corresponding to the local virtual address, the data in the write request is written into the local physical space corresponding to the local physical space address. In another case, if the first storage node determines that there is no local physical space address corresponding to the local virtual address, the local physical space is allocated to the target file, and the data in the write request is written to the allocation. In the local physical space.

In the embodiment of the present invention, optionally, when the access request is a read request, the first storage node is specifically configured to determine a local physical space address corresponding to the local virtual address, and read the local physical space address. Data in the local physical space.

In the embodiment of the present invention, the first storage node is further configured to receive a space allocation request sent by the management node 310, where the space allocation request is used to allocate a local virtual address space to the target file, where the space allocation request is Carrying a global virtual address of the first virtual space;

In the embodiment of the present invention, the first storage node is further configured to receive a space collection message sent by the management node 310, where the space collection message is used to recover the local virtual address space, where the space recovery message carries the first a global virtual address of a virtual space;

The distributed storage system 300 according to the embodiment of the present invention may correspond to the distributed storage system in the foregoing method embodiments, and the first storage node may perform the corresponding processes of the foregoing various methods. For brevity, no further details are provided herein.

FIG. 4 shows a schematic block diagram of a storage node 400 in accordance with an embodiment of the present invention. As shown in FIG. 4, the storage node 400 includes:

The receiving module 410 is configured to receive an access request of a host forwarded by a network device in the distributed storage system, where the access request is used to access a first virtual space of the target file, where the first virtual space is a management in the distributed storage system. The space allocated by the node in the global virtual address space of the distributed storage system for the target file, where the local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space, and the purpose of the access request The address is a global virtual address of the first virtual space;

The processing module 420 is configured to: according to the destination address and a local virtual address in the storage node Corresponding relationship, determining the local virtual address corresponding to the destination address of the access request, and accessing the target file according to the local virtual address.

The storage node of the embodiment of the present invention can access the distributed storage system by directly accessing the file in the distributed storage system according to the local virtual address corresponding to the destination address by receiving the access request with the destination address being the global virtual address. Files, which can improve the efficiency of file access.

In the embodiment of the present invention, optionally, the number of bits of the destination address is 2N, where N is the number of bits of the local virtual address, and the high N bits of the destination address include device information of the storage node, and the destination address is low. The N bit is the local virtual address, and the access request is forwarded by the network device to the storage node according to the device information of the storage node.

In the embodiment of the present invention, optionally, when the access request is a write request, the processing module 420 is specifically configured to determine that a local physical space address corresponding to the local virtual address exists, and write the data in the write request to The local physical space address corresponds to the local physical space; or,

Determining that there is no local physical space address corresponding to the local virtual address, allocating the local physical space address, and writing the data in the write request to the local physical space corresponding to the local physical space address.

In the embodiment of the present invention, when the access request is a read request, the processing module 420 is specifically configured to determine a local physical space address corresponding to the local virtual address, and read the local corresponding to the local physical space address. Data in physical space.

In the embodiment of the present invention, the receiving module 410 is further configured to receive a space allocation request sent by the management node, where the space allocation request is used to allocate a local virtual address space for the target file, where the space allocation request carries the The global virtual address of the first virtual space;

The processing module 420 is further configured to allocate the local virtual address space to the target file according to the space allocation request, and establish a correspondence between the destination address and a local virtual address of the local virtual address space.

In the embodiment of the present invention, the receiving module 410 is further configured to receive a space collection message sent by the management node, where the space recovery message is used to recover the local virtual address space, where the space recovery message carries the first virtual The global virtual address of the space;

The processing module 420 is further configured to delete the correspondence between the destination address and the local virtual address according to the space collection message, recover the local virtual address space, and recover the local physical space corresponding to the local virtual address space.

The storage node 400 according to an embodiment of the present invention may correspond to a first storage node in the method 200 of accessing a file according to an embodiment of the present invention, and the above and other operations and/or functions of the respective modules in the storage node 400 are respectively The corresponding processes of the various methods are not described here for brevity.

FIG. 5 illustrates a structure of a storage node provided by still another embodiment of the present invention, including at least one processor 502 (eg, a CPU), at least one network interface 505 or other communication interface, memory 506, and at least one communication bus 503. among them:

Communication bus 503 is used to implement connection communication between these devices.

The processor 502 is configured to execute executable modules, such as computer programs, stored in the memory 506.

The memory 506 may include a high speed random access memory (RAM: Random Access Memory) and may also include an NVM, such as at least one disk storage.

The storage node implements a communication connection with the network through at least one network interface 505, which may be wired or wireless.

In some embodiments, the memory 506 stores a program 5061 that is used to execute the program 5061 to implement the various methods in the foregoing method embodiments.

It should be understood that in the embodiment of the present invention, the term "and/or" is merely an association relationship describing an associated object, indicating that there may be three relationships. For example, A and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative For example, the division of the unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units or components may be combined or may be integrated into another system, or some features may be Ignore, or not execute. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. . The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto.

Claims

A method for accessing a file, the method being applied to a distributed storage system, the distributed storage system comprising a management node, a plurality of storage nodes, and a network device, each of the storage devices in the distributed storage system The local virtual address space of the node corresponds to a part of the global virtual address space of the distributed storage system, and the method includes:

Receiving, by the first storage node, an access request of the host forwarded by the network device, where the access request is used to access a first virtual space of the target file, where the first virtual space is the management node in the global virtual address space a space allocated for the target file, a destination address of the access request is a global virtual address of the first virtual space, and the first storage node is used to store the target file in the plurality of storage nodes Storage node

Determining, by the first storage node, the local virtual address corresponding to the destination address of the access request according to the correspondence between the destination address and the local virtual address in the first storage node;

The first storage node accesses the target file according to the local virtual address.
The method according to claim 1, wherein the number of bits of the destination address is 2N bits, wherein N is a number of bits of the local virtual address, and a high N bit of the destination address includes the first The device information of the storage node, where the lower N bits of the destination address are the local virtual address, and the access request is forwarded by the network device to the first storage node according to the device information of the first storage node.
The method according to claim 1 or 2, wherein, when the access request is a write request, the first storage node accesses the target file according to the local virtual address, including:

If the first storage node determines that there is a local physical space address corresponding to the local virtual address, the first storage node writes data in the write request to a local physical space corresponding to the local physical space address Medium; or,

If the first storage node determines that there is no local physical space address corresponding to the local virtual address, the first storage node allocates a local physical space for the target file, and writes data in the write request to Allocated in the local physical space.
The method according to claim 1 or 2, wherein, when the access request is a read request, the first storage node accesses the target file according to the local virtual address, including:

The first storage node determines a local physical space address corresponding to the local virtual address, and reads data in a local physical space corresponding to the local physical space address.
The method according to any one of claims 1 to 4, wherein before the first storage node receives an access request of the host forwarded by the network device, the method further includes:

Receiving, by the first storage node, a space allocation request sent by the management node, where the space allocation request is used to allocate a local virtual address space for the target file, where the space allocation request carries a global virtual space of the first virtual space address;

The first storage node allocates the local virtual address space to the target file according to the space allocation request;

The first storage node establishes a correspondence between the destination address and a local virtual address of the local virtual address space.
The method of claim 5, wherein the method further comprises:

Receiving, by the first storage node, a space reclaiming message sent by the management node, where the space reclaiming message is used to reclaim the local virtual address space, where the space reclaiming message carries a global virtual address of the first virtual space;

The first storage node deletes the correspondence between the destination address and the local virtual address according to the space collection message, and recovers the local virtual address space;

The first storage node reclaims a local physical space corresponding to the local virtual address space.
A distributed storage system, comprising: a management node, a plurality of storage nodes, and a network device;

A local virtual address space of each of the plurality of storage nodes corresponds to a portion of a global virtual address space of the distributed storage system;

The management node is configured to allocate a first virtual space to the target file in the global virtual address space;

The network device is configured to forward an access request of the host to the first storage node, where the access request is used to access the first virtual space, and the destination address of the access request is a global virtual address of the first virtual space. The first storage node is a storage node of the plurality of storage nodes for storing the target file;

The first storage node is configured to receive, by the network device, the access request, and determine the access according to a correspondence between the destination address and a local virtual address in the first storage node. The local virtual address corresponding to the requested destination address, and accessing the target file according to the local virtual address.
The distributed storage system according to claim 7, wherein the number of bits of the destination address is 2N bits, N is a number of bits of the local virtual address, and a high N bit of the destination address includes the number Device information of a storage node, where a lower N bits of the destination address is the local virtual address;

The network device is specifically configured to forward the access request to the first storage node according to the device information of the first storage node.
The distributed storage system according to claim 7 or 8, wherein when the access request is a write request, the first storage node is specifically configured to:

If it is determined that the local physical space address corresponding to the local virtual address exists, the data in the write request is written to the local physical space corresponding to the local physical space address; or

If it is determined that there is no local physical space address corresponding to the local virtual address, the local physical space is allocated to the target file, and the data in the write request is written into the allocated local physical space.
The distributed storage system according to claim 7 or 8, wherein the first storage node is specifically configured to determine a local physical space address corresponding to the local virtual address when the access request is a read request. Reading data in the local physical space corresponding to the local physical space address.
The distributed storage system according to any one of claims 7 to 10, wherein the first storage node is further configured to receive a space allocation request sent by the management node, where the space allocation request is used for The target file allocates a local virtual address space, and the space allocation request carries a global virtual address of the first virtual space;

Allocating the local virtual address space to the target file according to the space allocation request;

Establishing a correspondence between the destination address and a local virtual address of the local virtual address space.
The distributed storage system according to claim 11, wherein the first storage node is further configured to receive a space reclamation message sent by the management node, where the space reclamation message is used to recover the local virtual address space. The space reclamation message carries a global virtual address of the first virtual space;

Deleting the correspondence between the destination address and the local virtual address according to the space collection message Relationship, recycling the local virtual address space;

Reclaiming the local physical space corresponding to the local virtual address space.
A storage node, comprising:

a receiving module, configured to receive an access request of a host forwarded by a network device in the distributed storage system, where the access request is used to access a first virtual space of the target file, where the first virtual space is in the distributed storage system The management node allocates space for the target file in the global virtual address space of the distributed storage system, and the local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space The destination address of the access request is a global virtual address of the first virtual space;

a processing module, configured to determine, according to the correspondence between the destination address and a local virtual address in the storage node, the local virtual address corresponding to the destination address of the access request, and access the target according to the local virtual address file.
The storage node according to claim 13, wherein the number of bits of the destination address is 2N bits, N is the number of bits of the local virtual address, and the upper N bits of the destination address comprise the storage node. Device information, where the lower N bits of the destination address are the local virtual address, and the access request is forwarded by the network device to the storage node according to the device information of the storage node.
The storage node according to claim 13 or 14, wherein when the access request is a write request, the processing module is specifically configured to:

If it is determined that the local physical space address corresponding to the local virtual address exists, the data in the write request is written to the local physical space corresponding to the local physical space address; or

If it is determined that there is no local physical space address corresponding to the local virtual address, the local physical space is allocated to the target file, and the data in the write request is written into the allocated local physical space.
The storage node according to claim 13 or 14, wherein when the access request is a read request, the processing module is specifically configured to determine a local physical space address corresponding to the local virtual address, and read the location The data in the local physical space corresponding to the local physical space address.
The storage node according to any one of claims 13 to 16, wherein the receiving module is further configured to receive a space allocation request sent by the management node, where the space allocation request is used for the target file Allocating a local virtual address space, the space allocation request carrying the The global virtual address of the first virtual space;

The processing module is further configured to allocate the local virtual address space to the target file according to the space allocation request, and establish a correspondence between the destination address and a local virtual address of the local virtual address space.
The storage node according to claim 17, wherein the receiving module is further configured to receive a space reclamation message sent by the management node, where the space reclamation message is used to recover the local virtual address space, the space Retrieving a message carrying a global virtual address of the first virtual space;

The processing module is further configured to: delete the corresponding relationship between the destination address and the local virtual address according to the space collection message, reclaim the local virtual address space, and recover the local physical space corresponding to the local virtual address space.