Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in further detail below in conjunction with the accompanying drawings and the specific embodiments.
Core concept of the present invention is: based on the meta data server cluster metadata is managed, when storing metadata, utilize hash function to determine the address of storing metadata file, guarantee the load balancing of cluster; When carrying out metadata store, the corresponding wave file of storage in other servers is also made amendment to its copy when metadata is revised simultaneously, carries out the Coordination Treatment of fault when a server breaks down by a default server that is used to coordinate.So, can guarantee the load balancing of consistance, integrality and the cluster server of metadata under the concurrent visit.
Fig. 1 is the schematic flow sheet of the embodiment of the invention one, and as shown in Figure 1, this embodiment one mainly may further comprise the steps:
Step 101, distributed file system are set up M collection of metadata in advance, according to the first default hash function, adopt the mode of Hash hash that described first manifold is closed in each meta data server that disperses to be arranged in the metadata service cluster.
Described metadata (metadata), be meant the metadata information of file or catalogue, comprise the information such as name, size, modification time, access limit, the owner of file, for concrete file or catalogue, (inode) preserves these metadata informations by index node.Here, described meta data server is preserved a global set mapping table separately, described global set mapping table is used for the mapping relations of the metadata that each collection of metadata that the minute book server is provided with comprises with it, and all are not arranged on each collection of metadata and the mapping relations of meta data server under it in the book server.Particularly, the global set mapping table will be preserved the mapping relations of the global index number of file to affiliated collection of metadata, particularly, with this global set mapping table as Hash table, with the high H position of file global index number parameter, search the positional information that this global set mapping table can obtain collection of metadata under this file metadata as hash function.
Described collection of metadata (set), the aggregation units of forming by the metadata of many files, to preserve the state of set and the metadata mapping table of this set in the set, this metadata mapping table is used for writing down the global index number of each file of collection of metadata and the corresponding relation of its index node position, particularly, with this metadata mapping table is Hash table, with the low L position of file global index number parameter as hash function, search this metadata mapping table, can obtain the position of index node at the metadata place of this document, on the position of this index node with the metadata of in store file.
In this step,, can realize effectively that metadata set is combined in the equiblibrium mass distribution on each meta data server, help realizing the load balancing of cluster under the concurrent visit by the mode of Hash hash.
Step 102, when file is preserved in the meta data server S_1 request of client in the metadata service cluster, described meta data server S_1 is global index of described file allocation number, and according to this global index number, adopt the mode of Hash hash, determine the affiliated collection of metadata W of metadata of described file, and this metadata leaves the index node position D among the described collection of metadata W in, the global set mapping table that described meta data server S_1 preserves according to self is determined the meta data server S_i at described collection of metadata W place, triggers described meta data server S_i the metadata of described file is kept among the described meta data server S_i.
Here, described global index number is the unique identification of the metadata of each file in the distributed file system, is used for locating the set at this document metadata place and in the concrete preservation position of set.
Preferably, the described affiliated collection of metadata W of metadata that determines described file, and the index node position D that this metadata leaves among the described collection of metadata W is specifically as follows:
Described global index number is divided into high H position and low L position two parts, wherein, L+H=Z, described Z is the number of bits of default described global index number;
For the high H position of described global index number, carry out Hash calculation by the second default hash function, utilize resulting cryptographic hash to search described global set mapping table, obtain the positional information of described collection of metadata W; According to the positional information of described collection of metadata W, find described collection of metadata W;
For the low L position of described global index number, carry out Hash calculation by the 3rd default hash function, with resultant cryptographic hash as described index node position D.
Here, the setting of described H and L can be provided with according to actual needs by those skilled in the art, preferably, can be: H=16, L=16.
Preferably, the metadata of described file be kept among the described meta data server S_i be specifically as follows:
Meta data server S_i is the allocation index node on the D of index node position, and the metadata of described file is kept in the index node of the described index node position D in the book server;
The mapping relations of the global index of meta data server S_i described metadata of record in the described global set mapping table that self preserves number and the position of described collection of metadata W;
The mapping relations of the global index of meta data server S_i described metadata of record in the metadata mapping table of this collection of metadata W that self preserves number and the index node of described metadata.
Further, can also realize the backup of data,, particularly, can further comprise in the described step 101 with safety and the integrality of guaranteeing data by set up the mode of copy for each collection of metadata:
Described distributed file system is that a described M collection of metadata is set up the copy set respectively, and according to the principle of load balancing, the copy of a described M collection of metadata is gathered in each meta data server that is arranged in the described metadata service cluster, and it is inequality to guarantee that described collection of metadata and its copy are gathered the place meta data server, preserve the former copy mapping table of a set in described each meta data server respectively, the former copy mapping table of described set is used for preserving each collection of metadata and its copy that are arranged on book server and gathers the mapping relations of place meta data server, and each copy that is arranged in the book server is gathered each collection of metadata corresponding with it mapping relations of place meta data server originally.
After setting up that metadata set is fashionable and setting up copy set for it, operations such as the follow-up storage of carrying out file, visit the time, also need copy set to carry out synchronous operation, to guarantee the consistance of data.Particularly, when carrying out the preservation of file in the step 102, also need to carry out following steps:
Described meta data server S_i is according to the former copy mapping table of self preserving of set, determine the meta data server W_copy at the copy set place of described collection of metadata W, trigger described meta data server W_copy preserves described file in the copy set of described collection of metadata W metadata.
Here, the concrete grammar of preserving metadata in the copy set is identical with aforementioned metadata store method to its former set, the meta data server of promptly depositing copy set also needs to safeguard mapping table, a copy that a copy set and its comprise the global index number of the metadata index node address in gathering and the mapping table of file global index number, when carrying out the preservation of copy set, then need on these mapping tables, write down corresponding information.
Further, client can conduct interviews to file after being kept at file in the distributed file system as required, at this moment, also need the metadata of this document is carried out corresponding process operations, particularly, can utilize the visit of following method realization metadata:
When client need ask to visit the file F of appointment, described request is sent to arbitrary meta data server A in the described metadata service cluster, carry the ino of global index of described file F and the type of request visit in the described request;
Described meta data server A is according to the described ino of global index, determine the affiliated collection of metadata J of metadata of described file F, according to the described global set mapping table of self preserving, determine the meta data server S_J at described collection of metadata J place, if this meta data server S_J is a book server, then according to the type of described request visit, the metadata mapping table of the described collection of metadata J of the described ino of global index and self preservation, visit the metadata of described file F and carry out corresponding process operations, and according to the former copy mapping table of self preserving of described set, determine the meta data server at the copy set place of described collection of metadata J, the copy set that described processing operation is synchronized to the described collection of metadata J on this meta data server closes; Otherwise, the meta data server S_J at described collection of metadata J place is notified to described client, trigger described client and send described request to described meta data server S_J, described meta data server S_J is according to the type of described request visit, the metadata mapping table of the described collection of metadata J of the described ino of global index and self preservation, visit the metadata of described file F and carry out corresponding process operations, and according to the former copy mapping table of self preserving of described set, determine the meta data server at the copy set place of described collection of metadata J, the copy set that described processing operation is synchronized to the described collection of metadata J on this meta data server closes.
Here,, determine the concrete grammar of the collection of metadata J under the metadata of described file F according to the described ino of global index, ditto described.
According to the metadata mapping table of the type of described request visit, the described ino of global index and the described collection of metadata J that self preserves, the metadata of visiting described file F is specifically as follows:
Metadata mapping table with collection of metadata J is a Hash table, low L position with the described ino of global index is the parameter of described the 3rd hash function, carry out Hash calculation, utilize resulting cryptographic hash to search described metadata mapping table, obtain the position of index node of the metadata correspondence of described file F, can find the metadata of described file F based on this position, and can operate accordingly according to the type of described request visit.Concrete access type and the same existing system of corresponding process operations do not repeat them here.
Preferably, can preset a coordination server in the metadata service cluster of described distributed file system, this coordination server can be by sending heartbeat message termly, each meta data server in the described metadata service cluster is monitored, so that know out of order meta data server immediately, and then can in time carry out fault handling, guarantee the integrality that distributed file system is preserved metadata.Concrete grammar is as follows:
When described coordination server monitors arbitrary meta data server B when breaking down, described failure message is notified to other each meta data servers except that described meta data server B in the described metadata service cluster;
Receive each meta data server of described notice, the former copy mapping table of described set according to described notice and self preservation, judge the copy set of self whether having preserved the collection of metadata on the described meta data server B, if, then according to the principle of load balance, from other meta data servers that other do not break down, be identified for preserving the new meta data server of described copy set, trigger described new meta data server described copy set is preserved, and the set of described copy set cooperation for its correspondence is arranged in the book server originally.
Preferably, a described M collection of metadata with and copy set close also and can set up by above-mentioned coordination server.
Preferably, can be by a meta data server that adopts electoral machinery to elect in the meta data server in the metadata service cluster, as described coordination server.Described coordination server is a service routine, and itself does not preserve metadata information, and it accepts the meta data server registration, and passes through the status information of the dynamic generator data server of the log-on message cluster of meta data server.More preferably, on the angle of disposing, this coordination server can with the shared same hardware server of any one meta data server.
From technique scheme as can be seen, adopt the present invention can guarantee the load balancing of cluster under the consistance of metadata under the concurrent visit, the concurrent visit, when single metadata node breaks down, information synchronization problem when metadata integrity and metadata access, thereby can satisfy the demand of the concurrent visit of a large amount of clients, realize based on effective management of server cluster to metadata in the distributed file system.
In sum, more than be preferred embodiment of the present invention only, be not to be used to limit protection scope of the present invention.Within the spirit and principles in the present invention all, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.