CN101599994A - The method of transfer files data between distributed file system, access node and node - Google Patents
The method of transfer files data between distributed file system, access node and node Download PDFInfo
- Publication number
- CN101599994A CN101599994A CNA2009100858083A CN200910085808A CN101599994A CN 101599994 A CN101599994 A CN 101599994A CN A2009100858083 A CNA2009100858083 A CN A2009100858083A CN 200910085808 A CN200910085808 A CN 200910085808A CN 101599994 A CN101599994 A CN 101599994A
- Authority
- CN
- China
- Prior art keywords
- access node
- node
- file
- access
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides the method for transfer files data between a kind of distributed file system, access node and node, access node wherein, be applied to comprise the distributed file system of at least one service node, this access node comprises: first processing module, be used to receive user's file service request, judge locally when not having file data that the file service request asked, send first service node of a query requests at least one service node; Second processing module, be used to receive second access node tabulation that first service node returns, store the data block of the file data that the file service request asked in each second access node in second access node tabulation, and connect with each second access node, from second access node, obtain the file data that the file service request is asked, return to the user.The present invention has improved network transmission efficiency.
Description
Technical field
The present invention relates to a kind of distributed file system field, be meant the method for transfer files data between a kind of distributed file system, access node and node of cross-node store file data especially.
Background technology
As shown in Figure 1, the networking mode of the distributed file system of existing network operation at present, the general structure that adopts layering, for example: Centroid, the regional center node, fringe node etc., a Centroid is in charge of several regional center nodes, each regional center node is in charge of a plurality of fringe nodes, and the user generally inserts distributed file system by fringe node.
Because memory limitations, fringe node can't be preserved all files of the whole network.If can not find certain file that oneself needs on the fringe node of certain user under him, fringe node generally adopts dual mode to provide service for the user so:
First kind of mode is: user's tandem is had on the regional center node or Centroid of this document to upper level, provide service by even higher level of node;
The second way is: the fringe node under the user is responsible for having the node of this document to be transferred to this node from upper level this document, provides service for the user then.
If adopt above-mentioned first kind of mode, it is tandem has the node of this document to upper level mode, because file content can not be in time at the fringe node buffer memory, a large amount of users will be scheduled for regional center node or Centroid, can cause the Centroid overload overweight.
If adopt the above-mentioned second way, the file that is about to user's request is transferred to the mode of this fringe node from file source node (being regional center node or Centroid), and generic-document transmission all to adopt complete file be unit, this will produce following problem: if when file transfer will be finished, the file source node breaks down suddenly, just can cause this bust this, the data of having transmitted will be dropped, provide service if continue as the user, must seek new file source node, proceed file transfer, this makes between node the file transfer scheduling waste the network bandwidth and node processing ability, service second-rate.
In addition, in the existing distributed file system, though some system adopts: complete file is divided into some little data blocks stores transmission, reduced the transmission granularity to a certain extent, improve network bandwidth utilance, but the scheduled transmission method only is simply to carry out scheduled transmission based on the visit temperature of individual data piece, when not reaching heat degree threshold, data block can trigger data not dispatch, cause network transmission efficiency lower like this, the real-time of scheduling is relatively poor, can not well provide service for the user.
Summary of the invention
The technical problem to be solved in the present invention provides the method for transfer files data between a kind of distributed file system, access node and node, the file data transmission technical problem that real-time is relatively poor, network transmission efficiency is low between the Centroid overload that exists in the solution prior art, node.
For solving the problems of the technologies described above, the invention provides a kind of access node, be applied to comprise the distributed file system of at least one service node, comprising:
First processing module is used to receive user's file service request, judges locally when not having file data that described file service request asked, sends first service node of a query requests to described at least one service node;
Second processing module, be used to receive second access node tabulation that described first service node returns, store the data block of the file data that described file service request asked in each second access node in the tabulation of described second access node, and connect with described each second access node, from described second access node, obtain the file data that described file service request is asked, return to the user.
Wherein, described second processing module comprises:
Link block is used to receive second access node tabulation that described service node returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Acquisition module is used for from described each second access node, obtains the different pieces of information piece of the file data that described file service request asked respectively, and described different pieces of information piece is merged into the complete file data, returns to the user.
Wherein, above-mentioned access node also comprises:
Memory module is used for the data block of store file data, described file data and the tabulation of described at least one service node;
Described first processing module comprises:
Receiver module is used to receive user's file service request;
First processing sub is used for directly described file data being returned to the user when described memory module has the file data that described file service request asked;
Second processing sub is used for when described memory module does not have file data that described file service request asked, sends first service node of a query requests to the tabulation of described at least one service node.
Wherein, above-mentioned access node also comprises:
Switching Module is used for and two described access nodes at least, and other access node except that local access node exchanges the described file data in the described memory module and the tabulation of described at least one service node.
Wherein, above-mentioned access node also comprises:
Handover module is used for from the tabulation of described at least one service node, reselecting a service node, as the first new service node when described first service node is unavailable;
Described second processing sub sends described query requests to the described first new service node.
Wherein, above-mentioned access node also comprises:
First maintenance module, be used in described second processing module when described second access node obtains data block and finishes, calculate the Hash of described data block, if the Hash of described data block is different with the Hash that described second access node sends, then obtain described data block again, identical up to the Hash of described data block with the Hash that described second access node sends.
Wherein, above-mentioned access node also comprises:
When second maintenance module, the visit temperature that is used for the file data that obtains from described second access node in described second processing module are lower than a default heat degree threshold, described file data is carried out burin-in process.
Wherein, above-mentioned access node also comprises:
The third dimension is protected module, be used to safeguard that is downloaded a formation, when local access node obtains data block, adopt the principle of first-in first-out at a plurality of other access nodes, the data block of the file data that described other access node is asked returns to described other access node.
Wherein, above-mentioned access node also comprises:
Fourth dimension protects module, the same data block that is used for the file data of being asked in the described file service request of local access node, in the time of in the download formation of the second different access nodes, if described data block is downloaded from one of them second access node and is finished, then when described query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
The present invention also provides a kind of distributed file system, comprising: at least two access nodes and at least one service node;
First access node in described at least two access nodes comprises:
First processing module is used to receive user's file service request, judges locally when not having file data that described file service request asked, sends first service node of a query requests to described at least one service node;
Described first service node comprises:
Receiver module is used to receive the query requests that described first processing module sends;
Select module, be used for according to described query requests, from described at least two access nodes, obtain at least one second access node, form second access node tabulation, described second access node stores the data block of the file data that described file service request asked;
Sending module is used for described second access node tabulation is returned to described first access node;
Described first access node also comprises:
Second processing module is used for connecting with each second access node of described second access node tabulation, obtains the file data that described file service request is asked from described second access node, returns to the user.
Wherein, described second access node is during at least two, and described second processing module comprises:
Link block is used to receive second access node tabulation that described sending module returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Acquisition module is used for from described each second access node, obtains the different pieces of information piece of the file data that described file service request asked respectively, and described different pieces of information piece is merged into the complete file data, returns to the user.
The present invention also provides a kind of service node, is applied to include the distributed file system of at least two access nodes, comprising:
Receiver module is used for receiving the query requests that first access node of described at least two access nodes sends;
Acquisition module, be used for according to described query requests, from described at least two access nodes, obtain at least one second access node, form second access node tabulation, described second access node stores the data block of the file data that described file service request asked;
Sending module is used for described second access node tabulation is returned to described first access node.
Embodiments of the invention also provide the method for transfer files data between a kind of node, are applied to comprise the distributed file system of at least two access nodes and at least one service node, and described method comprises:
First access node in described at least two access nodes receives user's file service request, judge locally when not having file data that described file service request asked, send first service node of a query requests to described at least one service node;
Described first service node is according to described query requests, from described at least two access nodes, obtain at least one second access node, form the tabulation of second access node, and the tabulation of described second access node returned to described first access node, described second access node stores the data block of the file data that described file service request asked;
Each second access node in described first access node and the tabulation of described second access node connects, and obtains the file data that described file service request is asked from described second access node, returns to the user.
Wherein, described second access node is during at least two, each second access node in described first access node and the tabulation of described second access node connects, obtain the file data that described file service request is asked from described second access node, the step that returns to the user specifically comprises:
Described first access node receives second access node tabulation that described service node returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Described first access node obtains the different pieces of information piece of the file data that described file service request asked respectively from described each second access node, and described different pieces of information piece is merged into the complete file data, returns to the user.
Wherein, when described first access node obtains data block and finishes from described second access node, also comprise:
Calculate the Hash of described data block,, then obtain described data block again if the Hash of described data block is different with the Hash that described second access node sends, identical up to the Hash of described data block with the Hash that described second access node sends.
The beneficial effect of technique scheme of the present invention is as follows:
In the such scheme, first access node is being searched this locality when not having file data that the user asks, send a query requests to service node, after this service node receives this query requests, can be with in this distributed file system, the tabulation of second access node of a plurality of data blocks that store the file data that this document service request asked, return to this first access node, this first access node can be set up parallel the connection with these second access nodes, obtain the different pieces of information piece of the file data that this document service request asked simultaneously from these second access nodes, parallel transmission between a plurality of nodes of realization data block, improve network transmission efficiency, improve the real-time of data dispatch.
Description of drawings
Fig. 1 is the networking structure schematic diagram of existing distributed file system;
Fig. 2 is the networking structure schematic diagram of embodiments of the invention distributed file system;
Fig. 3 is the concrete networking structure schematic diagram of distributed file system shown in Figure 2;
Fig. 4 is in the distributed file system shown in Figure 3, a concrete structure schematic diagram of first access node;
Fig. 5 is another concrete structure schematic diagram of first access node;
Fig. 6 is the another concrete structure schematic diagram of first access node;
Fig. 7 is a concrete structure schematic diagram again of first access node;
Fig. 8 is a concrete structure schematic diagram again of first access node;
Fig. 9 is a concrete structure schematic diagram again of first access node;
Figure 10 is the concrete structure schematic diagram that first access node has the download formation maintenance function of second access node;
Figure 11 is a concrete structure schematic diagram again of first access node;
Figure 12 is in the above-mentioned distributed file system, a concrete file transfer schematic diagram;
Figure 13 is the method flow schematic diagram of transfer files data between node of the present invention.
Embodiment
For making the technical problem to be solved in the present invention, technical scheme and advantage clearer, be described in detail below in conjunction with the accompanying drawings and the specific embodiments.
The present invention is directed in the existing distributed file system, the network transmission efficiency of data is lower between each node, the real-time of scheduling is relatively poor, can not well provide service problem for the user, provide the network transmission efficiency of data between a kind of each node higher, the real-time of scheduling is the method for transfer files data between distributed file system, access node and node preferably.
As Fig. 2, shown in Figure 3, the embodiments of the invention distributed file system comprises: at least two access nodes 30 and at least one service node 40, not communication between these service nodes 40, the correctness of transfer of data between node in the assurance system;
Wherein, for convenience of description, will at least two in the access nodes, when this locality did not have file data that user's file service request asked, the access node of initiating request to service node was called first access node 31; Will at least two in the access nodes, store the data block of the file data that user's file service request asked, and, be called second access node 32 for this first access node 31 provides the access node of file data service;
In the actual distributed file system, this first access node 31 and this second access node 32 are same user's access nodes, and this user's access node is a subscription access server, and the user obtains the network file service by this access node; In this system, any access node all might be initiated request to service node 40, and any access node also all might provide the file data service for other access node in the system.
As shown in Figure 3, first access node 31 in above-mentioned at least two access nodes can specifically comprise:
This first service node 40 is servers, uses an internal database to preserve the metadata information of access node and file data; Service node is not preserved file data, and it is the center index of file location information, and concentrated index service is provided;
This first service node 40 can specifically comprise:
Sending module 403 is used for described second access node tabulation is returned to this first access node 31; Specifically, this sending module 403 is specially: the address sending module is used for the address of this each second access node of second access node tabulation is returned to this first access node 31; If that storage is the title or the ID of at least one second access node 32 in certainly above-mentioned second access node tabulation, then this sending module 403 is that the title or the ID of second access node 32 during this second access node is tabulated returns to first access node when first access node 31 returns the tabulation of second access node;
This first access node 31 also can comprise:
As shown in Figure 4, in this distributed file system, second access node is during at least 2, and this second processing module 312 can specifically comprise:
Among this embodiment, first access node 31 is being searched this locality when not having file data that the user asks, to sending a query requests for service node 40, this service node 40 can be with in this distributed file system according to this query requests, the tabulation of second access node 32 of a plurality of data blocks that store the file data that this document service request asked, return to this first access node 31, this first access node 31 can be set up parallel the connection with these second access nodes 32, obtain the different pieces of information piece of the file data that this document service request asked simultaneously from these second access nodes 32, parallel transmission between a plurality of nodes of realization data block, improve network transmission efficiency, improve the real-time of data dispatch.
As shown in Figure 5, in this distributed file system, this first access node 31 also can comprise:
This document data can be divided into data block, form storage with data block, when this first access node 31 other access node in for system provides the file data service, can data block be that granularity is transmitted, conveniently provide the file data service for a plurality of other access nodes simultaneously; Certainly the file data of storage can be for a plurality of in this memory module 313, and also should store the listed files of these file datas this moment in this memory module 313, so that service node is searched the needed file data of other access node on this access node;
Above-mentioned first processing module 311 comprises:
As shown in Figure 6, in above-mentioned distributed file system, this first access node 31 also can comprise:
Switching Module 3 14, be used at least two access nodes with this distributed file system, (this other access node can be above-mentioned second access node 32 to other access node except that this first access node, also can not be above-mentioned second access node 32), exchange the file data that above-mentioned memory module 313 stored and the tabulation of at least one service node.Like this, in this distributed file system, allow to exchange about the tabulation of service node and the information of other access nodes and file data, the load that can alleviate service node like this between access node.
As shown in Figure 7, in the above-mentioned distributed file system, this first access node 31 also can comprise:
That is to say, when above-mentioned first access node 31 starts, be connected to one of them user's service node according to the address in the tabulation of the service node of storage in the memory module 313 and obtain the network service, this connection will keep always, this first access node 31 can not be connected to a plurality of servers simultaneously, when having only current service node (i.e. this first service node 40) unavailable, just can switch to other service nodes in the tabulation of service node, guarantee the reliability of file data transmission like this.
As shown in Figure 8, in the above-mentioned distributed file system, this first access node 31 also can comprise:
As the data block of the damage that will download (for example: data block B), be divided into again plurality of small blocks (for example: b1, b2, b3...bn), then:
1. download b1 again, cover original b1, after the covering, data block B is carried out the Hash verification,, then repair successfully, otherwise carry out next step if verification is errorless with the b1 that downloads;
2. download b2 again, cover original b2, after the covering, data block B is carried out the Hash verification,, then repair successfully, otherwise continue to carry out if verification is errorless with the b2 that downloads;
3. up to data block B verification succeeds, the data block of the damage of soon having downloaded is repaired.
As shown in Figure 9, in the above-mentioned distributed file system, this first access node 31 also can comprise:
As shown in figure 10, in above-mentioned distributed file system, above-mentioned second access node 32 can comprise:
The third dimension is protected module 318, be used to safeguard that is downloaded a formation, at a plurality of first access nodes 31 when this second access node 32 obtains data block, adopt the principle of first-in first-out, the data block of the file data that described other first access node is asked returns to described other first access node; Specifically, at a plurality of first access nodes 31 when this second access node 32 obtains data block, the bottom that other first access node is afterwards added this download formation, when other first access node arrived the top of this download formation, the data block of the file data that other first access node is asked returned to this other first access node again.
Same, among the distributed file system embodiment of the present invention, first access node 31 can provide the file data service for other access node too, promptly this first access node 31 has the above-mentioned third dimension too and protects module 318, for its data block of shared file data safeguards that is downloaded a formation, the access node of downloading adds the bottom of this formation, advancing gradually then, and the top that reaches formation up to this access node begins file in download; When the access node of downloading has arrived the top of download formation of this access node 31, this first access node 31 set up one with being connected of the access node of downloading, the data block that is used for that it is had sends user's access node of needs download to.
As shown in figure 11, in the above-mentioned distributed file system, this first access node 31 also can comprise:
Fourth dimension protects module 319, the same data block that is used for the file data of being asked in the file service request of first access node 31 is when the download formation of the second different access nodes 32, if this data block is downloaded from one of them second access node 32 and is finished, then when this query requests arrived the top of download formation of other second access node 32, refusal was carried out the download of this data block.
That is to say, user's access node (as first access node 31) may be downloaded same file data simultaneously from a plurality of other user's access nodes (as second access node 32), obtain the different pieces of information piece of file from different user's access node (as second access node 32), the different part of file data just can be downloaded simultaneously like this;
User's access node (as first access node 31) also may be in a plurality of other download waiting lists of user's access node (as second access node 32), other user's access node at each (as second access node 32) is gone up and is downloaded same file data blocks, in fact having finished this data block when user's access node (as first access node 31) of this wait downloads, it can not notify those other user's access node (as second access node 32) to delete its download request, and only is that refusal is carried out and downloaded when being discharged to the formation top in the download formation of those other user's access node (as second access node 32) at its place; Certainly this user's access node also can be notified other its download request of user's access node deletion.
Below in conjunction with Figure 12, describe in the above-mentioned distributed file system browsing process of file data between node in detail:
When (1) user's access node (being first access node 31 in the foregoing description) starts, from the tabulation of storage service node within it, select a service node (being above-mentioned first service node 40), connect;
(2) when the user needs access file, user's access node is at first searched this document data in local memory module, if find, then directly provide file data to the user, otherwise user's access node is initiated query requests to the service node that connects;
(3) user's service node is the tabulation of user's access node at required file data place, i.e. file source node listing (being above-mentioned second access node tabulation), and for example node A, B issue the requesting users access node;
(4) user's access node while all nodes initiation connection requests in the file source node listing, promptly initiate connection request to file source access node A and file source access node B simultaneously, exchange information such as listed files then about the tabulation and the storage of user's service node, and the request file in download.
(5) each file source node joins download formation bottom with the download request of other user's access nodes, waits for downloads, and just begins file in download when other user's access nodes reach the top of formation;
(6) user's access node is from a plurality of file source nodes, for example node A, B, download same file simultaneously, obtain the different pieces of information piece of file from different file source nodes, for example obtain data block a from file source access node A, B obtains data block b from the file source access node, and the different part of file just has been downloaded simultaneously like this;
(7) after user's access node is finished the download of data block, the Hash of calculated data piece, the blocks of files Hash that this Hash and file source node are sent compares, if it is different, damaged with regard to the supporting paper piece, user's access node covers the blocks of files of damaging (for example 128kb) with the download of block-by-block, till Hash calculation shows that blocks of files has been repaired;
(8) data block of downloading successfully or asking when file has been filled up the required buffering area of user, and user's access node just can provide file service to the user.
To sum up, among the above-mentioned distributed file system embodiment of the present invention, first access node 31 is being searched this locality when not having file data that the user asks, send a query requests to service node 40, this service node 40 can be with in this distributed file system according to this query requests, the tabulation of second access node 32 of a plurality of data blocks that store the file data that this document service request asked, return to this first access node 31, this first access node 31 can be set up parallel the connection with these second access nodes 32, obtain the different pieces of information piece of the file data that this document service request asked simultaneously from these second access nodes 32, overcome the Centroid overload that prior art exists, the file transfer scheduling waste network bandwidth and disposal ability and problem such as network transmission efficiency is lower and the real-time of scheduling is relatively poor between node, parallel transmission between a plurality of nodes of realization data block, improve network transmission efficiency, improve the real-time of data dispatch.
Embodiments of the invention also provide a kind of access node, this access node is applied to comprise the distributed file system of at least one service node, this access node is a subscription access server, become data block to store file division, the user obtains the network file service by user's access node, the concrete structure of this access node such as above-mentioned Fig. 3 are to first access node 31 shown in Figure 10, this access node is identical with first access node 31 in the above-mentioned distributed file system, above-mentioned description all about first access node 31 is equally applicable to this access node, does not repeat them here.
Embodiments of the invention also provide a kind of service node, be applied to include the distributed file system of at least two access nodes, this service node is a client server, use an internal database to preserve the metadata information of user's access node and file, it is the center index of file location information, concentrated index service is provided, does not have communication between the different servers.As shown in Figure 3, this service node 40 comprises:
Sending module 403, be used for this second access node tabulation is returned to first access node, specifically, this sending module is specially: the address sending module is used for the address of this each second access node of second access node tabulation is returned to this first access node.Like this, first access node just can be set up parallel the connection with each second access node, obtains different data blocks simultaneously from these second access nodes, parallel transmission between a plurality of nodes of realization data block, improve network transmission efficiency, improve the real-time of data dispatch.
As shown in figure 13, embodiments of the invention also provide the method for transfer files data between a kind of node, are applied to comprise the distributed file system of at least two access nodes and at least one service node, and this method comprises:
Wherein, above-mentioned steps 133 can specifically comprise:
First access node receives second access node tabulation that described service node returns, and sets up parallel the connection with each second access node in described second access node tabulation;
First access node obtains the different pieces of information piece of the file data that described file service request asked respectively from described each second access node, and described different pieces of information piece is merged into the complete file data, returns to the user.
When first access node obtains data block and finishes from described second access node, also comprise:
Calculate the Hash of described data block, if the Hash of described data block is different with the Hash that described second access node sends, then notify described second access node to retransmit described data block, identical up to the Hash of described data block with the Hash that described second access node sends.
After the above-mentioned steps 133, when the visit temperature of complete file data is lower than a default heat degree threshold, described complete file data are carried out burin-in process.
In addition, in the above-mentioned steps 133, when first access node obtains data block from second access node, also comprise:
Second access node safeguards that is downloaded a formation, when a plurality of first access nodes and second access node obtain data block, and the principle of employing first-in first-out, the data block of the file data that other first access node is asked returns to other first access node; Specifically, the bottom that other first access node is afterwards added this download formation, when other first access node arrived the top of this download formation, the data block of the file data that other first access node is asked returned to other first access node again.
In the above-mentioned steps 133, also can comprise: the same data block of the file data that the file service request of first access node is asked, in the time of in the download formation of the second different access nodes, if this data block is downloaded from one of them second access node and is finished, then when this query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
This method embodiment, first access node is being searched this locality when not having file data that the user asks, send a query requests to service node, this service node can be with in this distributed file system according to this query requests, the tabulation of second access node of a plurality of data blocks that store the file data that this document service request asked, return to this first access node, this first access node can be set up parallel the connection with these second access nodes, obtain the different pieces of information piece of the file data that this document service request asked simultaneously from these second access nodes, overcome the Centroid overload that prior art exists, the file transfer scheduling waste network bandwidth and disposal ability and problem such as network transmission efficiency is lower and the real-time of scheduling is relatively poor between node, parallel transmission between a plurality of nodes of realization data block, improve network transmission efficiency, improve the real-time of data dispatch.
The above is a preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from principle of the present invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (15)
1. access node is applied to comprise it is characterized in that the distributed file system of at least one service node, comprising:
First processing module is used to receive user's file service request, judges locally when not having file data that described file service request asked, sends first service node of a query requests to described at least one service node;
Second processing module, be used to receive second access node tabulation that described first service node returns, store the data block of the file data that described file service request asked in each second access node in the tabulation of described second access node, and connect with described each second access node, from described second access node, obtain the file data that described file service request is asked, return to the user.
2. access node according to claim 1 is characterized in that, described second processing module comprises:
Link block is used to receive second access node tabulation that described service node returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Acquisition module is used for from described each second access node, obtains the different pieces of information piece of the file data that described file service request asked respectively, and described different pieces of information piece is merged into the complete file data, returns to the user.
3. access node according to claim 1 is characterized in that, also comprises:
Memory module is used for the data block of store file data, described file data and the tabulation of described at least one service node;
Described first processing module comprises:
Receiver module is used to receive user's file service request;
First processing sub is used for directly described file data being returned to the user when described memory module has the file data that described file service request asked;
Second processing sub is used for when described memory module does not have file data that described file service request asked, sends first service node of a query requests to the tabulation of described at least one service node.
4. access node according to claim 3 is characterized in that, also comprises:
Switching Module is used for and two described access nodes at least, and other access node except that local access node exchanges the described file data in the described memory module and the tabulation of described at least one service node.
5. access node according to claim 3 is characterized in that, also comprises:
Handover module is used for from the tabulation of described at least one service node, reselecting a service node, as the first new service node when described first service node is unavailable;
Described second processing sub sends described query requests to the described first new service node.
6. according to each described access node of claim 1-5, it is characterized in that, also comprise:
First maintenance module, be used in described second processing module when described second access node obtains data block and finishes, calculate the Hash of described data block, if the Hash of described data block is different with the Hash that described second access node sends, then obtain described data block again, identical up to the Hash of described data block with the Hash that described second access node sends.
7. according to each described access node of claim 1-5, it is characterized in that, also comprise:
When second maintenance module, the visit temperature that is used for the file data that obtains from described second access node in described second processing module are lower than a default heat degree threshold, described file data is carried out burin-in process.
8. according to each described access node of claim 1-5, it is characterized in that, also comprise:
The third dimension is protected module, be used to safeguard that is downloaded a formation, when local access node obtains data block, adopt the principle of first-in first-out at a plurality of other access nodes, the data block of the file data that described other access node is asked returns to described other access node.
9. according to each described access node of claim 1-5, it is characterized in that, also comprise:
Fourth dimension protects module, the same data block that is used for the file data of being asked in the described file service request of local access node, in the time of in the download formation of the second different access nodes, if described data block is downloaded from one of them second access node and is finished, then when described query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
10. a distributed file system comprises: two access nodes and at least one service node at least; It is characterized in that,
First access node in described at least two access nodes comprises:
First processing module is used to receive user's file service request, judges locally when not having file data that described file service request asked, sends first service node of a query requests to described at least one service node;
Described first service node comprises:
Receiver module is used to receive the query requests that described first processing module sends;
Select module, be used for according to described query requests, from described at least two access nodes, obtain at least one second access node, form second access node tabulation, described second access node stores the data block of the file data that described file service request asked;
Sending module is used for described second access node tabulation is returned to described first access node;
Described first access node also comprises:
Second processing module is used for connecting with each second access node of described second access node tabulation, obtains the file data that described file service request is asked from described second access node, returns to the user.
11. distributed file system according to claim 10 is characterized in that, described second access node is during at least two, and described second processing module comprises:
Link block is used to receive second access node tabulation that described sending module returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Acquisition module is used for from described each second access node, obtains the different pieces of information piece of the file data that described file service request asked respectively, and described different pieces of information piece is merged into the complete file data, returns to the user.
12. a service node is applied to include the distributed file system of at least two access nodes, it is characterized in that, comprising:
Receiver module is used for receiving the query requests that first access node of described at least two access nodes sends;
Acquisition module, be used for according to described query requests, from described at least two access nodes, obtain at least one second access node, form second access node tabulation, described second access node stores the data block of the file data that described file service request asked;
Sending module is used for described second access node tabulation is returned to described first access node.
13. the method for transfer files data between a node is applied to comprise the distributed file system of at least two access nodes and at least one service node it is characterized in that described method comprises:
First access node in described at least two access nodes receives user's file service request, judge locally when not having file data that described file service request asked, send first service node of a query requests to described at least one service node;
Described first service node is according to described query requests, from described at least two access nodes, obtain at least one second access node, form the tabulation of second access node, and the tabulation of described second access node returned to described first access node, described second access node stores the data block of the file data that described file service request asked;
Each second access node in described first access node and the tabulation of described second access node connects, and obtains the file data that described file service request is asked from described second access node, returns to the user.
14. method according to claim 13, it is characterized in that, described second access node is during at least two, each second access node in described first access node and the tabulation of described second access node connects, obtain the file data that described file service request is asked from described second access node, the step that returns to the user specifically comprises:
Described first access node receives second access node tabulation that described service node returns, and sets up parallel the connection with each second access node in described second access node tabulation;
Described first access node obtains the different pieces of information piece of the file data that described file service request asked respectively from described each second access node, and described different pieces of information piece is merged into the complete file data, returns to the user.
15. according to claim 13 or 14 described methods, it is characterized in that, when described first access node obtains data block and finishes from described second access node, also comprise:
Calculate the Hash of described data block,, then obtain described data block again if the Hash of described data block is different with the Hash that described second access node sends, identical up to the Hash of described data block with the Hash that described second access node sends.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100858083A CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100858083A CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101599994A true CN101599994A (en) | 2009-12-09 |
| CN101599994B CN101599994B (en) | 2012-07-18 |
Family
ID=41421238
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009100858083A Expired - Fee Related CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101599994B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103200242A (en) * | 2013-03-20 | 2013-07-10 | 成都康赛电子科大信息技术有限责任公司 | Method for constructing cross-layer face data analysis hub based on internet of things |
| WO2014183708A1 (en) * | 2013-12-02 | 2014-11-20 | 中兴通讯股份有限公司 | Method and system for realizing block storage of distributed file system |
| CN104580268A (en) * | 2013-10-09 | 2015-04-29 | 南京中兴新软件有限责任公司 | Method and device for transmitting file data |
| CN106888261A (en) * | 2017-02-27 | 2017-06-23 | 深圳市先河系统技术有限公司 | A kind of distributed document transmission method, server and private clound equipment |
| CN107229692A (en) * | 2017-05-19 | 2017-10-03 | 哈工大大数据产业有限公司 | A kind of distributed multi-table connecting method and system based on streamline |
| CN110023944A (en) * | 2017-01-03 | 2019-07-16 | 华为技术有限公司 | A kind of data processing method and relevant device |
| CN111291000A (en) * | 2018-05-18 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Block chain-based file acquisition method, equipment and storage medium |
| CN111464649A (en) * | 2017-04-19 | 2020-07-28 | 贵州白山云科技股份有限公司 | Access request source returning method and device |
| CN112491939A (en) * | 2019-09-12 | 2021-03-12 | 上海哔哩哔哩科技有限公司 | Multimedia resource scheduling method and system |
| CN112749004A (en) * | 2019-10-30 | 2021-05-04 | 中国移动通信集团安徽有限公司 | Data storage method and device based on node access heat |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110554919A (en) * | 2019-08-19 | 2019-12-10 | 武汉世纪楚林科技有限公司 | Edge device data processing method and device, terminal device and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101426193B (en) * | 2007-11-01 | 2011-01-12 | 华为技术有限公司 | Network resource information management method, communication apparatus and communication system |
| CN101442475A (en) * | 2007-11-24 | 2009-05-27 | 华为技术有限公司 | Distributed business acting method, network system and network appliance |
| CN101404649B (en) * | 2008-11-11 | 2012-01-11 | 阿里巴巴集团控股有限公司 | Data processing system based on CACHE and its method |
-
2009
- 2009-06-01 CN CN2009100858083A patent/CN101599994B/en not_active Expired - Fee Related
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103200242B (en) * | 2013-03-20 | 2016-04-06 | 成都康赛信息技术有限公司 | The method of cross-layer face data analysis hinge is built based on Internet of Things |
| CN103200242A (en) * | 2013-03-20 | 2013-07-10 | 成都康赛电子科大信息技术有限责任公司 | Method for constructing cross-layer face data analysis hub based on internet of things |
| CN104580268A (en) * | 2013-10-09 | 2015-04-29 | 南京中兴新软件有限责任公司 | Method and device for transmitting file data |
| WO2014183708A1 (en) * | 2013-12-02 | 2014-11-20 | 中兴通讯股份有限公司 | Method and system for realizing block storage of distributed file system |
| CN104679665A (en) * | 2013-12-02 | 2015-06-03 | 中兴通讯股份有限公司 | Method and system for achieving block storage of distributed file system |
| CN110023944A (en) * | 2017-01-03 | 2019-07-16 | 华为技术有限公司 | A kind of data processing method and relevant device |
| CN112929411A (en) * | 2017-02-27 | 2021-06-08 | 深圳市先河系统技术有限公司 | Distributed file transmission method, server and private cloud equipment |
| CN106888261A (en) * | 2017-02-27 | 2017-06-23 | 深圳市先河系统技术有限公司 | A kind of distributed document transmission method, server and private clound equipment |
| CN111464649A (en) * | 2017-04-19 | 2020-07-28 | 贵州白山云科技股份有限公司 | Access request source returning method and device |
| CN111464649B (en) * | 2017-04-19 | 2022-10-21 | 贵州白山云科技股份有限公司 | Access request source returning method and device |
| CN107229692A (en) * | 2017-05-19 | 2017-10-03 | 哈工大大数据产业有限公司 | A kind of distributed multi-table connecting method and system based on streamline |
| CN111291000A (en) * | 2018-05-18 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Block chain-based file acquisition method, equipment and storage medium |
| CN111291000B (en) * | 2018-05-18 | 2023-11-03 | 腾讯科技（深圳）有限公司 | File acquisition method, equipment and storage medium based on block chain |
| CN112491939A (en) * | 2019-09-12 | 2021-03-12 | 上海哔哩哔哩科技有限公司 | Multimedia resource scheduling method and system |
| CN112491939B (en) * | 2019-09-12 | 2022-12-27 | 上海哔哩哔哩科技有限公司 | Multimedia resource scheduling method and system |
| CN112749004A (en) * | 2019-10-30 | 2021-05-04 | 中国移动通信集团安徽有限公司 | Data storage method and device based on node access heat |
| CN112749004B (en) * | 2019-10-30 | 2023-09-05 | 中国移动通信集团安徽有限公司 | Data storage method and device based on node access heat |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101599994B (en) | 2012-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101599994B (en) | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes | |
| CN101741986B (en) | Page cache method for mobile communication equipment terminal | |
| CN107169083A (en) | Public security bayonet socket magnanimity vehicle data storage and retrieval method and device, electronic equipment | |
| CN103716343A (en) | Distributed service request processing method and system based on data cache synchronization | |
| CN102833352A (en) | Distributed cache management system and method for implementing distributed cache management | |
| CN105025053A (en) | Distributed file upload method based on cloud storage technology and system | |
| CN105005611B (en) | A kind of file management system and file management method | |
| CN103207841A (en) | Method and device for data reading and writing on basis of key-value buffer | |
| CN105868333A (en) | File processing method and device | |
| CN103685476B (en) | The method and web page template server that webpage shows are realized using web page template | |
| CN102375837A (en) | Data acquiring system and method | |
| CN103618787B (en) | A kind of webpage represents system and method | |
| CN101909068A (en) | Method, device and system for managing file copies | |
| CN102143212A (en) | Cache sharing method and device for content delivery network | |
| CN102104617A (en) | Method for storing massive picture data by website operating system | |
| CN102368776A (en) | Optimization function module of node list in content distribution/delivery network (CDN) | |
| CN102333130A (en) | Method and system for accessing cache server and intelligent cache scheduler | |
| CN102111301A (en) | General file transmission system and implementation method thereof | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| CN105207947A (en) | rogressive flow scheduling method and system capable of filtering vibration | |
| CN105068755A (en) | Data duplicate storage method facing cloud computing content distribution network | |
| CN102546674A (en) | Directory tree caching system and method based on network storage device | |
| CN102065136A (en) | P2P (Peer-to-Peer) network safety data transmission method and system | |
| CN106648471A (en) | Cloud platform data storage method and device | |
| CN102404372A (en) | Method, system and node device for storing content in WEB cache in distributed mode |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120718 Termination date: 20130601 |